ntf_r93.gsb file corrupted

13 messages Options
Embed this post
Permalink
Frank Warmerdam

ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Folks,

It appears the ntf_r93.gsb file used for grid shifts in Frace was corrupt
in the proj-datumgrid-1.4.zip file.  It would also have been corrupt if
extracted from subversion depending on the client OS and options.  I have
uploaded a new datumgrid zip file with the correct version.

   http://download.osgeo.org/proj/proj-datumgrid-1.5.zip

Details in:

   http://trac.osgeo.org/proj/ticket/52

I plan to update FWTools and OSGeo4W accordingly.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
RICHARD Didier

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Frank and all,

> Folks,
>
> It appears the ntf_r93.gsb file used for grid shifts in Frace was corrupt
> in the proj-datumgrid-1.4.zip file.  It would also have been corrupt if
> extracted from subversion depending on the client OS and options.  I have
> uploaded a new datumgrid zip file with the correct version.
>
>    http://download.osgeo.org/proj/proj-datumgrid-1.5.zip
>
> Details in:
>
>    http://trac.osgeo.org/proj/ticket/52
>

See my comments there.

> I plan to update FWTools and OSGeo4W accordingly.
>

Really nice move cause we've just identified the problem this afternoon.


We are going officially to tell the french community about this.

> Best regards,

Sincerely,

didier

> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> [hidden email]
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
> _______________________________________________
> Proj mailing list
> [hidden email]
> http://lists.maptools.org/mailman/listinfo/proj
>


--
RICHARD Didier - Chef du pôle technique du Géoportail
2/4, avenue Pasteur - 94165 Saint Mandé Cedex
Tél : +33 (0) 1 43 98 83 23
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
In reply to this post by Frank Warmerdam
A question :

Are not there some coordinate transformation tests which verifiy the
calculations ?

I don't know how often mistakes appears, but this type of tests don't seems
to be diificult to construct.

A no-regression test could have been mandatory, no ? Some unusual functions
might be broken without anybody pick out it.

Thanks

Jérôme


----- Original Message -----
From: "Frank Warmerdam" <[hidden email]>
To: "PROJ.4 and general Projections Discussions" <[hidden email]>
Sent: Wednesday, September 23, 2009 7:22 PM
Subject: [Proj] ntf_r93.gsb file corrupted


> Folks,
>
> It appears the ntf_r93.gsb file used for grid shifts in Frace was corrupt
> in the proj-datumgrid-1.4.zip file.  It would also have been corrupt if
> extracted from subversion depending on the client OS and options.  I have
> uploaded a new datumgrid zip file with the correct version.
>
>   http://download.osgeo.org/proj/proj-datumgrid-1.5.zip
>
> Details in:
>
>   http://trac.osgeo.org/proj/ticket/52
>
> I plan to update FWTools and OSGeo4W accordingly.
>
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> [hidden email]
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
> _______________________________________________
> Proj mailing list
> [hidden email]
> http://lists.maptools.org/mailman/listinfo/proj
>

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Frank Warmerdam

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Cuinet Jérôme wrote:
> A question :
>
> Are not there some coordinate transformation tests which verifiy the
> calculations ?
>
> I don't know how often mistakes appears, but this type of tests don't seems
> to be diificult to construct.

Jérôme,

There are regression tests, but they do not include ntf_r93.gsb even now.
I would add the problem corrupted the file in a relatively subtle way that
would only be detected for a grid shift in some places.

In fact I see there is a testIGNF but it was commented out in the past when
it wasn't obvious to determine whether we had the supporting gsb file or not.

I am updating the Makefile.am to include the testIGNF and testntv2 checks
if the corresponding grid shift files seem to be available.  I'll produce a
new beta later today with these changes.

Note these are only run on unix systems.

 > A no-regression test could have been mandatory, no ? Some unusual functions
 > might be broken without anybody pick out it.

The word "manditory" has limited application in open source.  It is very rare
that I get paid for any PROJ.4 maintenance, and I'm only doing this because
when I took it on there was no one else willing to do it.

There is certainly room for additional PROJ.4 developers and contributors.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Franck, thanks for these answers.

Fortunalately there are regression tests, quickly regarding source code I
have not seen them !

The "mandatory" term it isn't for maintenance. Updates concerns improvements
or corrections, no damaging ! Damages are on bêta corrections, on trunk
development tree. Release is an implicit declaration that the evolvement are
validated. Releases are not a "gamma version", and my system administrator
says me that update to the newest version is mandatory. I'm heretic if I
don't update my system.

It is principles that I think correct.

I know that no-regression detections are an hard task. But we can't damage a
software, we can also do improvements. Some users trust open source
developpers. If they can't, if an open source software is an always "gamma
software", a task is really never closed, we must say it !

I realize that this mail seems to be a troll, but I only want how I can
trust in open source software, and especially in proj4 library.

If that I think is wrong, so I will know that and I will not be able to
believe a software description :(


Best regards

Jérôme Cuinet


----- Original Message -----
From: "Frank Warmerdam" <[hidden email]>
To: "PROJ.4 and general Projections Discussions" <[hidden email]>
Sent: Thursday, September 24, 2009 8:49 PM
Subject: Re: [Proj] ntf_r93.gsb file corrupted


Cuinet Jérôme wrote:
> A question :
>
> Are not there some coordinate transformation tests which verifiy the
> calculations ?
>
> I don't know how often mistakes appears, but this type of tests don't
> seems
> to be diificult to construct.

Jérôme,

There are regression tests, but they do not include ntf_r93.gsb even now.
I would add the problem corrupted the file in a relatively subtle way that
would only be detected for a grid shift in some places.

In fact I see there is a testIGNF but it was commented out in the past when
it wasn't obvious to determine whether we had the supporting gsb file or
not.

I am updating the Makefile.am to include the testIGNF and testntv2 checks
if the corresponding grid shift files seem to be available.  I'll produce a
new beta later today with these changes.

Note these are only run on unix systems.

 > A no-regression test could have been mandatory, no ? Some unusual
functions
 > might be broken without anybody pick out it.

The word "manditory" has limited application in open source.  It is very
rare
that I get paid for any PROJ.4 maintenance, and I'm only doing this because
when I took it on there was no one else willing to do it.

There is certainly room for additional PROJ.4 developers and contributors.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
[hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Frank Warmerdam

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Cuinet Jérôme wrote:

> Franck, thanks for these answers.
>
> Fortunalately there are regression tests, quickly regarding source code I
> have not seen them !
>
> The "mandatory" term it isn't for maintenance. Updates concerns improvements
> or corrections, no damaging ! Damages are on bêta corrections, on trunk
> development tree. Release is an implicit declaration that the evolvement are
> validated. Releases are not a "gamma version", and my system administrator
> says me that update to the newest version is mandatory. I'm heretic if I
> don't update my system.
>
> It is principles that I think correct.
>
> I know that no-regression detections are an hard task. But we can't damage a
> software, we can also do improvements. Some users trust open source
> developpers. If they can't, if an open source software is an always "gamma
> software", a task is really never closed, we must say it !
>
> I realize that this mail seems to be a troll, but I only want how I can
> trust in open source software, and especially in proj4 library.
>
> If that I think is wrong, so I will know that and I will not be able to
> believe a software description :(

Jérôme,

I strongly encourage you and others to ensure that PROJ.4 or other lightly
maintained packages are suitable for your purposes if you depend on them
in a mission critical fashion.  Frankly, the project is not adequately
resourced to ensure that all good software development practices are
followed.

Even better maintained packages (like GDAL for instance) still have lots
of dark corners that are buggy and not regression tested.

So, implicit or explicit claims of validation are always "relative".  Little
is really certain and you are wise to confirm things meet your needs.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
A question to be sure :

Is Proj.4 linked in popular projects such as gdal, wich is linked in popular
desktop applications ?



----- Original Message -----
From: "Frank Warmerdam" <[hidden email]>
Jérôme,

I strongly encourage you and others to ensure that PROJ.4 or other lightly
maintained packages are suitable for your purposes if you depend on them
in a mission critical fashion.  Frankly, the project is not adequately
resourced to ensure that all good software development practices are
followed.

Even better maintained packages (like GDAL for instance) still have lots
of dark corners that are buggy and not regression tested.

So, implicit or explicit claims of validation are always "relative".  Little
is really certain and you are wise to confirm things meet your needs.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
[hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Frank Warmerdam

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Cuinet Jérôme wrote:
> A question to be sure :
>
> Is Proj.4 linked in popular projects such as gdal, wich is linked in popular
> desktop applications ?

Jérôme,

PROJ.4 is optionally used by GDAL.  It is used in popular applications
such as MapServer, and QGIS.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
OK,

So  if I can confirm some things, mostly can't do this.


Also, myself I do software with regressions. So I know how vicious are
regressions. But the purpose of my comment is no-regressions **could** be
mandatory. Today I don't depend on these software, so I can't work on these
projects. But tomorrow I don't know and it is why I do this technological
survey. And many people depends on them without knowing how to confirm
released version.


----- Original Message -----
From: "Frank Warmerdam" <[hidden email]>
To: "PROJ.4 and general Projections Discussions" <[hidden email]>
Sent: Thursday, September 24, 2009 10:21 PM
Subject: Re: [Proj] ntf_r93.gsb file corrupted


Cuinet Jérôme wrote:
> A question to be sure :
>
> Is Proj.4 linked in popular projects such as gdal, wich is linked in
> popular
> desktop applications ?

Jérôme,

PROJ.4 is optionally used by GDAL.  It is used in popular applications
such as MapServer, and QGIS.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
[hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
hamish-2

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Cuinet Jérôme wrote:
> Also, myself I do software with regressions. So I know how
> vicious are regressions. But the purpose of my comment is
> no-regressions **could** be mandatory. Today I don't depend
> on these software, so I can't work on these projects. But
> tomorrow I don't know and it is why I do this technological
> survey. And many people depends on them without knowing how
> to confirm released version.

I am sure that a donation to the project of a test-suite would
be most welcome.


regards,
Hamish



     
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
Yes,

I am also sure of that !

But could we validate a release until we are not sure that there is not
regression ? A test-suite do tests the quicklest. Not to have a test-suite
imply to do tests slowlier or not to do tests ?

Here, we have unix systems tests and tests disabled.

When I will use the product's project, I must donate, or I do as I could...


----- Original Message -----
From: "Hamish" <[hidden email]>
To: "PROJ.4 and general Projections Discussions" <[hidden email]>
Sent: Thursday, September 24, 2009 11:25 PM
Subject: Re: [Proj] ntf_r93.gsb file corrupted


Cuinet Jérôme wrote:
> Also, myself I do software with regressions. So I know how
> vicious are regressions. But the purpose of my comment is
> no-regressions **could** be mandatory. Today I don't depend
> on these software, so I can't work on these projects. But
> tomorrow I don't know and it is why I do this technological
> survey. And many people depends on them without knowing how
> to confirm released version.

I am sure that a donation to the project of a test-suite would
be most welcome.


regards,
Hamish




_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Eric Miller

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
In reply to this post by Frank Warmerdam
>>> On 9/24/2009 at 11:49 AM, Frank Warmerdam <[hidden email]>
wrote:
> Cuinet Jérôme wrote:
>> A question :
>>
>> Are not there some coordinate transformation tests which verifiy the

>> calculations ?
>>
>> I don't know how often mistakes appears, but this type of tests
don't seems
>> to be diificult to construct.
>
> Jérôme,
>
> There are regression tests, but they do not include ntf_r93.gsb even
now.
> I would add the problem corrupted the file in a relatively subtle way
that
> would only be detected for a grid shift in some places.

Wouldn't it be better to use md5 checksums or digital signatures to
verify that a data file's contents haven't been altered?  The md5 sum
approach doesn't provide much security, but it can be a quick
verification check that something hasn't gone horribly wrong with the
file.

md5sum ntf_r93.gsb > ntf_r93.gsb.md5sum

md5sum -c ntf_r93.gsb.md5sum

Or some variation...

--

Eric G. Miller
Staff Programmer
CA Dept. of Fish & Game

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Cuinet Jérôme

Re: ntf_r93.gsb file corrupted

Reply Threaded More More options
Print post
Permalink
MD5 test is to tests the file integrity. But if here the problem is due to
file integrity, further problem of regression may be due to another cause.

Development tests may include MD5 integrity test, but finally global test
could be made. The aim is to detect a problem, next to diagnostic it, is it
?


----- Original Message -----
From: "Eric Miller" <[hidden email]>
To: "PROJ.4 and general Projections Discussions" <[hidden email]>
Sent: Friday, September 25, 2009 12:17 AM
Subject: Re: [Proj] ntf_r93.gsb file corrupted


>>> On 9/24/2009 at 11:49 AM, Frank Warmerdam <[hidden email]>
wrote:
> Cuinet Jérôme wrote:
>> A question :
>>
>> Are not there some coordinate transformation tests which verifiy the

>> calculations ?
>>
>> I don't know how often mistakes appears, but this type of tests
don't seems
>> to be diificult to construct.
>
> Jérôme,
>
> There are regression tests, but they do not include ntf_r93.gsb even
now.
> I would add the problem corrupted the file in a relatively subtle way
that
> would only be detected for a grid shift in some places.

Wouldn't it be better to use md5 checksums or digital signatures to
verify that a data file's contents haven't been altered?  The md5 sum
approach doesn't provide much security, but it can be a quick
verification check that something hasn't gone horribly wrong with the
file.

md5sum ntf_r93.gsb > ntf_r93.gsb.md5sum

md5sum -c ntf_r93.gsb.md5sum

Or some variation...

--

Eric G. Miller
Staff Programmer
CA Dept. of Fish & Game

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj