genini fix

[Jürgen E. Fischer]

Hi,

I notices that the qgis nightly builds weren't picked up for the setup.ini.
This was due to the package version not being compared lexically, but not
numerically.

I applied following fix:

--- genini      2008-03-15 06:08:42.000000000 -0800
+++ /home/jef/genini    2009-08-02 21:45:03.000000000 -0700
@@ -170,7 +170,20 @@
     }

     return if $explicit;
-    my @files = sort grep{!/-src\.tar.bz2/} glob("$d/*.tar.bz2");
+    my @files = sort {
+                        my($an,$av,$ap) = ($a =~ /(.*)-([^-]+)-(\d+).tar.bz2$/);
+                        my($bn,$bv,$bp) = ($b =~ /(.*)-([^-]+)-(\d+).tar.bz2$/);
+
+                        if( defined $ap && defined $bp ) {
+                                return ($an ne $bn) ?
+                                        $an cmp $bn :
+                                        $av ne $bv ?
+                                                $av cmp $bv :
+                                                $ap <=> $bp;
+                        } else {
+                                return $a cmp $b;
+                        }
+                } grep { !/-src\.tar.bz2/ } glob("$d/*.tar.bz2");
     if (!@files) {
         myerror "not enough package files in $d";
         return;


That also explains why I couldn't use the revision number as package number
earlier.  Any opinions on switching to that now?


Jürgen

--
Jürgen E. Fischer         norBIT GmbH               Tel. [phone]
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. [phone]
Software Engineer         D-26506 Norden               http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
osgeo4w-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev

[Frank Warmerdam]

Jürgen E. Fischer wrote:

> Hi,
>
> I notices that the qgis nightly builds weren't picked up for the setup.ini.
> This was due to the package version not being compared lexically, but not
> numerically.
>
> I applied following fix:
>
> --- genini      2008-03-15 06:08:42.000000000 -0800
> +++ /home/jef/genini    2009-08-02 21:45:03.000000000 -0700
> @@ -170,7 +170,20 @@
>      }
>
>      return if $explicit;
> -    my @files = sort grep{!/-src\.tar.bz2/} glob("$d/*.tar.bz2");
> +    my @files = sort {
> +                        my($an,$av,$ap) = ($a =~ /(.*)-([^-]+)-(\d+).tar.bz2$/);
> +                        my($bn,$bv,$bp) = ($b =~ /(.*)-([^-]+)-(\d+).tar.bz2$/);
> +
> +                        if( defined $ap && defined $bp ) {
> +                                return ($an ne $bn) ?
> +                                        $an cmp $bn :
> +                                        $av ne $bv ?
> +                                                $av cmp $bv :
> +                                                $ap <=> $bp;
> +                        } else {
> +                                return $a cmp $b;
> +                        }
> +                } grep { !/-src\.tar.bz2/ } glob("$d/*.tar.bz2");
>      if (!@files) {
>          myerror "not enough package files in $d";
>          return;
>
>
> That also explains why I couldn't use the revision number as package number
> earlier.  Any opinions on switching to that now?

Jürgen,

I can't follow the change (my regex/perl foo is too weak).  Is the problem
that genini wasn't properly identifying which was the most recent version
due to the way versions are compared?  If so, there is a way of explicitly
listing the current and previous version in the setup.hint file and I wonder
if that would be a more appropriate way to control things.

If not, could you please confirm that this change does not affect the
setup.ini output for any other package?  I'm worried about it altering the
version for something else and us not realizing for some time.

I find genini very slow, and at some point I'd like to rewrite it in some
more efficient form.  I'm not really sure what part is the slow part.  But
Currently the genini step often times out on me if anything else is happening
on the machine at the same time.

Best regards,
--
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, [hidden email]
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent

_______________________________________________
osgeo4w-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev

[Jürgen E. Fischer]

Hi Frank,

On Mon, 03. Aug 2009 at 10:35:04 -0400, Frank Warmerdam wrote:
> I can't follow the change (my regex/perl foo is too weak).  Is the problem
> that genini wasn't properly identifying which was the most recent version due
> to the way versions are compared?  If so, there is a way of explicitly
> listing the current and previous version in the setup.hint file and I wonder
> if that would be a more appropriate way to control things.

Well, I'd say the patch shouldn't break anything.

The problem was that qgis-1.2.0-9.tar.bz2 was considered newer than
qgis-1.2.0-10.tar.bz2 as alphabetically "10" < "9".

The patch splits the file name into package name, version and numerical package
number. If that works it compares package name and version alphabetically and
the package number numerically.  It it doesn't, as the package number might not
be numerically, it falls back to the orignally behaviour.

> If not, could you please confirm that this change does not affect the
> setup.ini output for any other package?  I'm worried about it altering the
> version for something else and us not realizing for some time.

Already did.  It just affected the qgis-dev entry.

> I find genini very slow, and at some point I'd like to rewrite it in some
> more efficient form.  I'm not really sure what part is the slow part.  But
> Currently the genini step often times out on me if anything else is happening
> on the machine at the same time.

Ok.  I filed ticket #100 on that.  


Jürgen


--
Jürgen E. Fischer         norBIT GmbH               Tel. [phone]
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. [phone]
Software Engineer         D-26506 Norden               http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
osgeo4w-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev

[Matt Wilkie-2]

> I find genini very slow, and at some point I'd like to rewrite it in some
> more efficient form.  I'm not really sure what part is the slow part.  But
> Currently the genini step often times out on me if anything else is happening
> on the machine at the same time.
>  
When I was troubleshooting David's packing problem last week I noticed
that running regen.sh (genini) from a command shell seemed to be an
order of magnitude faster than from cgi. I ran the latter first though,
so perhaps the second and subsequent runs from shell were using memory
or disk cache, or something.

cheers,

-matt
_______________________________________________
osgeo4w-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev

[Jürgen E. Fischer]

Hi Frank,

On Mon, 03. Aug 2009 at 10:35:04 -0400, Frank Warmerdam wrote:
> I find genini very slow, and at some point I'd like to rewrite it in some
> more efficient form.  I'm not really sure what part is the slow part.  But
> Currently the genini step often times out on me if anything else is happening
> on the machine at the same time.

I added some comments.

The problem is that the md5sums of the packages are recalculated on each run.
That's taking >90% of the time.

#100 now has a patch that saves the md5sums along with timestamp and filesize
to a separate file.   So on re-runs the md5sum are reused unless the timestamp
or the file size changed.


Jürgen

--
Jürgen E. Fischer         norBIT GmbH               Tel. [phone]
Dipl.-Inf. (FH)           Rheinstraße 13            Fax. [phone]
Software Engineer         D-26506 Norden               http://www.norbit.de

--
norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH
Rheinstrasse 13, 26506 Norden
GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502

_______________________________________________
osgeo4w-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/osgeo4w-dev