PloneSoftwareCenter, pypi, re-releasing a product

[Maurits van Rees-3]

Hi,

I have been wondering about some PloneSoftwareCenter (PSC) stuff,
mostly the pypi additions and its effects on how project owners should
go about releasing a project on plone.org.

In previous versions of PloneSoftwareCenter when you had a final
version 1.2 you could click the re-release transition and the release
would be 1.2.1.  In the Poi project I have not seen that working for a
while: the latest (current) release is still called 1.2 even when I
have released 1.2.6 meanwhile and have clicked re-release each time I
made a new release.  It seems to at least have the effect of letting
Poi show op on top in the http://plone.org/products page.

The 1.2 release page has all 1.2.* releases, which is growing rather long:
http://plone.org/products/poi/releases/1.2

The releases page just lists the main releases, 1.1, 1.2 (both stable)
and 2.0 (experimental, not actually started on that yet):
http://plone.org/products/poi/releases

If I were to create a complete release of each 1.2.* minor release
then this releases page would become unwieldy again.

I guess that is how kupu does it though:
http://plone.org/products/kupu/releases lists 24 releases and I guess
that is readable enough actually.

Hm, then what is the difference between Poi and kupu here?  With Poi I
have set the distutils id in the edit form of the project.  I guess
kupu has not done that.  I am not actually using the distutils part of
PloneSoftwareCenter; I just upload to pypi and (for the 1.2 series) I
manually upload an old-style tarball with just the Poi directory.  So
I guess I might as well remove that distutils setting.


So some questions.

Do we know if the pypi possibilities of PSC are actively used?

What are the equivalants of http://pypi.python.org/simple/ and
http://pypi.python.org/simple/Products.Poi on plone.org?  What link
can be used as index instead of the official pypi?

Are the next two conclusions/recommendations valid?

- Only set the distutils id when you actually use that part of PSC
  (with collective.dist).  Do not use the rerelease transition in this
  case.  And I am not sure if the other transitions (alpha, beta, rc,
  final) make sense then.

- If you do not set the distutils id, the re-release transition works
  fine as it always has.

Cheers,

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Plone-website mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-website
The Wiki: http://plone.org/development/current/projects/WebsiteTeam

[Maurits van Rees-3]

Hi,

I did some tests locally with PSC trunk, so I will answer myself. :-)

Maurits van Rees, on 2009-10-23:

> Hi,
>
> I have been wondering about some PloneSoftwareCenter (PSC) stuff,
> mostly the pypi additions and its effects on how project owners should
> go about releasing a project on plone.org.
>
> In previous versions of PloneSoftwareCenter when you had a final
> version 1.2 you could click the re-release transition and the release
> would be 1.2.1.  In the Poi project I have not seen that working for a
> while:

I am not sure if there is a difference with previous PSC versions or
if I have just remembered it wrongly.  The re-release transition still
updates the release number, but *only* for releases in the alpha, beta
or release-candidate status, *not* for final releases.

The title of the project will become something like this:
Poi 1.2 (Alpha release 2)

If for release 1.2 you do 5 alpha releases, 6 betas, 7 rcs, 1 final
release and 8 re-releases of that final release (not that this would
make sense) this 1.2 will still remain one release.  All downloads
that you added during those stages will be listed on the release page.
For a bug fix release 1.2.1 you should create a completely new
release.  I have not done that up till now with Poi.  Mea culpa.  I'll
create a 1.2.7 release today to make up for that omission. :-)

> The releases page just lists the main releases, 1.1, 1.2 (both stable)
> and 2.0 (experimental, not actually started on that yet):
> http://plone.org/products/poi/releases
>
> If I were to create a complete release of each 1.2.* minor release
> then this releases page would become unwieldy again.
>
> I guess that is how kupu does it though:
> http://plone.org/products/kupu/releases lists 24 releases and I guess
> that is readable enough actually.

And on the main page of the project only the latest stable release
will be listed.  For Poi that is now 1.2, but will soon be 1.2.7.

> Hm, then what is the difference between Poi and kupu here?  With Poi I
> have set the distutils id in the edit form of the project.  I guess
> kupu has not done that.

The distutils id setting has nothing to do with it.

> I am not actually using the distutils part of
> PloneSoftwareCenter; I just upload to pypi and (for the 1.2 series) I
> manually upload an old-style tarball with just the Poi directory.  So
> I guess I might as well remove that distutils setting.

It is not harming after all, so I'll keep it.

If you use the distutils option and you upload your packages with
'python2.4 setup.py mregister sdist mupload' (with collective.dist
installed), the difference with the non-distutils old-style manual
handling is that alpha, beta, rc and final releases for a release 1.2
are not grouped together in one release, but they are really separate
releases.  Same as the official pypi really, so that is fine.

PSC actually does a good job of recognizing that you are uploading an
alpha, beta or rc version and will set the correct review state, at
least when your version numbering is sane.

> So some questions.
>
> Do we know if the pypi possibilities of PSC are actively used?
>
> What are the equivalants of http://pypi.python.org/simple/ and
> http://pypi.python.org/simple/Products.Poi on plone.org?  What link
> can be used as index instead of the official pypi?

In a local instance this works:
http://localhost:8080/test/products/links
where .../products is the url of the main PSC object.

Hm, except that release candidates are not listed...  I fixed this in
r100237.

On plone.org it would be this: http://plone.org/products/links
This has a redirect to http://dist.plone.org/packages/

The structure of that page differs from http://pypi.python.org/simple/
but I guess it can be used as index.

The links page on old.plone.org has no redirect:
http://old.plone.org/products/links
But this page is empty apart from the old layout.  No links to
packages here.

> Are the next two conclusions/recommendations valid?
>
> - Only set the distutils id when you actually use that part of PSC
>   (with collective.dist).  Do not use the rerelease transition in this
>   case.  And I am not sure if the other transitions (alpha, beta, rc,
>   final) make sense then.

Nah, distutils id is fine, even when you are not using it.

If you really upload and register from the command line with setup.py
then you won't need the transitions, so just ignore them.

> - If you do not set the distutils id, the re-release transition works
>   fine as it always has.

But not for final releases, but I guess that has always been the case.

Input is still welcome, but it's much clearer to me now. :-)

Oh, and on PSC trunk I get a few test failures, so some eyes are
welcome there.

Cheers,

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Plone-website mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-website
The Wiki: http://plone.org/development/current/projects/WebsiteTeam

[Maurits van Rees-3]

Maurits van Rees, on 2009-10-23:
> For a bug fix release 1.2.1 you should create a completely new
> release.  I have not done that up till now with Poi.  Mea culpa.  I'll
> create a 1.2.7 release today to make up for that omission. :-)

Done. :-)


I used zest.releaser to do a fullrelease on pypi and plone.org.  For
reference, this is my ~/.pypirc:

===========================
[distutils]
index-servers =
  pypi
  plone

[pypi]
#pypi.python.org
username:maurits
password:secret

[plone]
repository:http://plone.org/products
username:maurits
password:secret
===========================

In the Poi 1.2.x series I am uploading old-style tarballs to plone.org
with just the Poi directory, so not with Products.Poi/Products/Poi.
So I removed the uploaded egg and manually created and uploaded such
an old-style tarball.  So I did not use the complete pypi integration,
but the registering of the new long description went fine.

I'll add some random notes, which someone might find helpful or might
inspire someone to do some fixes.

I got a UnicodeDecodeError uploading to the official pypi with
'python2.4 setup.py sdist upload' (and the proper [server-login]
section in my .pypirc), probably caused by an umlaut that I recently
added in the HISTORY.txt.  With 'python2.4 setup.py sdist mupload'
(using collective.dist) this went fine.

For testing I used my local PloneSoftwareCenter instance.  A
'python2.4 setup.py mregister' failed with Unauthorized.  
'python2.4 setup.py mregister sdist upload' however succeeded.
Nothing appeared in the error logs, even though I removed Unauthorized
from the ignored exceptions, was running the Plone instance in the
foreground and had this in zope.conf:
debug-mode on
security-policy-implementation python
verbose-security on

You can still manually create a release, say 2.0, in PSC for planning
purposes without adding any downloads.  When you do a release 2.0 from
the command line PSC will happily add your file in that existing
release.  Any alpha, beta or rc releases you register and upload from
the command line are not added to that 2.0 release, as PSC does not
realize they belong together.

>> Hm, then what is the difference between Poi and kupu here?  With Poi I
>> have set the distutils id in the edit form of the project.  I guess
>> kupu has not done that.
>
> The distutils id setting has nothing to do with it.

Since I added Products.Poi as distutils id in
http://plone.org/products/poi PSC correctly associated the setup.py
mregister/mupload of Products.Poi with the existing poi project.

Cheers,

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Plone-website mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-website
The Wiki: http://plone.org/development/current/projects/WebsiteTeam

[Maurits van Rees-3]

Maurits van Rees, on 2009-10-23:
> You can still manually create a release, say 2.0, in PSC for planning
> purposes without adding any downloads.  When you do a release 2.0 from
> the command line PSC will happily add your file in that existing
> release.  Any alpha, beta or rc releases you register and upload from
> the command line are not added to that 2.0 release, as PSC does not
> realize they belong together.

I am doing some cleanup in the Poi project on plone.org.  I have just
given the 2.0 release the hidden state.  The result is that this
hidden release is now listed on the top of the page at
http://plone.org/products ...

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Plone-website mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-website
The Wiki: http://plone.org/development/current/projects/WebsiteTeam