Things we need to do to ZopeSkel

[Martin Aspeli-2]

Hi guys,

First of all, if you haven't checked out ZopeSkel local commands, you
should. Get the latest version:

  $ easy_install -U ZopeSkel

and then do:

  $ paster create -t archetype my.contenttype
  (answer questions)

  $ cd my.contenttype
  $ paster addcontent contenttype
  (answer questions)

  $ paster addcontent view
  (answer questions)

  $ paster addcontent portlet
  (answer questions)

This stuff is really great. :) It saves a ton of typing, it allows us to
automate some best-practice stuff, and it's pretty slick.

To make it *very* slick here are a few things we should do. Some people
(Tarek, Mustapha, David, others) are doing some great work on ZopeSkel.
If we can polish a tiny bit more, then this will go a long way towards
helping people get started as developers.

  1. Document this on plone.org better.

  2. Make sure each time you generate a package, there's a README.txt
that says "now here are some things you could do" and explain the basic
layout of the generated code.

  3. Clean up some ZopeSkel/paster warts:

        - Things sometimes have bad defaults, e.g. an "archetype" package has
to be a Zope 2 product, but it defaults to False.

        - zip_safe should not be an option - it should always be False, since
Zope 3 can't read ZCML inside a zipped egg. This is nasty, since you
don't notice during development when you're in develop mode!

        - If you create an egg, you have to make sure that the foldername and
the namespace + package names match up. e.g. "paster create -t plone
my.package" means you *have* to say namespace = "my" and package =
"package".

        - Typing "True" and "False" sucks. We need to ensure we can accept t,
f, T, F, y, n, Y N etc.

        - The description text is something badly formatted and hard to read.
If we could fix the layout a bit so that the prompt is always on a new
line and the text is wrapped properly, that'd be nice

        - You always get a tests.py, and sometimes you get a tests/ package as
well. One will beat the other. tests.py is also a bit limited for most
purposes.

There are probably a few more like these.

  4. Make sure there are sufficient inline comments (but too many!) for
the things we generate so people understand what they do.

  5. Think about new things we could add as local commands:

        - I'd like to see something for generating basic workflows, maybe where
you optionally pick the states and transtions and permissions you want
to manage.

        - An option to create a viewlet would be nice

        - The 'view' local command should allow you to set the context type, maybe

        - We could maybe have some local commands in the plone3_theme template
for adding a customisation of a particular view, viewlet or skin
resource. If we could get something where you paste a name from the
browser into paster, that'd be very helpful.

        - Commands to add tests!

  6. Think about new skeletons we'd like to support.

        - Maybe a basic "policy" package that has a GS profile but not a
content type.


ZopeSkel is not a panacea. It only really works if you have a pretty
good idea what's going on - but it lowers the bar, it makes it easier to
get started, and it saves a lot of typing. It also makes it easier to
write documentation, since you can skip a lot of boilerplate detail that
people are likely to screw up anyway. :)

Cheers,
Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Martin Aspeli-2]

Martin Aspeli wrote:

<snip>

> This stuff is really great. :) It saves a ton of typing, it allows us to
> automate some best-practice stuff, and it's pretty slick.
>
> To make it *very* slick here are a few things we should do. Some people
> (Tarek, Mustapha, David, others) are doing some great work on ZopeSkel.
> If we can polish a tiny bit more, then this will go a long way towards
> helping people get started as developers.

<snip>

> ZopeSkel is not a panacea. It only really works if you have a pretty
> good idea what's going on - but it lowers the bar, it makes it easier to
> get started, and it saves a lot of typing. It also makes it easier to
> write documentation, since you can skip a lot of boilerplate detail that
> people are likely to screw up anyway. :)

Oh, and if it wasn't obvious ... I was hoping for volunteers. ;-)

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Wichert Akkerman]

Previously Martin Aspeli wrote:
> - The description text is something badly formatted and hard to read.
> If we could fix the layout a bit so that the prompt is always on a new
> line and the text is wrapped properly, that'd be nice

I would love to see the templates changed to use README.txt as
(long) description, possibly including docs/HISTORY.txt. That's the de
facto standard anyway and encourages people to properly document their
package.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Tarek Ziadé]

wichert wrote:

Previously Martin Aspeli wrote:
> - The description text is something badly formatted and hard to read.
> If we could fix the layout a bit so that the prompt is always on a new
> line and the text is wrapped properly, that'd be nice

I would love to see the templates changed to use README.txt as
(long) description, possibly including docs/HISTORY.txt. That's the de
facto standard anyway and encourages people to properly document their
package.

Yeah, that what we did with the recipe skeleton: we reused the same
standard than Jim to have a proper PyPI page.

Another thing I think it's important is to provide doctest browser
tests everywhere, with some boiler plate code that allows people to start
writing some doctests right away. Even a small example would be great in
the generated doctest

Tarek

[Tarek Ziadé]

Martin Aspeli wrote:

<snip>

> ZopeSkel is not a panacea. It only really works if you have a pretty
> good idea what's going on - but it lowers the bar, it makes it easier to
> get started, and it saves a lot of typing. It also makes it easier to
> write documentation, since you can skip a lot of boilerplate detail that
> people are likely to screw up anyway. :)

Oh, and if it wasn't obvious ... I was hoping for volunteers. ;-)

I'm in for these tasks.

If some other people are showing up, we can split the tasks on some irc meeting.
Paris sprint will probably have a  ZopeSkel topic as well

Tarek

[Florian Schulze-2]

On Wed, 20 Feb 2008 09:39:12 +0100, Wichert Akkerman  
<[hidden email]> wrote:

> Previously Martin Aspeli wrote:
>> - The description text is something badly formatted and hard to read.
>> If we could fix the layout a bit so that the prompt is always on a new
>> line and the text is wrapped properly, that'd be nice
>
> I would love to see the templates changed to use README.txt as
> (long) description, possibly including docs/HISTORY.txt. That's the de
> facto standard anyway and encourages people to properly document their
> package.

Is there a way to test the result before upload? So you don't get rest  
errors and need to retry?

Regards,
Florian Schulze


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Florian Schulze-2]

On Wed, 20 Feb 2008 02:43:22 +0100, Martin Aspeli  
<[hidden email]> wrote:

> - zip_safe should not be an option - it should always be False, since
> Zope 3 can't read ZCML inside a zipped egg. This is nasty, since you
> don't notice during development when you're in develop mode!

Is it possible to fix this? I remember that some people made some tests  
with zipped Products in Zope 2 and they loaded much faster, because the IO  
is the bottleneck for the startup time in many cases. We should at least  
try to gradually improve this, and I think the chance to get this working  
with Zope3 packages is much higher than with Zope2 products.

Regards,
Florian Schulze


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Tarek Ziadé]

Florian Schulze-2 wrote:

Is there a way to test the result before upload? So you don't get rest  
errors and need to retry?

Great idea !

I think we can do it on on a lower level in distutils, in the register command,
and avoid bugging the skeleton with that, because this feature is missing for all eggs.
As a matter of fact, the task I am working on from the strategic summit
is changing this command. (iw.dist)

I can add there a check on long_description to display a error if it
is not reST-compliant.

++
Tarek

[Reinout van Rees]

Wichert Akkerman schreef:
> Previously Martin Aspeli wrote:
>> - The description text is something badly formatted and hard to read.
>> If we could fix the layout a bit so that the prompt is always on a new
>> line and the text is wrapped properly, that'd be nice
>
> I would love to see the templates changed to use README.txt as
> (long) description, possibly including docs/HISTORY.txt. That's the de
> facto standard anyway and encourages people to properly document their
> package.

Agreed.

And I'd like to see some standard on where to place the README. If you
have a two-level collective.something package, the readme gets placed in
collective.something or collective.something/docs.

What I'm used to is to have it in
collective.something/collective/something/README.txt


version.txt is also commonly found in
collective.something/collective/something/version.txt, and that's
another candidate imho for including automatically in setup.py's version
field.

Reinout

--
Reinout van Rees          Blog: http://vanrees.org/weblog/
[hidden email]          Work: http://zestsoftware.nl/
http://vanrees.org          Video: http://reinout.blip.tv/


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Wichert Akkerman]

Previously Reinout van Rees wrote:

> Wichert Akkerman schreef:
> > Previously Martin Aspeli wrote:
> >> - The description text is something badly formatted and hard to read.
> >> If we could fix the layout a bit so that the prompt is always on a new
> >> line and the text is wrapped properly, that'd be nice
> >
> > I would love to see the templates changed to use README.txt as
> > (long) description, possibly including docs/HISTORY.txt. That's the de
> > facto standard anyway and encourages people to properly document their
> > package.
>
> Agreed.
>
> And I'd like to see some standard on where to place the README. If you
> have a two-level collective.something package, the readme gets placed in
> collective.something or collective.something/docs.

The root of the package is the standard place for all documentation
for both non-Zope and zope packages, so lets use that.

> version.txt is also commonly found in
> collective.something/collective/something/version.txt, and that's
> another candidate imho for including automatically in setup.py's version
> field.

Or we can just make that file go away. When needed we should be able to
extract the version number from the setuptools workingset data.

Wichert.

--
Wichert Akkerman <[hidden email]>    It is simple to make things.
http://www.wiggy.net/                   It is hard to make things simple.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Maurits van Rees-3]

Florian Schulze, on 2008-02-20:
> Is there a way to test the result before upload? So you don't get rest  
> errors and need to retry?

For checking rest I usually do "rst2html README.txt > readme.html"

On Ubuntu this is included in the python-docutils package.  Works for
me.

--
Maurits van Rees | http://maurits.vanrees.org/
            Work | http://zestsoftware.nl/
"This is your day, don't let them take it away." [Barlow Girl]


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Damien Baty (ML)]

        Hello,

Le 20/02/08 17:16, Maurits van Rees a écrit :
> Florian Schulze, on 2008-02-20:
>> Is there a way to test the result before upload? So you don't get rest  
>> errors and need to retry?
>
> For checking rest I usually do "rst2html README.txt > readme.html"
>
> On Ubuntu this is included in the python-docutils package.  Works for
> me.

   I would also recommend "restview" by Marius Gedminas. Very handy:

     http://mg.pov.lt/restview

   Hope it helps.

        Regards,

--
Damien Baty
Pilot Systems - 9, rue Desargues - 75011 Paris - France
Telephone : [phone] - http://www.pilotsystems.net
Free Zope & Plone hosting - http://www.objectis.org

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Martin Aspeli-2]

Florian Schulze wrote:

> On Wed, 20 Feb 2008 02:43:22 +0100, Martin Aspeli  
> <[hidden email]> wrote:
>
>> - zip_safe should not be an option - it should always be False, since
>> Zope 3 can't read ZCML inside a zipped egg. This is nasty, since you
>> don't notice during development when you're in develop mode!
>
> Is it possible to fix this? I remember that some people made some tests  
> with zipped Products in Zope 2 and they loaded much faster, because the IO  
> is the bottleneck for the startup time in many cases. We should at least  
> try to gradually improve this, and I think the chance to get this working  
> with Zope3 packages is much higher than with Zope2 products.

In theory it should be easy to fix the ZCML parser to use the setuptools
API to load resources. I have no idea how hard it is in practice, though.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[jjmurre]

Martin Aspeli wrote:
> Hi guys,
>
<snip>

>
> ZopeSkel is not a panacea. It only really works if you have a pretty
> good idea what's going on - but it lowers the bar, it makes it easier to
> get started, and it saves a lot of typing. It also makes it easier to
> write documentation, since you can skip a lot of boilerplate detail that
> people are likely to screw up anyway. :)
>
> Cheers,
> Martin
>

</snip>

It would be nice if some of the ZopeSkel templates had the possibility
to generate > 2 level namespace packages, like eg. 'plone.app.whatever'.

Regards, Jan


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Martin Aspeli-2]

Tarek Ziadé wrote:
>
> jjmurre wrote:
>> It would be nice if some of the ZopeSkel templates had the possibility
>> to generate > 2 level namespace packages, like eg. 'plone.app.whatever'.
>>
>
> +1
>
> The recipe skeleton has a 3 level namespace, but we should add a generic one

The plone_app template is a three-level thing as well. It's pretty
painful to maintain a two-level and a three-level version of the same
template. Two-level is still more common and useful in my experience and
opinion.

Martin

--
Author of `Professional Plone Development`, a book for developers who
want to work with Plone. See http://martinaspeli.net/plone-book


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Andreas Zeidler]

On Feb 20, 2008, at 10:11 AM, Tarek Ziadé wrote:
> Another thing I think it's important is to provide doctest browser
> tests everywhere, with some boiler plate code that allows people to  
> start
> writing some doctests right away. Even a small example would be  
> great in
> the generated doctest

a while ago i've created a test module that scans a given directory  
for *.txt (and *.rst) files and automatically sets them up as doctests  
for your package.  with this the only thing you need to do to wire up  
a new test is to create a file with the name pattern "test*.txt"  
within that directory.  it also scans subdirectories and respects the  
"-t" option to the testrunner, making it possible to specify a reg-ex  
to conveniently run only a subset of your tests or a single one.  i've  
worked with this in a project for which we wrote around 180 doctests  
and it was very helpful.

i've also reused the module for plone.app.linkintegrity, so you can  
actually have a look at it[1].  there are some additional helpers in  
the test class, but lines 64ff should give you the general idea.  i'd  
be happy to integrate this into ZopeSkel if people are interested.

cheers,


andi

[1] http://dev.plone.org/plone/browser/plone.app.linkintegrity/trunk/plone/app/linkintegrity/tests/testDocTests.py?rev=16482

--
zeidler it consulting - http://zitc.de/ - [hidden email]
friedelstraße 31 - 12047 berlin - telefon [phone]
pgp key at http://zitc.de/pgp - http://wwwkeys.de.pgp.net/
plone 3.0.6 released! -- http://plone.org/products/plone



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Plone-developers mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/plone-developers

[Tarek Ziadé]

Andreas Zeidler wrote:

a while ago i've created a test module that scans a given directory  
for *.txt (and *.rst) files and automatically sets them up as doctests  
for your package.  with this the only thing you need to do to wire up  
a new test is to create a file with the name pattern "test*.txt"  
within that directory.  it also scans subdirectories and respects the  
"-t" option to the testrunner, making it possible to specify a reg-ex  
to conveniently run only a subset of your tests or a single one.  i've  
worked with this in a project for which we wrote around 180 doctests  
and it was very helpful.

Yes exactly.

That's what I have added in ZopeSkel, all .txt files in docs/ folder are
launched, so the -t option works.

I have added an example in README.txt


Tarek