Chandler Desktop packaging in Debian and Ubuntu

8 messages Options
Embed this post
Permalink
Matt Schafer-2

Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink
Howdy,

After building the current trunk of Chandler Desktop for Ubuntu 9.04
(Jaunty) [1], I decided to find out what it would take to package this
into a .deb that would be available on the Ubuntu repositories.  After
some initial investigation, I found that someone has started a
packaging effort in Debian [2].  He and I have decided to team up to
do the Debian packaging.  It will give me a chance to learn from an
experienced maintainer and give him some manpower to figure out the
chandler source tree.

As for Ubuntu.  They pick up all packages in Debian unstable prior to
their DebianImportFreeze date [3].  They generally do not include new
packages or updates to packages after the Ubuntu release except for
security fixes and, rarely, major functionality fixes.  For the next
release of Ubuntu (9.10, codenamed Karmic Koala), DebianImportFreeze
is tomorrow [4].  Since Ubuntu has a 6-month release cycle, I would
guess that gives us at most 6 months to get the package incorporated
into Debian in time for Ubuntu Karmic+1.

In addition to the official Debian and Ubuntu repositories, we can
create packages and upload them to a custom repository hosted on the
OSAF web site or using a PPA hosted in Launchpad [5].  This would
allow us to get the packages out sooner to Chandler users and to
upload new fixes and releases between the Ubuntu releases.

Hopefully, we can get this package created by Karmic+1.  At any rate,
I'd be happy to work directly with the Chandler project and devs with
any fixes and suggestions, if you're interested.

Please let me know if this plan is not desirable.

Matt

PS:  Sorry for the ever-changing email addresses and IRC nicknames.
(I was [hidden email].)  I've finally settled on what I want to
move forward with.  This is where you'll see me in the future.

References:
[1]  http://markmail.org/thread/u4zybvu7womnwxmp
[2]  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=449031
[3]  https://wiki.ubuntu.com/DebianImportFreeze
[4]  https://wiki.ubuntu.com/KarmicReleaseSchedule
[5]  http://launchpad.net/ is the package maintenance system for Ubuntu.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
Jos Yule-3

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink
Wow, that sounds great! Thanks for spending the time getting this
together, and please carry on. It would be great to have a .deb for Ubuntu.

+1

jos

--
Jos Yule
Digital Hyakugei
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
Grant Baillie

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink
In reply to this post by Matt Schafer-2
Hi, Matt

That plan sounds great. Overall, I'm fine with using a launchpad PPA  
if that's simplest; also I think Jared set up one @ OSAF a while back.

I suspect that there is a fair amount of work to do (as you can see by  
running lintian on the deb you produced :o). There are a few areas I  
can think of:

1) Chandler's dependence on patched external libraries, viz wxPython  
and Berkeley DB (package libdbX.Y). So far as those go:

- I have finally gotten a chance to work on creating patches (against  
wx trunk) for the various wx enhancements Chandler needs. The plan is  
to submit those as enhancement requests/bugs that should come out in  
wx 2.9.x. Hopefully that is consistent with your Karmic+1 plan.

- A while back, Jacob Floyd did some work on creating Chandler  
packages for OpenSUSE (IIRC, it could have been another non-Debian  
Linux). He basically created a libchandlerdb package out of Chandler's  
patched Berkeley DB (with the library names changes to avoid file  
conflicts). Possibly this is a route to consider for Ubuntu.

- Otherwise, as I remember it, there's a thread on the Berkeley DB dev  
list somewhere about a patch Andi submitted that somehow appeared in  
their trunk in a form that wasn't thread-safe. I can hunt around for  
the link if we want to try to get them to change their code. Or,  
possibly Andi remembers more.

2) The current Chandler build process uses Python's package management  
system (setuptools/pypi.python.org) to check for dependencies and  
install python packages. (Examples are EggTranslations and zanshin).  
Presumably, we should create .debs for these, and I think Jared Rhine  
started the process of doing that, e.g. http://svn.osafoundation.org/sandbox/packaging/deb/zanshin/trunk 
.

3) I suspect that the localization mechanism (plugins and  
EggTranslations) is violating Debian policy in some ways; there might  
have to be changes to the i18n code if this is the case. Or we could  
distribute binary python eggs for all the translatable data (icons and/
or gettext files)

4) Possibly lower down on the totem pole is that some of our  
dependencies (e.g. epydoc, docutils, pylint) are really only there to  
run tests and/or create documentation. In a world where I had a lot of  
time I would break chandler up into chandler and chandler-dev  
packages, where the chandler package just had the stuff you need to  
run the app.

--Grant

On 24 Jun, 2009, at 15:11, Matt Schafer wrote:

> Howdy,
>
> After building the current trunk of Chandler Desktop for Ubuntu 9.04
> (Jaunty) [1], I decided to find out what it would take to package this
> into a .deb that would be available on the Ubuntu repositories.  After
> some initial investigation, I found that someone has started a
> packaging effort in Debian [2].  He and I have decided to team up to
> do the Debian packaging.  It will give me a chance to learn from an
> experienced maintainer and give him some manpower to figure out the
> chandler source tree.
>
> As for Ubuntu.  They pick up all packages in Debian unstable prior to
> their DebianImportFreeze date [3].  They generally do not include new
> packages or updates to packages after the Ubuntu release except for
> security fixes and, rarely, major functionality fixes.  For the next
> release of Ubuntu (9.10, codenamed Karmic Koala), DebianImportFreeze
> is tomorrow [4].  Since Ubuntu has a 6-month release cycle, I would
> guess that gives us at most 6 months to get the package incorporated
> into Debian in time for Ubuntu Karmic+1.
>
> In addition to the official Debian and Ubuntu repositories, we can
> create packages and upload them to a custom repository hosted on the
> OSAF web site or using a PPA hosted in Launchpad [5].  This would
> allow us to get the packages out sooner to Chandler users and to
> upload new fixes and releases between the Ubuntu releases.
>
> Hopefully, we can get this package created by Karmic+1.  At any rate,
> I'd be happy to work directly with the Chandler project and devs with
> any fixes and suggestions, if you're interested.
>
> Please let me know if this plan is not desirable.
>
> Matt
>
> PS:  Sorry for the ever-changing email addresses and IRC nicknames.
> (I was [hidden email].)  I've finally settled on what I want to
> move forward with.  This is where you'll see me in the future.
>
> References:
> [1]  http://markmail.org/thread/u4zybvu7womnwxmp
> [2]  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=449031
> [3]  https://wiki.ubuntu.com/DebianImportFreeze
> [4]  https://wiki.ubuntu.com/KarmicReleaseSchedule
> [5]  http://launchpad.net/ is the package maintenance system for  
> Ubuntu.
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Open Source Applications Foundation "chandler-dev" mailing list
> http://lists.osafoundation.org/mailman/listinfo/chandler-dev

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
Andi Vajda

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink

On Mon, 29 Jun 2009, Grant Baillie wrote:

> - Otherwise, as I remember it, there's a thread on the Berkeley DB dev
> list somewhere about a patch Andi submitted that somehow appeared in
> their trunk in a form that wasn't thread-safe. I can hunt around for
> the link if we want to try to get them to change their code. Or,
> possibly Andi remembers more.

It's about a callback I added to the DBT structure in order to avoid
multiple byte array copies when loading strings into python.

They took the patch and moved the callback to the DBENV (?) structure since
that's all they needed to use the idea for their Java interface. I need to
be able to use different callbacks, not globally, but for the current
operation's DBT.

Andi..
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
Jacob Floyd

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink
In reply to this post by Grant Baillie
Hey there,
I'm Jacob Floyd, aka techgurufloyd. I was working for several months on getting Chandler to build on Gentoo. Gentoo is source-based, and to get it into the main tree, I couldn't include the dependencies in the build... It had to use the system libs wherever possible.
My comments inline:

> 1) Chandler's dependence on patched external libraries, viz wxPython
> and Berkeley DB (package libdbX.Y). So far as those go:
>
> - I have finally gotten a chance to work on creating patches (against
> wx trunk) for the various wx enhancements Chandler needs. The plan is
> to submit those as enhancement requests/bugs that should come out in
> wx 2.9.x. Hopefully that is consistent with your Karmic+1 plan.

Sweet. Making those patches bogged me down and my efforts had to focus elsewhere. Where can I get your patches? Like a tarball of them or something.
At least for gentoo, It was way too much hastle to create a separate chandlerwx because of the way python packages work (conversely, I did build db as a separate chandlerdb-lib package, as the libs lent themself well to this technique). So, I figured I'd build wx as a part of Chandler, at least until upstream gets the patches. To do so, I wanted to use the vanilla source tarballs, and apply a patch for each feature, so that as they're included upstream, I can drop the patch, until the point we can use a system version of wx. This also allows me to apply each of the gentoo specific patches, and make sure it's easy (or at least easier) for the Security team to apply any patches to both sources (vanilla wx and the chandler variant) as necessary. Maybe something similar will be needed on Ubuntu.

> - A while back, Jacob Floyd did some work on creating Chandler
> packages for OpenSUSE (IIRC, it could have been another non-Debian
> Linux). He basically created a libchandlerdb package out of Chandler's
> patched Berkeley DB (with the library names changes to avoid file
> conflicts). Possibly this is a route to consider for Ubuntu.

I was working on Gentoo. However, I've not been able to spend a lot of time on that for a month or two. If anyone's interested in what I did, I can help. However, my focus right now is to get a paid job.
I made a package I called chandlerdb-lib that installs the db libs concurrently with the system db keeping everything separate. That was an interesting adventure. I can show the build process if desired.

> - Otherwise, as I remember it, there's a thread on the Berkeley DB dev
> list somewhere about a patch Andi submitted that somehow appeared in
> their trunk in a form that wasn't thread-safe. I can hunt around for
> the link if we want to try to get them to change their code. Or,
> possibly Andi remembers more.

I've put a link to that thread on http://chandlerproject.org/Developers/GentooBuildNotes
DB_DBT_USERCOPY: http://forums.oracle.com/forums/thread.jspa?messageID=1504262�
OSAF Bug#11547: http://bugzilla.osafoundation.org/show_bug.cgi?id=11547

===[SNIP]===
> 4) Possibly lower down on the totem pole is that some of our
> dependencies (e.g. epydoc, docutils, pylint) are really only there to
> run tests and/or create documentation. In a world where I had a lot of
> time I would break chandler up into chandler and chandler-dev
> packages, where the chandler package just had the stuff you need to
> run the app.

I've outlined the deps, at least as far as the gentoo packages are concerned, on the page: http://chandlerproject.org/Developers/GentooBuildNotes

Another thing I was going to do is divide chandler into two packages: chandler and chandler-dev. Chandler-dev would have some things that would only interest those that would be developing with chandler. I was also going to create a separate package for each of the plugins, with Use-based (ie optional) dependencies from chandler to those plugins. That way someone could install it if they wished.

Anyway, I hope that helps. If not, just ignore it. ;-)

Jacob Floyd

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev

signature.asc (279 bytes) Download Attachment
Grant Baillie

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink

On 29 Jun, 2009, at 13:48, Jacob Floyd wrote:

> Hey there,
> I'm Jacob Floyd, aka techgurufloyd. I was working for several months  
> on getting Chandler to build on Gentoo. Gentoo is source-based, and  
> to get it into the main tree, I couldn't include the dependencies in  
> the build... It had to use the system libs wherever possible.
> My comments inline:
>
>> 1) Chandler's dependence on patched external libraries, viz wxPython
>> and Berkeley DB (package libdbX.Y). So far as those go:
>>
>> - I have finally gotten a chance to work on creating patches (against
>> wx trunk) for the various wx enhancements Chandler needs. The plan is
>> to submit those as enhancement requests/bugs that should come out in
>> wx 2.9.x. Hopefully that is consistent with your Karmic+1 plan.
>
> Sweet. Making those patches bogged me down and my efforts had to  
> focus elsewhere. Where can I get your patches? Like a tarball of  
> them or something.

Hi, Jacob

For now, I have put the patches up on people.osafoundation.org:

The original changes (i.e. wx 2.8.something against Chandler's wx):

     http://people.osafoundation.org/grant/patches-r55849.tar.gz

Ported over to current wx trunk:

     http://people.osafoundation.org/grant/patches-r61245.tar.gz

Just in case it's not clear, see

     http://people.osafoundation.org/grant/wx-checkout-info-r61245.txt

which is how I did the checkouts to patch.

This last set I haven't even compiled; there have been some  
rearrangements in the source base, so I just got done porting the  
patches. I'm about to go on vacation, but I'll probably try doing that  
once I get back (next week).

> At least for gentoo, It was way too much hastle to create a separate  
> chandlerwx because of the way python packages work (conversely, I  
> did build db as a separate chandlerdb-lib package, as the libs lent  
> themself well to this technique). So, I figured I'd build wx as a  
> part of Chandler, at least until upstream gets the patches. To do  
> so, I wanted to use the vanilla source tarballs, and apply a patch  
> for each feature, so that as they're included upstream, I can drop  
> the patch, until the point we can use a system version of wx. This  
> also allows me to apply each of the gentoo specific patches, and  
> make sure it's easy (or at least easier) for the Security team to  
> apply any patches to both sources (vanilla wx and the chandler  
> variant) as necessary. Maybe something similar will be needed on  
> Ubuntu.
>
>> - A while back, Jacob Floyd did some work on creating Chandler
>> packages for OpenSUSE (IIRC, it could have been another non-Debian
>> Linux). He basically created a libchandlerdb package out of  
>> Chandler's
>> patched Berkeley DB (with the library names changes to avoid file
>> conflicts). Possibly this is a route to consider for Ubuntu.
>
> I was working on Gentoo. However, I've not been able to spend a lot  
> of time on that for a month or two. If anyone's interested in what I  
> did, I can help. However, my focus right now is to get a paid job.
> I made a package I called chandlerdb-lib that installs the db libs  
> concurrently with the system db keeping everything separate. That  
> was an interesting adventure. I can show the build process if desired.

I think that would be interesting, as it could be ported over to  
Debian (Assuming it's OK with all the various licenses and policies).

>
>> - Otherwise, as I remember it, there's a thread on the Berkeley DB  
>> dev
>> list somewhere about a patch Andi submitted that somehow appeared in
>> their trunk in a form that wasn't thread-safe. I can hunt around for
>> the link if we want to try to get them to change their code. Or,
>> possibly Andi remembers more.
>
> I've put a link to that thread on http://chandlerproject.org/Developers/GentooBuildNotes
> DB_DBT_USERCOPY: http://forums.oracle.com/forums/thread.jspa?messageID=1504262�
> OSAF Bug#11547: http://bugzilla.osafoundation.org/show_bug.cgi?
> id=11547

(Oh great, I knew there was a link to that somewhere).

>
> ===[SNIP]===
>> 4) Possibly lower down on the totem pole is that some of our
>> dependencies (e.g. epydoc, docutils, pylint) are really only there to
>> run tests and/or create documentation. In a world where I had a lot  
>> of
>> time I would break chandler up into chandler and chandler-dev
>> packages, where the chandler package just had the stuff you need to
>> run the app.
>
> I've outlined the deps, at least as far as the gentoo packages are  
> concerned, on the page: http://chandlerproject.org/Developers/GentooBuildNotes
>
> Another thing I was going to do is divide chandler into two  
> packages: chandler and chandler-dev. Chandler-dev would have some  
> things that would only interest those that would be developing with  
> chandler. I was also going to create a separate package for each of  
> the plugins, with Use-based (ie optional) dependencies from chandler  
> to those plugins. That way someone could install it if they wished.

Yeah, that makes total sense.

Cheers,
--Grant


_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev
Jacob Floyd

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink
Comments included below:

On Tue, Jun 30, 2009 at 18:22, Grant Baillie<[hidden email]> wrote:
>
===[SNIP]===

> Hi, Jacob
>
> For now, I have put the patches up on people.osafoundation.org:
>
> The original changes (i.e. wx 2.8.something against Chandler's wx):
>
>     http://people.osafoundation.org/grant/patches-r55849.tar.gz
>
> Ported over to current wx trunk:
>
>     http://people.osafoundation.org/grant/patches-r61245.tar.gz
>
> Just in case it's not clear, see
>
>     http://people.osafoundation.org/grant/wx-checkout-info-r61245.txt
>
> which is how I did the checkouts to patch.
>
> This last set I haven't even compiled; there have been some
> rearrangements in the source base, so I just got done porting the
> patches. I'm about to go on vacation, but I'll probably try doing that
> once I get back (next week).
OK, I'll check the patches out when I get a chance.

===[SNIP]===
>> I was working on Gentoo. However, I've not been able to spend a lot
>> of time on that for a month or two. If anyone's interested in what I
>> did, I can help. However, my focus right now is to get a paid job.
>> I made a package I called chandlerdb-lib that installs the db libs
>> concurrently with the system db keeping everything separate. That
>> was an interesting adventure. I can show the build process if desired.
>
> I think that would be interesting, as it could be ported over to
> Debian (Assuming it's OK with all the various licenses and policies).

I just looked through my build script, and there are a couple of changes that need to be made so that you can install it alongside the system db. They are:

Before you compile db run:
sed -i -e "s/_base=\tlibdb/_base=\tlibchandlerdb/g" Makefile.in

Add the following to the configure command:
--libdir="/usr/lib" (*see below)
--includedir="/usr/include/chandlerdb-lib"
--with-uniquename=_CHANDLERDB

With these changes, you need to make sure that chandlerdb knows which library to access:
sed -i \
        -e "s/PREFIX, 'db', 'lib'/PREFIX, 'lib'/" \ (* see below)
        -e "s/PREFIX, 'db', 'include'/PREFIX, 'include', 'chandlerdb-lib'/" \
        -e "s/'db-%s'/'chandlerdb-%s'/" \
        setup.py || die "sed -i setup.py failed"

I also use PREFIX="/usr" when running setup.py

*Note that the /usr/lib dir needs to be updated for 64 bit. On gentoo I use a gentoo script called "get_libdir" (eg '--libdir="${ROOT}usr/$(get_libdir)"' for the configure and "PREFIX, '$(get_libdir)'" in the sed line ) so that I don't have to worry about it. When it compiles, it compiles specifically for the local architecture. I don't know what they do in ubuntu.

===[SNIP]===

Hope that Helps,
Jacob Floyd

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Open Source Applications Foundation "chandler-dev" mailing list
http://lists.osafoundation.org/mailman/listinfo/chandler-dev

signature.asc (279 bytes) Download Attachment
Graham Perrin

Re: Chandler Desktop packaging in Debian and Ubuntu

Reply Threaded More More options
Print post
Permalink