[[e-mail]: Re: [DebianGIS-dev] Bug#519575: hdf5-1.8.3]

[Francesco P. Lovergine]

This is the whole thread that's ongoing in private in the last few days.


----- Forwarded message from Stuart Layton <[hidden email]> -----

Date: Tue, 16 Jun 2009 13:55:19 -0400
From: Stuart Layton <[hidden email]>
To: "Francesco P. Lovergine" <[hidden email]>
Cc: Eric Jonas <[hidden email]>
Subject: Re: [DebianGIS-dev] Bug#519575: hdf5-1.8.3
X-Mailer: Evolution 2.26.1

Not really, the --enable-threadsafe and --enable-cxx configure flags are
incompatible in 1.8.3

My solution was actually much simpler and it doesn't really address this
problem. We only needed the serial, serial-dev, hdf5-tools packages  and
won't need threadsafe, so I only packaged up those packages and disabled
threadsafe.


On Tue, 2009-06-16 at 19:35 +0200, Francesco P. Lovergine wrote:

> On Tue, Jun 16, 2009 at 12:30:12PM -0400, Stuart Layton wrote:
> > I've packed up hdf5-serial, hdf5-serial-dev, and hdf5-tools version
> > 1.8.3 for ubuntu, I'd be willing to work with somebody who wants to
> > migrate the package to debian.
> >
>
> It is basically ready for debian too since ages (1.8.2), and tagged for experimental.
> Unfortunately there are still pending issues due to upstream XXX:
> in order to generate the thread-safe C library one has to drop
> C++/Fortran support, that renders the whole thing unviable. AND the MPI
> support requires thread safety AFAIK. So now i have only to choose
> if dropping C++/Fortran support or multithread-support. After unuseful
> discussion with upstream it seems that the only possibility would be
> diverging from upstream and creating  different flavors with different
> library names and SONAMEs for that: a thread-safe C library, a non-thread-safe
> C/C++/Fortan libraries set. That would force people to support different
> names for Debian only which is not something I like so much. The thing
> could be partially mitigated by the usual h5cc & C tools, but still
> would imply having multiple conflicting -dev packages.
>
> Did you find any better solution for that in 1.8.3?
>

----- End forwarded message -----
----- Forwarded message from Stuart Layton <[hidden email]> -----

Date: Wed, 17 Jun 2009 09:28:50 -0400
From: Stuart Layton <[hidden email]>
To: "Francesco P. Lovergine" <[hidden email]>
Subject: Re: [DebianGIS-dev] Bug#519575: hdf5-1.8.3
X-Mailer: Evolution 2.26.1

I've done a bit of digging and I have a bit of information that might
help a little bit.

FC 11 has hdf5-1.8.3 in their repositories and it appears that they have
just disabled threadsafe and maintained the c++/fortran bindings.

http://koji.fedoraproject.org/koji/rpminfo?rpmID=1256531

I'll also forward you some email exchanges we've had with the hdf5
team.  

-Stuart



On Tue, 2009-06-16 at 19:35 +0200, Francesco P. Lovergine wrote:

> On Tue, Jun 16, 2009 at 12:30:12PM -0400, Stuart Layton wrote:
> > I've packed up hdf5-serial, hdf5-serial-dev, and hdf5-tools version
> > 1.8.3 for ubuntu, I'd be willing to work with somebody who wants to
> > migrate the package to debian.
> >
>
> It is basically ready for debian too since ages (1.8.2), and tagged for experimental.
> Unfortunately there are still pending issues due to upstream XXX:
> in order to generate the thread-safe C library one has to drop
> C++/Fortran support, that renders the whole thing unviable. AND the MPI
> support requires thread safety AFAIK. So now i have only to choose
> if dropping C++/Fortran support or multithread-support. After unuseful
> discussion with upstream it seems that the only possibility would be
> diverging from upstream and creating  different flavors with different
> library names and SONAMEs for that: a thread-safe C library, a non-thread-safe
> C/C++/Fortan libraries set. That would force people to support different
> names for Debian only which is not something I like so much. The thing
> could be partially mitigated by the usual h5cc & C tools, but still
> would imply having multiple conflicting -dev packages.
>
> Did you find any better solution for that in 1.8.3?
>

----- End forwarded message -----
----- Forwarded message from "Francesco P. Lovergine" <[hidden email]> -----

Date: Wed, 17 Jun 2009 17:13:59 +0200
From: "Francesco P. Lovergine" <[hidden email]>
To: Stuart Layton <[hidden email]>
Cc: "Francesco P. Lovergine" <[hidden email]>
Subject: Re: [DebianGIS-dev] Bug#519575: hdf5-1.8.3
User-Agent: Mutt/1.5.19 (2009-01-05)

On Wed, Jun 17, 2009 at 09:28:50AM -0400, Stuart Layton wrote:

> I've done a bit of digging and I have a bit of information that might
> help a little bit.
>
> FC 11 has hdf5-1.8.3 in their repositories and it appears that they have
> just disabled threadsafe and maintained the c++/fortran bindings.
>
> http://koji.fedoraproject.org/koji/rpminfo?rpmID=1256531
>
> I'll also forward you some email exchanges we've had with the hdf5
> team.  
>

Yes, I know that thread, I'm not too happy of supporting
a flavor not supported by upstream. If people had issues
the standard answer would be: why are using threading if
it is not supported? Who is guilty? Of course that is a
regression in respect with 1.6, so a decision needs to be
taken... Cutting the configuration check about thread-safety
is not a solution, only a dirty manner to have something
working, but probably (?) not stable.

> -Stuart
>
>
>
> On Tue, 2009-06-16 at 19:35 +0200, Francesco P. Lovergine wrote:
> > On Tue, Jun 16, 2009 at 12:30:12PM -0400, Stuart Layton wrote:
> > > I've packed up hdf5-serial, hdf5-serial-dev, and hdf5-tools version
> > > 1.8.3 for ubuntu, I'd be willing to work with somebody who wants to
> > > migrate the package to debian.
> > >
> >
> > It is basically ready for debian too since ages (1.8.2), and tagged for experimental.
> > Unfortunately there are still pending issues due to upstream XXX:
> > in order to generate the thread-safe C library one has to drop
> > C++/Fortran support, that renders the whole thing unviable. AND the MPI
> > support requires thread safety AFAIK. So now i have only to choose
> > if dropping C++/Fortran support or multithread-support. After unuseful
> > discussion with upstream it seems that the only possibility would be
> > diverging from upstream and creating  different flavors with different
> > library names and SONAMEs for that: a thread-safe C library, a non-thread-safe
> > C/C++/Fortan libraries set. That would force people to support different
> > names for Debian only which is not something I like so much. The thing
> > could be partially mitigated by the usual h5cc & C tools, but still
> > would imply having multiple conflicting -dev packages.
> >
> > Did you find any better solution for that in 1.8.3?
> >

--
Francesco P. Lovergine

----- End forwarded message -----
----- Forwarded message from Stuart Layton <[hidden email]> -----

Date: Wed, 17 Jun 2009 17:28:25 -0400
From: Stuart Layton <[hidden email]>
To: "Francesco P. Lovergine" <[hidden email]>
Cc: Eric Jonas <[hidden email]>
Subject: [Fwd: Re: [hdf-forum] HDF5 --enable-cxx --enable-threadsafe
        conflict, ubuntu/debian packages]
X-Mailer: Evolution 2.26.1

We've had more talks with the hdf group and it doesn't look like they
are going to make the c++  bindings threadsafe anytime soon (see
attached).

The issue appears to be that, basically, the c++ libs aren't threadsafe.
That's not that uncommon, lots of people wrap thread-safe c libraries in
non-thread-safe C++ bindings. The HDF5 people claim that they were
getting a lot of comments from confused users. I mean, I can understand
how the users were confused as the HDF5 group didn't really put any
warnings anyplace that this was the case, and then just provided two
config options

--enable-cxx
--enable-threadsafe

when the latter should have been --enable-threadsafe-c-only or something
like that.

Again, just to be clear, it's not a stability issue or poor interaction
or whatever with --enable-c++ and --enable-threadsafe (as we originally
thought, and had had suggested to us earlier).

What are the options here? I mean people who install from source are
going to be required to deal with the conflicting compiles if they need
both threadsafe and c++ bindings.

My personal vote would be either to make two packages but have them
conflict with each other, with a note to file a bug with upstream if
they need both features or just provide the library with the configure
script altered to allow for both --enable-threadsafe and --enable-c++.

It's important to note that this will not break anyone's existing
software, as no one who is using the C++ bindings is assuming that they
are threadsafe.

Upstream has really put us and everybody else involved in a really bad
situation but I feel that there are a lot of people looking/waiting for
hdf5-1.8.3 to be packaged up.

Like I said in my first email the packaging job I did is extremely
simple and doesn't provide any of the libraries except the serial
library.  


If you'd be willing to provide me with the debian dir you use to package
up 1.8.2 I'd be willing to host the code in my PPA noting the problems
with threadsafe and c++ bindings.  I could either compile two competing
packages that can't be installed in parallel or just disable the check
in configure and provide a big flag saying that the c++ bindings are not
threadsafe.

Again I know this is suboptimal but I feel that there are quite a lot of
people who could rely benefit from the newer library and seeing as how
upstream doesn't seem to be doing anything to resolve this I'd like to
help push things forward.

-Stuart



Content-Description: Forwarded message - Re: [hdf-forum] HDF5 --enable-cxx --enable-threadsafe conflict, ubuntu/debian packages
Date: Wed, 17 Jun 2009 15:54:10 -0500
From: Quincey Koziol <[hidden email]>
To: Eric Jonas <[hidden email]>
Cc: [hidden email], Stuart Layton <[hidden email]>
Subject: Re: [hdf-forum] HDF5 --enable-cxx --enable-threadsafe conflict,
        ubuntu/debian packages
X-Mailer: Apple Mail (2.935.3)

Hi Eric,

On Jun 17, 2009, at 3:39 PM, Eric Jonas wrote:

> On Wed, 2009-06-17 at 15:21 -0500, Quincey Koziol wrote:
>> Hi Eric,
>>
>> On Jun 17, 2009, at 10:52 AM, Eric Jonas wrote:
>>
>>> Hey guys, I've been working with some friends who are trying to get
>>> HDF5
>>> 1.8.x packaged up for debian/ubuntu, and at issue is the conflict
>>> between --enable-cxx and --enable-threadsafe in the 1.8.x series.
>>>
>>> I know I've talked to people before and received answers from "We're
>>> not
>>> sure" to "Just disable the check in the configure.ac". Fedora just
>>> hacked up the configure and ships with threadsafe-enabled c++ libs.
>>> I'm
>>> hoping people have another recommendation, as the Debian packagers  
>>> are
>>> wary of deviating from upstream too much. And of course, shipping a
>>> separate thread-safe and non-thread-safe HDF5 lib such that we can
>>> then
>>> also ship the c++ non-thread-safe lib seems really suboptimal.
>>
>> The threadsafety in the HDF5 library is implemented by grabbing a
>> mutex whenever a thread enters a C API routine.  However, there is no
>> equivalent mutex for the C++ (or FORTRAN) API wrappers.  Because  
>> those
>> wrappers generate objects which are not safe from getting smashed by
>> multiple threads, we ship the HDF5 library with the configure script
>> detecting and preventing users from enabling both of those configure
>> flags.  I wouldn't consider it safe for a distribution to ship with
>> both options enabled, since it requires an application developer to
>> put wrappers (or some other threadsafety mechanism) around the HDF5  
>> C+
>> + objects before the C++ API can be used in a threaded application.
>>
>> Quincey
>
> Ahh, so the point is that it appears that your concerns are that
> --enable-threadsafe will be interpreted as implying that the C++ api
> wrappers are threadsafe, when in reality it's just the underlying C API
> that would be thread safe, and additional effort would be necessary to
> have the C++ bindings be thread safe.

        Yes, exactly.

> The problem is that at the moment this means that a distribution either
> has to abandon thread safe options entirely (and there are presumably
> some applications / users of the C interface that depend on this thread
> safety guarantee) or abandon the C++ bindings entirely (and there are
> applications, such as my own, which depend on the C++ libs, but not in a
> thread-safe capacity).

        These are certainly valid use cases.  :-)

> Would it not be preferable to let you build the c++ option with
> --enable-threadsafe, but just have a warning in the c++ docs that the C
> ++ bindings themselves are not thread safe? C++ programmers are used to
> having various higher-level libraries have differing (let's say
> "nuanced") thread-safety guarantees, from the STL on down (up?).

        Well, we tried it that way for a long time and it generated a lot of  
questions to our helpdesk, so eventually we just took the gun off the  
table, so people couldn't shoot themselves in the foot (so to speak :-).  
I'd actually prefer to make the C++ bindings thread safe...  :-/  It's not
a _huge_ amount of work, but our time is tied up with other projects
currently.  Patches from knowledgeable folks in the community would be
welcome...

        Quincey





----- End forwarded message -----

--
Francesco P. Lovergine

_______________________________________________
Pkg-grass-general mailing list
[hidden email]
http://lists.alioth.debian.org/mailman/listinfo/pkg-grass-general