RE: PATCH: Raster stability fixes, ticket #462

6 messages Options
Embed this post
Permalink
Jason Birch

RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
I think that this patch should be implemented for two reasons.

First, I believe that this is no more of a sledgehammer approach than the existing guard.

Second, getting coordination between a skilled MapGuide developers that can work though the threading issues and the GDAL Provider developer has been attempted in the past and failed because nobody stepped up to the plate from the MapGuide community.

Checking this in as a temporary stability measure allows us to have a reasonable (if slow) raster story for the 2.1 release.

Of course, if there are developers from both the FDO and MapGuide communities that are able to work together to properly resolve this issue before the FDO 3.4 release (it's at RC3) and the MapGuide 2.1 release, then that would be ideal.

Jason

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bruce Dechant
Sent: March-27-09 8:55 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I totally agree - the locking needs to be in the provider.

Thanks,
Bruce


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Friday, March 27, 2009 9:40 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462


It does look like a sledgehammer approach -- what if some day there is a raster provider that works ok and doesn't need the locks? If anything, the lock should be in the FDO provider.

Traian


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Walt Welton-Lair
Sent: Friday, March 27, 2009 11:11 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

The memory patch (mapguide_raster_unalloc.5.patch) looks good, and I'll submit that.

The stability patch (mapguide_raster_stability.patch) looks reasonable, but I'd like another 1 or 2 people to also look at it.  It's significant since it limits MapGuide to processing one raster stylization request at a time, but I guess that's better than having a dead lock.  Anyone know where the thread-unsafe code actually is?  Is it in the provider itself?  Can the mutex be moved closer to the actual thread-unsafe code?

Walt

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 3:10 AM
To: [hidden email]
Subject: [mapguide-internals] PATCH: Raster stability fixes, ticket #462

Hi,

Haris has been working on the raster stability problem against MapGuide 2.0.2.  I was having the same problems, and was able to duplicate on 2.1.  I have worked with Haris to create patches against trunk that address a couple of the problems and increase stability immensely.

Could someone please review the patches attached to:

http://trac.osgeo.org/mapguide/ticket/462

and commit them to trunk if appropriate?

My feeling is that the connection management stuff may need some additional work, since limiting the connection pool to 1 for single-threaded providers doesn't appear to have the desired effect.  However, getting both of these patches into 2.1 would at least improve stability for Raster users.

Jason_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
Trevor Wekel-3

RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
Resending.  My original post bounced on FDOInternals.

-----Original Message-----
From: Trevor Wekel
Sent: Friday, March 27, 2009 11:45 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: RE: PATCH: Raster stability fixes, ticket #462

Just to add a little more detail, the guard in question will also impact WMS consumption in MapGuide.  Querying and rendering of WMS layers from external data sources will become completely single threaded.

Moving the guard forces most of the stylization/rendering work to be single threaded.  I suspect these operations are fairly lengthy.  From what I recall, the Fdo ReadNext() call is also done within StylizeGridLayer.  If ReadNext() is "slow" for WMS and GDAL then there will be an even greater performance impact to multi-user performance when the guard is moved.

The current guard only protects setup and execution of the MgFeatureService::SelectFeatures call.  From an Fdo perspective, I believe SelectFeatures is fairly fast so guarding it does not significantly affect multi-user performance.

In my opinion, moving the guard in MapGuide will impose a heavy price performance wise.  The best option is to add guards to the GDAL Raster provider.  However, Jason is bang on with his analysis of the likelihood of any collaboration happening short term.  One possible solution would be to put a guard around all of the public Fdo APIs in the GDAL Provider.  That would not require much collaboration and would be "drop in" testable.

Thanks,
Trevor


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 11:07 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I think that this patch should be implemented for two reasons.

First, I believe that this is no more of a sledgehammer approach than the existing guard.

Second, getting coordination between a skilled MapGuide developers that can work though the threading issues and the GDAL Provider developer has been attempted in the past and failed because nobody stepped up to the plate from the MapGuide community.

Checking this in as a temporary stability measure allows us to have a reasonable (if slow) raster story for the 2.1 release.

Of course, if there are developers from both the FDO and MapGuide communities that are able to work together to properly resolve this issue before the FDO 3.4 release (it's at RC3) and the MapGuide 2.1 release, then that would be ideal.

Jason

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bruce Dechant
Sent: March-27-09 8:55 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I totally agree - the locking needs to be in the provider.

Thanks,
Bruce


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Friday, March 27, 2009 9:40 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462


It does look like a sledgehammer approach -- what if some day there is a raster provider that works ok and doesn't need the locks? If anything, the lock should be in the FDO provider.

Traian


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Walt Welton-Lair
Sent: Friday, March 27, 2009 11:11 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

The memory patch (mapguide_raster_unalloc.5.patch) looks good, and I'll submit that.

The stability patch (mapguide_raster_stability.patch) looks reasonable, but I'd like another 1 or 2 people to also look at it.  It's significant since it limits MapGuide to processing one raster stylization request at a time, but I guess that's better than having a dead lock.  Anyone know where the thread-unsafe code actually is?  Is it in the provider itself?  Can the mutex be moved closer to the actual thread-unsafe code?

Walt

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 3:10 AM
To: [hidden email]
Subject: [mapguide-internals] PATCH: Raster stability fixes, ticket #462

Hi,

Haris has been working on the raster stability problem against MapGuide 2.0.2.  I was having the same problems, and was able to duplicate on 2.1.  I have worked with Haris to create patches against trunk that address a couple of the problems and increase stability immensely.

Could someone please review the patches attached to:

http://trac.osgeo.org/mapguide/ticket/462

and commit them to trunk if appropriate?

My feeling is that the connection management stuff may need some additional work, since limiting the connection pool to 1 for single-threaded providers doesn't appear to have the desired effect.  However, getting both of these patches into 2.1 would at least improve stability for Raster users.

Jason_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
Jason Birch

RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
Is there any way at that point in the code to identify the provider being used for the Raster operation, and only apply the guard if it's the GDAL provider?

I believe that there was a fair amount of effort put into guarding all of the FDO APIs in the GDAL provider already, but perhaps some got missed.  I'll be interested to hear back from Haris on his investigations into the connection manager to see if this is a combination of factors, not just the GDAL provider.

BTW, yes, having the guard in that location is somewhat slow with multiple layers/users. But it also means that my MapGuide server doesn't crash after a minute of operation once I get a few concurrent sessions going.  These patches mean that it takes at least ten minutes (I'm seeing the problem of exhausting the client connections too )...

In an ideal world, I'd be happier if there was a bare bones, multithreaded, non-configurable, seat of your pants GDAL provider that supported VRTs.  However, just being able to publicly launch the City's new web maps after a year and a half, countless hours of my time, and over $80k in external development costs would be nice :)  I've definitely had cause to regret jumping on this bandwagon early.

Jason

-----Original Message-----
From: Trevor Wekel
Sent: Friday, March 27, 2009 11:45 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: RE: PATCH: Raster stability fixes, ticket #462

Just to add a little more detail, the guard in question will also impact WMS consumption in MapGuide.  Querying and rendering of WMS layers from external data sources will become completely single threaded.

Moving the guard forces most of the stylization/rendering work to be single threaded.  I suspect these operations are fairly lengthy.  From what I recall, the Fdo ReadNext() call is also done within StylizeGridLayer.  If ReadNext() is "slow" for WMS and GDAL then there will be an even greater performance impact to multi-user performance when the guard is moved.

The current guard only protects setup and execution of the MgFeatureService::SelectFeatures call.  From an Fdo perspective, I believe SelectFeatures is fairly fast so guarding it does not significantly affect multi-user performance.

In my opinion, moving the guard in MapGuide will impose a heavy price performance wise.  The best option is to add guards to the GDAL Raster provider.  However, Jason is bang on with his analysis of the likelihood of any collaboration happening short term.  One possible solution would be to put a guard around all of the public Fdo APIs in the GDAL Provider.  That would not require much collaboration and would be "drop in" testable.

Thanks,
Trevor


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 11:07 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I think that this patch should be implemented for two reasons.

First, I believe that this is no more of a sledgehammer approach than the existing guard.

Second, getting coordination between a skilled MapGuide developers that can work though the threading issues and the GDAL Provider developer has been attempted in the past and failed because nobody stepped up to the plate from the MapGuide community.

Checking this in as a temporary stability measure allows us to have a reasonable (if slow) raster story for the 2.1 release.

Of course, if there are developers from both the FDO and MapGuide communities that are able to work together to properly resolve this issue before the FDO 3.4 release (it's at RC3) and the MapGuide 2.1 release, then that would be ideal.

Jason

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bruce Dechant
Sent: March-27-09 8:55 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I totally agree - the locking needs to be in the provider.

Thanks,
Bruce


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Friday, March 27, 2009 9:40 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462


It does look like a sledgehammer approach -- what if some day there is a raster provider that works ok and doesn't need the locks? If anything, the lock should be in the FDO provider.

Traian


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Walt Welton-Lair
Sent: Friday, March 27, 2009 11:11 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

The memory patch (mapguide_raster_unalloc.5.patch) looks good, and I'll submit that.

The stability patch (mapguide_raster_stability.patch) looks reasonable, but I'd like another 1 or 2 people to also look at it.  It's significant since it limits MapGuide to processing one raster stylization request at a time, but I guess that's better than having a dead lock.  Anyone know where the thread-unsafe code actually is?  Is it in the provider itself?  Can the mutex be moved closer to the actual thread-unsafe code?

Walt

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 3:10 AM
To: [hidden email]
Subject: [mapguide-internals] PATCH: Raster stability fixes, ticket #462

Hi,

Haris has been working on the raster stability problem against MapGuide 2.0.2.  I was having the same problems, and was able to duplicate on 2.1.  I have worked with Haris to create patches against trunk that address a couple of the problems and increase stability immensely.

Could someone please review the patches attached to:

http://trac.osgeo.org/mapguide/ticket/462

and commit them to trunk if appropriate?

My feeling is that the connection management stuff may need some additional work, since limiting the connection pool to 1 for single-threaded providers doesn't appear to have the desired effect.  However, getting both of these patches into 2.1 would at least improve stability for Raster users.

Jason
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
Trevor Wekel-3

RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
Hmm...  It's a bit hacky but you could call GetResourceContent on the feature source and look for OSGeo.GDAL in the xml document that comes back.

Don't jump ship!  Who is going to keep us in line if you leave?? ;)

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 12:35 PM
To: 'FDO Internals Mail List'; MapGuide Internals Mail List
Subject: [fdo-internals] RE: PATCH: Raster stability fixes, ticket #462

Is there any way at that point in the code to identify the provider being used for the Raster operation, and only apply the guard if it's the GDAL provider?

I believe that there was a fair amount of effort put into guarding all of the FDO APIs in the GDAL provider already, but perhaps some got missed.  I'll be interested to hear back from Haris on his investigations into the connection manager to see if this is a combination of factors, not just the GDAL provider.

BTW, yes, having the guard in that location is somewhat slow with multiple layers/users. But it also means that my MapGuide server doesn't crash after a minute of operation once I get a few concurrent sessions going.  These patches mean that it takes at least ten minutes (I'm seeing the problem of exhausting the client connections too )...

In an ideal world, I'd be happier if there was a bare bones, multithreaded, non-configurable, seat of your pants GDAL provider that supported VRTs.  However, just being able to publicly launch the City's new web maps after a year and a half, countless hours of my time, and over $80k in external development costs would be nice :)  I've definitely had cause to regret jumping on this bandwagon early.

Jason

-----Original Message-----
From: Trevor Wekel
Sent: Friday, March 27, 2009 11:45 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: RE: PATCH: Raster stability fixes, ticket #462

Just to add a little more detail, the guard in question will also impact WMS consumption in MapGuide.  Querying and rendering of WMS layers from external data sources will become completely single threaded.

Moving the guard forces most of the stylization/rendering work to be single threaded.  I suspect these operations are fairly lengthy.  From what I recall, the Fdo ReadNext() call is also done within StylizeGridLayer.  If ReadNext() is "slow" for WMS and GDAL then there will be an even greater performance impact to multi-user performance when the guard is moved.

The current guard only protects setup and execution of the MgFeatureService::SelectFeatures call.  From an Fdo perspective, I believe SelectFeatures is fairly fast so guarding it does not significantly affect multi-user performance.

In my opinion, moving the guard in MapGuide will impose a heavy price performance wise.  The best option is to add guards to the GDAL Raster provider.  However, Jason is bang on with his analysis of the likelihood of any collaboration happening short term.  One possible solution would be to put a guard around all of the public Fdo APIs in the GDAL Provider.  That would not require much collaboration and would be "drop in" testable.

Thanks,
Trevor


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 11:07 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I think that this patch should be implemented for two reasons.

First, I believe that this is no more of a sledgehammer approach than the existing guard.

Second, getting coordination between a skilled MapGuide developers that can work though the threading issues and the GDAL Provider developer has been attempted in the past and failed because nobody stepped up to the plate from the MapGuide community.

Checking this in as a temporary stability measure allows us to have a reasonable (if slow) raster story for the 2.1 release.

Of course, if there are developers from both the FDO and MapGuide communities that are able to work together to properly resolve this issue before the FDO 3.4 release (it's at RC3) and the MapGuide 2.1 release, then that would be ideal.

Jason

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Bruce Dechant
Sent: March-27-09 8:55 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

I totally agree - the locking needs to be in the provider.

Thanks,
Bruce


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Traian Stanev
Sent: Friday, March 27, 2009 9:40 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462


It does look like a sledgehammer approach -- what if some day there is a raster provider that works ok and doesn't need the locks? If anything, the lock should be in the FDO provider.

Traian


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Walt Welton-Lair
Sent: Friday, March 27, 2009 11:11 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket #462

The memory patch (mapguide_raster_unalloc.5.patch) looks good, and I'll submit that.

The stability patch (mapguide_raster_stability.patch) looks reasonable, but I'd like another 1 or 2 people to also look at it.  It's significant since it limits MapGuide to processing one raster stylization request at a time, but I guess that's better than having a dead lock.  Anyone know where the thread-unsafe code actually is?  Is it in the provider itself?  Can the mutex be moved closer to the actual thread-unsafe code?

Walt

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 3:10 AM
To: [hidden email]
Subject: [mapguide-internals] PATCH: Raster stability fixes, ticket #462

Hi,

Haris has been working on the raster stability problem against MapGuide 2.0.2.  I was having the same problems, and was able to duplicate on 2.1.  I have worked with Haris to create patches against trunk that address a couple of the problems and increase stability immensely.

Could someone please review the patches attached to:

http://trac.osgeo.org/mapguide/ticket/462

and commit them to trunk if appropriate?

My feeling is that the connection management stuff may need some additional work, since limiting the connection pool to 1 for single-threaded providers doesn't appear to have the desired effect.  However, getting both of these patches into 2.1 would at least improve stability for Raster users.

Jason
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
Jason Birch

RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
Heh.

At this point I'm probably too invested to jump ship, but I definitely get frustrated at times, usually right around project status meetings with my supervisor or when dealing with members of the public I promised a cross-browser map to months ago.

I think somewhere else in the code there's an equally hacky method used to compare test the feature class name (Image vs Raster) to differentiate WMS from Raster providers.  That wouldn't help if the ADSK raster provider is multithreaded though, or if there are other raster providers we don't "know" about.

Jason

-----Original Message-----
From: Trevor Wekel
Sent: March-27-09 11:49 AM
To: FDO Internals Mail List; MapGuide Internals Mail List
Subject: [fdo-internals] RE: PATCH: Raster stability fixes, ticket #462

Hmm...  It's a bit hacky but you could call GetResourceContent on the feature source and look for OSGeo.GDAL in the xml document that comes back.

Don't jump ship!  Who is going to keep us in line if you leave?? ;)
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
Haris Kurtagic

RE: RE: PATCH: Raster stability fixes, ticket #462

Reply Threaded More More options
Print post
Permalink
In reply to this post by Jason Birch
It is not provider itself. I think I know exactly ( I just posted to MG
internals ) what is going on and where problems are.
Pacth will not make it slower.

And most importantly we will solve it :)

Haris

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Jason Birch
Sent: Friday, March 27, 2009 7:35 PM
To: 'FDO Internals Mail List'; MapGuideInternals Mail List
Subject: [fdo-internals] RE: PATCH: Raster stability fixes, ticket #462

Is there any way at that point in the code to identify the provider
being used for the Raster operation, and only apply the guard if it's
the GDAL provider?

I believe that there was a fair amount of effort put into guarding all
of the FDO APIs in the GDAL provider already, but perhaps some got
missed.  I'll be interested to hear back from Haris on his
investigations into the connection manager to see if this is a
combination of factors, not just the GDAL provider.

BTW, yes, having the guard in that location is somewhat slow with
multiple layers/users. But it also means that my MapGuide server doesn't
crash after a minute of operation once I get a few concurrent sessions
going.  These patches mean that it takes at least ten minutes (I'm
seeing the problem of exhausting the client connections too )...

In an ideal world, I'd be happier if there was a bare bones,
multithreaded, non-configurable, seat of your pants GDAL provider that
supported VRTs.  However, just being able to publicly launch the City's
new web maps after a year and a half, countless hours of my time, and
over $80k in external development costs would be nice :)  I've
definitely had cause to regret jumping on this bandwagon early.

Jason

-----Original Message-----
From: Trevor Wekel
Sent: Friday, March 27, 2009 11:45 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: RE: PATCH: Raster stability fixes, ticket #462

Just to add a little more detail, the guard in question will also impact
WMS consumption in MapGuide.  Querying and rendering of WMS layers from
external data sources will become completely single threaded.

Moving the guard forces most of the stylization/rendering work to be
single threaded.  I suspect these operations are fairly lengthy.  From
what I recall, the Fdo ReadNext() call is also done within
StylizeGridLayer.  If ReadNext() is "slow" for WMS and GDAL then there
will be an even greater performance impact to multi-user performance
when the guard is moved.

The current guard only protects setup and execution of the
MgFeatureService::SelectFeatures call.  From an Fdo perspective, I
believe SelectFeatures is fairly fast so guarding it does not
significantly affect multi-user performance.

In my opinion, moving the guard in MapGuide will impose a heavy price
performance wise.  The best option is to add guards to the GDAL Raster
provider.  However, Jason is bang on with his analysis of the likelihood
of any collaboration happening short term.  One possible solution would
be to put a guard around all of the public Fdo APIs in the GDAL
Provider.  That would not require much collaboration and would be "drop
in" testable.

Thanks,
Trevor


-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Jason
Birch
Sent: Friday, March 27, 2009 11:07 AM
To: 'MapGuide Internals Mail List'; 'FDO Internals Mail List'
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket
#462

I think that this patch should be implemented for two reasons.

First, I believe that this is no more of a sledgehammer approach than
the existing guard.

Second, getting coordination between a skilled MapGuide developers that
can work though the threading issues and the GDAL Provider developer has
been attempted in the past and failed because nobody stepped up to the
plate from the MapGuide community.

Checking this in as a temporary stability measure allows us to have a
reasonable (if slow) raster story for the 2.1 release.

Of course, if there are developers from both the FDO and MapGuide
communities that are able to work together to properly resolve this
issue before the FDO 3.4 release (it's at RC3) and the MapGuide 2.1
release, then that would be ideal.

Jason

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Bruce
Dechant
Sent: March-27-09 8:55 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket
#462

I totally agree - the locking needs to be in the provider.

Thanks,
Bruce


-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Traian
Stanev
Sent: Friday, March 27, 2009 9:40 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket
#462


It does look like a sledgehammer approach -- what if some day there is a
raster provider that works ok and doesn't need the locks? If anything,
the lock should be in the FDO provider.

Traian


-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Walt
Welton-Lair
Sent: Friday, March 27, 2009 11:11 AM
To: MapGuide Internals Mail List
Subject: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket
#462

The memory patch (mapguide_raster_unalloc.5.patch) looks good, and I'll
submit that.

The stability patch (mapguide_raster_stability.patch) looks reasonable,
but I'd like another 1 or 2 people to also look at it.  It's significant
since it limits MapGuide to processing one raster stylization request at
a time, but I guess that's better than having a dead lock.  Anyone know
where the thread-unsafe code actually is?  Is it in the provider itself?
Can the mutex be moved closer to the actual thread-unsafe code?

Walt

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Jason
Birch
Sent: Friday, March 27, 2009 3:10 AM
To: [hidden email]
Subject: [mapguide-internals] PATCH: Raster stability fixes, ticket #462

Hi,

Haris has been working on the raster stability problem against MapGuide
2.0.2.  I was having the same problems, and was able to duplicate on
2.1.  I have worked with Haris to create patches against trunk that
address a couple of the problems and increase stability immensely.

Could someone please review the patches attached to:

http://trac.osgeo.org/mapguide/ticket/462

and commit them to trunk if appropriate?

My feeling is that the connection management stuff may need some
additional work, since limiting the connection pool to 1 for
single-threaded providers doesn't appear to have the desired effect.
However, getting both of these patches into 2.1 would at least improve
stability for Raster users.

Jason
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/fdo-internals