#163: Simultaneous reharvest fail
-----------------------+----------------------------------------------------
Reporter: ddnebert | Owner:
[hidden email]
Type: defect | Status: new
Priority: major | Milestone: v2.5.0
Component: General | Version: v2.4.2
Resolution: | Keywords: harvest, concurrency, run
-----------------------+----------------------------------------------------
Comment (by simonp):
Hi Doug,
Which database are you using when you get this bug?
I have also come across the concurrent serializable transaction conflict
in the recent bug on Dbms retries and before that I used to get it in lots
of situations with ORACLE (eg. doing a big batch import etc).
I think the problem is in part due to the rather restrictive (but very
safe) way in which access to data in the database is synchronized between
different threads. At the moment (except for ORACLE) we have exclusive
locks ie. one writer or one reader can have a lock at any time (isolation
level SERIALIZABLE). This appears to be in part due to McKoi which doesn't
support any other isolation level and because this is the safest way (in
terms of consistency) to handle this issue.
We could change the default isolation level for all databases except McKoi
to allow data to be read at any time as long as there is no write in
progress (isolation level READ_COMMITTED - for all except McKoi which
shouldn't be used in production anyway) which would help with this problem
and also improve performance too. However we should check our database
interaction to see whether there are any issues.
The other issue with the harvest is that by my reckoning, each harvest
process uses one connection to the database - if you attempt to start 35
at the same time by selecting them all and choosing run then I think your
harvest will fail anyway - you need to schedule them in groups according
to the database pool size for your database (see <resources><resource
enabled="true">..... in web/geonetwork/WEB-INF/config.xml) or more simply,
just increase the poolsize - default poolsize is 10.
Cheers,
Simon
--
Ticket URL: <
http://trac.osgeo.org/geonetwork/ticket/163#comment:4>
GeoNetwork opensource Developer website <
http://trac.osgeo.org/geonetwork>
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-develGeoNetwork OpenSource is maintained at
http://sourceforge.net/projects/geonetwork
