OSGeo.org › OSGeo Software and Data Projects › GeoNetwork opensource › GeoNetwork developer

"unable to open resource main-db"

7 messages

"unable to open resource main-db"

Reply Threaded

hello lists,

I'm seeing this error, every so often :

"unable to open resource main-db after 20attempts"

This happens when you're using the editor for a bit. When it happens, you must either restart GeoNetwork or wait a while (session timeout?).

It started to happen for us after moving to Postgres and moving to a GN 2.4-based application.

A search on Google shows that quite lots of GeoNetwork nodes have it. Moreover, from a WHO node that displays a stacktrace, it seems they have it when using MySQL, not Postgres (http://209.85.229.132/search?q=cache:5Z0sxBf6fS8J:apps.who.int/geonetwork/srv/en/metadata.show%3Fid%3D74%26currTab%3Dsimple+%22unable+to+open+resource+main-db%22+%22Communications+link+failure+due+to+underlying+exception:%22&cd=1&hl=en&ct=clnk).

Does anybody know what to do to prevent this from ever happening ? My JDBC settings in config.xml are poolSize=10, reconnectTime=undefined (defaulting to Jeeves).

Thank you so much

Kind regards
Heikki Doeleman

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

ntrocino

Re: "unable to open resource main-db"

Reply Threaded

We are using GN 2.2 and postgres and we have the same problem.

Anyone has suggestion/idea on this one?

Kind regards,
Nicola Trocino

Heikki Doeleman wrote:

hello lists,

I'm seeing this error, every so often :

**"unable to open resource main-db after 20attempts"

This happens when you're using the editor for a bit. When it happens, you
must either restart GeoNetwork or wait a while (session timeout?).

It started to happen for us after moving to Postgres and moving to a GN
2.4-based application.

James Wilson

Re: "unable to open resource main-db"

Reply Threaded

In reply to this post by Heikki Doeleman

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle, with pool size = 10 and reconnectTime=undefined. Problem is not easily repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is supposed to work, and resources get released. By my reading, the only place connections get released is by calling close on connection pool object (DbmsPool) with the resource (Dbms) as an argument, which unlocks the resource.

However, in the code, this never seems to be done. The default pattern seems to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the error....

Anybody else got any ideas?

James

Heikki Doeleman wrote:

hello lists,

I'm seeing this error, every so often :

**"unable to open resource main-db after 20attempts"

This happens when you're using the editor for a bit. When it happens, you
must either restart GeoNetwork or wait a while (session timeout?).

It started to happen for us after moving to Postgres and moving to a GN
2.4-based application.

A search on Google shows that quite lots of GeoNetwork nodes have it.
Moreover, from a WHO node that displays a stacktrace, it seems they have it
when using MySQL, not Postgres (
http://209.85.229.132/search?q=cache:5Z0sxBf6fS8J:apps.who.int/geonetwork/srv/en/metadata.show%3Fid%3D74%26currTab%3Dsimple+%22unable+to+open+resource+main-db%22+%22Communications+link+failure+due+to+underlying+exception:%22&cd=1&hl=en&ct=clnk
).

Does anybody know what to do to prevent this from ever happening ? My JDBC
settings in config.xml are poolSize=10, reconnectTime=undefined (defaulting
to Jeeves).

Thank you so much

Kind regards
Heikki Doeleman

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
GeoNetwork-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

SimonPigot

Re: "unable to open resource main-db"

Reply Threaded

I suspect this is due to the Update service (which calls EditUtils) not reusing a single Dbms connection but opening two or even three to do its work (left over from the way it used to work maybe) - this, coupled with the 'leave the resource to go out of scope' approach ends up eventually with all Dbms connections in the pool being locked and the error message 'unable to open resource....' - I'm testing now to confirm - but I don't think there is any reason why Update and EditUtils should not reuse one Dbms connection so this will be committed anyway.

Cheers,
Simon
________________________________________
From: James Wilson [[hidden email]]
Sent: Wednesday, 21 October 2009 3:02 AM
To: [hidden email]
Subject: Re: [GeoNetwork-devel] "unable to open resource main-db"

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle,
with pool size = 10 and reconnectTime=undefined. Problem is not easily
repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is
supposed to work, and resources get released. By my reading, the only place
connections get released is by calling close on connection pool object
(DbmsPool) with the resource (Dbms) as an argument, which unlocks the
resource.

However, in the code, this never seems to be done. The default pattern seems
to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the
error....

Anybody else got any ideas?

James

Heikki Doeleman wrote:

>
> hello lists,
>
> I'm seeing this error, every so often :
>
> **"unable to open resource main-db after 20attempts"
>
> This happens when you're using the editor for a bit. When it happens, you
> must either restart GeoNetwork or wait a while (session timeout?).
>
> It started to happen for us after moving to Postgres and moving to a GN
> 2.4-based application.
>
> A search on Google shows that quite lots of GeoNetwork nodes have it.
> Moreover, from a WHO node that displays a stacktrace, it seems they have
> it
> when using MySQL, not Postgres (
> http://209.85.229.132/search?q=cache:5Z0sxBf6fS8J:apps.who.int/geonetwork/srv/en/metadata.show%3Fid%3D74%26currTab%3Dsimple+%22unable+to+open+resource+main-db%22+%22Communications+link+failure+due+to+underlying+exception:%22&cd=1&hl=en&ct=clnk
> ).
>
> Does anybody know what to do to prevent this from ever happening ? My JDBC
> settings in config.xml are poolSize=10, reconnectTime=undefined
> (defaulting
> to Jeeves).
>
>
> Thank you so much
>
> Kind regards
> Heikki Doeleman
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry® Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9-12, 2009. Register
> now!
> http://p.sf.net/sfu/devconf
> _______________________________________________
> GeoNetwork-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>

SimonPigot

Re: "unable to open resource main-db"

Reply Threaded

Just as a clarification: I believe its fine to leave the resource (Dbms) as is at the end of a service call because it doesn't really go out of scope - jeeves will close/commit or abort/rollback on the Dbms resource depending on whether the service dispatch succeeded or returned an error. More after testing to see whether multiple Dbms opening in Update/EditUtils is the issue or not.
________________________________________
From: [hidden email] [[hidden email]]
Sent: Wednesday, 21 October 2009 2:25 PM
To: [hidden email]; [hidden email]
Subject: [ExternalEmail] Re: [GeoNetwork-devel] "unable to open resource main-db"

I suspect this is due to the Update service (which calls EditUtils) not reusing a single Dbms connection but opening two or even three to do its work (left over from the way it used to work maybe) - this, coupled with the 'leave the resource to go out of scope' approach ends up eventually with all Dbms connections in the pool being locked and the error message 'unable to open resource....' - I'm testing now to confirm - but I don't think there is any reason why Update and EditUtils should not reuse one Dbms connection so this will be committed anyway.

Cheers,
Simon
________________________________________
From: James Wilson [[hidden email]]
Sent: Wednesday, 21 October 2009 3:02 AM
To: [hidden email]
Subject: Re: [GeoNetwork-devel] "unable to open resource main-db"

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle,
with pool size = 10 and reconnectTime=undefined. Problem is not easily
repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is
supposed to work, and resources get released. By my reading, the only place
connections get released is by calling close on connection pool object
(DbmsPool) with the resource (Dbms) as an argument, which unlocks the
resource.

However, in the code, this never seems to be done. The default pattern seems
to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the
error....

Anybody else got any ideas?

James

Heikki Doeleman wrote:

--
View this message in context: http://n2.nabble.com/unable-to-open-resource-main-db-tp3728570p3859528.html
Sent from the GeoNetwork developer mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork

SimonPigot

Re: "unable to open resource main-db"

Reply Threaded

In reply to this post by SimonPigot

Sigh - nice idea but no :-) - its not due to what I've described below. Back to the drawing board....

Cheers,
Simon
________________________________________
From: [hidden email] [[hidden email]]
Sent: Wednesday, 21 October 2009 2:25 PM
To: [hidden email]; [hidden email]
Subject: [ExternalEmail] Re: [GeoNetwork-devel] "unable to open resource main-db"

I suspect this is due to the Update service (which calls EditUtils) not reusing a single Dbms connection but opening two or even three to do its work (left over from the way it used to work maybe) - this, coupled with the 'leave the resource to go out of scope' approach ends up eventually with all Dbms connections in the pool being locked and the error message 'unable to open resource....' - I'm testing now to confirm - but I don't think there is any reason why Update and EditUtils should not reuse one Dbms connection so this will be committed anyway.

Cheers,
Simon
________________________________________
From: James Wilson [[hidden email]]
Sent: Wednesday, 21 October 2009 3:02 AM
To: [hidden email]
Subject: Re: [GeoNetwork-devel] "unable to open resource main-db"

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle,
with pool size = 10 and reconnectTime=undefined. Problem is not easily
repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is
supposed to work, and resources get released. By my reading, the only place
connections get released is by calling close on connection pool object
(DbmsPool) with the resource (Dbms) as an argument, which unlocks the
resource.

However, in the code, this never seems to be done. The default pattern seems
to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the
error....

Anybody else got any ideas?

James

Heikki Doeleman wrote:

SimonPigot

Re: "unable to open resource main-db"

Reply Threaded

In reply to this post by James Wilson

Looking at Heikki's original message and references to stack traces in google - its actually taking place in the services metadata.show and iso19139.xml. I can reproduce it with curl and thumping the server with lots of requests to the metadata.show service.

Seems that some code, somewhere was opening a Dbms resource but wasn't going through the usual Jeeves clean up of Dbms resources (via commit on success or rollback on exception) at the end of a service, hence the 10 Dbms connections in the pool would eventually end up locked and an exception with message 'unable to open main-db after 20 attempts' would be given. The code appears to be the timertask thread that is used to increase popularity asynchronously in metadata.show (its also called by iso19139.xml). One fix is to add:

context.getResourceManager().close();

as the last line of the try block in the run method of the IncreasePopularityTask class in src/org/fao/geonet/services/metadata/Show.java

I'm not sure this is good though as when thumping the server (albeit with a somewhat unrealistic load) as described above, the thread that commits the increase in popularity tends to overlap with other threads that attempt to select from the Metadata table - the select threads fail because of pending commits. Safest might be to remove the async popularity update task and just run dm.increasePopularity(dbms, id); synchronously (no exceptions there)?

Cheers,
Simon
________________________________________
From: James Wilson [[hidden email]]
Sent: Wednesday, 21 October 2009 3:02 AM
To: [hidden email]
Subject: Re: [GeoNetwork-devel] "unable to open resource main-db"

Heikki,

I'm also seeing this problem after doing repeated saves. I'm using oracle,
with pool size = 10 and reconnectTime=undefined. Problem is not easily
repeatable, but happens regularly.

Is this a problem with the connection pool running out of objects?

Reading the code, I'm somewhat confused as to how the connection pool is
supposed to work, and resources get released. By my reading, the only place
connections get released is by calling close on connection pool object
(DbmsPool) with the resource (Dbms) as an argument, which unlocks the
resource.

However, in the code, this never seems to be done. The default pattern seems
to be to get a new resource with

dbms = context.getResourceManager().open(Geonet.Res.MAIN_DB)

and then to leave the resource to go out of scope

Yours, mystified, and frustrated at not being able to track down the
error....

Anybody else got any ideas?

James

Heikki Doeleman wrote: