CSW GetRecords supposed to return csw:SummaryRecord instead of dc:metada?

3 messages Options
Embed this post
Permalink
wgrunberg

CSW GetRecords supposed to return csw:SummaryRecord instead of dc:metada?

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
Deer deegree team,

I am using GeoNetwork 2.4.1 to harvest CSW records from our deegree 2.3pre (August 2009) build and I am encountering dificulties due to what may be a bug in deegree_csw.

Geonetwork sends a GetRecords request with <csw:ElementSetName>summary</csw:ElementSetName> but without an outputSchema. The default outputSchema for a GetRecords request is http://www.opengis.net/cat/csw/2.0.2. deegree returns dc:metada but apparently the response is suposed to be csw:SummaryRecord.

I looked at the CSW 2.0.2 specifications again but I am only more confused now.
Following is the thread I kicked off on the GeoNetwork developer forum.
I hope the GeoNetwork folks make more sense.

Ciao, Wolfgang




-------- Original Message --------
Subject: Re: [GeoNetwork-devel] harvester fails with deegree CSW 2.0.2 service
Date: Wed, 30 Sep 2009 13:14:15 +0200
From: heikki [hidden email]
Reply-To: [hidden email]
To: Francois Prunayre [hidden email]
CC: Wolfgang Grunberg [hidden email], [hidden email]
References: [hidden email] [hidden email] [hidden email]


hello Wolfgang, François, and list :

I've looked at it a bit more with Jose Garcia, and we found :

(1) GeoNetwork did not send a outputSchema in its GetRecords request. This means that the default per the CSW202 spec should be used by the deegree server, which I think means we should receive results of type csw:SummaryRecord, as says François;

(2) GeoNetwork should actually put a "more preferred" ISO outputSchema in the request, if the Capabilities advertizes it (as does the deegree server's Capabilities). Jose is committing a fix for this to GeoNetwork today;

(3) GeoNetwork can only handle GetRecordsResponse that contains SearchResults of type csw:SummaryRecord (CSW) or gmd:MD_Metadata (ISO). Other types, like the dc:metadata receivede from deegree, leads to the message "Skipped record not in supported format :" you got

(4) sending GetRecords requests to the deegree server with various outputSchema I get :

outputSchema="http://www.isotc211.org/2005/gmd"  --> response: ISO records, correct

outputSchema="http://www.opengis.net/cat/csw/2.0.2" --> response: dc:metada, expected response: csw:SummaryRecord

<absent outputSchema> -->  response: dc:metada, expected response: csw:SummaryRecord

Hopefully things are more clear ?

Kind regards
Heikki Doeleman






On Wed, Sep 30, 2009 at 12:18 PM, Francois Prunayre <[hidden email]> wrote:
Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <[hidden email]>:
> GeoNetwork 2.41's CSW harvester fails with our deegree 2.3pre CSW service
> (http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).
>
>
> GeoNetwork's GetRecords transaction request returns what appears to be a
> valid CSW and Dublin Core GetRecordsResponse response. I have attached the
> truncated response. This is a great improvement because deegree didn't
> accept the GeoNetwork response in the past.
>
> However, GeoNetwork's log gives me a "[geonetwork.harvester] - Skipped
> record not in supported format : metadata" error and I am having
> difficulties figuring out what the problem is.
It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
"Default value is http://www.opengis.net/cat/csw/2.0.2" CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema="http://www.opengis.net/cat/csw/2.0.2"
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:
 * GetRecords in summary format first to retrieve ids and date
 * GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois



> What kind of response does GeoNetwork require?
> Also, how do I enable more verbose logging in GeoNetwork?
>
> Thank you for your time!
>
> POST request to http://75.101.143.247:8080/deegree-csw/services :
> <?xml version="1.0" encoding="UTF-8"?>
> <csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
> service="CSW" version="2.0.2" resultType="results" startPosition="1301"
> maxRecords="20">
>  <csw:Query typeNames="csw:Record">
>    <csw:ElementSetName>summary</csw:ElementSetName>
>  </csw:Query>
> </csw:GetRecords>
>
> Response:
> See attachment with a truncated response.
>
> --
> _______________________________
> Wolfgang Grunberg
> Arizona Geological Survey
> [hidden email]
> 520-770-3500
>
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> GeoNetwork-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>

------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


-- 
_______________________________
Wolfgang Grunberg
Arizona Geological Survey
[hidden email]
520-770-3500



------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
deegree-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-users
wgrunberg

Re: CSW GetRecords supposed to return csw:SummaryRecord instead of dc:metada?

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
Correction:
>From what I understand, both csw:SummaryRecord and csw:GetRecordsResponse responses should have DC formated metadata if no ouputSchema is defined.
However, should a GetRecords request with a "summary " ElementSetName return a csw:SummaryRecord response and not a csw:GetRecordsResponse response?

WG

Wolfgang Grunberg wrote:
Deer deegree team,

I am using GeoNetwork 2.4.1 to harvest CSW records from our deegree 2.3pre (August 2009) build and I am encountering dificulties due to what may be a bug in deegree_csw.

Geonetwork sends a GetRecords request with <csw:ElementSetName>summary</csw:ElementSetName> but without an outputSchema. The default outputSchema for a GetRecords request is http://www.opengis.net/cat/csw/2.0.2. deegree returns dc:metada but apparently the response is suposed to be csw:SummaryRecord.

I looked at the CSW 2.0.2 specifications again but I am only more confused now.
Following is the thread I kicked off on the GeoNetwork developer forum.
I hope the GeoNetwork folks make more sense.

Ciao, Wolfgang




-------- Original Message --------
Subject: Re: [GeoNetwork-devel] harvester fails with deegree CSW 2.0.2 service
Date: Wed, 30 Sep 2009 13:14:15 +0200
From: heikki [hidden email]
Reply-To: [hidden email]
To: Francois Prunayre [hidden email]
CC: Wolfgang Grunberg [hidden email], [hidden email]
References: [hidden email] [hidden email] [hidden email]


hello Wolfgang, François, and list :

I've looked at it a bit more with Jose Garcia, and we found :

(1) GeoNetwork did not send a outputSchema in its GetRecords request. This means that the default per the CSW202 spec should be used by the deegree server, which I think means we should receive results of type csw:SummaryRecord, as says François;

(2) GeoNetwork should actually put a "more preferred" ISO outputSchema in the request, if the Capabilities advertizes it (as does the deegree server's Capabilities). Jose is committing a fix for this to GeoNetwork today;

(3) GeoNetwork can only handle GetRecordsResponse that contains SearchResults of type csw:SummaryRecord (CSW) or gmd:MD_Metadata (ISO). Other types, like the dc:metadata receivede from deegree, leads to the message "Skipped record not in supported format :" you got

(4) sending GetRecords requests to the deegree server with various outputSchema I get :

outputSchema="http://www.isotc211.org/2005/gmd"  --> response: ISO records, correct

outputSchema="http://www.opengis.net/cat/csw/2.0.2" --> response: dc:metada, expected response: csw:SummaryRecord

<absent outputSchema> -->  response: dc:metada, expected response: csw:SummaryRecord

Hopefully things are more clear ?

Kind regards
Heikki Doeleman






On Wed, Sep 30, 2009 at 12:18 PM, Francois Prunayre <[hidden email]> wrote:
Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <[hidden email]>:
> GeoNetwork 2.41's CSW harvester fails with our deegree 2.3pre CSW service
> (http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).
>
>
> GeoNetwork's GetRecords transaction request returns what appears to be a
> valid CSW and Dublin Core GetRecordsResponse response. I have attached the
> truncated response. This is a great improvement because deegree didn't
> accept the GeoNetwork response in the past.
>
> However, GeoNetwork's log gives me a "[geonetwork.harvester] - Skipped
> record not in supported format : metadata" error and I am having
> difficulties figuring out what the problem is.
It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
"Default value is http://www.opengis.net/cat/csw/2.0.2" CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema="http://www.opengis.net/cat/csw/2.0.2"
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:
 * GetRecords in summary format first to retrieve ids and date
 * GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois



> What kind of response does GeoNetwork require?
> Also, how do I enable more verbose logging in GeoNetwork?
>
> Thank you for your time!
>
> POST request to http://75.101.143.247:8080/deegree-csw/services :
> <?xml version="1.0" encoding="UTF-8"?>
> <csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
> service="CSW" version="2.0.2" resultType="results" startPosition="1301"
> maxRecords="20">
>  <csw:Query typeNames="csw:Record">
>    <csw:ElementSetName>summary</csw:ElementSetName>
>  </csw:Query>
> </csw:GetRecords>
>
> Response:
> See attachment with a truncated response.
>
> --
> _______________________________
> Wolfgang Grunberg
> Arizona Geological Survey
> [hidden email]
> 520-770-3500
>
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> GeoNetwork-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>

------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


-- 
_______________________________
Wolfgang Grunberg
Arizona Geological Survey
[hidden email]
520-770-3500

  


------------------------------------------------------------------------------ Come build with us! The BlackBerry&reg; Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33; http://p.sf.net/sfu/devconf

_______________________________________________ deegree-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/deegree-users

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
deegree-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-users
wgrunberg

Re: CSW GetRecords supposed to return csw:SummaryRecord instead of dc:metada?

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)
Heikki Doeleman posted at [hidden email]

hi Wolfgang,

the tests I did were already using your URL at http://75.101.143.247:8080/deegree-csw/services. For good measure I tried them again, both with your online CSW client that you mentioned and with another HTTP client tool (Poster), yielding the same results :

outputSchema="http://www.isotc211.org/2005/gmd"  --> response: ISO records, correct

outputSchema="http://www.opengis.net/cat/csw/2.0.2" --> response: dc:metada, expected response: csw:SummaryRecord

<absent outputSchema> -->  response: dc:metada, expected response: csw:SummaryRecord


So if I'm not mistaken, a bug in the deegree CSW server is that outputSchema="http://www.opengis.net/cat/csw/2.0.2" is not handled correctly. Maybe you do handle the case of a missing outputSchema correctly in the sense that you treat it as "http://www.opengis.net/cat/csw/2.0.2" which would explain that we get the same results ?


As for the contents that should be in the output when outputSchema is absent or is "http://www.opengis.net/cat/csw/2.0.2", it definitely still should be a csw:GetRecordsResponse, but one that inside its descendant csw:SearchResults contains a list of csw:SummaryRecord elements (and not as it does now, a list of dc:metadata elements).

Inside those csw:SummaryRecord elements I would expect dc: formatted data, yes, starting with a dc:identifier or a dct:bibliographicCitation element. For the full spec see the CSW202 schema at http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd.


Does this make sense ?

Kind regards
Heikki Doeleman
_______________________________
Wolfgang Grunberg
Arizona Geological Survey
[hidden email]
520-770-3500



Wolfgang Grunberg wrote:
Correction:
>From what I understand, both csw:SummaryRecord and csw:GetRecordsResponse responses should have DC formated metadata if no ouputSchema is defined.
However, should a GetRecords request with a "summary " ElementSetName return a csw:SummaryRecord response and not a csw:GetRecordsResponse response?

WG

Wolfgang Grunberg wrote:
Deer deegree team,

I am using GeoNetwork 2.4.1 to harvest CSW records from our deegree 2.3pre (August 2009) build and I am encountering dificulties due to what may be a bug in deegree_csw.

Geonetwork sends a GetRecords request with <csw:ElementSetName>summary</csw:ElementSetName> but without an outputSchema. The default outputSchema for a GetRecords request is http://www.opengis.net/cat/csw/2.0.2. deegree returns dc:metada but apparently the response is suposed to be csw:SummaryRecord.

I looked at the CSW 2.0.2 specifications again but I am only more confused now.
Following is the thread I kicked off on the GeoNetwork developer forum.
I hope the GeoNetwork folks make more sense.

Ciao, Wolfgang




-------- Original Message --------
Subject: Re: [GeoNetwork-devel] harvester fails with deegree CSW 2.0.2 service
Date: Wed, 30 Sep 2009 13:14:15 +0200
From: heikki [hidden email]
Reply-To: [hidden email]
To: Francois Prunayre [hidden email]
CC: Wolfgang Grunberg [hidden email], [hidden email]
References: [hidden email] [hidden email] [hidden email]


hello Wolfgang, François, and list :

I've looked at it a bit more with Jose Garcia, and we found :

(1) GeoNetwork did not send a outputSchema in its GetRecords request. This means that the default per the CSW202 spec should be used by the deegree server, which I think means we should receive results of type csw:SummaryRecord, as says François;

(2) GeoNetwork should actually put a "more preferred" ISO outputSchema in the request, if the Capabilities advertizes it (as does the deegree server's Capabilities). Jose is committing a fix for this to GeoNetwork today;

(3) GeoNetwork can only handle GetRecordsResponse that contains SearchResults of type csw:SummaryRecord (CSW) or gmd:MD_Metadata (ISO). Other types, like the dc:metadata receivede from deegree, leads to the message "Skipped record not in supported format :" you got

(4) sending GetRecords requests to the deegree server with various outputSchema I get :

outputSchema="http://www.isotc211.org/2005/gmd"  --> response: ISO records, correct

outputSchema="http://www.opengis.net/cat/csw/2.0.2" --> response: dc:metada, expected response: csw:SummaryRecord

<absent outputSchema> -->  response: dc:metada, expected response: csw:SummaryRecord

Hopefully things are more clear ?

Kind regards
Heikki Doeleman






On Wed, Sep 30, 2009 at 12:18 PM, Francois Prunayre <[hidden email]> wrote:
Hello Wolfgang,

2009/9/30 Wolfgang Grunberg <[hidden email]>:
> GeoNetwork 2.41's CSW harvester fails with our deegree 2.3pre CSW service
> (http://75.101.143.247:8080/deegree-csw/services?REQUEST=GetCapabilities&version=2.0.2&service=CSW).
>
>
> GeoNetwork's GetRecords transaction request returns what appears to be a
> valid CSW and Dublin Core GetRecordsResponse response. I have attached the
> truncated response. This is a great improvement because deegree didn't
> accept the GeoNetwork response in the past.
>
> However, GeoNetwork's log gives me a "[geonetwork.harvester] - Skipped
> record not in supported format : metadata" error and I am having
> difficulties figuring out what the problem is.
It looks like dc:metadata elements are returned in deegree CSW
response. The harvester ask for a summary response with default schema
"Default value is http://www.opengis.net/cat/csw/2.0.2" CSW 2.0.2 spec.
I think we should get csw:SummaryRecord instead. Not sure if the
dc:metadata elements are 2.0.1 spec elements ?

Also if you set outputSchema="http://www.opengis.net/cat/csw/2.0.2"
in the http://testing.deegree.org/deegree-csw/client/client.html you
get ISO records.
If you did not set outputSchema, you get an exception.

We discussed that point with Heikki, and the harvester is doing the following:
 * GetRecords in summary format first to retrieve ids and date
 * GetRecordById in ISO or OGC format to load the records.

More test needed.

Francois



> What kind of response does GeoNetwork require?
> Also, how do I enable more verbose logging in GeoNetwork?
>
> Thank you for your time!
>
> POST request to http://75.101.143.247:8080/deegree-csw/services :
> <?xml version="1.0" encoding="UTF-8"?>
> <csw:GetRecords xmlns:csw="http://www.opengis.net/cat/csw/2.0.2"
> service="CSW" version="2.0.2" resultType="results" startPosition="1301"
> maxRecords="20">
>  <csw:Query typeNames="csw:Record">
>    <csw:ElementSetName>summary</csw:ElementSetName>
>  </csw:Query>
> </csw:GetRecords>
>
> Response:
> See attachment with a truncated response.
>
> --
> _______________________________
> Wolfgang Grunberg
> Arizona Geological Survey
> [hidden email]
> 520-770-3500
>
>
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
> http://p.sf.net/sfu/devconf
> _______________________________________________
> GeoNetwork-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>

------------------------------------------------------------------------------
Come build with us! The BlackBerry&reg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33;
http://p.sf.net/sfu/devconf
_______________________________________________
GeoNetwork-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork


-- 
_______________________________
Wolfgang Grunberg
Arizona Geological Survey
[hidden email]
520-770-3500

  


------------------------------------------------------------------------------ Come build with us! The BlackBerry&reg; Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33; http://p.sf.net/sfu/devconf

_______________________________________________ deegree-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/deegree-users

------------------------------------------------------------------------------ Come build with us! The BlackBerry&reg; Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9&#45;12, 2009. Register now&#33; http://p.sf.net/sfu/devconf

_______________________________________________ deegree-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/deegree-users

------------------------------------------------------------------------------
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
http://p.sf.net/sfu/devconf
_______________________________________________
deegree-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-users