wiki-nature geodata aggregators

[Jo-2]

dear all,

I've recently been amazed at how both geonames and OpenStreetmap are
now acting as data aggregators for a lot of other sources:

http://www.geonames.org/data-sources.html
http://wiki.openstreetmap.org/wiki/Potential_Datasources

Great that this is happening, but also provoking an "issue" insofaras:
- such sites have "wiki-nature" interfaces to accept modifications and
improvements
- there's no mechanism to feed changes back to help fix and improve
the original sources

The admin burden in feeding changes back would be pretty high for a
community run, open site.

Open geodata is still open to accusations of data quality problems -
variability in coverage, variable generailty in precision.
"It would be nice" to see a data quality push - not in terms of
comparison to proprietary sources, but in terms of peer review and of
spreading, well, quality assurance, through the network back up to
original data providers. How to do this without bureaucratic overhead
(or while funding bureaucratic overhead)? Answers on a postcard...


love, jo
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Martin Spott]

Jo Walsh wrote:

> "It would be nice" to see a data quality push - not in terms of
> comparison to proprietary sources, but in terms of peer review and of
> spreading, well, quality assurance, through the network back up to
> original data providers. How to do this without bureaucratic overhead
> (or while funding bureaucratic overhead)? Answers on a postcard...

No idea, as the conceptual 'design' behind all these data providers
requires their sources to be "authoritative" in some way - a criteria
that is unlikely to be met by any effort which relies primarily on
crowdsourcing  :-)

Cheers,
        Martin.
--
 Unix _IS_ user friendly - it's just selective about who its friends are !
--------------------------------------------------------------------------
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Andrew Turner-2]

On Sat, Jul 25, 2009 at 11:57 AM, Martin Spott<[hidden email]> wrote:

> Jo Walsh wrote:
>
>> "It would be nice" to see a data quality push - not in terms of
>> comparison to proprietary sources, but in terms of peer review and of
>> spreading, well, quality assurance, through the network back up to
>> original data providers. How to do this without bureaucratic overhead
>> (or while funding bureaucratic overhead)? Answers on a postcard...
>
> No idea, as the conceptual 'design' behind all these data providers
> requires their sources to be "authoritative" in some way - a criteria
> that is unlikely to be met by any effort which relies primarily on
> crowdsourcing  :-)

I disagree. You're seeing OSM and GeoNames powering the data systems
of more companies and organizations. They are leveraging scale,
development and communities that none of these individual groups could
begin to enable themselves.

There have been efforts to incorporate data back into original sources
- a problem that has social, legal, and technical hurdles. Yet the
onus can be on the receiving organization to incorporate this
potentially very valuable data instead of putting that burden on the
larger community itself to attempt to cater to each group.

What I think we'll see more of are organizations just embracing these
projects and data sources as their primary source itself and
enhancing. The problem here is the licensing for databases like OSM is
"viral", so any changes would have to be opened back up - so the goal
here would be to convince government and other agencies that their
value doesn't lie in their hoarding the data, but curating and
ensuring it's coverage and accuracy in their specific areas of
interest. This removes the technical burden of them developing tools
(their not tool shops) or pulling together various domains and
connecting with regions outside their own area. The added benefit is
it engages them directly with the community that also cares in using
the data through these aggregation hubs.

Andrew

>
> Cheers,
>        Martin.
> --
>  Unix _IS_ user friendly - it's just selective about who its friends are !
> --------------------------------------------------------------------------
> _______________________________________________
> Geodata mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/geodata
>

--
Andrew Turner
mobile: [phone]
[hidden email]
http://highearthorbit.com

http://geocommons.com           Helping build the Geospatial Web
Introduction to Neogeography - http://oreilly.com/catalog/neogeography
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Mikel Maron, OSM]

Some javascript/style in this post has been disabled (why?)

Well said Andrew.

I'll also add that just because a geodata set has a stamp or "authority", and some apparent quality assurance process,
does not mean that the resulting data is any better, or the process any more full proof, than the many eyes approach of OSM.
The "man behind the curtain" of authoritative geodata is that the processes are actually less comprehensive (a certain NMA
that shall remain nameless does not even track which individual surveyor & cartographer were responsible for which bits
of data) and often of lower quality. And that's with tremendous amount of more money and time than OSM has had!

Not to say that there's nothing OSM can learn from traditional map makers .. we're not that full of ourselves!
But certainly there's widespread acknowledgement that "they" can learn from "us" as well. The efforts that Andrew
mentions, to feedback crowdsourced data into official sources, are some of the most interesting problems in our field
right now.

-Mikel

---

From: Andrew Turner <[hidden email]>
To: Martin Spott <[hidden email]>
Cc: [hidden email]
Sent: Saturday, July 25, 2009 10:03:41 AM
Subject: Re: [Geodata] wiki-nature geodata aggregators

On Sat, Jul 25, 2009 at 11:57 AM, Martin Spott<[hidden email]> wrote:

> Jo Walsh wrote:
>
>> "It would be nice" to see a data quality push - not in terms of
>> comparison to proprietary sources, but in terms of peer review and of
>> spreading, well, quality assurance, through the network back up to
>> original data providers. How to do this without bureaucratic overhead
>> (or while funding bureaucratic overhead)? Answers on a postcard...
>
> No idea, as the conceptual 'design' behind all these data providers
> requires their sources to be "authoritative" in some way - a criteria
> that is unlikely to be met by any effort which relies primarily on
> crowdsourcing  :-)

I disagree. You're seeing OSM and GeoNames powering the data systems
of more companies and organizations. They are leveraging scale,
development and communities that none of these individual groups could
begin to enable themselves.

There have been efforts to incorporate data back into original sources
- a problem that has social, legal, and technical hurdles. Yet the
onus can be on the receiving organization to incorporate this
potentially very valuable data instead of putting that burden on the
larger community itself to attempt to cater to each group.

What I think we'll see more of are organizations just embracing these
projects and data sources as their primary source itself and
enhancing. The problem here is the licensing for databases like OSM is
"viral", so any changes would have to be opened back up - so the goal
here would be to convince government and other agencies that their
value doesn't lie in their hoarding the data, but curating and
ensuring it's coverage and accuracy in their specific areas of
interest. This removes the technical burden of them developing tools
(their not tool shops) or pulling together various domains and
connecting with regions outside their own area. The added benefit is
it engages them directly with the community that also cares in using
the data through these aggregation hubs.

Andrew

>
> Cheers,
>        Martin.
> --
>  Unix _IS_ user friendly - it's just selective about who its friends are !
> --------------------------------------------------------------------------
> _______________________________________________
> Geodata mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/geodata
>



--
Andrew Turner
mobile: [phone]
[hidden email]
http://highearthorbit.com

http://geocommons.com          Helping build the Geospatial Web
Introduction to Neogeography - http://oreilly.com/catalog/neogeography
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Arnulf Christl (OSGeo)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Turner schrieb:

> On Sat, Jul 25, 2009 at 11:57 AM, Martin Spott<[hidden email]> wrote:
>> Jo Walsh wrote:
>>
>>> "It would be nice" to see a data quality push - not in terms of
>>> comparison to proprietary sources, but in terms of peer review and of
>>> spreading, well, quality assurance, through the network back up to
>>> original data providers. How to do this without bureaucratic overhead
>>> (or while funding bureaucratic overhead)? Answers on a postcard...
>> No idea, as the conceptual 'design' behind all these data providers
>> requires their sources to be "authoritative" in some way - a criteria
>> that is unlikely to be met by any effort which relies primarily on
>> crowdsourcing  :-)
>
> I disagree. You're seeing OSM and GeoNames powering the data systems
> of more companies and organizations. They are leveraging scale,
> development and communities that none of these individual groups could
> begin to enable themselves.

I agree with Andrew on this. The mobile dept of Nokia plans to integrate
OSM as the core dataset for their handhelds. Link anyone?

> There have been efforts to incorporate data back into original sources
> - a problem that has social, legal, and technical hurdles. Yet the
> onus can be on the receiving organization to incorporate this
> potentially very valuable data instead of putting that burden on the
> larger community itself to attempt to cater to each group.
>
> What I think we'll see more of are organizations just embracing these
> projects and data sources as their primary source itself and
> enhancing. The problem here is the licensing for databases like OSM is
> "viral", so any changes would have to be opened back up - so the goal
> here would be to convince government and other agencies that their
> value doesn't lie in their hoarding the data, but curating and
> ensuring it's coverage and accuracy in their specific areas of
> interest.

We have some encouraging efforts to that effect going on in Germany. The
viral effect of OSM can be remedied with a late bound, distributed
infrastructure, this Blog may help explain what I mean:
http://arnulf.us/sevendipity/archives/20-Copyright-in-a-Shrapnel-Shell.html

And the new license is practically in place already.

Regards,

> This removes the technical burden of them developing tools
> (their not tool shops) or pulling together various domains and
> connecting with regions outside their own area. The added benefit is
> it engages them directly with the community that also cares in using
> the data through these aggregation hubs.
>
> Andrew
>
>> Cheers,
>>        Martin.
>> --
>>  Unix _IS_ user friendly - it's just selective about who its friends are !
>> --------------------------------------------------------------------------
>> _______________________________________________
>> Geodata mailing list
>> [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/geodata
>>
>

- --
Arnulf Christl
Spatial Systems Architect
WhereGroup www.wheregroup.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpvB0oACgkQhcSN09gHfZyAYwCeO7XPQhKL5q5RR+3BzRDTUPRu
TVMAnAqR+4T5rV7ApcoSPjzIvqCWdMXe
=OFGZ
-----END PGP SIGNATURE-----
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Arnulf Christl (OSGeo)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Mikel Maron, OSM schrieb:
> Well said Andrew.
>
> I'll also add that just because a geodata set has a stamp or
> "authority", and some apparent quality assurance process, does not
> mean that the resulting data is any better, or the process any more
> full proof, than the many eyes approach of OSM.

True. At the same time for some it does make a difference whether an
authority of some kind stamps information or not. OSGeo may well (be)
develop(ed) into a kind of authority because it is not ...

> The "man behind the
> curtain"

...but values openness and operates with a crowd sourced, transparent
but all the same traceable process. And it is in some ways independent
of the OSM die-hards. Independency is highly valued for stamping things.

> of authoritative geodata is that the processes are actually
> less comprehensive (a certain NMA that shall remain nameless does not
> even track which individual surveyor & cartographer were responsible
> for which bits of data) and often of lower quality. And that's with
> tremendous amount of more money and time than OSM has had!

Beware of such statements, I am pretty unsure whether it is ecologically
viable to move a crowd to map something or have one professional do the
same thing with - well - "professional" efficiency. This is not said to
devalue what OSMembers do nor overestimate what certain unnamed
OSurveyors do.

> Not to say that there's nothing OSM can learn from traditional map
> makers .. we're not that full of ourselves! But certainly there's
> widespread acknowledgement that "they" can learn from "us" as well.
> The efforts that Andrew mentions, to feedback crowdsourced data into
> official sources, are some of the most interesting problems in our
> field right now.
>
> -Mikel

Well said.

Regards,

> ________________________________ From: Andrew Turner
> <[hidden email]> To: Martin Spott <[hidden email]> Cc:
> [hidden email] Sent: Saturday, July 25, 2009 10:03:41 AM
> Subject: Re: [Geodata] wiki-nature geodata aggregators
>
> On Sat, Jul 25, 2009 at 11:57 AM, Martin
> Spott<[hidden email]> wrote:
>> Jo Walsh wrote:
>>
>>> "It would be nice" to see a data quality push - not in terms of
>>> comparison to proprietary sources, but in terms of peer review
>>> and of spreading, well, quality assurance, through the network
>>> back up to original data providers. How to do this without
>>> bureaucratic overhead (or while funding bureaucratic overhead)?
>>> Answers on a postcard...
>> No idea, as the conceptual 'design' behind all these data providers
>>  requires their sources to be "authoritative" in some way - a
>> criteria that is unlikely to be met by any effort which relies
>> primarily on crowdsourcing  :-)
>
> I disagree. You're seeing OSM and GeoNames powering the data systems
> of more companies and organizations. They are leveraging scale,
> development and communities that none of these individual groups
> could begin to enable themselves.
>
> There have been efforts to incorporate data back into original
> sources - a problem that has social, legal, and technical hurdles.
> Yet the onus can be on the receiving organization to incorporate this
>  potentially very valuable data instead of putting that burden on the
>  larger community itself to attempt to cater to each group.
>
> What I think we'll see more of are organizations just embracing these
>  projects and data sources as their primary source itself and
> enhancing. The problem here is the licensing for databases like OSM
> is "viral", so any changes would have to be opened back up - so the
> goal here would be to convince government and other agencies that
> their value doesn't lie in their hoarding the data, but curating and
> ensuring it's coverage and accuracy in their specific areas of
> interest. This removes the technical burden of them developing tools
> (their not tool shops) or pulling together various domains and
> connecting with regions outside their own area. The added benefit is
> it engages them directly with the community that also cares in using
> the data through these aggregation hubs.
>
> Andrew
>
>> Cheers, Martin. -- Unix _IS_ user friendly - it's just selective
>> about who its friends are !
>> --------------------------------------------------------------------------
>>  _______________________________________________ Geodata mailing
>> list [hidden email]
>> http://lists.osgeo.org/mailman/listinfo/geodata
>>
>
>
>
>
> ------------------------------------------------------------------------
>
>
> _______________________________________________ Geodata mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/geodata

- --
Arnulf Christl
Spatial Systems Architect
WhereGroup www.wheregroup.com

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpvClsACgkQhcSN09gHfZz5QgCdH8t5B8irTgOw0aRXuuib1bGZ
cFQAoJwa0rCFwBA5p7e9KRBNZhIpYrsi
=pRZM
-----END PGP SIGNATURE-----
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[David William Bitner-2]

On Tue, Jul 28, 2009 at 9:25 AM, Arnulf Christl <[hidden email]> wrote:

> I'll also add that just because a geodata set has a stamp or
> "authority", and some apparent quality assurance process, does not
> mean that the resulting data is any better, or the process any more
> full proof, than the many eyes approach of OSM.

True. At the same time for some it does make a difference whether an
authority of some kind stamps information or not. OSGeo may well (be)
develop(ed) into a kind of authority because it is not ...

Often times when liability is concerned sometimes -- often more wrongly than right -- it is necessary to use an "official" version of something merely for the reason that you can then pass-the-buck if there are problems.  Along the same lines, when you are doing work that could end up in a courtroom or more simply end up getting questioned or audited or .... it is necessary to build off something that is at least a snapshot such that those who are questioning/auditing/whatevering you can go back and recreate your work from the same version of the data that you had used. 

As with source code, versioning, having snapshots, sometimes having a branch away from trunk that gets released (think a snapshot of OSM at a certain time that has been deemed by a local authority as acceptable for planning work and tagged as such) can be very valuable things that can help to bridge the gap between the "wilds" of crowd sourcing and the need for control of "official" datasets.  Along with this,  as an entity considers tagging a blessed version for whatever purpose, as the trunk continues to change, having good "diff" tools becomes important for the authorities such that they can monitor what is changing and can respond accordingly (accepting the edits, going back in and saying "no, we surveyed this with super new fangled gadget and know that it is right" etc).

_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Andrew Turner]

Some javascript/style in this post has been disabled (why?)

David William Bitner wrote:

On Tue, Jul 28, 2009 at 9:25 AM, Arnulf Christl <[hidden email]> wrote:

> I'll also add that just because a geodata set has a stamp or
> "authority", and some apparent quality assurance process, does not
> mean that the resulting data is any better, or the process any more
> full proof, than the many eyes approach of OSM.

True. At the same time for some it does make a difference whether an
authority of some kind stamps information or not. OSGeo may well (be)
develop(ed) into a kind of authority because it is not ...

Often times when liability is concerned sometimes -- often more wrongly than right -- it is necessary to use an "official" version of something merely for the reason that you can then pass-the-buck if there are problems.  Along the same lines, when you are doing work that could end up in a courtroom or more simply end up getting questioned or audited or .... it is necessary to build off something that is at least a snapshot such that those who are questioning/auditing/whatevering you can go back and recreate your work from the same version of the data that you had used. 

As with source code, versioning, having snapshots, sometimes having a branch away from trunk that gets released (think a snapshot of OSM at a certain time that has been deemed by a local authority as acceptable for planning work and tagged as such) can be very valuable things that can help to bridge the gap between the "wilds" of crowd sourcing and the need for control of "official" datasets.  Along with this,  as an entity considers tagging a blessed version for whatever purpose, as the trunk continues to change, having good "diff" tools becomes important for the authorities such that they can monitor what is changing and can respond accordingly (accepting the edits, going back in and saying "no, we surveyed this with super new fangled gadget and know that it is right" etc).

There has been some good brainstorming on this idea at a few recent conferences and sessions. An entity could "sign" a snapshot/region/way. Calculate the MD5 hash, apply your GPG public key, and you now have an 'official', or accepted, version of data. This is similar to how Git (and other DVCS tools) work.

_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[nicolas chavent]

"Along with this,  as an entity considers tagging a blessed version for whatever purpose, as the trunk continues to change, having good "diff" tools becomes important for the authorities such that they can monitor what is changing and can respond accordingly (accepting the edits, going back in and saying "no, we surveyed this with super new fangled gadget and know that it is right" etc)."

Curious about experienced proven solutions for these "diff tools".
Curious as well of documented cases where by "Authorities" are using VGI datasets (of which OSM can be a paradigm) the way outlined above by David. This aspect of data sharing has been touched in some disussions at State Of The Map 2009. And something similar to what David outlined would be being thought of by the canadian NMA which shared its base map data (Geobase) such a with OSM. In such a scheme, OSM edits to the shared version of Geobase, would help the NMA into managing their update cycle more efficiently.

Thanks in advance for any element on this.

Best
N.

On Tue, Jul 28, 2009 at 6:59 PM, Andrew Turner <[hidden email]> wrote:

David William Bitner wrote:

On Tue, Jul 28, 2009 at 9:25 AM, Arnulf Christl <[hidden email]> wrote:

> I'll also add that just because a geodata set has a stamp or
> "authority", and some apparent quality assurance process, does not
> mean that the resulting data is any better, or the process any more
> full proof, than the many eyes approach of OSM.

True. At the same time for some it does make a difference whether an
authority of some kind stamps information or not. OSGeo may well (be)
develop(ed) into a kind of authority because it is not ...

Often times when liability is concerned sometimes -- often more wrongly than right -- it is necessary to use an "official" version of something merely for the reason that you can then pass-the-buck if there are problems.  Along the same lines, when you are doing work that could end up in a courtroom or more simply end up getting questioned or audited or .... it is necessary to build off something that is at least a snapshot such that those who are questioning/auditing/whatevering you can go back and recreate your work from the same version of the data that you had used. 

As with source code, versioning, having snapshots, sometimes having a branch away from trunk that gets released (think a snapshot of OSM at a certain time that has been deemed by a local authority as acceptable for planning work and tagged as such) can be very valuable things that can help to bridge the gap between the "wilds" of crowd sourcing and the need for control of "official" datasets.  Along with this,  as an entity considers tagging a blessed version for whatever purpose, as the trunk continues to change, having good "diff" tools becomes important for the authorities such that they can monitor what is changing and can respond accordingly (accepting the edits, going back in and saying "no, we surveyed this with super new fangled gadget and know that it is right" etc).

There has been some good brainstorming on this idea at a few recent conferences and sessions. An entity could "sign" a snapshot/region/way. Calculate the MD5 hash, apply your GPG public key, and you now have an 'official', or accepted, version of data. This is similar to how Git (and other DVCS tools) work.

_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

--
Nicolas Chavent
Norplan, Abu Dhabi
GIS Expert and Project Manager
Mobile (UAE): + 971 56 602 35 15
Email: [hidden email]
Skype: c_nicolas

_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata

[Jo-2]

dear all,

On Sun, Jul 26, 2009 at 9:53 PM, Mikel Maron, OSM<[hidden email]>
> The "man behind the curtain" of authoritative geodata is that the processes
> are actually less comprehensive (a certain NMA
> that shall remain nameless does not even track which individual surveyor &
> cartographer were responsible for which bits
> of data)

In the foyer of the National Library of Scotland they're selling
facsimiles of John Thomson's Atlas of Scotland.
http://www.nls.uk/maps/atlas/thomson/index.html

Each set of pages has a different attestation - this map attested by
So-and-So, Land Surveyor, So-and-So, Factor to the Duke of
Such-and-Such. I enjoyed that; the curtain is opened, data
contributions endorsed by, dare I say it, a social network.

Sample attestations here (one grainy cameraphoto from me,
one tile stolen from the NLIS map service):
http://zool.posterous.com/map-attestation

Harder to gain traction for attestation with mutable vector data -
"This line segment is attested by..." or with data that may get
ever increasing resolution - one could attest to completeness
at different scales, for different tiles? A lot easier with geonames...

cheers,


jo
_______________________________________________
Geodata mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/geodata