Quick and Dirty CSV File

8 messages Options
Embed this post
Permalink
Landon Blake

Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)

I busted out a quick and dirty CSV file for discussion. I attached it as a text file with comments. I also attached it as a Microsoft Excel Spreadhseet without comments and with some formatting. I can also make it available in Open Office Format.

 

A couple of things came up when I was working on the file:

 

-          I provided a suggested 3D tolerance in the units of the expected coordinates. Does anyone see a problem with using these units instead of the source coordinates?

-          I added a comments section. I figured we could add comments there, without commas, of course. This will make a good “metadata” field for each row if needed.

-          I didn’t have to work with degrees-minutes-seconds ordinate values, but some rows might need to. I would suggest we use a format like 37-57-23.66554 and -121-45-18.55265 for ordinate values that need to be in this format.

-          I also propose we use the negative sign to indicate west longitude values and south latitude values.

 

I will await discussion before making revisions to this file.

 

Landon

 

 



Warning:
Information provided via electronic media is not guaranteed against defects including translation and transmission errors.
If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.




_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs

Test_Data_File.csv (2K) Download Attachment
Test_Data_File.xls (19K) Download Attachment
Norm Olsen

RE: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
Some javascript/style in this post has been disabled (why?)

I’ve never used a Wiki before.  I have posted a comment under the discussion tab.  Is that what one is supposed to do to comment on the current proposal?

 

Norm

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Landon Blake
Sent: Thursday, November 05, 2009 9:29 AM
To: [hidden email]
Subject: [MetaCRS] Quick and Dirty CSV File

 

I busted out a quick and dirty CSV file for discussion. I attached it as a text file with comments. I also attached it as a Microsoft Excel Spreadhseet without comments and with some formatting. I can also make it available in Open Office Format.

 

A couple of things came up when I was working on the file:

 

-       I provided a suggested 3D tolerance in the units of the expected coordinates. Does anyone see a problem with using these units instead of the source coordinates?

-       I added a comments section. I figured we could add comments there, without commas, of course. This will make a good “metadata” field for each row if needed.

-       I didn’t have to work with degrees-minutes-seconds ordinate values, but some rows might need to. I would suggest we use a format like 37-57-23.66554 and -121-45-18.55265 for ordinate values that need to be in this format.

-       I also propose we use the negative sign to indicate west longitude values and south latitude values.

 

I will await discussion before making revisions to this file.

 

Landon

 

 

 

Warning:
Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.


_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Martin Davis

Re: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
In reply to this post by Landon Blake
I'm going to post my comments here:

- I'm not crazy about the "Northing:nnnn" format. Seems hard to parse,
and what exactly is being communicated by this prefix? I think Norm is
on the money with his suggestion that the CRS determines the
interpretation of the ordinate values.

- Slim down the column names. Eg
Expected Destination CRS Ordinate 1 -> Dest Ord 1
Suggest 3D Distance Tolerance -> Distance Tol

- Is there agreement about the concept of a 3D distance tolerance? It
seems to me that it's better to break out the tolerances for each axis,
if that's desired.

- Instead of saying "None" or "Not Provided" just leave the field empty
(null)

- Would it be better to normalize the authority prefix out into a
separate column? This will make the table more amenable to querying in
database environments.

Martin

Landon Blake wrote:

>
> I busted out a quick and dirty CSV file for discussion. I attached it
> as a text file with comments. I also attached it as a Microsoft Excel
> Spreadhseet without comments and with some formatting. I can also make
> it available in Open Office Format.
>
> A couple of things came up when I was working on the file:
>
> - I provided a suggested 3D tolerance in the units of the expected
> coordinates. Does anyone see a problem with using these units instead
> of the source coordinates?
>
> - I added a comments section. I figured we could add comments there,
> without commas, of course. This will make a good “metadata” field for
> each row if needed.
>
> - I didn’t have to work with degrees-minutes-seconds ordinate values,
> but some rows might need to. I would suggest we use a format like
> 37-57-23.66554 and -121-45-18.55265 for ordinate values that need to
> be in this format.
>
> - I also propose we use the negative sign to indicate west longitude
> values and south latitude values.
>
> I will await discussion before making revisions to this file.
>
> Landon
>
>
>
> *Warning:
> *Information provided via electronic media is not guaranteed against
> defects including translation and transmission errors. If the reader
> is not the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication is
> strictly prohibited. If you have received this information in error,
> please notify the sender immediately.
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> MetaCRS mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/metacrs
>  

--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022

_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Landon Blake

RE: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
In reply to this post by Landon Blake
This is harder than I thought. :]

Martin wrote: "- I'm not crazy about the "Northing:nnnn" format. Seems
hard to parse, and what exactly is being communicated by this prefix? I
think Norm is on the money with his suggestion that the CRS determines
the
interpretation of the ordinate values."

The ordinate prefix eliminates the need to remember the order of
ordinate values, and would also give the parser a clue to the format for
the value that follows. I don't need to worry about parsing a value in
DMS format for an ellipsoid height that is always in meters.

I don't know about Python or Javascript, but you can split the ordinate
prefix from the ordinate value pretty simply with the built in
String.split function.

Still, if the consensus is that this it too hard to parse, we can drop
the prefix.

Martin wrote: "Slim down the column names. Eg
Expected Destination CRS Ordinate 1 -> Dest Ord 1 Suggest 3D Distance
Tolerance -> Distance Tol"

I don't see why this is necessary. Abbreviations make things harder to
understand, and we are just dealing with plain text files. If there is
no file format limitation on column name length, why not spell it out?

Martin wrote: "Is there agreement about the concept of a 3D distance
tolerance? It seems to me that it's better to break out the tolerances
for each axis, if that's desired."

I have no preference. This was just my suggestion. I can change it to a
tolerance for each axis.

Martin wrote: "Instead of saying "None" or "Not Provided" just leave the
field empty (null)"

Now this makes parsing harder! If we leave "none" or "not provided" each
row will always have the same number of values. If we don't include a
value each row could have a different number of values or cells. This
could make parsing a lot more difficult. For example, if I use the
String.split function in Java I can get an array of Strings where the
comma determines what goes in each element of the array. I can then use
the same index position to always access the desired array element
without having to look at the other elements in the array. I hope this
makes sense.

Martin wrote: "Would it be better to normalize the authority prefix out
into a separate column? This will make the table more amenable to
querying in database environments."

This is a good suggestion. I can make this change.

Landon
Office Phone Number: (209) 946-0268
Cell Phone Number: (209) 992-0658
 
 
-----Original Message-----
From: Martin Davis [mailto:[hidden email]]
Sent: Thursday, November 05, 2009 10:11 AM
To: Landon Blake
Subject: Re: [MetaCRS] Quick and Dirty CSV File

I'm going to post my comments here:

- I'm not crazy about the "Northing:nnnn" format. Seems hard to parse,
and what exactly is being communicated by this prefix? I think Norm is
on the money with his suggestion that the CRS determines the
interpretation of the ordinate values.

- Slim down the column names. Eg
Expected Destination CRS Ordinate 1 -> Dest Ord 1
Suggest 3D Distance Tolerance -> Distance Tol

- Is there agreement about the concept of a 3D distance tolerance? It
seems to me that it's better to break out the tolerances for each axis,
if that's desired.

- Instead of saying "None" or "Not Provided" just leave the field empty
(null)

- Would it be better to normalize the authority prefix out into a
separate column? This will make the table more amenable to querying in
database environments.

Martin


Landon Blake wrote:
>
> I busted out a quick and dirty CSV file for discussion. I attached it
> as a text file with comments. I also attached it as a Microsoft Excel
> Spreadhseet without comments and with some formatting. I can also make

> it available in Open Office Format.
>
> A couple of things came up when I was working on the file:
>
> - I provided a suggested 3D tolerance in the units of the expected
> coordinates. Does anyone see a problem with using these units instead
> of the source coordinates?
>
> - I added a comments section. I figured we could add comments there,
> without commas, of course. This will make a good "metadata" field for
> each row if needed.
>
> - I didn't have to work with degrees-minutes-seconds ordinate values,
> but some rows might need to. I would suggest we use a format like
> 37-57-23.66554 and -121-45-18.55265 for ordinate values that need to
> be in this format.
>
> - I also propose we use the negative sign to indicate west longitude
> values and south latitude values.
>
> I will await discussion before making revisions to this file.
>
> Landon
>
>
>
> *Warning:
> *Information provided via electronic media is not guaranteed against
> defects including translation and transmission errors. If the reader
> is not the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication is
> strictly prohibited. If you have received this information in error,
> please notify the sender immediately.
>
>
------------------------------------------------------------------------
>
> _______________________________________________
> MetaCRS mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/metacrs
>  

--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022



Warning:
Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.
_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Landon Blake

RE: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
Just remembered reading something about "painting a bike shed". The
point of the illustration was that open source developers debate the
most over the least important things.

If Martin or Norm wish, they can tweak the CSV file and post it to the
list again. I will accept whatever changes they feel are appropriate.

I am, after all, still very much a rookie programmer and rookie
geodesist. :]

Landon
Office Phone Number: (209) 946-0268
Cell Phone Number: (209) 992-0658
 
 
-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Landon Blake
Sent: Thursday, November 05, 2009 10:13 AM
To: Martin Davis
Cc: [hidden email]
Subject: RE: [MetaCRS] Quick and Dirty CSV File

This is harder than I thought. :]

Martin wrote: "- I'm not crazy about the "Northing:nnnn" format. Seems
hard to parse, and what exactly is being communicated by this prefix? I
think Norm is on the money with his suggestion that the CRS determines
the
interpretation of the ordinate values."

The ordinate prefix eliminates the need to remember the order of
ordinate values, and would also give the parser a clue to the format for
the value that follows. I don't need to worry about parsing a value in
DMS format for an ellipsoid height that is always in meters.

I don't know about Python or Javascript, but you can split the ordinate
prefix from the ordinate value pretty simply with the built in
String.split function.

Still, if the consensus is that this it too hard to parse, we can drop
the prefix.

Martin wrote: "Slim down the column names. Eg
Expected Destination CRS Ordinate 1 -> Dest Ord 1 Suggest 3D Distance
Tolerance -> Distance Tol"

I don't see why this is necessary. Abbreviations make things harder to
understand, and we are just dealing with plain text files. If there is
no file format limitation on column name length, why not spell it out?

Martin wrote: "Is there agreement about the concept of a 3D distance
tolerance? It seems to me that it's better to break out the tolerances
for each axis, if that's desired."

I have no preference. This was just my suggestion. I can change it to a
tolerance for each axis.

Martin wrote: "Instead of saying "None" or "Not Provided" just leave the
field empty (null)"

Now this makes parsing harder! If we leave "none" or "not provided" each
row will always have the same number of values. If we don't include a
value each row could have a different number of values or cells. This
could make parsing a lot more difficult. For example, if I use the
String.split function in Java I can get an array of Strings where the
comma determines what goes in each element of the array. I can then use
the same index position to always access the desired array element
without having to look at the other elements in the array. I hope this
makes sense.

Martin wrote: "Would it be better to normalize the authority prefix out
into a separate column? This will make the table more amenable to
querying in database environments."

This is a good suggestion. I can make this change.

Landon
Office Phone Number: (209) 946-0268
Cell Phone Number: (209) 992-0658
 
 
-----Original Message-----
From: Martin Davis [mailto:[hidden email]]
Sent: Thursday, November 05, 2009 10:11 AM
To: Landon Blake
Subject: Re: [MetaCRS] Quick and Dirty CSV File

I'm going to post my comments here:

- I'm not crazy about the "Northing:nnnn" format. Seems hard to parse,
and what exactly is being communicated by this prefix? I think Norm is
on the money with his suggestion that the CRS determines the
interpretation of the ordinate values.

- Slim down the column names. Eg
Expected Destination CRS Ordinate 1 -> Dest Ord 1
Suggest 3D Distance Tolerance -> Distance Tol

- Is there agreement about the concept of a 3D distance tolerance? It
seems to me that it's better to break out the tolerances for each axis,
if that's desired.

- Instead of saying "None" or "Not Provided" just leave the field empty
(null)

- Would it be better to normalize the authority prefix out into a
separate column? This will make the table more amenable to querying in
database environments.

Martin


Landon Blake wrote:
>
> I busted out a quick and dirty CSV file for discussion. I attached it
> as a text file with comments. I also attached it as a Microsoft Excel
> Spreadhseet without comments and with some formatting. I can also make

> it available in Open Office Format.
>
> A couple of things came up when I was working on the file:
>
> - I provided a suggested 3D tolerance in the units of the expected
> coordinates. Does anyone see a problem with using these units instead
> of the source coordinates?
>
> - I added a comments section. I figured we could add comments there,
> without commas, of course. This will make a good "metadata" field for
> each row if needed.
>
> - I didn't have to work with degrees-minutes-seconds ordinate values,
> but some rows might need to. I would suggest we use a format like
> 37-57-23.66554 and -121-45-18.55265 for ordinate values that need to
> be in this format.
>
> - I also propose we use the negative sign to indicate west longitude
> values and south latitude values.
>
> I will await discussion before making revisions to this file.
>
> Landon
>
>
>
> *Warning:
> *Information provided via electronic media is not guaranteed against
> defects including translation and transmission errors. If the reader
> is not the intended recipient, you are hereby notified that any
> dissemination, distribution or copying of this communication is
> strictly prohibited. If you have received this information in error,
> please notify the sender immediately.
>
>
------------------------------------------------------------------------
>
> _______________________________________________
> MetaCRS mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/metacrs
>  

--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022



Warning:
Information provided via electronic media is not guaranteed against
defects including translation and transmission errors. If the reader is
not the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication is strictly
prohibited. If you have received this information in error, please
notify the sender immediately.
_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Martin Davis

Re: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
In reply to this post by Landon Blake


Landon Blake wrote:

>
> Martin wrote: "Instead of saying "None" or "Not Provided" just leave the
> field empty (null)"
>
> Now this makes parsing harder! If we leave "none" or "not provided" each
> row will always have the same number of values. If we don't include a
> value each row could have a different number of values or cells. This
> could make parsing a lot more difficult. For example, if I use the
> String.split function in Java I can get an array of Strings where the
> comma determines what goes in each element of the array. I can then use
> the same index position to always access the desired array element
> without having to look at the other elements in the array. I hope this
> makes sense.
>
>  
This isn't a problem in CSV files.  An empty field still has a field
delimiter, so the number of columns is unchanged.

--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022

_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Landon Blake

RE: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
I'm about 4 shades of red right now. You are correct. When saving the
file with no value from Excel the empty cell was preserved.

My bad.

Landon
Office Phone Number: (209) 946-0268
Cell Phone Number: (209) 992-0658
 
 

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Martin Davis
Sent: Thursday, November 05, 2009 10:37 AM
Cc: [hidden email]
Subject: Re: [MetaCRS] Quick and Dirty CSV File



Landon Blake wrote:
>
> Martin wrote: "Instead of saying "None" or "Not Provided" just leave
the
> field empty (null)"
>
> Now this makes parsing harder! If we leave "none" or "not provided"
each
> row will always have the same number of values. If we don't include a
> value each row could have a different number of values or cells. This
> could make parsing a lot more difficult. For example, if I use the
> String.split function in Java I can get an array of Strings where the
> comma determines what goes in each element of the array. I can then
use
> the same index position to always access the desired array element
> without having to look at the other elements in the array. I hope this
> makes sense.
>
>  
This isn't a problem in CSV files.  An empty field still has a field
delimiter, so the number of columns is unchanged.

--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022

_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs


Warning:
Information provided via electronic media is not guaranteed against defects including translation and transmission errors. If the reader is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this information in error, please notify the sender immediately.
_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs
Martin Davis

Re: Quick and Dirty CSV File

Reply Threaded More More options
Print post
Permalink
In reply to this post by Landon Blake


Landon Blake wrote:

> This is harder than I thought. :]
>
> Martin wrote: "- I'm not crazy about the "Northing:nnnn" format. Seems
> hard to parse, and what exactly is being communicated by this prefix? I
> think Norm is on the money with his suggestion that the CRS determines
> the
> interpretation of the ordinate values."
>
> The ordinate prefix eliminates the need to remember the order of
> ordinate values, and would also give the parser a clue to the format for
> the value that follows. I don't need to worry about parsing a value in
> DMS format for an ellipsoid height that is always in meters.
>
> I don't know about Python or Javascript, but you can split the ordinate
> prefix from the ordinate value pretty simply with the built in
> String.split function.
>
> Still, if the consensus is that this it too hard to parse, we can drop
> the prefix.
>
>  
It's easy to write a custom parser to do this, but as Norm said one of
the advantages of using CSV is that it can be consumed by numerous tools
- many of which won't be able to easily parse out prefixes.

Also, you will have to rigorously define the set of prefixes used and
their semantics.  This is more documentation to maintain and more
conceptual weight for users to understand.

If this information is desired, I would suggest breaking it out into
extra columns.


--
Martin Davis
Senior Technical Architect
Refractions Research, Inc.
(250) 383-3022

_______________________________________________
MetaCRS mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/metacrs