OSGeo.org › OSGeo Software and Data Projects › GRASS › Grass - Users

v.in.ascii slowness in GRASS 6.4.0

10 messages

v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

List:

I have been using v.in.ascii for some time and performance has been very
good through GRASS 6.3.0. However, with GRASS 6.4.0 importing ascii
points is painfully slow. What is causing this; can I speed-up the
process with any options? The -b option (Don't build topology points
does not seem to help.

Thanks,
Tom

--
Thomas E Adams
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

EMAIL: [hidden email]

VOICE: 937-383-0528
FAX: 937-383-0033

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Martin Landa

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

2009/11/4 Thomas Adams <[hidden email]>:
> I have been using v.in.ascii for some time and performance has been very
> good through GRASS 6.3.0. However, with GRASS 6.4.0 importing ascii points
> is painfully slow. What is causing this; can I speed-up the process with any
> options? The -b option (Don't build topology points does not seem to help.

testing dataset would be good.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Martin Landa

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

Hi,

2009/11/4 Thomas Adams <[hidden email]>:
> Thank you for your response. The columns are: longitude|latitude|value. With
> previous versions of GRASS, the import process might be ~30 seconds, at
> most. Now, 10 -to- 15 minutes. This is radar based precipitation estimates
> for 1-hr over the Ohio R valley in the U.S.

seems to be OK here (6.4.svn)

$ time v.in.ascii in=xmrg0106200804z.ascii out=test --q

real 0m19.592s
user 0m12.269s
sys 0m3.116s

$ v.info test -t | grep points
points=65000

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Thomas Adams

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

Martin,

Very interesting… Could the region settings make a difference? Here are
mine:

projection: 3 (Latitude-Longitude)
zone: 0
datum: wgs72
ellipsoid: wgs72
north: 49:56:15N
south: 24:03:45N
west: 125:01:14.988W
east: 66:28:45.0048W
nsres: 0:02:30
ewres: 0:02:29.999988
rows: 621
cols: 1405
cells: 872505

What are you using?

Tom

Martin Landa wrote:

> Hi,
>
> 2009/11/4 Thomas Adams <[hidden email]>:
>
>> Thank you for your response. The columns are: longitude|latitude|value. With
>> previous versions of GRASS, the import process might be ~30 seconds, at
>> most. Now, 10 -to- 15 minutes. This is radar based precipitation estimates
>> for 1-hr over the Ohio R valley in the U.S.
>>
>
> seems to be OK here (6.4.svn)
>
> $ time v.in.ascii in=xmrg0106200804z.ascii out=test --q
>
> real 0m19.592s
> user 0m12.269s
> sys 0m3.116s
>
> $ v.info test -t | grep points
> points=65000
>
> Martin
>
>

--
Thomas E Adams
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

EMAIL: [hidden email]

VOICE: 937-383-0528
FAX: 937-383-0033

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Martin Landa

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

2009/11/4 Thomas Adams <[hidden email]>:
> Very interesting… Could the region settings make a difference? Here are
> mine:

AFAIK, the region settings doesn't make a difference.

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

hamish-2

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

In reply to this post by Thomas Adams

Thomas Adams wrote:
> I have been using v.in.ascii for some time and performance
> has been very good through GRASS 6.3.0. However, with GRASS
> 6.4.0 importing ascii points is painfully slow. What is
> causing this; can I speed-up the process with any options?
> The -b option (Don't build topology points does not seem to
> help.

how many points in the file? (wc -l filename.txt)

what database backend?

Hamish

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Martin Landa

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

2009/11/4 Hamish <[hidden email]>:
> how many points in the file? (wc -l filename.txt)

65k

> what database backend?

in my case dbf (as default for 6.x).

Martin

--
Martin Landa <landa.martin gmail.com> * http://gama.fsv.cvut.cz/~landa
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

Thomas Adams

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

In reply to this post by hamish-2

Hamish,

Here are the answers to your questions:

points=65000
database backend is dbf

Thanks,
Tom

Hamish wrote:

> Thomas Adams wrote:
>
>> I have been using v.in.ascii for some time and performance
>> has been very good through GRASS 6.3.0. However, with GRASS
>> 6.4.0 importing ascii points is painfully slow. What is
>> causing this; can I speed-up the process with any options?
>> The -b option (Don't build topology points does not seem to
>> help.
>>
>
> how many points in the file? (wc -l filename.txt)
>
> what database backend?
>
>
> Hamish
>
>
>
>
>

Thomas Adams

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

In reply to this post by Martin Landa

Martin & Hamish,

I believe I know what the problem is (to a degree). I am running GRASS
in a network environment where GRASS is installed on one machine, my
user files are on a 2nd, my GRASS database is on a 3rd, and I am logged
into a 4th (all on Redhat Linux). So, reading my file over the -network-
must be causing the big slowdown. One week ago we installed a new
version of RedHat Linux on all of our networked machines (of which there
are a combination of 10 workstations and ~10 servers); something with
this new installation did something to slowdown what I was doing
previously. I can ssh into an independent machine where everything is
local and my import times are comparable to what Martin got:

$ time v.in.ascii in=xmrg0106200804z.ascii out=test --q

real 0m19.592s
user 0m12.269s
sys 0m3.116s

$ v.info test -t | grep points
points=65000

This is a problem for me because I was dependent on using v.in.ascii for
automatically processing some data and analyzing it in an operational
setting. Looks like I have to find a new combination of machines to use
within my environment.

Thanks for your help!

Tom

Martin Landa wrote:

> 2009/11/4 Hamish <[hidden email]>:
>
>> how many points in the file? (wc -l filename.txt)
>>
>
> 65k
>
>
>> what database backend?
>>
>
> in my case dbf (as default for 6.x).
>
> Martin
>
>

Markus Neteler

Re: v.in.ascii slowness in GRASS 6.4.0

Reply Threaded

On Thu, Nov 5, 2009 at 1:33 AM, Thomas Adams <[hidden email]> wrote:

> Martin & Hamish,
>
> I believe I know what the problem is (to a degree). I am running GRASS in a
> network environment where GRASS is installed on one machine, my user files
> are on a 2nd, my GRASS database is on a 3rd, and I am logged into a 4th (all
> on Redhat Linux). So, reading my file over the -network- must be causing the
> big slowdown. One week ago we installed a new version of RedHat Linux on all
> of our networked machines (of which there are a combination of 10
> workstations and ~10 servers); something with this new installation did
> something to slowdown what I was doing previously.

<OT>
I guess that you use NFS. There are a few options to tune NFS, perhaps
during update your optimized settings got lost?

Some random links:
* http://www.linuxconfig.org/HowTo_configure_NFS
* http://blogs.techrepublic.com.com/opensource/?p=64
* http://atmail.com/kb/category/multiserver/
</OT>

cheers
Markus
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user