GRASS64 sql 'where' statement character/array limit?

2 messages Options
Embed this post
Permalink
Stephen.King

GRASS64 sql 'where' statement character/array limit?

Reply Threaded More More options
Print post
Permalink
Greetings all,

I am a new listserver member with a grass64 problem.  I am hoping somebody can help me out:

I have a batch script that uses ps.map plot to select and plot areas from a vector coverage.  The script uses the sql 'where' statement similar to this example:

SQL_STRING="ID=xxxx1 OR ID=xxxx2 OR ID=xxxx3"

ps.map output << EOF
vareas
 where $SQL_STRING
 <plot commands go here>
end
EOF

The SQL_STRING is read in from a file, and can contain more than 100 unique IDs.

This works great, BUT there appears to be a limit to the number of ID's (or characters) that are allowed in the string.  The command fails when I have more than 64 IDs in the statement (just under 1000 characters).  Later in the script, I use a similar 'where' statement in a v.extract command and have the same problem, so its not limited to ps.map.

My question: are there known length limits to the SQL select statement in GRASS?  Or perhaps I some other problem?

Thanks for any help you can provide!  Let me know if you need more info.

Steve
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user
hamish-2

Re: GRASS64 sql 'where' statement character/array limit?

Reply Threaded More More options
Print post
Permalink
Stephen.King wrote:

> I am a new listserver member with a grass64 problem. 
> I am hoping somebody can help me out:
>
> I have a batch script that uses ps.map plot to select and
> plot areas from a vector coverage.  The script uses the
> sql 'where' statement similar to this example:
>
> SQL_STRING="ID=xxxx1 OR ID=xxxx2 OR ID=xxxx3"
>
> ps.map output << EOF
> vareas
>  where $SQL_STRING
>  <plot commands go here>
> end
> EOF
>
> The SQL_STRING is read in from a file, and can contain more
> than 100 unique IDs.
>
> This works great, BUT there appears to be a limit to the
> number of ID's (or characters) that are allowed in the
> string.  The command fails when I have more than 64 IDs
> in the statement (just under 1000 characters).  Later
> in the script, I use a similar 'where' statement in a
> v.extract command and have the same problem, so its not
> limited to ps.map.
>
> My question: are there known length limits to the SQL
> select statement in GRASS?  Or perhaps I some other
> problem?


12 chars per ID * 100 IDs = 1200 chars

Both methods seem to be arbitrarily limited to 1023 characters
in the source code.

For ps.map as part of buf[] and G_getl2() and gobble_input() in
input.c, and for v.extract by lib/db/dbmi_client/select.c's
db_select_int() with buf[] and G_snprintf().

You could try replacing those 1024s with 4096s in the source
code and recompile..

maybe there are other places too..?


Maybe easier: Can you recreate the v.extract step with
db.execute? (with IDs read from a file)

or, are IDs the key column? (change with v.reclass)
If so v.extract list=1,2,3,4,...,n and ps.map/vareas
"cats 1,2,3,4,...,n" might help.

or, break it into two (repeat) steps, with 50 IDs per step.


Hamish




_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user