[GRASS GIS] #7: Location wizard: should predefine DB connection for new location

25 messages Options
Embed this post
Permalink
1 2
GRASS GIS

[GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
#7: Location wizard: should predefine DB connection for new location
---------------------+------------------------------------------------------
 Reporter:  neteler  |       Owner:  [hidden email]
     Type:  defect   |      Status:  new                      
 Priority:  major    |   Milestone:  6.3.0                    
Component:  Python   |     Version:  unspecified              
 Keywords:           |  
---------------------+------------------------------------------------------
 The Location wizard does not predefine the DB connection for new location
 which leads to problems with several modules (needs to generate MAPSET/VAR
 file). Probably we should predefine SQLite here.

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7>
GRASS GIS <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  Python   |     Version:  unspecified              
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by cmbarton):

 Replying to [ticket:7 neteler]:
 > The Location wizard does not predefine the DB connection for new
 location which leads to problems with several modules (needs to generate
 MAPSET/VAR file). Probably we should predefine SQLite here.

 I agree. However, except for xy locations, all are created using g.proj.
 So this is an issue for g.proj. That is, the location wizard does not
 operate 'inside' a location where it could use GRASS commands to do this.

 Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:1>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  Python   |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Changes (by neteler):

 * cc: pkelly, cmbarton (added)
  * version:  unspecified => svn-trunk

Comment:

 Paul,

 could you please take a look? It affects g.proj which should call
 functionality from Vect_default_field_info() to define the VAR
 file for the DBMI connection in a newly created location
 (see lib/vector/Vlib/field.c). I guess that implementation in
 g.proj takes 10 minutes max... :)

 Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:2>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.4.0                    
 Component:  Python   |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Changes (by neteler):

  * milestone:  6.3.0 => 6.4.0

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:3>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.4.0                    
 Component:  Python   |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by hamish):

 As mentioned before, I do not think that the VAR file should be a part of
 the default MAPSET folder specification.

 see  http://thread.gmane.org/gmane.comp.gis.grass.devel/22676

 e.g. for users who mainly work with rasters this is unneeded filesystem
 noise.

 C modules which require a DB to be set should be exiting with an error if
 none is found (suggesting to run db.connect). Scripts like v.db.add* can
 run that automatically if needed. Any other module bugs can be fixed,
 exiting with an error is not much work if that doesn't happen already,
 just check a return code.

 Since I commented out the auto-set to DBF in 6.3cvs's init.sh 3 months ago
 there have been no complaints about the change. This is not done in 6.2
 either, no complaints there.


 Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:4>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  default  |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Changes (by neteler):

  * component:  Python => default
  * milestone:  6.4.0 => 6.3.0

Comment:

 I disagree and think that new locations should be valid locations - or -
 *all* modules should be able to work with locations lacking the VAR file.
 I see no point in having this trivial thing lacking and then make the user
 stumble about several commands not functioning.

 The "file system noise" is in average 70 bytes (!) per MAPSET which can be
 just ignored as a problem.

 If we have a default DBMI driver (which is yet DBF), we also have to set
 default settings. No complaints in 6.2 probably because use cases are
 different.

 Scripts like v.db.add* cannot run that automatically because we don't want
 to hardcode DBF (or whatever driver) many times scattered all over the
 source code but only in a single place (which was the init script and I
 still complain that this was removed leading to broken DBMI settings - see
 the archive for this).

 Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:5>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  default  |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by cmbarton):

 Replying to [comment:5 neteler]:
 > I disagree and think that new locations should be valid locations - or -
 *all* modules should be able to work with locations lacking the VAR file.
 I see no point in having this trivial thing lacking and then make the user
 stumble about several commands not functioning.
 >
 > The "file system noise" is in average 70 bytes (!) per MAPSET which can
 be just ignored as a problem.
 >
 > If we have a default DBMI driver (which is yet DBF), we also have to set
 default settings. No complaints in 6.2 probably because use cases are
 different.
 >
 > Scripts like v.db.add* cannot run that automatically because we don't
 want to hardcode DBF (or whatever driver) many times scattered all over
 the source code but only in a single place (which was the init script and
 I still complain that this was removed leading to broken DBMI settings -
 see the archive for this).
 >
 > Markus

 I agree with Markus. All locations should be equal. This extra file is
 irrelevant to those working only with raster, essentially invisible, and
 takes up no space. The alternative is fine also (i.e., the lack of VAR
 causes no problem with any module; it is simply created when needed). But
 we should not be creating locations that have hidden gotchas.

 Michael

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:6>
GRASS-trac <http://grass.osgeo.org>
Commonly referred to as GRASS, this is a Geographic Information System (GIS) used for geospatial data management and analysis, image processing, graphics/maps production, spatial modeling, and visualization. GRASS is currently used in academic and commercial settings around the world, as well as by many governmental agencies and environmental consulting companies.
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  default  |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by hamish):

 re. create VAR file for each mapset if it doesn't exist
 (see also default DB -> SQLite)


 Markus:
 > I disagree and think that new locations should be valid locations -
 ..

 Michael:
 > But we should not be creating locations that have hidden gotchas.

 In that case, wouldn't all locations / datasets created with earlier
 versions of grass be considered invalid?
 (if so we need to re-add auto-creation of VAR in init.sh)


 Markus:
 > or - *all* modules should be able to work with locations lacking
 > the VAR file.

 This is the solution I would like to pursue. Can anyone give examples of
 which modules break? AFAIK they have all been fixed (v.db.add*).
 Do any remain?



 > No complaints in 6.2 probably because use cases are different.

 How so? I don't understand what changed. Changes to/new v.db.add*??


 For fixing C modules it should be as hard as testing a return value and
 calling G_fatal_error() with "DB unset. Please run db.select".
 For scripts it should be as simple as either calling db.select in the
 script if unset or just letting the C module's G_fatal_error() spit out
 the appropriate error.



 > I see no point in having this trivial thing lacking and then make
 > the user stumble about several commands not functioning.
 ..
 > If we have a default DBMI driver (which is yet DBF), we also have
 > to set default settings.
 ..
 > Scripts like v.db.add* cannot run that automatically because we
 > don't want to hardcode DBF (or whatever driver) many times
 > scattered all over the source code but only in a single place

 I agree it should be set in a single place and it is annoying to go
 through the step manually.

 v.in.ascii (.c) creates dbf/ and sets it as default if it doesn't exist:
 [new mapset]
 G6.3svn> echo "1 2 foo bar" | v.in.ascii out=foomap fs=space x=1 y=2
 Scanning input for column types...
 Maximum input row length: 12
 Maximum number of columns: 4
 Minimum number of columns: 4
 WARNING: Default driver / database set to:
          driver: dbf
          database: $GISDBASE/$LOCATION_NAME/$MAPSET/dbf/
 Importing points...
 ...


 this happens in.c (aka v.in.ascii's main.c),
  Fi = Vect_default_field_info(&Map, 1, NULL, GV_1TABLE);

 which is from lib/vector/Vlib/field.c  Vect_default_field_info()
  (hardcoded to DBF)

 so any C module which creates a new table should use/check that?
 AFAICT that seems to be the case for the C modules.
 grass63/vector$ grep -rI Vect_default_field_info * | \
   grep -v \.svn | cut -f1 -d/


 If db.select + default values is not sufficient for a single place there
 could be a maintenance script which tests and sets the default DB when
 needed. ? v.db.add* scripts could call that. I am happy to author that if
 it is decided to go that way.



 > (which was the init script and I still complain that this was
 > removed leading to broken DBMI settings - see the archive for
 > this).

 - I hadn't understood the reason for putting it in init.sh was that was
 supposed to be the central default DB set point. If that is to be the case
 it's better to put that in a support script than in init.sh

 - Initially removed due to nasty scripting bug which overwrote VAR
 unconditionally, so if anyone decides to revert the removal, please
 take care not to reintroduce that.


 > The "file system noise" is in average 70 bytes (!) per MAPSET
 > which can be just ignored as a problem.

 I don't know if the joke translates well from English, but there is an old
 saying "cleanliness is next to impossible." But still a worthy goal.


 Vect_default_field_info() limits it to scripts which don't create tables
 without using other C modules, and we can code a support script module to
 handle those cases with only 1 line a module.


 Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:7>
GRASS-trac <http://grass.osgeo.org>
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Moritz Lennert

Re: Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
On 21/01/08 04:47, GRASS-trac wrote:
> #7: Location wizard: should predefine DB connection for new location
> ----------------------+-----------------------------------------------------
>   Reporter:  neteler  |       Owner:  [hidden email]
>       Type:  defect   |      Status:  new                      
>   Priority:  major    |   Milestone:  6.3.0                    
>  Component:  default  |     Version:  svn-trunk                
> Resolution:           |    Keywords:                          
> ----------------------+-----------------------------------------------------
> Comment (by hamish):

>  For fixing C modules it should be as hard as testing a return value and
>  calling G_fatal_error() with "DB unset. Please run db.select".
>  For scripts it should be as simple as either calling db.select in the
>  script if unset or just letting the C module's G_fatal_error() spit out
>  the appropriate error.

I imagine you mean db.connect instead of db.select...

Moritz
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS-trac] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  default  |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by neteler):

 Comment (by hamish):
 >  In that case, wouldn't all locations / datasets created with earlier
 >  versions of grass be considered invalid?
 >  (if so we need to re-add auto-creation of VAR in init.sh)

 No, because it was there unless you took it out...

 >  Markus:
 >  > or - *all* modules should be able to work with locations lacking
 >  > the VAR file.
 >
 >  This is the solution I would like to pursue. Can anyone give examples
 of
 >  which modules break? AFAIK they have all been fixed (v.db.add*).
 >  Do any remain?

 AFAIK all db.* modules and scripts at least. Also I suspect the v.*
 scripts
 to face a problem.


 ...
 >  v.in.ascii (.c) creates dbf/ and sets it as default if it doesn't
 exist:

 g.proj does not for example.
 Likewise possibly v.in.ogr and r.in.gdal (and other modules which
 generate locations)


 >  so any C module which creates a new table should use/check that?
 >  AFAICT that seems to be the case for the C modules.
 >  grass63/vector$ grep -rI Vect_default_field_info * | \
 >    grep -v \.svn | cut -f1 -d/

 Please also check all pure db.* modules.


 ...
 >  > The "file system noise" is in average 70 bytes (!) per MAPSET
 >  > which can be just ignored as a problem.
 >
 >  I don't know if the joke translates well from English, but there is an
 old
 >  saying "cleanliness is next to impossible." But still a worthy goal.

 I think we have quite some more severe other problems... think about all
 the todo's concerning buffer sizes.

 >
 >  Vect_default_field_info() limits it to scripts which don't create
 tables
 >  without using other C modules, and we can code a support script module
 to
 >  handle those cases with only 1 line a module.

 Still someone has to check all modules/scripts...
 And as mentioned: the db.* modules do not call Vect_*().

 Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:8>
GRASS-trac <http://grass.osgeo.org>
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  [hidden email]
      Type:  defect   |      Status:  new                      
  Priority:  major    |   Milestone:  6.3.0                    
 Component:  default  |     Version:  svn-trunk                
Resolution:           |    Keywords:                          
----------------------+-----------------------------------------------------
Comment (by neteler):

 Hi,

 any news here? With the upcoming release we'll get tons of
 requests if we don't solve this problem...

 I still don't understand why Hamish insists to do it the
 complicated way.

 Markus

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:9>
GRASS GIS <http://grass.osgeo.org>
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Paul Kelly

Re: Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
Sorry, just noticed there was a call to action by me in that bug report
which I missed. Anyway, I think the key issue here is whether or not the
database connection file is considered an essential part of a valid GRASS
location.
1. If it *is*, then creation of it should be added to the places that
crate locations: lib/init/mke_mapset.c and lib/gis/make_mapset.c.
2. If it is *not*, then the vector/db library functions that need to
access it should be updated to either gracefully fail or create default
settings on finding it absent, and this behaviour should be made
consistent in all places.

Whatever way it is, I think adding a check to it in Init.sh is an ugly
hack that just works around the problem rather than fixing it.

BUT: It seems option 1 is already partially implemented!
lib/init/mke_mapset.c (called if you create a new location using
g.setproj, or the text-based startup) does indeed create the VAR file. So
it looks to me that the anomaly is that this has been left out of
lib/gis/make_mapset.c (called if you create a new location using g.proj,
r.in.gdal or v.in.ogr or the GUI startup).

If we add creation of the VAR file to lib/gis/make_mapset.c, ideally we
should also do an audit of the vector modules and remove the functionality
that creates a default VAR file if not present, as it will no longer be
needed. What do you think of this solution, Hamish?

Paul
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Michael Barton

Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS

On Feb 18, 2008, at 4:00 AM, GRASS GIS wrote:

> #7: Location wizard: should predefine DB connection for new location
> ----------------------
> +-----------------------------------------------------
>   Reporter:  neteler  |       Owner:  [hidden email]
>       Type:  defect   |      Status:  new
>   Priority:  major    |   Milestone:  6.3.0
>  Component:  default  |     Version:  svn-trunk
> Resolution:           |    Keywords:
> ----------------------
> +-----------------------------------------------------
> Comment (by neteler):
>
>  Hi,
>
>  any news here? With the upcoming release we'll get tons of
>  requests if we don't solve this problem...
>
>  I still don't understand why Hamish insists to do it the
>  complicated way.
>
>  Markus
>
> --
> Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:9>
> GRASS GIS <http://grass.osgeo.org>
> GRASS Geographic Information System (GRASS GIS) - http://
> grass.osgeo.org/

Markus,

This really needs to be done in g.proj, although I could hack it into  
the location wizard if needed. From what I was following of the  
discussion, there were some *philosophical* reasons for not doing it,  
but the cost for making a var file is essentially nil and the cost of  
not having one potentially high.

Michael
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  hamish  
      Type:  defect   |      Status:  assigned
  Priority:  major    |   Milestone:  6.4.0    
 Component:  default  |     Version:  svn-trunk
Resolution:           |    Keywords:          
----------------------+-----------------------------------------------------
Changes (by hamish):

  * status:  new => assigned
  * owner:  [hidden email] => hamish
 * cc: [hidden email] (added)
  * milestone:  6.3.0 => 6.4.0

Comment:

 for 6.3.0 I have re-enabled automatic VAR creation if it doesn't exist.


 more soon.....

 Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:10>
GRASS GIS <http://grass.osgeo.org>
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Paul Kelly

Re: Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
On Thu, 13 Mar 2008, GRASS GIS wrote:

> for 6.3.0 I have re-enabled automatic VAR creation if it doesn't exist.
>
>
> more soon.....

Attached is a patch (completely untested) that illustrates my idea for
solving the problem. I think there should also be something in
lib/gis/make_mapset.c (used by g.mapsets) but there are some complications
there in relation to inheriting the DB settings; see my earlier mail.

Paul
[make_loc.diff]

Index: lib/gis/make_loc.c
===================================================================
--- lib/gis/make_loc.c (revision 29515)
+++ lib/gis/make_loc.c (working copy)
@@ -48,7 +48,7 @@
     FILE *report_file )
 
 {
-    char path[2048];
+    char path[GPATH_MAX];
     int         out_stat;
 
     /* Try to create the location directory, under the gisdbase. */
@@ -69,6 +69,19 @@
     G__put_window( wind, "", "DEFAULT_WIND" );
     G__put_window( wind, "", "WIND" );
 
+    /* Create database settings file */
+    {
+        struct Key_Value *dbsettings = G_create_key_value();
+      
+        G_set_key_value("DB_DRIVER", "dbf", dbsettings);
+        G_set_key_value("DB_DATABASE", "$GISDBASE/$LOCATION_NAME/$MAPSET/dbf/", dbsettings);
+        G__file_name( path, "", "VAR", "PERMANENT" );
+        G_write_key_value_file( path, proj_info, &out_stat );
+        if( out_stat != 0 )
+            return -1;
+        G_free_key_value(dbsettings);
+    }
+  
     /* Write out the PROJ_INFO, and PROJ_UNITS if available. */
     if( proj_info != NULL )
     {


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#7: Location wizard: should predefine DB connection for new location
----------------------+-----------------------------------------------------
  Reporter:  neteler  |       Owner:  hamish  
      Type:  defect   |      Status:  assigned
  Priority:  major    |   Milestone:  6.4.0    
 Component:  default  |     Version:  svn-trunk
Resolution:           |    Keywords:          
----------------------+-----------------------------------------------------
Comment (by hamish):

 Eric wrote:
 > I created a new location using today's svn source, and noticed that
 > db.connect doesn't have its database parameter pre-populated with
 > $GISDBASE/$LOCATION_NAME/$MAPSET/dbf as it used to; it is now blank.
 > Possible bug? Would it possible to get the module to at least populate
 > $GISDBASE/$LOCATION_NAME/$MAPSET for this parameter?

 db.connect takes its default driver, database, schema, and group settings
 from the $MAPSET/VAR file by way of the db_get_default_*_name() fns.
 (lib/db/dbmi_base/default_name.c)  This is a circular chicken-or-egg
 situation. (the default is to set it back to what it already is, which is
 a redundant task)


 Martin:
 > yes, see
 > http://trac.osgeo.org/grass/ticket/7
 >
 > It seems to me that the VAR file should defined for newly created
 > mapset by default to avoid possible problems.

 that approach deals with the symptoms, but doesn't address the underlying
 problem: what to do when the default DB settings are unknown. To be
 robust it needs to be dealt with at module run-time and it needs to fail
 in a safe way if the VAR file is somehow damaged.


 A secondary issue is abstracting the default DB settings back to one place
 so that when we make the switch to sqlite-as-default, we don't have to
 change ad hoc VAR file & dbf/ directory creation code in a dozen places.

 - For C code this was already condensed into Vect_default_field_info() but
 could be abstracted further*,

 - For shell scripts we can add a new flag to db.connect* to check if the
 DB is set, and if not then set it. This would then be called from all
 scripts which create vector tables outside of v.* modules, and could be
 used in init.sh too. (although if we have done everything correctly then
 we won't need to put it in init.sh or any other mapset creation code;
 thus I suggest we keep it out of init.sh)

 [*] now in SVN/trunk as DB_DEFAULT_DRIVER, db_set_default_connection(),
 and 'db.connect -c'. (r30545) Init.sh and various scripts are now updated
 to use that.


 Eric:
 > Thanks for the pointer. I'll hardcode my database paths for now.

 see the db.connect help page example for DBF (I usually just cut and
 paste that to the command line when needed). It accepts variable
 LOCATION, MAPSET, you don't need to hardcode it. (Be sure to keep the
 path with $VARIABLES in single quotes!)


 Martin:
 > anyway if you run some command which calls Vect_default_field_info()
 > from Vlib, VAR file is created and default driver set up to dbf.
 >
 > e.g. g.copy vect=

 i.e. any C vector module which goes down that street should do the trick.
 The vector lib will set the default database if it goes to find it and
 finds it unset.


 Paul:
 > Also I posted a suggested solution to the dev list a while ago, but
 > there were no comments.

 could you post a link to that in the bug report?


 > > It seems to me that the VAR file should defined for newly created
 > > mapset by default to avoid possible problems.
 >
 > For every mapset, or just for each location? If for every mapset, should
 > it be inherited from whatever the setting for PERMANENT is, or just
 > blindly set to dbf every time a new mapset is created? If it does have
 > to be created for every mapset and inherited from PERMANENT, presumably
 > the state of the DB connection for PERMANENT can't be determined
 > without using Vectlib functions? But all the mapset/location-handling
 > functions are in GISlib, and it would really not be a good idea to have
 > GIS dependent on Vectlib functions (I think). So this is really quite
 > complicated. Perhaps the functions that deal with the default database
 > settings could be moved out of Vectlib and into GISlib.

 whoa, keep it simple. if it is requested and not set, then set it. as
 long as C vector modules are funneled through a single vector lib
 function which checks & sets, and scripts doing low level stuff don't
 make assumptions, we should be ok.

 Michael:
 > I'm with Marcus and Martin on this. The potential for problems is high
 > if this is not done; the cost to do it is very small. Can we just go
 > ahead and make this change to g.proj?

 But it's not high. I commented out the auto VAR file creation sometime
 before Christmas, and this is the first actual-usage complaint I've heard.
 And it's not even a complaint, Eric just noticed that it wasn't set and
 was wondering why.

 [now I've made it live again in init.sh, but I still don't really think it
 should be there; at least the 'mkdir dbf/' stuff is out of init.sh now]

 AFAIK we are not (primarily) talking about g.proj, we are talking about
 Init.sh. If mapset/location creation is a problem all we have to is keep
 the 'db.connect -c' in init.sh and it will get the VAR file the first time
 the mapset is accessed.


 Paul:
 > Attached is a patch (completely untested) that illustrates my idea for
 > solving the problem.

 now obviated?

 > I think there should also be something in lib/gis/make_mapset.c (used by
 > g.mapsets) but there are some complications there in relation to
 > inheriting the DB settings; see my earlier mail.

 (*g.mapset* not *g.mapsets*)

 .... I still don't think this belongs in any mapset creation code and
 don't think a VAR file should be included in the minimum requirements for
 the definition of a valid mapset (ie 'dir/WIND').



 Hamish

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/7#comment:11>
GRASS GIS <http://grass.osgeo.org>
GRASS Geographic Information System (GRASS GIS) - http://grass.osgeo.org/
_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Eric Patton

RE: Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink



 Eric:
 >> Thanks for the pointer. I'll hardcode my database paths for now.

Hamish:
> see the db.connect help page example for DBF (I usually just cut and
> paste that to the command line when needed). It accepts variable
> LOCATION, MAPSET, you don't need to hardcode it. (Be sure to keep the
> path with $VARIABLES in single quotes!)

Should each of $GISDABSE, $LOCATION_NAME, and $MAPSET have populated values
on location startup? These variables always remain blank for each location that
I try:

~ >echo $GISDBASE

~ >echo $LOCATION_NAME

~ >echo $MAPSET

~ >

are these variables only available from the shell if I export `g.gisenv` or eval `g.gisenv`
first?

~ Eric.




 


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev

winmail.dat (4K) Download Attachment
Eric Patton

RE: Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
Eric:
 >> Thanks for the pointer. I'll hardcode my database paths for now.

Hamish:
> see the db.connect help page example for DBF (I usually just cut and
> paste that to the command line when needed). It accepts variable
> LOCATION, MAPSET, you don't need to hardcode it. (Be sure to keep the
> path with $VARIABLES in single quotes!)

Should each of $GISDABSE, $LOCATION_NAME, and $MAPSET have populated values
on location startup? These variables always remain blank for each location that
I try:

~ >echo $GISDBASE

~ >echo $LOCATION_NAME

~ >echo $MAPSET

~ >

are these variables only available from the shell if I export `g.gisenv` or eval `g.gisenv`
first?

~ Eric.




_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev

winmail.dat (4K) Download Attachment
Michael Barton

Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS

> #7: Location wizard: should predefine DB connection for new location
> ----------------------
> +-----------------------------------------------------
>   Reporter:  neteler  |       Owner:  hamish
>       Type:  defect   |      Status:  assigned
>   Priority:  major    |   Milestone:  6.4.0
>  Component:  default  |     Version:  svn-trunk
> Resolution:           |    Keywords:
> ----------------------
> +-----------------------------------------------------
> Comment (by hamish):
>
>  Eric wrote:
>> I created a new location using today's svn source, and noticed that
>> db.connect doesn't have its database parameter pre-populated with
>> $GISDBASE/$LOCATION_NAME/$MAPSET/dbf as it used to; it is now blank.
>> Possible bug? Would it possible to get the module to at least  
>> populate
>> $GISDBASE/$LOCATION_NAME/$MAPSET for this parameter?
>
>  db.connect takes its default driver, database, schema, and group  
> settings
>  from the $MAPSET/VAR file by way of the db_get_default_*_name() fns.
>  (lib/db/dbmi_base/default_name.c)  This is a circular chicken-or-egg
>  situation. (the default is to set it back to what it already is,  
> which is
>  a redundant task)
>
>
>  Martin:
>> yes, see
>> http://trac.osgeo.org/grass/ticket/7
>>
>> It seems to me that the VAR file should defined for newly created
>> mapset by default to avoid possible problems.
>
>  that approach deals with the symptoms, but doesn't address the  
> underlying
>  problem: what to do when the default DB settings are unknown. To be
>  robust it needs to be dealt with at module run-time and it needs  
> to fail
>  in a safe way if the VAR file is somehow damaged.
>
>
>  A secondary issue is abstracting the default DB settings back to  
> one place
>  so that when we make the switch to sqlite-as-default, we don't  
> have to
>  change ad hoc VAR file & dbf/ directory creation code in a dozen  
> places.
>
>  - For C code this was already condensed into  
> Vect_default_field_info() but
>  could be abstracted further*,
>
>  - For shell scripts we can add a new flag to db.connect* to check  
> if the
>  DB is set, and if not then set it. This would then be called from all
>  scripts which create vector tables outside of v.* modules, and  
> could be
>  used in init.sh too. (although if we have done everything  
> correctly then
>  we won't need to put it in init.sh or any other mapset creation code;
>  thus I suggest we keep it out of init.sh)
>
>  [*] now in SVN/trunk as DB_DEFAULT_DRIVER,  
> db_set_default_connection(),
>  and 'db.connect -c'. (r30545) Init.sh and various scripts are now  
> updated
>  to use that.
>
>
>  Eric:
>> Thanks for the pointer. I'll hardcode my database paths for now.
>
>  see the db.connect help page example for DBF (I usually just cut and
>  paste that to the command line when needed). It accepts variable
>  LOCATION, MAPSET, you don't need to hardcode it. (Be sure to keep the
>  path with $VARIABLES in single quotes!)
>
>
>  Martin:
>> anyway if you run some command which calls Vect_default_field_info()
>> from Vlib, VAR file is created and default driver set up to dbf.
>>
>> e.g. g.copy vect=
>
>  i.e. any C vector module which goes down that street should do the  
> trick.
>  The vector lib will set the default database if it goes to find it  
> and
>  finds it unset.
>
>
>  Paul:
>> Also I posted a suggested solution to the dev list a while ago, but
>> there were no comments.
>
>  could you post a link to that in the bug report?
>
>
>>> It seems to me that the VAR file should defined for newly created
>>> mapset by default to avoid possible problems.
>>
>> For every mapset, or just for each location? If for every mapset,  
>> should
>> it be inherited from whatever the setting for PERMANENT is, or just
>> blindly set to dbf every time a new mapset is created? If it does  
>> have
>> to be created for every mapset and inherited from PERMANENT,  
>> presumably
>> the state of the DB connection for PERMANENT can't be determined
>> without using Vectlib functions? But all the mapset/location-handling
>> functions are in GISlib, and it would really not be a good idea to  
>> have
>> GIS dependent on Vectlib functions (I think). So this is really quite
>> complicated. Perhaps the functions that deal with the default  
>> database
>> settings could be moved out of Vectlib and into GISlib.
>
>  whoa, keep it simple. if it is requested and not set, then set it. as
>  long as C vector modules are funneled through a single vector lib
>  function which checks & sets, and scripts doing low level stuff don't
>  make assumptions, we should be ok.
>
>  Michael:
>> I'm with Marcus and Martin on this. The potential for problems is  
>> high
>> if this is not done; the cost to do it is very small. Can we just go
>> ahead and make this change to g.proj?
>
>  But it's not high. I commented out the auto VAR file creation  
> sometime
>  before Christmas, and this is the first actual-usage complaint  
> I've heard.
>  And it's not even a complaint, Eric just noticed that it wasn't  
> set and
>  was wondering why.
>
>  [now I've made it live again in init.sh, but I still don't really  
> think it
>  should be there; at least the 'mkdir dbf/' stuff is out of init.sh  
> now]
>
>  AFAIK we are not (primarily) talking about g.proj, we are talking  
> about
>  Init.sh. If mapset/location creation is a problem all we have to  
> is keep
>  the 'db.connect -c' in init.sh and it will get the VAR file the  
> first time
>  the mapset is accessed.
>
>
>  Paul:
>> Attached is a patch (completely untested) that illustrates my idea  
>> for
>> solving the problem.
>
>  now obviated?
>
>> I think there should also be something in lib/gis/make_mapset.c  
>> (used by
>> g.mapsets) but there are some complications there in relation to
>> inheriting the DB settings; see my earlier mail.
>
>  (*g.mapset* not *g.mapsets*)
>
>  .... I still don't think this belongs in any mapset creation code and
>  don't think a VAR file should be included in the minimum  
> requirements for
>  the definition of a valid mapset (ie 'dir/WIND').
>
>
>
>  Hamish

If we want this to be easily changeable in the future, it should be  
dealt with in a central place (g.proj, init.sh [or its replacement],  
etc) so that changing it once will switch from a dbf to sqlite default.

This has come up periodically over the past year, as lack of a VAR  
file seems to cause different scattered modules to fail, and is  
usually puzzling to users as to what is going wrong.

It sounds like this can be fixed 2 ways: change g.proj to create a  
VAR file when it creates a location (this is how locations are  
created in the location wizard) or change modules to gracefully deal  
with the lack of a VAR file. The ongoing discussion about which of  
these two ways to fix it has left this unfixed for months. I'm  
agnostic about which way is best, except that I don't think that we  
should hack it into the location wizard wxPython code.

Michael

_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
hamish-2

RE: Re: [GRASS GIS] #7: Location wizard: should predefine DB connection for new location

Reply Threaded More More options
Print post
Permalink
In reply to this post by Eric Patton
> Eric:
> >> Thanks for the pointer. I'll hardcode my database paths for now.
>
> Hamish:
> > see the db.connect help page example for DBF (I usually just cut
> > and paste that to the command line when needed). It accepts
> > variable LOCATION, MAPSET, you don't need to hardcode it. (Be sure
> > to keep the path with $VARIABLES in single quotes!)

Eric:

> Should each of $GISDABSE, $LOCATION_NAME, and $MAPSET have populated
> values on location startup? These variables always remain blank for
> each location that I try:
>
> ~ >echo $GISDBASE
> ~ >echo $LOCATION_NAME
> ~ >echo $MAPSET
> ~ >
>
> are these variables only available from the shell if I export
> `g.gisenv` or eval `g.gisenv` first?

If you want to use them as variables from the shell you should use
  eval `g.gisenv`
first. Back in the GRASS 5 days they used to be set all the time but
this caused problems if you had the GUI running and then from the
terminal window you changed e.g. the mapset with g.mapset. The already
launched GUI would not and could not know about the change. So g.gisenv
was created to hold those variables.


But this has nothing (or at least little) to do with the task at hand.
db.connect will parse those variables itself, which is why you have to
enclose them in single quotes when given on the command line- to
prevent the shell from attempting to expand them (into nothing).

so just cut and paste:
 db.connect driver=dbf database='$GISDBASE/$LOCATION_NAME/$MAPSET/dbf/'

and it works.


Hamish



      ____________________________________________________________________________________
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 


_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
1 2