[GRASS GIS] #520: r.terraflow/iostream does not respect STREAM_DIR option

23 messages Options
Embed this post
Permalink
1 2
GRASS GIS

Re: [GRASS GIS] #520: r.terraflow/iostream does not respect STREAM_DIR option

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#520: r.terraflow/iostream does not respect STREAM_DIR option
---------------------+------------------------------------------------------
  Reporter:  mmetz   |       Owner:  [hidden email]
      Type:  defect  |      Status:  new                      
  Priority:  major   |   Milestone:  6.4.0                    
 Component:  Raster  |     Version:  svn-develbranch6        
Resolution:          |    Keywords:  r.terraflow, tmp        
  Platform:  All     |         Cpu:  All                      
---------------------+------------------------------------------------------
Comment (by mmetz):

 Replying to [comment:19 hamish]:
 > Replying to [comment:18 mmetz]:
 >
 > > The default answer of /var/tmp or another system TEMP folder
 > > is IMHO not a good idea because of lack of space. Why not
 > > leaving it blank, if no answer is given, use the GRASS temp
 > > directory? That directory should always exist and I guess there
 > > is most of the time enough space.
 >
 > this has been discussed before, check the archives. ISTR that the
 Terraflow team (Andrew? Laura?) explained why they used that a long time
 ago (I now forget exact reason),

 Picking a directory whose contents are deleted automatically if I remember
 right, which would be the case for $MAPSET/.tmp/$MACHINE.

 Whatever directory is used as default in r.terraflow, it should at least
 exist on all supported platforms.

 > Note that these temp files may be >2gig

 I get regularly >20GB, having to work on somewhat larger datasets.
 Thinking about Doug's datasets, it would go way above 100GB.

 > (use the memory= option to lower those needs), and they may be
 potentially useful(?) for inspection/archive/debugging/reuse later, so
 keeping them completely hidden may not be desired.

 These files are deleted upon successful completion of the module using the
 iostream lib, nothing left to inspect. They are probably only useful for
 the iostream lib developers for debugging. Same for the segment lib,
 intermediate files are probably only useful for developers working on the
 segment lib. I don't see a reason why files created by the iostream
 library should be treated differently to files created by the segment
 library, or vice versa, when it comes to their storage location.

 This discussion is now moving away from r.terraflow towards where to store
 large temporary files, maybe this ticket can be closed and the discussion
 be continued somewhere else if needed?

 Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/520#comment:20>
GRASS GIS <http://grass.osgeo.org>

_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
GRASS GIS

Re: [GRASS GIS] #520: r.terraflow/iostream does not respect STREAM_DIR option

Reply Threaded More More options
Print post
Permalink
In reply to this post by GRASS GIS
#520: r.terraflow/iostream does not respect STREAM_DIR option
---------------------+------------------------------------------------------
  Reporter:  mmetz   |       Owner:  [hidden email]
      Type:  defect  |      Status:  new                      
  Priority:  major   |   Milestone:  6.4.0                    
 Component:  Raster  |     Version:  svn-develbranch6        
Resolution:          |    Keywords:  r.terraflow, tmp        
  Platform:  All     |         Cpu:  All                      
---------------------+------------------------------------------------------
Comment (by neteler):

 Replying to [comment:20 mmetz]:
 > Replying to [comment:19 hamish]:
 > > Replying to [comment:18 mmetz]:
 > > ISTR that the Terraflow team (Andrew? Laura?) explained why they used
 that a long time ago (I now forget exact reason),

 Most likely because r.terraflow wasn't originally written for GRASS/was
 written also as stand-alone version.

 > Picking a directory whose contents are deleted automatically if I
 remember right, which would be the case for $MAPSET/.tmp/$MACHINE.

 Perfect. I assume that 99% of the users will never inspect any tmp file of
 r.terraflow. So that is a perfect default. If a specialist wants to
 inspect, then define a different directory +in that case*.

 > > Note that these temp files may be >2gig
 >
 > I get regularly >20GB, having to work on somewhat larger datasets.
 Thinking about Doug's datasets, it would go way above 100GB.

 Yes, and I always forget to look in /var/tmp/ for undeleted rubbish (but
 check instead my HOME for delete candidates). Having auto-deletion of tmp
 stuff left by interrupted/crashed r.terraflow runs when leaving GRASS
 would be a major advantage for most users.

 > > (use the memory= option to lower those needs), and they may be
 potentially useful(?) for inspection/archive/debugging/reuse later, so
 keeping them completely hidden may not be desired.

 As said, who wants to analyse these files is also able to change a default
 setting.

 > These files are deleted upon successful completion of the module using
 the iostream lib, nothing left to inspect. They are probably only useful
 for the iostream lib developers for debugging. Same for the segment lib,
 intermediate files are probably only useful for developers working on the
 segment lib. I don't see a reason why files created by the iostream
 library should be treated differently to files created by the segment
 library, or vice versa, when it comes to their storage location.

 +1

 > This discussion is now moving away from r.terraflow towards where to
 store large temporary files, maybe this ticket can be closed and the
 discussion be continued somewhere else if needed?

 No, still /var/tmp/ is the default also in GRASS 7. This should be
 changed.

 MarkusN

 > Markus M

--
Ticket URL: <http://trac.osgeo.org/grass/ticket/520#comment:21>
GRASS GIS <http://grass.osgeo.org>

_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
Helena Mitasova

Re: Re: [GRASS GIS] #520: r.terraflow/iostream does not respect STREAM_DIR option

Reply Threaded More More options
Print post
Permalink

>
>> These files are deleted upon successful completion of the module  
>> using
>  the iostream lib, nothing left to inspect. They are probably only  
> useful
>  for the iostream lib developers for debugging. Same for the  
> segment lib,
>  intermediate files are probably only useful for developers working  
> on the
>  segment lib. I don't see a reason why files created by the iostream
>  library should be treated differently to files created by the segment
>  library, or vice versa, when it comes to their storage location.
>
>  +1
>
>> This discussion is now moving away from r.terraflow towards where to
>  store large temporary files, maybe this ticket can be closed and the
>  discussion be continued somewhere else if needed?
>
>  No, still /var/tmp/ is the default also in GRASS 7. This should be
>  changed.

yes, please, this needs to be changed, it has been a long standing issue

Helena

>
>  MarkusN
>
>> Markus M
>
> --
> Ticket URL: <http://trac.osgeo.org/grass/ticket/520#comment:21>
> GRASS GIS <http://grass.osgeo.org>
> _______________________________________________
> grass-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/grass-dev

_______________________________________________
grass-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-dev
1 2