speed of GRASS/R interface

[Carlos Grohmann-2]

I've been thinking about the time it takes to import a raster from GRASS to R. I have some rasters with about 2000 rows and 2500 columns. importing starts fine, creates the header, then goes to 100%, and it hangs there for about 10 minutes, before get ready for the next command.
Any way of decrease this waiting time?

cheers

Carlos

--
+-----------------------------------------------------------+
              Carlos Henrique Grohmann - Guano  
  Visiting Researcher at Kingston University London - UK
  Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
Linux User #89721  - carlos dot grohmann at gmail dot com
+-----------------------------------------------------------+
_________________
"Good morning, doctors. I have taken the liberty of removing Windows 95 from my hard drive."
--The winning entry in a "What were HAL's first words" contest judged by 2001: A SPACE ODYSSEY creator Arthur C. Clarke

Can't stop the signal.
_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Carlos Grohmann-2]

Thanks for the quick reply, Augustin.

I am doing a multi-scale study of surface roughness. So I have 180 rasters, created using 3 different methods, 5 different resolutions, and with 12 moving-window sizes. In R, I am importing the data to produce histograms, density plots and so on. More as a summary statistics tool than really analysis. In R I can plot density curves for various maps in one graphic, and see what is the behaviour of the morphometric variable when resolution or moving-window size changes.


Carlos

On 10/2/07, Agustin Lobo <[hidden email]> wrote:

It makes sense, R puts everything in memory.
In general, unless you have small raster layers,
the best is to use the gis for the processing, get
intermediate data out of the raster and process these data in R.
Then back to gis for displaying if required.
In other words, no way of using R instead of r.mapcalc
unless the raster is very small.

If you can let us know what you want to do with the raster
we can try to be more specific on how a combined use of grass/r could
solve the problem (hopefully).

Agus

Carlos "Guâno" Grohmann escribió:

> I've been thinking about the time it takes to import a raster from GRASS
> to R. I have some rasters with about 2000 rows and 2500 columns.
> importing starts fine, creates the header, then goes to 100%, and it
> hangs there for about 10 minutes, before get ready for the next command.
> Any way of decrease this waiting time?
>
> cheers
>
> Carlos
>
> --
> +-----------------------------------------------------------+
>               Carlos Henrique Grohmann - Guano
>   Visiting Researcher at Kingston University London - UK
>   Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
> Linux User #89721  - carlos dot grohmann at gmail dot com
> +-----------------------------------------------------------+
> _________________
> "Good morning, doctors. I have taken the liberty of removing Windows 95
> from my hard drive."
> --The winning entry in a "What were HAL's first words" contest judged by
> 2001: A SPACE ODYSSEY creator Arthur C. Clarke
>
> Can't stop the signal.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>

--
Dr. Agustin Lobo
Institut de Ciencies de la Terra "Jaume Almera" (CSIC)
LLuis Sole Sabaris s/n
08028 Barcelona
Spain
Tel. [phone]
Fax. [phone]
email: [hidden email]
http://www.ija.csic.es/gt/obster

--
+-----------------------------------------------------------+
              Carlos Henrique Grohmann - Guano  
  Visiting Researcher at Kingston University London - UK
  Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
Linux User #89721  - carlos dot grohmann at gmail dot com
+-----------------------------------------------------------+
_________________
"Good morning, doctors. I have taken the liberty of removing Windows 95 from my hard drive."
--The winning entry in a "What were HAL's first words" contest judged by 2001: A SPACE ODYSSEY creator Arthur C. Clarke

Can't stop the signal.
_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Agustin Lobo-2]

Carlos,

Just as an example, the attached R script which uses
the following grass shell script stats_aux.scr:

r.statistics base=$1 cover=$2 method=average output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.m1
r.statistics base=$1 cover=$2 method=variance output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.v1
r.statistics base=$1 cover=$2 method=sum output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.s1
#
r.statistics base=$1 cover=$3 method=average output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.m2
r.statistics base=$1 cover=$3 method=variance output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.v2
#
r.statistics base=$1 cover=$4 method=average output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.m3
r.statistics base=$1 cover=$4 method=variance output=delme --overwrite
cp /home/alobo/PERU/seg1/2001u/cats/delme delme.v3
#
g.remove rast=delme


The script calculates a table of stratified
statistics (one reference raster as strata and 3 rasters for the
data). I.e., You can calculate the average, sd etc of 3 bands for each
class on a given landcover map. There are probably other
(better) ways of doing this, probably using r.stats instead of
r.statistics, but this could be a guide. The idea is
use r.stats and get that input into R instead of the
actual rasters. For example
r.stats -c in=miraster out=delme.txt

and then get delme.txt into R with read.table().For stratified stats:

r.stats -c in=reference,band out=delme.txt



Agus



Carlos "Guâno" Grohmann escribió:

> Thanks for the quick reply, Augustin.
>
> I am doing a multi-scale study of surface roughness. So I have 180
> rasters, created using 3 different methods, 5 different resolutions, and
> with 12 moving-window sizes. In R, I am importing the data to produce
> histograms, density plots and so on. More as a summary statistics tool
> than really analysis. In R I can plot density curves for various maps in
> one graphic, and see what is the behaviour of the morphometric variable
> when resolution or moving-window size changes.
>
>
> Carlos
>
>
>
>
> On 10/2/07, *Agustin Lobo* <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     It makes sense, R puts everything in memory.
>     In general, unless you have small raster layers,
>     the best is to use the gis for the processing, get
>     intermediate data out of the raster and process these data in R.
>     Then back to gis for displaying if required.
>     In other words, no way of using R instead of r.mapcalc
>     unless the raster is very small.
>
>     If you can let us know what you want to do with the raster
>     we can try to be more specific on how a combined use of grass/r could
>     solve the problem (hopefully).
>
>     Agus
>
>     Carlos "Guâno" Grohmann escribió:
>      > I've been thinking about the time it takes to import a raster
>     from GRASS
>      > to R. I have some rasters with about 2000 rows and 2500 columns.
>      > importing starts fine, creates the header, then goes to 100%, and it
>      > hangs there for about 10 minutes, before get ready for the next
>     command.
>      > Any way of decrease this waiting time?
>      >
>      > cheers
>      >
>      > Carlos
>      >
>      > --
>      > +-----------------------------------------------------------+
>      >               Carlos Henrique Grohmann - Guano
>      >   Visiting Researcher at Kingston University London - UK
>      >   Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
>      > Linux User #89721  - carlos dot grohmann at gmail dot com
>      > +-----------------------------------------------------------+
>      > _________________
>      > "Good morning, doctors. I have taken the liberty of removing
>     Windows 95
>      > from my hard drive."
>      > --The winning entry in a "What were HAL's first words" contest
>     judged by
>      > 2001: A SPACE ODYSSEY creator Arthur C. Clarke
>      >
>      > Can't stop the signal.
>      >
>      >
>      >
>     ------------------------------------------------------------------------
>      >
>      > _______________________________________________
>      > statsgrass mailing list
>      > [hidden email] <mailto:[hidden email]>
>      > http://grass.itc.it/mailman/listinfo/statsgrass
>      >
>
>     --
>     Dr. Agustin Lobo
>     Institut de Ciencies de la Terra "Jaume Almera" (CSIC)
>     LLuis Sole Sabaris s/n
>     08028 Barcelona
>     Spain
>     Tel. [phone]
>     Fax. [phone]
>     email: [hidden email] <mailto:[hidden email]>
>     http://www.ija.csic.es/gt/obster
>
>
>
>
> --
> +-----------------------------------------------------------+
>               Carlos Henrique Grohmann - Guano  
>   Visiting Researcher at Kingston University London - UK
>   Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
> Linux User #89721  - carlos dot grohmann at gmail dot com
> +-----------------------------------------------------------+
> _________________
> "Good morning, doctors. I have taken the liberty of removing Windows 95
> from my hard drive."
> --The winning entry in a "What were HAL's first words" contest judged by
> 2001: A SPACE ODYSSEY creator Arthur C. Clarke
>
> Can't stop the signal.
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>

--
Dr. Agustin Lobo
Institut de Ciencies de la Terra "Jaume Almera" (CSIC)
LLuis Sole Sabaris s/n
08028 Barcelona
Spain
Tel. [phone]
Fax. [phone]
email: [hidden email]
http://www.ija.csic.es/gt/obster

"sgr.stats" <- function(nom.ref="t1pc1e3c", nompc1="t1pc1", nompc2="t1pc2",nompc3="t1pc3")
{
print("Executing sgr.stats...")
require(spgrass6)
G <- gmeta6()
comm <- paste("/home/alobo/PERU/stats_aux.scr", nom.ref, nompc1, nompc2, nompc3)
print(comm)
print("Grass...")
system(comm)
print("back to R...")
print(paste("    command executed in grass:",comm))

#Calculates the stat file for imorm
m1 <- read.table("delme.m1", skip=4,sep=":")
m1[m1=="*"] <- NA
m2 <- read.table("delme.m2", skip=4,sep=":")
m2[m2=="*"] <- NA
m3 <- read.table("delme.m3", skip=4,sep=":")
m3[m3=="*"] <- NA
v1 <- read.table("delme.v1", skip=4,sep=":")
v1[v1=="*"] <- NA
v2 <- read.table("delme.v2", skip=4,sep=":")
v2[v2=="*"] <- NA
v3 <- read.table("delme.v3", skip=4,sep=":")
v3[v3=="*"] <- NA
v1[,2] <- sqrt(v1[,2])
v2[,2] <- sqrt(v2[,2])
v3[,2] <- sqrt(v3[,2])
s1 <- read.table("delme.s1", skip=4,sep=":")
s1[s1=="*"] <- NA
n <- s1[,2]/m1[,2]
lin <- colu <- n*0
cv1 <- v1[,2]*100/m1[,2]
cv2 <- v2[,2]*100/m2[,2]
cv3 <- v3[,2]*100/m3[,2]
res <- cbind(m1[,1],lin, colu,n,
                              m1[,2],m2[,2],m3[,2],
                              v1[,2],v2[,2],v3[,2],
                              cv1,cv2,cv3)
dimnames(res)[[2]] <- c("etiq","lin", "col", "num",
                    "media 1", "media 2", "media 3",
                    "sd 1", "sd 2", "sd 3",
                    "cv 1", "cv 2", "cv 3")
system (paste("rm delme.m*", "delme.s*", "delme.v*"))
res
}

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Matteo Dall'Amico]

Some javascript/style in this post has been disabled (why?)

Hello,

I would like to raise a question about plotting: I would like to plot on the SAME graph the temperature with time of sensors at different depth.

That means, I would like to have in the same graph on the x-axis the time, y-axis the temperature, and on the graph several curves with different colors representing the different depth.

I have a file with the first column representing the time, column 2 representing the temperature at depth1, column 3 the temperature at depth2... and so on.

Does someone know how I could do that?

Thanks a lot in advance,

Matteo

---------------------------------

Matteo Dall'Amico

PhD candidate

CUDAM and department of Civil and Environmental Engineering

Universita' degli Studi di Trento

Via Mesiano 77 - 38050 Trento - Italy

email: [hidden email]

Tel. [phone]

Fax [phone]

http://www.dottorato.it/trento/

---------------------------------

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Edzer J. Pebesma-2]

Some javascript/style in this post has been disabled (why?)

Matteo, you might try:

 d = data.frame(a = rnorm(5), b = rnorm(5), c = rnorm(5), t = as.POSIXct(Sys.time()+1:5))
 library(lattice)
xyplot(a+b+c~t, d, auto.key = TRUE)

Hope this helps,
--
Edzer

Matteo Dall'Amico wrote:

Hello,

I would like to raise a question about plotting: I would like to plot on the SAME graph the temperature with time of sensors at different depth.

That means, I would like to have in the same graph on the x-axis the time, y-axis the temperature, and on the graph several curves with different colors representing the different depth.

I have a file with the first column representing the time, column 2 representing the temperature at depth1, column 3 the temperature at depth2... and so on.

Does someone know how I could do that?

Thanks a lot in advance,

Matteo

---------------------------------

Matteo Dall'Amico

PhD candidate

CUDAM and department of Civil and Environmental Engineering

Universita' degli Studi di Trento

Via Mesiano 77 - 38050 Trento - Italy

email: [hidden email]

Tel. [phone]

Fax [phone]

http://www.dottorato.it/trento/

---------------------------------

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass
  

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Jarosław Jasiewicz]

Matteo Dall'Amico napisał(a):

> Hello,
> I would like to raise a question about plotting: I would like to plot
> on the SAME graph the temperature with time of sensors at different depth.
> That means, I would like to have in the same graph on the x-axis the
> time, y-axis the temperature, and on the graph several curves with
> different colors representing the different depth.
> I have a file with the first column representing the time, column 2
> representing the temperature at depth1, column 3 the temperature at
> depth2... and so on.
> Does someone know how I could do that?
> Thanks a lot in advance,
>
> Matteo
>
> ---------------------------------
> Matteo Dall'Amico
> PhD candidate
> CUDAM and department of Civil and Environmental Engineering
> Universita' degli Studi di Trento
> Via Mesiano 77 - 38050 Trento - Italy
> email: [hidden email]
> <mailto:[hidden email]>
> Tel. [phone]
> Fax [phone]
> http://www.dottorato.it/trento/
> ---------------------------------
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>statsgrass mailing list
>[hidden email]
>http://grass.itc.it/mailman/listinfo/statsgrass
>  
>

plot(first_graph, col=red, xlim=c(min_time_of_whole_data,
max_time_of_whole_data), ylim=c(min_temp_of_whole_data,
max_temp_of_whole_data),....)
plot(second_graph, col="green", add=TRUE)
.....
plot(n_graph, col="anyone", add=TRUE)

hope you need this

Jarek

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Matteo Dall'Amico]

Thank you Jaroslaw,
I tried but "plot" doesn't seem to accept "add" as an argument:

 > plot(temp[,2],add=TRUE)
Warning messages:
1: "add" is not a graphical parameter in: plot.window(xlim, ylim,  
log, asp, ...)
2: "add" is not a graphical parameter in: plot.xy(xy, type, pch, lty,  
col, bg, cex, lwd, ...)
3: "add" is not a graphical parameter in: axis(side, at,  
as.graphicsAnnot(labels), tick, line, pos, outer,
4: "add" is not a graphical parameter in: axis(side, at,  
as.graphicsAnnot(labels), tick, line, pos, outer,
5: "add" is not a graphical parameter in: box(which = which, lty =  
lty, ...)
6: "add" is not a graphical parameter in: title(main, sub, xlab,  
ylab, line, outer, ...)

Suggestions?
Thanks,

Matteo

On 5 Oct 2007, at 12:26, Jarosław Jasiewicz wrote:

> Matteo Dall'Amico napisał(a):
>
>> Hello,
>> I would like to raise a question about plotting: I would like to  
>> plot on the SAME graph the temperature with time of sensors at  
>> different depth.
>> That means, I would like to have in the same graph on the x-axis  
>> the time, y-axis the temperature, and on the graph several curves  
>> with different colors representing the different depth.
>> I have a file with the first column representing the time, column  
>> 2 representing the temperature at depth1, column 3 the temperature  
>> at depth2... and so on.
>> Does someone know how I could do that?
>> Thanks a lot in advance,
>>
>> Matteo
>>
>> ---------------------------------
>> Matteo Dall'Amico
>> PhD candidate
>> CUDAM and department of Civil and Environmental Engineering
>> Universita' degli Studi di Trento
>> Via Mesiano 77 - 38050 Trento - Italy
>> email: [hidden email]  
>> <mailto:[hidden email]>
>> Tel. [phone]
>> Fax [phone]
>> http://www.dottorato.it/trento/
>> ---------------------------------
>>
>>
>> ---------------------------------------------------------------------
>> ---
>>
>> _______________________________________________
>> statsgrass mailing list
>> [hidden email]
>> http://grass.itc.it/mailman/listinfo/statsgrass
>>
> plot(first_graph, col=red, xlim=c(min_time_of_whole_data,  
> max_time_of_whole_data), ylim=c(min_temp_of_whole_data,  
> max_temp_of_whole_data),....)
> plot(second_graph, col="green", add=TRUE)
> .....
> plot(n_graph, col="anyone", add=TRUE)
>
> hope you need this
>
> Jarek
>

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Roger Bivand]

On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:

> I've been thinking about the time it takes to import a raster from GRASS to
> R. I have some rasters with about 2000 rows and 2500 columns. importing
> starts fine, creates the header, then goes to 100%, and it hangs there for
> about 10 minutes, before get ready for the next command.
> Any way of decrease this waiting time?

I'm looking into this, as far as I can see as of now it is somewhere in
the new("SpatialGridDataFrame", ...) call in
res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
Rprof() output points at unlist() taking 90% of the total run time. I'll
let you know where I get.

Roger

>
> cheers
>
> Carlos
>
>

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: [phone]; fax [phone]
e-mail: [hidden email]

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Jarosław Jasiewicz]

Roger!

I also notice that of Carlos wrote.

That hangs not occured when spgrass6 was based on r.out.ascii instead of
binary input-output and in fact it was realably faster. Maybe back to
previouis solution be good temporal solution. I also exchange data
between grass and R very often so that hang make me sometimes inpatient

greetengs and regards
Jarek


6/10/2007, "Roger Bivand" <[hidden email]> napisał/a:

>On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>
>> I've been thinking about the time it takes to import a raster from GRASS to
>> R. I have some rasters with about 2000 rows and 2500 columns. importing
>> starts fine, creates the header, then goes to 100%, and it hangs there for
>> about 10 minutes, before get ready for the next command.
>> Any way of decrease this waiting time?
>
>I'm looking into this, as far as I can see as of now it is somewhere in
>the new("SpatialGridDataFrame", ...) call in
>res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>Rprof() output points at unlist() taking 90% of the total run time. I'll
>let you know where I get.
>
>Roger
>
>>
>> cheers
>>
>> Carlos
>>
>>
>
>--
>Roger Bivand
>Economic Geography Section, Department of Economics, Norwegian School of
>Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>Norway. voice: [phone]; fax [phone]
>e-mail: [hidden email]

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Roger Bivand]

On Sat, 6 Oct 2007, [hidden email] wrote:

>
> Roger!
>
> I also notice that of Carlos wrote.
>
> That hangs not occured when spgrass6 was based on r.out.ascii instead of
> binary input-output and in fact it was realably faster. Maybe back to
> previouis solution be good temporal solution. I also exchange data
> between grass and R very often so that hang make me sometimes inpatient
>
> greetengs and regards
> Jarek

The problem is in new("SpatialGrid", SPix):

library(sp)
sessionInfo()
cellcentre.offset <- c(-16.821667, 13.060307)
cellsize <- c(0.000833, 0.000833)
cells.dim <- c(3629.000000, 921.00000)
grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
pts <- sp:::boguspoints(grd)
pts@bbox[,1] = pts@bbox[,1] - 0.5 * grd@cellsize
pts@bbox[,2] = pts@bbox[,2] + 0.5 * grd@cellsize
proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
   "+no_defs +towgs84=0.000,0.000,0.000"))
SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
SG <- new("SpatialGrid", SPix)

with rapidly increasing memory use. I have no idea why. My system:

> sessionInfo()
R version 2.6.0 (2007-10-03)
i686-pc-linux-gnu

locale:
LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] sp_0.9-14

loaded via a namespace (and not attached):
[1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15

and the message after Ctrl-C:

Error in coordinates(coords) :
   error in evaluating the argument 'obj' in selecting a method for function 'coordinates'
Error in bbox(object) :
   error in evaluating the argument 'obj' in selecting a method for function 'bbox'

which makes me suspect that the new() is (now?) going to completely
senseless places. This has the same effect:

sG <- new("SpatialGrid", bbox=pts@bbox, proj4string=pts@proj4string,
   coords=pts@coords, grid=grd, grid.index=integer(0))

unfortunately. It also impacts the rgdal route, because its call to make
the output SGDF is the same.

However, replacing the call to new("SpatialGrid", ...) to one of building
an empty object with just new(), and stuffing the slots manually with
slot() <- reduces import from over 12 minutes to 6 seconds (!). Perhaps
the problem is that S4 classes have got "cleverer" and dislike the
definition of SpatialGrid as only having a "SpatialPixels" representation?

I have committed the very draft readRAST6() to CVS on sourceforge, but the
real solution needs doing in sp itself (end of the SpatialGrid() function
in SpatialGrid-methods.R).

I'll carry on playing ...

Roger

>
>
> 6/10/2007, "Roger Bivand" <[hidden email]> napisa?/a:
>
>> On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>>
>>> I've been thinking about the time it takes to import a raster from GRASS to
>>> R. I have some rasters with about 2000 rows and 2500 columns. importing
>>> starts fine, creates the header, then goes to 100%, and it hangs there for
>>> about 10 minutes, before get ready for the next command.
>>> Any way of decrease this waiting time?
>>
>> I'm looking into this, as far as I can see as of now it is somewhere in
>> the new("SpatialGridDataFrame", ...) call in
>> res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>> Rprof() output points at unlist() taking 90% of the total run time. I'll
>> let you know where I get.
>>
>> Roger
>>
>>>
>>> cheers
>>>
>>> Carlos
>>>
>>>
>>
>> --
>> Roger Bivand
>> Economic Geography Section, Department of Economics, Norwegian School of
>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> Norway. voice: [phone]; fax [phone]
>> e-mail: [hidden email]
>

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: [phone]; fax [phone]
e-mail: [hidden email]
_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Roger Bivand]

Hi,

Modification also committed in sp on sourceforge, speedup from from ~ 12
minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion.
I'm seeing similar speedups in rgdal. Still no idea why - a change in the
methods package? When we release a fresh sp, the need for special fixes
will go away, I hope.

Could somebody try the current sourceforge CVS of sp and see if you see
the same speedup? May need to wait for the anonymous CVS to synch.

Best wishes,

Roger


On Sat, 6 Oct 2007, Roger Bivand wrote:

> On Sat, 6 Oct 2007, [hidden email] wrote:
>
>>
>>  Roger!
>>
>>  I also notice that of Carlos wrote.
>>
>>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
>>  binary input-output and in fact it was realably faster. Maybe back to
>>  previouis solution be good temporal solution. I also exchange data
>>  between grass and R very often so that hang make me sometimes inpatient
>>
>>  greetengs and regards
>>  Jarek
>
> The problem is in new("SpatialGrid", SPix):
>
> library(sp)
> sessionInfo()
> cellcentre.offset <- c(-16.821667, 13.060307)
> cellsize <- c(0.000833, 0.000833)
> cells.dim <- c(3629.000000, 921.00000)
> grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
> pts <- sp:::boguspoints(grd)
> pts@bbox[,1] = pts@bbox[,1] - 0.5 * grd@cellsize
> pts@bbox[,2] = pts@bbox[,2] + 0.5 * grd@cellsize
> proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
>  "+no_defs +towgs84=0.000,0.000,0.000"))
> SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
> SG <- new("SpatialGrid", SPix)
>
> with rapidly increasing memory use. I have no idea why. My system:
>
>>  sessionInfo()
> R version 2.6.0 (2007-10-03)
> i686-pc-linux-gnu
>
> locale:
> LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] sp_0.9-14
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
>
> and the message after Ctrl-C:
>
> Error in coordinates(coords) :
>  error in evaluating the argument 'obj' in selecting a method for function
> 'coordinates'
> Error in bbox(object) :
>   error in evaluating the argument 'obj' in selecting a method for function
>   'bbox'
>
> which makes me suspect that the new() is (now?) going to completely senseless
> places. This has the same effect:
>
> sG <- new("SpatialGrid", bbox=pts@bbox, proj4string=pts@proj4string,
>   coords=pts@coords, grid=grd, grid.index=integer(0))
>
> unfortunately. It also impacts the rgdal route, because its call to make the
> output SGDF is the same.
>
> However, replacing the call to new("SpatialGrid", ...) to one of building an
> empty object with just new(), and stuffing the slots manually with slot() <-
> reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is
> that S4 classes have got "cleverer" and dislike the definition of SpatialGrid
> as only having a "SpatialPixels" representation?
>
> I have committed the very draft readRAST6() to CVS on sourceforge, but the
> real solution needs doing in sp itself (end of the SpatialGrid() function in
> SpatialGrid-methods.R).
>
> I'll carry on playing ...
>
> Roger
>
>
>>
>>
>>  6/10/2007, "Roger Bivand" <[hidden email]> napisa?/a:
>>
>> >  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>> >
>> > >  I've been thinking about the time it takes to import a raster from
>> > >  GRASS to
>> > >  R. I have some rasters with about 2000 rows and 2500 columns.
>> > >  importing
>> > >  starts fine, creates the header, then goes to 100%, and it hangs there
>> > >  for
>> > >  about 10 minutes, before get ready for the next command.
>> > >  Any way of decrease this waiting time?
>> >
>> >  I'm looking into this, as far as I can see as of now it is somewhere in
>> >  the new("SpatialGridDataFrame", ...) call in
>> >  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>> >  Rprof() output points at unlist() taking 90% of the total run time. I'll
>> >  let you know where I get.
>> >
>> >  Roger
>> >
>> > >
>> > >  cheers
>> > >
>> > >  Carlos
>> > >
>> > >
>> >
>> >  --
>> >  Roger Bivand
>> >  Economic Geography Section, Department of Economics, Norwegian School of
>> >  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> >  Norway. voice: [phone]; fax [phone]
>> >  e-mail: [hidden email]
>>
>
>

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: [phone]; fax [phone]
e-mail: [hidden email]

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Carlos Grohmann-2]

Hi,

I just downloaded/compiled/installed the current sourceforge CVS for
sp and spgrass6, and it's WAY faster than before.

thanks, Roger!

Carlos


On 10/6/07, Roger Bivand <[hidden email]> wrote:

> Hi,
>
> Modification also committed in sp on sourceforge, speedup from from ~ 12
> minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion.
> I'm seeing similar speedups in rgdal. Still no idea why - a change in the
> methods package? When we release a fresh sp, the need for special fixes
> will go away, I hope.
>
> Could somebody try the current sourceforge CVS of sp and see if you see
> the same speedup? May need to wait for the anonymous CVS to synch.
>
> Best wishes,
>
> Roger
>
>
> On Sat, 6 Oct 2007, Roger Bivand wrote:
>
> > On Sat, 6 Oct 2007, [hidden email] wrote:
> >
> >>
> >>  Roger!
> >>
> >>  I also notice that of Carlos wrote.
> >>
> >>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
> >>  binary input-output and in fact it was realably faster. Maybe back to
> >>  previouis solution be good temporal solution. I also exchange data
> >>  between grass and R very often so that hang make me sometimes inpatient
> >>
> >>  greetengs and regards
> >>  Jarek
> >
> > The problem is in new("SpatialGrid", SPix):
> >
> > library(sp)
> > sessionInfo()
> > cellcentre.offset <- c(-16.821667, 13.060307)
> > cellsize <- c(0.000833, 0.000833)
> > cells.dim <- c(3629.000000, 921.00000)
> > grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
> > pts <- sp:::boguspoints(grd)
> > pts@bbox[,1] = pts@bbox[,1] - 0.5 * grd@cellsize
> > pts@bbox[,2] = pts@bbox[,2] + 0.5 * grd@cellsize
> > proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
> >  "+no_defs +towgs84=0.000,0.000,0.000"))
> > SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
> > SG <- new("SpatialGrid", SPix)
> >
> > with rapidly increasing memory use. I have no idea why. My system:
> >
> >>  sessionInfo()
> > R version 2.6.0 (2007-10-03)
> > i686-pc-linux-gnu
> >
> > locale:
> > LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
> >
> > attached base packages:
> > [1] stats     graphics  grDevices utils     datasets  methods   base
> >
> > other attached packages:
> > [1] sp_0.9-14
> >
> > loaded via a namespace (and not attached):
> > [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
> >
> > and the message after Ctrl-C:
> >
> > Error in coordinates(coords) :
> >  error in evaluating the argument 'obj' in selecting a method for function
> > 'coordinates'
> > Error in bbox(object) :
> >   error in evaluating the argument 'obj' in selecting a method for function
> >   'bbox'
> >
> > which makes me suspect that the new() is (now?) going to completely senseless
> > places. This has the same effect:
> >
> > sG <- new("SpatialGrid", bbox=pts@bbox, proj4string=pts@proj4string,
> >   coords=pts@coords, grid=grd, grid.index=integer(0))
> >
> > unfortunately. It also impacts the rgdal route, because its call to make the
> > output SGDF is the same.
> >
> > However, replacing the call to new("SpatialGrid", ...) to one of building an
> > empty object with just new(), and stuffing the slots manually with slot() <-
> > reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is
> > that S4 classes have got "cleverer" and dislike the definition of SpatialGrid
> > as only having a "SpatialPixels" representation?
> >
> > I have committed the very draft readRAST6() to CVS on sourceforge, but the
> > real solution needs doing in sp itself (end of the SpatialGrid() function in
> > SpatialGrid-methods.R).
> >
> > I'll carry on playing ...
> >
> > Roger
> >
> >
> >>
> >>
> >>  6/10/2007, "Roger Bivand" <[hidden email]> napisa?/a:
> >>
> >> >  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
> >> >
> >> > >  I've been thinking about the time it takes to import a raster from
> >> > >  GRASS to
> >> > >  R. I have some rasters with about 2000 rows and 2500 columns.
> >> > >  importing
> >> > >  starts fine, creates the header, then goes to 100%, and it hangs there
> >> > >  for
> >> > >  about 10 minutes, before get ready for the next command.
> >> > >  Any way of decrease this waiting time?
> >> >
> >> >  I'm looking into this, as far as I can see as of now it is somewhere in
> >> >  the new("SpatialGridDataFrame", ...) call in
> >> >  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
> >> >  Rprof() output points at unlist() taking 90% of the total run time. I'll
> >> >  let you know where I get.
> >> >
> >> >  Roger
> >> >
> >> > >
> >> > >  cheers
> >> > >
> >> > >  Carlos
> >> > >
> >> > >
> >> >
> >> >  --
> >> >  Roger Bivand
> >> >  Economic Geography Section, Department of Economics, Norwegian School of
> >> >  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> >> >  Norway. voice: [phone]; fax [phone]
> >> >  e-mail: [hidden email]
> >>
> >
> >
>
> --
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: [phone]; fax [phone]
> e-mail: [hidden email]
>

--
+-----------------------------------------------------------+
              Carlos Henrique Grohmann - Guano
  Visiting Researcher at Kingston University London - UK
  Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
Linux User #89721  - carlos dot grohmann at gmail dot com
+-----------------------------------------------------------+
_________________
"Good morning, doctors. I have taken the liberty of removing Windows
95 from my hard drive."
--The winning entry in a "What were HAL's first words" contest judged
by 2001: A SPACE ODYSSEY creator Arthur C. Clarke

Can't stop the signal.

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Carlos Grohmann-2]

It took about 6 seconds to read a raster with 2200x2500 cells, and
about 3 secs to build a truehist().

don't have any categorical map, though.

Carlos



On 10/8/07, Agustin Lobo <[hidden email]> wrote:

> How long it takes computing the histogram of the 900*3600 raster in R?
> If you have a categorical map for the same region, could you check
> how long it takes computing the stratified statistics?
>
> Thanks!
>
> Agus
>
> Carlos "Guâno" Grohmann escribió:
> > Hi,
> >
> > I just downloaded/compiled/installed the current sourceforge CVS for
> > sp and spgrass6, and it's WAY faster than before.
> >
> > thanks, Roger!
> >
> > Carlos
> >
> >
> > On 10/6/07, Roger Bivand <[hidden email]> wrote:
> >> Hi,
> >>
> >> Modification also committed in sp on sourceforge, speedup from from ~ 12
> >> minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion.
> >> I'm seeing similar speedups in rgdal. Still no idea why - a change in the
> >> methods package? When we release a fresh sp, the need for special fixes
> >> will go away, I hope.
> >>
> >> Could somebody try the current sourceforge CVS of sp and see if you see
> >> the same speedup? May need to wait for the anonymous CVS to synch.
> >>
> >> Best wishes,
> >>
> >> Roger
> >>
> >>
> >> On Sat, 6 Oct 2007, Roger Bivand wrote:
> >>
> >>> On Sat, 6 Oct 2007, [hidden email] wrote:
> >>>
> >>>>  Roger!
> >>>>
> >>>>  I also notice that of Carlos wrote.
> >>>>
> >>>>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
> >>>>  binary input-output and in fact it was realably faster. Maybe back to
> >>>>  previouis solution be good temporal solution. I also exchange data
> >>>>  between grass and R very often so that hang make me sometimes inpatient
> >>>>
> >>>>  greetengs and regards
> >>>>  Jarek
> >>> The problem is in new("SpatialGrid", SPix):
> >>>
> >>> library(sp)
> >>> sessionInfo()
> >>> cellcentre.offset <- c(-16.821667, 13.060307)
> >>> cellsize <- c(0.000833, 0.000833)
> >>> cells.dim <- c(3629.000000, 921.00000)
> >>> grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
> >>> pts <- sp:::boguspoints(grd)
> >>> pts@bbox[,1] = pts@bbox[,1] - 0.5 * grd@cellsize
> >>> pts@bbox[,2] = pts@bbox[,2] + 0.5 * grd@cellsize
> >>> proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
> >>>  "+no_defs +towgs84=0.000,0.000,0.000"))
> >>> SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
> >>> SG <- new("SpatialGrid", SPix)
> >>>
> >>> with rapidly increasing memory use. I have no idea why. My system:
> >>>
> >>>>  sessionInfo()
> >>> R version 2.6.0 (2007-10-03)
> >>> i686-pc-linux-gnu
> >>>
> >>> locale:
> >>> LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
> >>>
> >>> attached base packages:
> >>> [1] stats     graphics  grDevices utils     datasets  methods   base
> >>>
> >>> other attached packages:
> >>> [1] sp_0.9-14
> >>>
> >>> loaded via a namespace (and not attached):
> >>> [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
> >>>
> >>> and the message after Ctrl-C:
> >>>
> >>> Error in coordinates(coords) :
> >>>  error in evaluating the argument 'obj' in selecting a method for function
> >>> 'coordinates'
> >>> Error in bbox(object) :
> >>>   error in evaluating the argument 'obj' in selecting a method for function
> >>>   'bbox'
> >>>
> >>> which makes me suspect that the new() is (now?) going to completely senseless
> >>> places. This has the same effect:
> >>>
> >>> sG <- new("SpatialGrid", bbox=pts@bbox, proj4string=pts@proj4string,
> >>>   coords=pts@coords, grid=grd, grid.index=integer(0))
> >>>
> >>> unfortunately. It also impacts the rgdal route, because its call to make the
> >>> output SGDF is the same.
> >>>
> >>> However, replacing the call to new("SpatialGrid", ...) to one of building an
> >>> empty object with just new(), and stuffing the slots manually with slot() <-
> >>> reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is
> >>> that S4 classes have got "cleverer" and dislike the definition of SpatialGrid
> >>> as only having a "SpatialPixels" representation?
> >>>
> >>> I have committed the very draft readRAST6() to CVS on sourceforge, but the
> >>> real solution needs doing in sp itself (end of the SpatialGrid() function in
> >>> SpatialGrid-methods.R).
> >>>
> >>> I'll carry on playing ...
> >>>
> >>> Roger
> >>>
> >>>
> >>>>
> >>>>  6/10/2007, "Roger Bivand" <[hidden email]> napisa?/a:
> >>>>
> >>>>>  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
> >>>>>
> >>>>>>  I've been thinking about the time it takes to import a raster from
> >>>>>>  GRASS to
> >>>>>>  R. I have some rasters with about 2000 rows and 2500 columns.
> >>>>>>  importing
> >>>>>>  starts fine, creates the header, then goes to 100%, and it hangs there
> >>>>>>  for
> >>>>>>  about 10 minutes, before get ready for the next command.
> >>>>>>  Any way of decrease this waiting time?
> >>>>>  I'm looking into this, as far as I can see as of now it is somewhere in
> >>>>>  the new("SpatialGridDataFrame", ...) call in
> >>>>>  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
> >>>>>  Rprof() output points at unlist() taking 90% of the total run time. I'll
> >>>>>  let you know where I get.
> >>>>>
> >>>>>  Roger
> >>>>>
> >>>>>>  cheers
> >>>>>>
> >>>>>>  Carlos
> >>>>>>
> >>>>>>
> >>>>>  --
> >>>>>  Roger Bivand
> >>>>>  Economic Geography Section, Department of Economics, Norwegian School of
> >>>>>  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> >>>>>  Norway. voice: [phone]; fax [phone]
> >>>>>  e-mail: [hidden email]
> >>>
> >> --
> >> Roger Bivand
> >> Economic Geography Section, Department of Economics, Norwegian School of
> >> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> >> Norway. voice: [phone]; fax [phone]
> >> e-mail: [hidden email]
> >>
> >
> >
>
> --
> Dr. Agustin Lobo
> Institut de Ciencies de la Terra "Jaume Almera" (CSIC)
> LLuis Sole Sabaris s/n
> 08028 Barcelona
> Spain
> Tel. [phone]
> Fax. [phone]
> email: [hidden email]
> http://www.ija.csic.es/gt/obster
>

--
+-----------------------------------------------------------+
              Carlos Henrique Grohmann - Guano
  Visiting Researcher at Kingston University London - UK
  Geologist M.Sc  - Doctorate Student at IGc-USP - Brazil
Linux User #89721  - carlos dot grohmann at gmail dot com
+-----------------------------------------------------------+
_________________
"Good morning, doctors. I have taken the liberty of removing Windows
95 from my hard drive."
--The winning entry in a "What were HAL's first words" contest judged
by 2001: A SPACE ODYSSEY creator Arthur C. Clarke

Can't stop the signal.

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Roger Bivand]

On Mon, 8 Oct 2007, Carlos "Guâno" Grohmann wrote:

> Hi,
>
> I just downloaded/compiled/installed the current sourceforge CVS for
> sp and spgrass6, and it's WAY faster than before.

Thanks, I'll back out of the spgrass6 changes, which are in sp too. I see
very big speed-ups in rgdal too, and have R CMD check'ed the released
gstat for problems. I have no idea why the methods recursed into the new
SpatialGrid object, but they obviously did.

Roger

>
> thanks, Roger!
>
> Carlos
>
>
> On 10/6/07, Roger Bivand <[hidden email]> wrote:
>> Hi,
>>
>> Modification also committed in sp on sourceforge, speedup from from ~ 12
>> minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion.
>> I'm seeing similar speedups in rgdal. Still no idea why - a change in the
>> methods package? When we release a fresh sp, the need for special fixes
>> will go away, I hope.
>>
>> Could somebody try the current sourceforge CVS of sp and see if you see
>> the same speedup? May need to wait for the anonymous CVS to synch.
>>
>> Best wishes,
>>
>> Roger
>>
>>
>> On Sat, 6 Oct 2007, Roger Bivand wrote:
>>
>>> On Sat, 6 Oct 2007, [hidden email] wrote:
>>>
>>>>
>>>>  Roger!
>>>>
>>>>  I also notice that of Carlos wrote.
>>>>
>>>>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
>>>>  binary input-output and in fact it was realably faster. Maybe back to
>>>>  previouis solution be good temporal solution. I also exchange data
>>>>  between grass and R very often so that hang make me sometimes inpatient
>>>>
>>>>  greetengs and regards
>>>>  Jarek
>>>
>>> The problem is in new("SpatialGrid", SPix):
>>>
>>> library(sp)
>>> sessionInfo()
>>> cellcentre.offset <- c(-16.821667, 13.060307)
>>> cellsize <- c(0.000833, 0.000833)
>>> cells.dim <- c(3629.000000, 921.00000)
>>> grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
>>> pts <- sp:::boguspoints(grd)
>>> pts@bbox[,1] = pts@bbox[,1] - 0.5 * grd@cellsize
>>> pts@bbox[,2] = pts@bbox[,2] + 0.5 * grd@cellsize
>>> proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
>>>  "+no_defs +towgs84=0.000,0.000,0.000"))
>>> SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
>>> SG <- new("SpatialGrid", SPix)
>>>
>>> with rapidly increasing memory use. I have no idea why. My system:
>>>
>>>>  sessionInfo()
>>> R version 2.6.0 (2007-10-03)
>>> i686-pc-linux-gnu
>>>
>>> locale:
>>> LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
>>>
>>> attached base packages:
>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>> other attached packages:
>>> [1] sp_0.9-14
>>>
>>> loaded via a namespace (and not attached):
>>> [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
>>>
>>> and the message after Ctrl-C:
>>>
>>> Error in coordinates(coords) :
>>>  error in evaluating the argument 'obj' in selecting a method for function
>>> 'coordinates'
>>> Error in bbox(object) :
>>>   error in evaluating the argument 'obj' in selecting a method for function
>>>   'bbox'
>>>
>>> which makes me suspect that the new() is (now?) going to completely senseless
>>> places. This has the same effect:
>>>
>>> sG <- new("SpatialGrid", bbox=pts@bbox, proj4string=pts@proj4string,
>>>   coords=pts@coords, grid=grd, grid.index=integer(0))
>>>
>>> unfortunately. It also impacts the rgdal route, because its call to make the
>>> output SGDF is the same.
>>>
>>> However, replacing the call to new("SpatialGrid", ...) to one of building an
>>> empty object with just new(), and stuffing the slots manually with slot() <-
>>> reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is
>>> that S4 classes have got "cleverer" and dislike the definition of SpatialGrid
>>> as only having a "SpatialPixels" representation?
>>>
>>> I have committed the very draft readRAST6() to CVS on sourceforge, but the
>>> real solution needs doing in sp itself (end of the SpatialGrid() function in
>>> SpatialGrid-methods.R).
>>>
>>> I'll carry on playing ...
>>>
>>> Roger
>>>
>>>
>>>>
>>>>
>>>>  6/10/2007, "Roger Bivand" <[hidden email]> napisa?/a:
>>>>
>>>>>  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>>>>>
>>>>>>  I've been thinking about the time it takes to import a raster from
>>>>>>  GRASS to
>>>>>>  R. I have some rasters with about 2000 rows and 2500 columns.
>>>>>>  importing
>>>>>>  starts fine, creates the header, then goes to 100%, and it hangs there
>>>>>>  for
>>>>>>  about 10 minutes, before get ready for the next command.
>>>>>>  Any way of decrease this waiting time?
>>>>>
>>>>>  I'm looking into this, as far as I can see as of now it is somewhere in
>>>>>  the new("SpatialGridDataFrame", ...) call in
>>>>>  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>>>>>  Rprof() output points at unlist() taking 90% of the total run time. I'll
>>>>>  let you know where I get.
>>>>>
>>>>>  Roger
>>>>>
>>>>>>
>>>>>>  cheers
>>>>>>
>>>>>>  Carlos
>>>>>>
>>>>>>
>>>>>
>>>>>  --
>>>>>  Roger Bivand
>>>>>  Economic Geography Section, Department of Economics, Norwegian School of
>>>>>  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>>>>>  Norway. voice: [phone]; fax [phone]
>>>>>  e-mail: [hidden email]
>>>>
>>>
>>>
>>
>> --
>> Roger Bivand
>> Economic Geography Section, Department of Economics, Norwegian School of
>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> Norway. voice: [phone]; fax [phone]
>> e-mail: [hidden email]
>>
>
>
>

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: [phone]; fax [phone]
e-mail: [hidden email]
_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Markus Neteler-3]

Today I downloaded 'sp' and 'spgrass6' from CVS (http://r-spatial.sf.net/),
steps here to enable more people to try:

# use empty password == <enter> key:
cvs -d:pserver:[e-mail]:/cvsroot/r-spatial login
# fetch classes:
cvs -z3 -d:pserver:[e-mail]:/cvsroot/r-spatial co -P sp
cvs -z3 -d:pserver:[e-mail]:/cvsroot/r-spatial co -P spgrass6

# install from the same directory:
R CMD INSTALL sp
R CMD INSTALL spgrass6

Then I tried the new classes: INCREDIBLY FAST!

Seems that I can now start working with my satellite data
in R - thanks for this speedup!

best wishes,
Markus

guano wrote:

It took about 6 seconds to read a raster with 2200x2500 cells, and
about 3 secs to build a truehist().

don't have any categorical map, though.

Carlos

[Cesare Furlanello]

Hi Markus / Roger
is this upgrade being propagated to the R CRAN repository?
-- cesare


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Markus Neteler
Sent: lunedì 15 ottobre 2007 14.58
To: [hidden email]
Subject: Re: [STATSGRASS] [R-spatial-devel] speed of GRASS/R interface


Today I downloaded 'sp' and 'spgrass6' from CVS (http://r-spatial.sf.net/),
steps here to enable more people to try:

# use empty password == <enter> key:
cvs -d:pserver:[hidden email]:/cvsroot/r-spatial
login
# fetch classes:
cvs -z3
-d:pserver:[hidden email]:/cvsroot/r-spatial co -P
sp
cvs -z3
-d:pserver:[hidden email]:/cvsroot/r-spatial co -P
spgrass6

# install from the same directory:
R CMD INSTALL sp
R CMD INSTALL spgrass6

Then I tried the new classes: INCREDIBLY FAST!

Seems that I can now start working with my satellite data
in R - thanks for this speedup!

best wishes,
Markus


guano wrote:
>
> It took about 6 seconds to read a raster with 2200x2500 cells, and
> about 3 secs to build a truehist().
>
> don't have any categorical map, though.
>
> Carlos
>

--
View this message in context: http://www.nabble.com/speed-of-GRASS-R-interface-tf4555475.html#a13212226
Sent from the Grass - Stats mailing list archive at Nabble.com.

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

------------------
ITC -> dall'1 marzo 2007 Fondazione Bruno Kessler
ITC -> since 1 March 2007 Fondazione Bruno Kessler
------------------

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Edzer Pebesma-2]

I just submitted sp 0.9-16 (the cvs version) to CRAN.
--
Edzer

Cesare Furlanello wrote:

> Hi Markus / Roger
> is this upgrade being propagated to the R CRAN repository?
> -- cesare
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Markus Neteler
> Sent: lunedì 15 ottobre 2007 14.58
> To: [hidden email]
> Subject: Re: [STATSGRASS] [R-spatial-devel] speed of GRASS/R interface
>
>
> Today I downloaded 'sp' and 'spgrass6' from CVS (http://r-spatial.sf.net/),
> steps here to enable more people to try:
>
> # use empty password == <enter> key:
> cvs -d:pserver:[hidden email]:/cvsroot/r-spatial
> login
> # fetch classes:
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> sp
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> spgrass6
>
> # install from the same directory:
> R CMD INSTALL sp
> R CMD INSTALL spgrass6
>
> Then I tried the new classes: INCREDIBLY FAST!
>
> Seems that I can now start working with my satellite data
> in R - thanks for this speedup!
>
> best wishes,
> Markus
>
>
> guano wrote:
>  
>> It took about 6 seconds to read a raster with 2200x2500 cells, and
>> about 3 secs to build a truehist().
>>
>> don't have any categorical map, though.
>>
>> Carlos
>>
>>    
>
> --
> View this message in context: http://www.nabble.com/speed-of-GRASS-R-interface-tf4555475.html#a13212226
> Sent from the Grass - Stats mailing list archive at Nabble.com.
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>
> ------------------
> ITC -> dall'1 marzo 2007 Fondazione Bruno Kessler
> ITC -> since 1 March 2007 Fondazione Bruno Kessler
> ------------------
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>  

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Thomas Adams]

Markus,

Thanks for the information; I have never tried this before, but when I
do try, I get:

cvs login: warning: failed to open /Users/adams/.cvspass for reading: No
such file or directory

With subsequent attempts I just get returned to my local prompt.

Regards,
Tom


Markus Neteler wrote:

> Today I downloaded 'sp' and 'spgrass6' from CVS (http://r-spatial.sf.net/),
> steps here to enable more people to try:
>
> # use empty password == <enter> key:
> cvs -d:pserver:[hidden email]:/cvsroot/r-spatial
> login
> # fetch classes:
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> sp
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> spgrass6
>
> # install from the same directory:
> R CMD INSTALL sp
> R CMD INSTALL spgrass6
>
> Then I tried the new classes: INCREDIBLY FAST!
>
> Seems that I can now start working with my satellite data
> in R - thanks for this speedup!
>
> best wishes,
> Markus
>
>
> guano wrote:
>  
>> It took about 6 seconds to read a raster with 2200x2500 cells, and
>> about 3 secs to build a truehist().
>>
>> don't have any categorical map, though.
>>
>> Carlos
>>
>>    
>
>  

--
Thomas E Adams
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

EMAIL: [hidden email]

VOICE: 937-383-0528
FAX: [phone]

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Markus Neteler-3]

Tom,

I have no clue about this error but Edzer just posted
that he sent the new version to CRAN. So, an update from there
should do it now as well (once propagated, not sure how
much time that takes).

best,
Markus

On Mon, Oct 15, 2007 at 03:24:41PM +0200, Thomas Adams wrote:

> Markus,
>
> Thanks for the information; I have never tried this before, but when I
> do try, I get:
>
> cvs login: warning: failed to open /Users/adams/.cvspass for reading: No
> such file or directory
>
> With subsequent attempts I just get returned to my local prompt.
>
> Regards,
> Tom
>
> --
> Thomas E Adams
> National Weather Service
> Ohio River Forecast Center
> 1901 South State Route 134
> Wilmington, OH 45177
>
> EMAIL:  [hidden email]
>
> VOICE:  937-383-0528
> FAX:    [phone]
>

--
Markus Neteler  <neteler itc it>  http://mpa.itc.it/markus/
FBK-irst -  Centro per la Ricerca Scientifica e Tecnologica
MPBA - Predictive Models for Biol. & Environ. Data Analysis
Via Sommarive, 18        -       38050 Povo (Trento), Italy

------------------
ITC -> dall'1 marzo 2007 Fondazione Bruno Kessler
ITC -> since 1 March 2007 Fondazione Bruno Kessler
------------------

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass

[Roger Bivand]

On Mon, 15 Oct 2007, Cesare Furlanello wrote:

> Hi Markus / Roger
> is this upgrade being propagated to the R CRAN repository?

We will try to release sp to CRAN soon. The problem has been that the
new("SpatialGrid", ...) method has called the validObject() method for the
SpatialGrid object, which recurses into all of the constituent objects,
even though the documentation of new style classes and methods says that
it should accept the SpatialPixels object that is passed to it as already
verified. We have been trying to find a "proper" solution, but maybe we'll
just release sp as it is, and make further changes if this release breaks
anything.

Roger

> -- cesare
>
>
> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On Behalf Of Markus Neteler
> Sent: lunedì 15 ottobre 2007 14.58
> To: [hidden email]
> Subject: Re: [STATSGRASS] [R-spatial-devel] speed of GRASS/R interface
>
>
> Today I downloaded 'sp' and 'spgrass6' from CVS (http://r-spatial.sf.net/),
> steps here to enable more people to try:
>
> # use empty password == <enter> key:
> cvs -d:pserver:[hidden email]:/cvsroot/r-spatial
> login
> # fetch classes:
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> sp
> cvs -z3
> -d:pserver:[hidden email]:/cvsroot/r-spatial co -P
> spgrass6
>
> # install from the same directory:
> R CMD INSTALL sp
> R CMD INSTALL spgrass6
>
> Then I tried the new classes: INCREDIBLY FAST!
>
> Seems that I can now start working with my satellite data
> in R - thanks for this speedup!
>
> best wishes,
> Markus
>
>
> guano wrote:
>>
>> It took about 6 seconds to read a raster with 2200x2500 cells, and
>> about 3 secs to build a truehist().
>>
>> don't have any categorical map, though.
>>
>> Carlos
>>
>
> --
> View this message in context: http://www.nabble.com/speed-of-GRASS-R-interface-tf4555475.html#a13212226
> Sent from the Grass - Stats mailing list archive at Nabble.com.
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>
> ------------------
> ITC -> dall'1 marzo 2007 Fondazione Bruno Kessler
> ITC -> since 1 March 2007 Fondazione Bruno Kessler
> ------------------
>
> _______________________________________________
> statsgrass mailing list
> [hidden email]
> http://grass.itc.it/mailman/listinfo/statsgrass
>

--
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: [phone]; fax [phone]
e-mail: [hidden email]

_______________________________________________
statsgrass mailing list
[hidden email]
http://grass.itc.it/mailman/listinfo/statsgrass