Problem with Lidar Tools

6 messages Options
Embed this post
Permalink
kaipi

Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
Dear list,

I am quite new to Grass and need to generate a DSM from Lidar data. Here is my workflow so far:

v.in.ascii input=f:\lidar_test\single.csv output=single fs=, x=1 y=2 z=3 -z -b -t --o
v.build single
g.region vect=single
v.in.ascii input=f:\lidar_test\first.csv output=first fs=, x=1 y=2 z=3 -z -b -t --o
v.outlier input=single output=inlier_s outlier=outlier_s thres_o=5 soe=1 son=1 lambda_i=0.1 --o
v.outlier input=first output=inlier_f outlier=outlier_f thres_o=5 soe=1 son=1 lambda_i=0.1 --o
v.lidar.edgedetection input=inlier_s output=edgedetected see=400 sen=400 lambda_g=0.01 tgh=6 tgl=3 theta_g=0.26 lambda_r=1 --o
v.lidar.growing input=edgedetected output=growing first=inlier_f  tj=0.25 td=0.6 --o
v.lidar.correction input=growing output=classified terrain=terrain sce=5 scn=5 lambda_c=1 tch=2 tcl=1 --o
v.lidar.correction input=classified output=classified_it2 terrain=terrain_it2 sce=5 scn=5 lambda_c=1 tch=2 tcl=1 --o
v.build terrain_it2
v.out.ogr in=terrain_it2 dsn=F:\lidar_test\terrain_it2 format=ESRI_Shapefile  type=point lco=SHPT=POINTZ -e


v.lidar.edgedetection classifies the data into cat 1 and cat 2 (=Terrain or Edge ?). This seams to work. The next step is v.lidar.growing. This tool should classify into 4 categories: TERRAIN SINGLE PULSE, TERRAIN DOUBLE PULSE, OBJECT SINGLE PULSE or OBJECT DOUBLE PULSE. But this does not happen. The cats still remain the same (cat1 and cat2) and the output looks identical to the output from v.lidar.edgedetection. v.lidar.growing seams to have no effect.
Am I doing something wrong ?

I have another problem with the parameters of the lidar tools (although I read Brovellis "Managing and processing LIDAR data within GRASS"). Can you give me some hints how to adjust the default values to very high res Lidar data (=50 points/m²). I raised see and sen parameter in v.lidar.edgedetection to 400 with the result, that the detected edges are now so wide, that they cover complete buildings. This way I can get rid of the most buildings but the result is not really satisfying and I guess the problem is false classification of v.lidar.growing

Let me know if you need more infos.

I am using GRass 6.3.0 on Windows. Can I expect improvements switching to 6.4 or switching to Linux ?

Thank you very much !!

kaipi

John Tate-2

Re: Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
Hi Kaipi,

I've only been doing rural LiDAR processing, so can't help regards building
removal for the DTM.

One thing that isn't mentioned in the Brovelli, 2004 article or the online
documention on the wiki, is that spline 'step' relates to metres (1 step =
1m). Obvious I know, but wasn't to me when I started fiddling with the
parametersand the LiDAR data for the first time. Therefore:
-see and sen parameter in v.lidar.edgedetection to 400
is 400m! and hence:
-the detected edges are now so wide, that they cover complete buildings

The spline step parameters should be double the average distance between
points. So for 50pts/m² the spline steps chould be 0.04 (2*(1/50)) for
example. Play with these either side of 0.04 in the parameters until you get
a good result. These modules were designed a few years ago now, and not
really designed for such high resolution data, though should still work.
Though I am not entirely sure what you are attempting to do (e.g. purpose of
data: remove building for DTM/ or create 3D model of the building DSM).

You will find that v.outlier is the biggest restriction on how many points
you can process at one time. I was working with roughly 1.5pts/m² and found
that I had to work in 1km tiles due to the v.outlier restriction. Something
to be wary of when you decrease the spline step incase you start getting
errors.

For reference, I was using:
GRASS GIS (v.6.4.0svn, library Revision: 37101 (2009-05-10)) (windows
version)


----- Original Message -----
From: "kaipi" <[hidden email]>
To: <[hidden email]>
Sent: Friday, October 30, 2009 12:43 PM
Subject: [GRASS-user] Problem with Lidar Tools



Dear list,

I am quite new to Grass and need to generate a DSM from Lidar data. Here is
my workflow so far:

v.in.ascii input=f:\lidar_test\single.csv output=single fs=, x=1 y=2 z=3 -z
-b -t --o
v.build single
g.region vect=single
v.in.ascii input=f:\lidar_test\first.csv output=first fs=, x=1 y=2 z=3 -z -b
-t --o
v.outlier input=single output=inlier_s outlier=outlier_s thres_o=5 soe=1
son=1 lambda_i=0.1 --o
v.outlier input=first output=inlier_f outlier=outlier_f thres_o=5 soe=1
son=1 lambda_i=0.1 --o
v.lidar.edgedetection input=inlier_s output=edgedetected see=400 sen=400
lambda_g=0.01 tgh=6 tgl=3 theta_g=0.26 lambda_r=1 --o
v.lidar.growing input=edgedetected output=growing first=inlier_f  tj=0.25
td=0.6 --o
v.lidar.correction input=growing output=classified terrain=terrain sce=5
scn=5 lambda_c=1 tch=2 tcl=1 --o
v.lidar.correction input=classified output=classified_it2
terrain=terrain_it2 sce=5 scn=5 lambda_c=1 tch=2 tcl=1 --o
v.build terrain_it2
v.out.ogr in=terrain_it2 dsn=F:\lidar_test\terrain_it2 format=ESRI_Shapefile
type=point lco=SHPT=POINTZ -e

v.lidar.edgedetection classifies the data into cat 1 and cat 2 (=Terrain or
Edge ?). This seams to work. The next step is v.lidar.growing. This tool
should classify into 4 categories: TERRAIN SINGLE PULSE, TERRAIN DOUBLE
PULSE, OBJECT SINGLE PULSE or OBJECT DOUBLE PULSE. But this does not happen.
The cats still remain the same (cat1 and cat2) and the output looks
identical to the output from v.lidar.edgedetection. v.lidar.growing seams to
have no effect.
Am I doing something wrong ?

I have another problem with the parameters of the lidar tools (although I
read Brovellis "Managing and processing LIDAR data within GRASS"). Can you
give me some hints how to adjust the default values to very high res Lidar
data (=50 points/m²). I raised see and sen parameter in
v.lidar.edgedetection to 400 with the result, that the detected edges are
now so wide, that they cover complete buildings. This way I can get rid of
the most buildings but the result is not really satisfying and I guess the
problem is false classification of v.lidar.growing

Let me know if you need more infos.

I am using GRass 6.3.0 on Windows. Can I expect improvements switching to
6.4 or switching to Linux ?

Thank you very much !!

kaipi


--
View this message in context:
http://n2.nabble.com/Problem-with-Lidar-Tools-tp3918684p3918684.html
Sent from the Grass - Users mailing list archive at Nabble.com.
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user
Markus Neteler

Re: Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
In reply to this post by kaipi
On Fri, Oct 30, 2009 at 1:43 PM, kaipi <[hidden email]> wrote:
> I am quite new to Grass and need to generate a DSM from Lidar data. Here is
> my workflow so far:
...
> v.lidar.edgedetection classifies the data into cat 1 and cat 2 (=Terrain or
> Edge ?). This seams to work.

Since you use 6.3 there is some confusion which has been fixed in 6.4 and
later.  A minute ago I have backported the correction for numbering of
terrain classification which is correctly as follows:

TERRAIN = 1
EDGE = 2
UNKNOWN = 3

(see also
http://grass.osgeo.org/grass64/manuals/html64_user/v.lidar.edgedetection.html
)

> The next step is v.lidar.growing. This tool
> should classify into 4 categories: TERRAIN SINGLE PULSE, TERRAIN DOUBLE
> PULSE, OBJECT SINGLE PULSE or OBJECT DOUBLE PULSE. But this does not happen.
> The cats still remain the same (cat1 and cat2) and the output looks
> identical to the output from v.lidar.edgedetection. v.lidar.growing seams to
> have no effect.
> Am I doing something wrong ?

Would you mind to update to 6.4? We have made a couple of fixes since 6.3.
If the problem persists, we could check it.

Markus
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user
Markus Neteler

Re: Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
In reply to this post by John Tate-2
Hi John,

On Sat, Oct 31, 2009 at 12:39 PM, John Tate <[hidden email]> wrote:

> Hi Kaipi,
>
> I've only been doing rural LiDAR processing, so can't help regards building
> removal for the DTM.
>
> One thing that isn't mentioned in the Brovelli, 2004 article or the online
> documention on the wiki, is that spline 'step' relates to metres (1 step =
> 1m). Obvious I know, but wasn't to me when I started fiddling with the
> parametersand the LiDAR data for the first time. Therefore:
> -see and sen parameter in v.lidar.edgedetection to 400
> is 400m! and hence:
> -the detected edges are now so wide, that they cover complete buildings
>
> The spline step parameters should be double the average distance between
> points. So for 50pts/m² the spline steps chould be 0.04 (2*(1/50)) for
> example. Play with these either side of 0.04 in the parameters until you get
> a good result. These modules were designed a few years ago now, and not
> really designed for such high resolution data, though should still work.
> Though I am not entirely sure what you are attempting to do (e.g. purpose of
> data: remove building for DTM/ or create 3D model of the building DSM).
>
> You will find that v.outlier is the biggest restriction on how many points
> you can process at one time. I was working with roughly 1.5pts/m² and found
> that I had to work in 1km tiles due to the v.outlier restriction. Something
> to be wary of when you decrease the spline step incase you start getting
> errors.
>
> For reference, I was using:
> GRASS GIS (v.6.4.0svn, library Revision: 37101 (2009-05-10)) (windows
> version)

These comments are very intesting!

(Un)related: I have started a small, incomplete, Lidar tutorial:
http://grass.osgeo.org/wiki/LIDAR#Micro-tutorial_for_LIDAR_data_analysis

It would be great if you could integrate your comments in this Wiki
page (Wiki syntax is pretty easy) for the benefit of other users. In
the mailing list archive things get easily "lost".

thanks
Markus
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user
kaipi

Re: Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
Thanks for your help ! Unfortunately I still don't get satisfying results with the Lidar tools. Here is what I have done so far:

- Updated to 6.4 on Windows: v.outlier doesn't work (see my post in Win Grass forum)
- Installed Grass 6.4 RC 5 on Ubuntu (Ubuntu package): same results like on Windows Grass 6.3
- Decreased spline steps to 0.04: v.outlier and v.lidar.edgedetection give errors. My results are better the higher the spline steps are (> 20). Brovelli recommends spline step 3 up to 4 times of the planimetric resolution. AFAIK planimetric resolution means points per m². If I have 50 pts/m² this would mean a spline step of 200 ?? With spline steps about 20 I get nice edges and with spline step of 400 complete buildings are detected as edges.
- Tried the Lidar demo data (nc_spm_08_2008_may18) and followed the instructions on the Micro-tutorial_for_LIDAR_data_analysis: v.lidar.edgedetection took so long, that I canceled process after 1 h (input file has only 2000-3000 points - so something must be wrong)

I have also a problem to understand the definitions of first and last return data. Here is a part of my Lidar data (ASCII format converted from raw LIdar binary):

0 lon lat height intensity last first
1 3360422.27 5825275.62   37.16 255 1 1
2 3360421.94 5825275.60   38.21  56 0 1
3 3360422.48 5825275.60   37.18 255 1 0
4 3360422.17 5825275.58   38.18 141 0 1
5 3360422.65 5825275.58   37.26 255 1 0
6 3360422.84 5825275.56   37.31 255 1 1
7 3360423.05 5825275.54   37.32 255 1 1
8 3360423.36 5825275.52   37.13 255 1 1
9 3360423.06 5825275.50   38.11  12 0 1
10 3360423.60 5825275.50   37.09 255 1 0
11 3360423.70 5825275.49   37.32 255 1 1

First and last returns can be true (1) or false (0) or both. My understanding is, that all points that have first return=true are first returns and all returns that have last=true are last returns. When first and last are true we have a single pulse otherwise we have double pulses.
If I understand the microtutorial correctly I need for v.lidar.growing as input the last return single pulses (last=true and first=false) and as first all pulses with first=true.
But I get better results when I use as input all last return pulses (only last=true) and as input only the first single pulses (first=true and last=false).

I still have the problem that v.lidar.growing does not change categories. Output has only 2 categories - same as output of v.lidar.edgedetection.

I have attached two screenshots of my results so far (using the parameters in my first post). I have not yet run v.surf.bspline.

DSM - DTM

Classification after v.lidar.edgedetection:
Classification after v.lidar.edgedetection

Any ideas regarding v.lidar.growing ?

Thanks - Kaipi
John Tate-2

Re: Problem with Lidar Tools

Reply Threaded More More options
Print post
Permalink
Hi Kaipi,

Planimetric/horizontal resolution is essentially the end resolution of the
data (grid resolution).
For example:
If you have 2pt/m² (on average the distance between points is 0.5m (1/2)),
the end resolution is 0.5m.
So with 50pts/m², on average the distance between points is 1/50 = 0.02m.

You are absolutely right though (my mistake), Brovelli et al (2004) state 3
or 4 times the planimetric resolution for v.outlier and
v.lidar.edgedetection, so 0.08 (m) would be a favourable spline step in
their opinion, though not necessarily best for your data.

Have you tried using the r.in.xyz function on the first return data at the
grid resolution of your data (0.02m or 0.04m so there are fewer gaps in the
surface) to see what it looks like (maybe use v.surf.rst to fill any
remaining gaps (see micro-tutorial)). This will give you an indication of
the look of the detail that you have at the resolution of your data before
you try to remove the objects.

I would also recommend using the top left or bottom right of your data (an
area with a building), say a 20m x 20m area with the lower spline steps so
you don't get the errors. I found that roughly 2 million points was about as
much as the dbf driver could handle. However, I used (v.in.ascii -z -b -r ),
never built topology (-b) and allowed it to create a table in points mode
(not using -t). You need to connect the database first (v6.4) to prevent the
error in later modules (ERROR: Unable to read name of database):

#connect database (has to be manually created). Do before importing data.

db.connect driver=dbf database='$GISDBASE/$LOCATION_NAME/$MAPSET/dbf/'



#and display (has to be a separate command and displayed or it does not
work! bug anyone?)

db.connect -p



I'm afraid I have not seen that type of LiDAR data before, so cannot help
there. I've only dealt with pure first and last return data (xyzixyzi).



Also, I cannot comment on the v.lidar.growing issue as I never looked into
the data tables. Hopefully Markus can help you there.


Hope this helps.

Cheers

John

----- Original Message -----
From: "kaipi" <[hidden email]>
To: <[hidden email]>
Sent: Sunday, November 01, 2009 7:20 PM
Subject: Re: [GRASS-user] Problem with Lidar Tools



Thanks for your help ! Unfortunately I still don't get satisfying results
with the Lidar tools. Here is what I have done so far:

- Updated to 6.4 on Windows: v.outlier doesn't work (see my post in Win
Grass forum)
- Installed Grass 6.4 RC 5 on Ubuntu (Ubuntu package): same results like on
Windows Grass 6.3
- Decreased spline steps to 0.04: v.outlier and v.lidar.edgedetection give
errors. My results are better the higher the spline steps are (> 20).
Brovelli recommends spline step 3 up to 4 times of the planimetric
resolution. AFAIK planimetric resolution means points per m². If I have 50
pts/m² this would mean a spline step of 200 ?? With spline steps about 20 I
get nice edges and with spline step of 400 complete buildings are detected
as edges.
- Tried the Lidar demo data (nc_spm_08_2008_may18) and followed the
instructions on the Micro-tutorial_for_LIDAR_data_analysis:
v.lidar.edgedetection took so long, that I canceled process after 1 h (input
file has only 2000-3000 points - so something must be wrong)

I have also a problem to understand the definitions of first and last return
data. Here is a part of my Lidar data (ASCII format converted from raw LIdar
binary):

0 lon lat height intensity last first
1 3360422.27 5825275.62   37.16 255 1 1
2 3360421.94 5825275.60   38.21  56 0 1
3 3360422.48 5825275.60   37.18 255 1 0
4 3360422.17 5825275.58   38.18 141 0 1
5 3360422.65 5825275.58   37.26 255 1 0
6 3360422.84 5825275.56   37.31 255 1 1
7 3360423.05 5825275.54   37.32 255 1 1
8 3360423.36 5825275.52   37.13 255 1 1
9 3360423.06 5825275.50   38.11  12 0 1
10 3360423.60 5825275.50   37.09 255 1 0
11 3360423.70 5825275.49   37.32 255 1 1

First and last returns can be true (1) or false (0) or both. My
understanding is, that all points that have first return=true are first
returns and all returns that have last=true are last returns. When first and
last are true we have a single pulse otherwise we have double pulses.
If I understand the microtutorial correctly I need for v.lidar.growing as
input the last return single pulses (last=true and first=false) and as first
all pulses with first=true.
But I get better results when I use as input all last return pulses (only
last=true) and as input only the first single pulses (first=true and
last=false).

I still have the problem that v.lidar.growing does not change categories.
Output has only 2 categories - same as output of v.lidar.edgedetection.

I have attached two screenshots of my results so far (using the parameters
in my first post). I have not yet run v.surf.bspline.

http://n2.nabble.com/file/n3928180/lidar_before_after.jpg

Classification after v.lidar.edgedetection:
http://n2.nabble.com/file/n3928180/edgedetected.jpg

Any ideas regarding v.lidar.growing ?

Thanks - Kaipi

--
View this message in context:
http://n2.nabble.com/Problem-with-Lidar-Tools-tp3918684p3928180.html
Sent from the Grass - Users mailing list archive at Nabble.com.

Hi Kaipi,

I've only been doing rural LiDAR processing, so can't help regards building
removal for the DTM.

One thing that isn't mentioned in the Brovelli, 2004 article or the online
documention on the wiki, is that spline 'step' relates to metres (1 step =
1m). Obvious I know, but wasn't to me when I started fiddling with the
parametersand the LiDAR data for the first time. Therefore:
-see and sen parameter in v.lidar.edgedetection to 400
is 400m! and hence:
-the detected edges are now so wide, that they cover complete buildings

The spline step parameters should be double the average distance between
points. So for 50pts/m² the spline steps chould be 0.04 (2*(1/50)) for
example. Play with these either side of 0.04 in the parameters until you get
a good result. These modules were designed a few years ago now, and not
really designed for such high resolution data, though should still work.
Though I am not entirely sure what you are attempting to do (e.g. purpose of
data: remove building for DTM/ or create 3D model of the building DSM).

You will find that v.outlier is the biggest restriction on how many points
you can process at one time. I was working with roughly 1.5pts/m² and found
that I had to work in 1km tiles due to the v.outlier restriction. Something
to be wary of when you decrease the spline step incase you start getting
errors.

For reference, I was using:
GRASS GIS (v.6.4.0svn, library Revision: 37101 (2009-05-10)) (windows
version)


----- Original Message -----
From: "kaipi" <[hidden email]>
To: <[hidden email]>
Sent: Friday, October 30, 2009 12:43 PM
Subject: [GRASS-user] Problem with Lidar Tools



Dear list,

I am quite new to Grass and need to generate a DSM from Lidar data. Here is
my workflow so far:

v.in.ascii input=f:\lidar_test\single.csv output=single fs=, x=1 y=2 z=3 -z
-b -t --o
v.build single
g.region vect=single
v.in.ascii input=f:\lidar_test\first.csv output=first fs=, x=1 y=2 z=3 -z -b
-t --o
v.outlier input=single output=inlier_s outlier=outlier_s thres_o=5 soe=1
son=1 lambda_i=0.1 --o
v.outlier input=first output=inlier_f outlier=outlier_f thres_o=5 soe=1
son=1 lambda_i=0.1 --o
v.lidar.edgedetection input=inlier_s output=edgedetected see=400 sen=400
lambda_g=0.01 tgh=6 tgl=3 theta_g=0.26 lambda_r=1 --o
v.lidar.growing input=edgedetected output=growing first=inlier_f  tj=0.25
td=0.6 --o
v.lidar.correction input=growing output=classified terrain=terrain sce=5
scn=5 lambda_c=1 tch=2 tcl=1 --o
v.lidar.correction input=classified output=classified_it2
terrain=terrain_it2 sce=5 scn=5 lambda_c=1 tch=2 tcl=1 --o
v.build terrain_it2
v.out.ogr in=terrain_it2 dsn=F:\lidar_test\terrain_it2 format=ESRI_Shapefile
type=point lco=SHPT=POINTZ -e

v.lidar.edgedetection classifies the data into cat 1 and cat 2 (=Terrain or
Edge ?). This seams to work. The next step is v.lidar.growing. This tool
should classify into 4 categories: TERRAIN SINGLE PULSE, TERRAIN DOUBLE
PULSE, OBJECT SINGLE PULSE or OBJECT DOUBLE PULSE. But this does not happen.
The cats still remain the same (cat1 and cat2) and the output looks
identical to the output from v.lidar.edgedetection. v.lidar.growing seams to
have no effect.
Am I doing something wrong ?

I have another problem with the parameters of the lidar tools (although I
read Brovellis "Managing and processing LIDAR data within GRASS"). Can you
give me some hints how to adjust the default values to very high res Lidar
data (=50 points/m²). I raised see and sen parameter in
v.lidar.edgedetection to 400 with the result, that the detected edges are
now so wide, that they cover complete buildings. This way I can get rid of
the most buildings but the result is not really satisfying and I guess the
problem is false classification of v.lidar.growing

Let me know if you need more infos.

I am using GRass 6.3.0 on Windows. Can I expect improvements switching to
6.4 or switching to Linux ?

Thank you very much !!

kaipi


--
View this message in context:
http://n2.nabble.com/Problem-with-Lidar-Tools-tp3918684p3918684.html
Sent from the Grass - Users mailing list archive at Nabble.com.
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user