AVIRIS data in GRASS

[Dylan Beaudette-2]

Hi,

I have been searching mailing lists for a while, and have not been
able to discover any approaches for working with AVIRIS data in GRASS.
Does anyone on the list know how to either convert or extract data as
delivered in the "free data" section [1] of the AVIRIS home page? I
have tried working with the "radiance" data, which when uncompressed
comes with several .img files-- however my copy of GDAL does not know
what to make of it. It looks like it may be possible [2] to import the
data using r.in.bin... I was not able to get this working.



1. http://aviris.jpl.nasa.gov/html/aviris.freedata.html
2. http://n2.nabble.com/AVIRIS---grass-td1864545.html

Thanks!

Dylan
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Michael Perdue]

I don't think you can use r.in.bin because there is no option to  
specify the number of bands and how those bands are interleaved. In  
the past I've used Multispec and Matlab to read the data but you might  
be able to manipulate GDAL to read it. If you can create a header file  
for a generic binary file that will provide GDAL with the number of  
rows, columns, byte order, #channels, # of bytes/sample and the  
interleave method then you can tell GDAL how to read it. Unfortunately  
I can't hit the GDAL website right now so I can't tell if any of the  
generic binary formats allow for all those definitions.

Cheers,

Mike

PS. Also note that none of the tiles provided in the "free" area have  
been geometrically rectified and some have severe distortions due to  
aircraft dynamics.

On 5-Jul-09, at 3:49 PM, Dylan Beaudette wrote:

> Hi,
>
> I have been searching mailing lists for a while, and have not been
> able to discover any approaches for working with AVIRIS data in GRASS.
> Does anyone on the list know how to either convert or extract data as
> delivered in the "free data" section [1] of the AVIRIS home page? I
> have tried working with the "radiance" data, which when uncompressed
> comes with several .img files-- however my copy of GDAL does not know
> what to make of it. It looks like it may be possible [2] to import the
> data using r.in.bin... I was not able to get this working.
>
>
>
> 1. http://aviris.jpl.nasa.gov/html/aviris.freedata.html
> 2. http://n2.nabble.com/AVIRIS---grass-td1864545.html
>
> Thanks!
>
> Dylan
> _______________________________________________
> grass-user mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/grass-user

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[hamish-2]

Michael Perdue wrote:
> I don't think you can use r.in.bin
> because there is no option to specify the number of bands
> and how those bands are interleaved. In the past I've used
> Multispec and Matlab to read the data but you might be able
> to manipulate GDAL to read it.

if you have a small m-file to load it and save in a format
ready for GRASS's r.in.mat it could be useful to post. Users
without Matlab could always use GNU Octave to do the job.

also I suspect Python can do binary reads in a clean way, which
may help if the expected format was known.


Hamish



     

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Andrew MacIntyre-2]

Hamish wrote:
> also I suspect Python can do binary reads in a clean way, which
> may help if the expected format was known.

Python's struct and array modules are a good starting point for
handling binary data.

Numpy (http://numpy.scipy.org/, can be built with NetCDF support)
is a third party module of considerable utility.

Andrew.

-------------------------> "These thoughts are mine alone!" <---------
Andrew MacIntyre           National Licensing and Allocations Branch
tel:   [phone]     Inputs to Industry Division
fax:   [phone]     Australian Communications & Media Authority
email: [hidden email]            http://www.acma.gov.au/ 

If you have received this email in error, please notify the sender immediately and erase all copies of the email and any attachments to it. The information contained in this email and any attachments may be private, confidential and legally privileged or the subject of copyright. If you are not the addressee it may be illegal to review, disclose, use, forward, or distribute this email and/or its contents.
 
Unless otherwise specified, the information in the email and any attachments is intended as a guide only and should not be relied upon as legal or technical advice or regarded as a substitute for legal or technical advice in individual cases. Opinions contained in this email or any of its attachments do not necessarily reflect the opinions of ACMA.
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Markus Neteler]

On Sun, Jul 5, 2009 at 11:49 PM, Dylan
Beaudette<[hidden email]> wrote:
> Hi,
>
> I have been searching mailing lists for a while, and have not been
> able to discover any approaches for working with AVIRIS data in GRASS.
> Does anyone on the list know how to either convert or extract data as
> delivered in the "free data" section [1] of the AVIRIS home page? I
> have tried working with the "radiance" data, which when uncompressed
> comes with several .img files-- however my copy of GDAL does not know
> what to make of it.

I would try to create a VRT file
http://www.gdal.org/gdal_vrttut.html

Then it might be readable with r.in.gdal. The GDAL list might be of help if
needed (please keep us posted!).

Markus
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Michael Perdue]

Some javascript/style in this post has been disabled (why?)

On 5-Jul-09, at 9:29 PM, Hamish wrote:

Michael Perdue wrote:

I don't think you can use r.in.bin

because there is no option to specify the number of bands

and how those bands are interleaved. In the past I've used

Multispec and Matlab to read the data but you might be able

to manipulate GDAL to read it.

if you have a small m-file to load it and save in a format
ready for GRASS's r.in.mat it could be useful to post. Users
without Matlab could always use GNU Octave to do the job.

also I suspect Python can do binary reads in a clean way, which
may help if the expected format was known.

Unfortunately, my computer and all my backup drives were stolen in a home invasion about 6 months ago so if I had made a m-file, it's gone. However, it's fairly straight forward in matlab/octave.

I just did;

fid = fopen ('f960705t01p02_r05_sc01.c.img','r');

AVIRIS = int16(fread(fid, 'int16', 'ieee-be'));

fclose(fid)

AVIRIS2 = reshape (AVIRIS, 224,614,512);

AVIRIS3 = permute (AVIRIS2,[3 2 1]);

This created the correct output on my system. Of course this assumes that the original file has 224 bands, 614 columns and 512 lines. That should be true for all except the last tile in every flight line which will be less that 512 lines. I'd put more effort into writing an m-file for these files but I don't think it's of a lot of value unless you also read in the *.nav files and geometrically correct the files at the same time. As I've mentioned, the distortions due to aircraft dynamics can be rather extreme. Unfortunately, that is a lot less trivial. Attached is an example from one of the sample datasets.

Cheers,

Mike

_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Michael Perdue]

I was able to generate a ENVI style header that will allow you to read  
the files with GDAL. Modify as necessary and you should be able to  
work with the *.img files.


On 5-Jul-09, at 6:56 PM, Michael Perdue wrote:

> ... you might be able to manipulate GDAL to read it. If you can  
> create a header file for a generic binary file that will provide  
> GDAL with the number of rows, columns, byte order, #channels, # of  
> bytes/sample and the interleave method then you can tell GDAL how to  
> read it.
>
> On 5-Jul-09, at 3:49 PM, Dylan Beaudette wrote:
>>  I
>> have tried working with the "radiance" data, which when uncompressed
>> comes with several .img files-- however my copy of GDAL does not know
>> what to make of it. It looks like it may be possible [2] to import  
>> the
>> data using r.in.bin... I was not able to get this working.
>>

Cheers,

Mike





_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[Dylan Beaudette]

On Monday 06 July 2009, Michael Perdue wrote:
> I was able to generate a ENVI style header that will allow you to read
> the files with GDAL. Modify as necessary and you should be able to
> work with the *.img files.

Fantastic. Thanks for the tips Mike. I will give this a try. For the record,
Frank W. over on the GDAL mailing list suggested the code samples below. A
script could be used to generate a single file for all 224 bands.

# make one of these for the entire set of data
# add more VRTRasterBand elements for the bands you want:
#
<VRTDataset rasterXSize="614" rasterYSize="512">
   <VRTRasterBand dataType="Int16" band="1" subClass="VRTRawRasterBand">
     <SourceFilename
relativetoVRT="1">f970619t01p02_r02_sc01.c.img</SourceFilename>
     <ImageOffset>0</ImageOffset>
     <PixelOffset>448</PixelOffset>
     <LineOffset>275072</LineOffset>
     <ByteOrder>MSB</ByteOrder>
   </VRTRasterBand>
</VRTDataset>

# multi-band example:
<VRTDataset rasterXSize="614" rasterYSize="512">
   <VRTRasterBand dataType="Int16" band="1" subClass="VRTRawRasterBand">
     <SourceFilename
relativetoVRT="1">f970619t01p02_r02_sc01.c.img</SourceFilename>
     <ImageOffset>0</ImageOffset>
     <PixelOffset>448</PixelOffset>
     <LineOffset>275072</LineOffset>
     <ByteOrder>MSB</ByteOrder>
   </VRTRasterBand>
   <VRTRasterBand dataType="Int16" band="2" subClass="VRTRawRasterBand">
     <SourceFilename
relativetoVRT="1">f970619t01p02_r02_sc01.c.img</SourceFilename>
     <ImageOffset>2</ImageOffset>
     <PixelOffset>448</PixelOffset>
     <LineOffset>275072</LineOffset>
     <ByteOrder>MSB</ByteOrder>
   </VRTRasterBand>
</VRTDataset>

This works well in QGIS-- and I imagine in GRASS as well. The tricky part from
here is applying the rectification and "navigation" parameters...

Cheers,
Dylan

> On 5-Jul-09, at 6:56 PM, Michael Perdue wrote:
> > ... you might be able to manipulate GDAL to read it. If you can
> > create a header file for a generic binary file that will provide
> > GDAL with the number of rows, columns, byte order, #channels, # of
> > bytes/sample and the interleave method then you can tell GDAL how to
> > read it.
> >
> > On 5-Jul-09, at 3:49 PM, Dylan Beaudette wrote:
> >>  I
> >> have tried working with the "radiance" data, which when uncompressed
> >> comes with several .img files-- however my copy of GDAL does not know
> >> what to make of it. It looks like it may be possible [2] to import
> >> the
> >> data using r.in.bin... I was not able to get this working.
>
> Cheers,
>
> Mike

--
Dylan Beaudette
Soil Resource Laboratory
http://casoilresource.lawr.ucdavis.edu/
University of California at Davis
530.754.7341
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user

[hamish-2]

Dylan wrote:
> The tricky part from here is applying the rectification and
> "navigation" parameters...

perhaps something from the MB-System code could help?

http://www.ldeo.columbia.edu/res/pi/MB-System/

it is essentially exactly what that software does (rectification of swath bathymetry from a ships nav stream), in another medium.
It's GPL.


Hamish



     
_______________________________________________
grass-user mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/grass-user