ARCSI parameter fullimgouts problem

[PI TI]

Hi all ARCSI geeks.

I am a new user to ARCSI so I would like to congratulate everyone get involved to the project for the great work.

Now to the point,  I am facing a problem I would like to share. (I am using arcsi version 3.2.4) 

I run the following command for landsat8 images:

arcsi.py -s ls8 -f KEA --stats --fullimgouts -p RAD SREF FOOTPRINT --aeropro Maritime --atmospro MidlatitudeSummer --aot 0.25 -o D:\\GDAL_DATA\\IMG_SAMPLE\\L8\\LC08_L1TP_183032_20190821_20190903_01_T1\\ARCSI_IMG -i D:\\GDAL_DATA\\IMG_SAMPLE\\L8\\LC08_L1TP_183032_20190821_20190903_01_T1\\LC08_L1TP_183032_20190821_20190903_01_T1_MTL.txt

Everything work as expected and I got my images. However I have noticed that a mask is applied to the final product which I can not really underastand why this mask is applied.

So my first question is what is this mask?????

Here is screenshot to understand the problem: (I do place on the background the original image and spot with red arrows the masked( cropped ) areas.

Then I realize that using the --fullimgouts parameter should not delete the unmasked image.

Unfotunatlly this didnt work. No matter how hard a tried everytime my unmasked image was deleted.

So going through the code of arcsirun.py file I found that --fullimgouts parameter is not used when deleting the unmasked image file.

So I maanged to overcome the issue using the following workaround

I did a small change to your code on line 778

I have changed this block:

rsgisUtils.deleteFileWithBasename(paramsObj.radianceImage

To this:

if not paramsObj.fullimgouts:    
    rsgisUtils.deleteFileWithBasename(paramsObj.radianceImage)

So now whenever I pass the --fullimgouts argument in my command It does what I expect it to do , meaning it doesnt delete the unmasked image.


I am not sure whether this is a bug or something I can not get clear.

If anyone knows something more please let me know.


So far I can keep doing my work with your great arcsi product without any problems.


But if someone else is facing the same problem he might use my workaround.

Thanks




[Peter Bunting]

Hi, 

The areas removed are based on the view angle. The cut is made as Landsat has some edge effects between the different bands, you can see it really clearly in Landsat 5 and 7 not so much in Landsat 8. This masking is done to ensure the resulting image files have fully valid data, so it is intentional rather than a bug. 

The code which is creating this mask is in the generateValidImageDataMask function within the arcsisensorlandsat8.py file. 

If this causing a problem for your application?

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743
* Email: p...@aber.ac.uk
* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

On 28 Jan 2020, at 13:38, PI TI <p.ts...@gmail.com> wrote:

Hi all ARCSI geeks.

I am a new user to ARCSI so I would like to congratulate everyone get involved to the project for the great work.

Now to the point,  I am facing a problem I would like to share. (I am using arcsi version 3.2.4) 

I run the following command for landsat8 images:

arcsi.py -s ls8 -f KEA --stats --fullimgouts -p RAD SREF FOOTPRINT --aeropro Maritime --atmospro MidlatitudeSummer --aot 0.25 -o D:\\GDAL_DATA\\IMG_SAMPLE\\L8\\LC08_L1TP_183032_20190821_20190903_01_T1\\ARCSI_IMG -i D:\\GDAL_DATA\\IMG_SAMPLE\\L8\\LC08_L1TP_183032_20190821_20190903_01_T1\\LC08_L1TP_183032_20190821_20190903_01_T1_MTL.txt

Everything work as expected and I got my images. However I have noticed that a mask is applied to the final product which I can not really underastand why this mask is applied.

So my first question is what is this mask?????

Here is screenshot to understand the problem: (I do place on the background the original image and spot with red arrows the masked( cropped ) areas.

<Auto Generated Inline Image 1.png>

Then I realize that using the --fullimgouts parameter should not delete the unmasked image.

Unfotunatlly this didnt work. No matter how hard a tried everytime my unmasked image was deleted.

So going through the code of arcsirun.py file I found that --fullimgouts parameter is not used when deleting the unmasked image file.

So I maanged to overcome the issue using the following workaround

I did a small change to your code on line 778

I have changed this block:

rsgisUtils.deleteFileWithBasename(paramsObj.radianceImage

To this:

if not paramsObj.fullimgouts:    
    rsgisUtils.deleteFileWithBasename(paramsObj.radianceImage)

So now whenever I pass the --fullimgouts argument in my command It does what I expect it to do , meaning it doesnt delete the unmasked image.


I am not sure whether this is a bug or something I can not get clear.

If anyone knows something more please let me know.


So far I can keep doing my work with your great arcsi product without any problems.


But if someone else is facing the same problem he might use my workaround.

Thanks




--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/fb540ae7-439a-4756-9ad8-758abfc1fbb1%40googlegroups.com.
<Auto Generated Inline Image 1.png>

[PI TI]

Hi Pete, thanks for the quick responce and the explanation about the masks applied.

As I have to create a mosaic of Atmosperic corrected landsat images, this mask will probably 'cause problems.

To be honest I have only try with a single set of images so long. So I am not 100% sure if the mask will 'cause me problems.

I will try to preview neighboring images using the "masked" outputs.

I ll also try the to edit the .py file you suggest and remove the mask at this point.

Thanks again for you help, really appreciate it.




Pavlos Tsagkis

On Tuesday, 28 January 2020 17:17:42 UTC+2, Peter Bunting wrote:

Hi, 

The areas removed are based on the view angle. The cut is made as Landsat has some edge effects between the different bands, you can see it really clearly in Landsat 5 and 7 not so much in Landsat 8. This masking is done to ensure the resulting image files have fully valid data, so it is intentional rather than a bug. 

The code which is creating this mask is in the generateValidImageDataMask function within the arcsisensorlandsat8.py file. 

If this causing a problem for your application?

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743

* Email...@aber.ac.uk

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib...@googlegroups.com.

[Peter Bunting]

Dear Pavlos, 

I have never had a problem with this mask causing images not to overlap. I regularly put images through ARCSI and then mosaic / composite them. From my experience, it is a bigger problem to have invalid pixels on the edges of images. However, it you do find this causes gaps please let me know and I’ll look to see if it could be resolved.

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743

* Email: p...@aber.ac.uk

* ORCID: http://orcid.org/0000-0002-7435-0148
****************************************************

"Please consider the environment before printing this email or any documents attached”

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rsgislib-support/1f04fcdc-7a99-42af-b200-8008bb3525f6%40googlegroups.com.

[PI TI]

Hi Pete, I ll give a try to mosaic the "masked" images and observe the results.

I will come back to share the result.

Thanks for the information.

Pavlos

On Wednesday, 29 January 2020 00:39:12 UTC+2, Peter Bunting wrote:

Dear Pavlos, 

I have never had a problem with this mask causing images not to overlap. I regularly put images through ARCSI and then mosaic / composite them. From my experience, it is a bigger problem to have invalid pixels on the edges of images. However, it you do find this causes gaps please let me know and I’ll look to see if it could be resolved.

Best wishes, 

Pete

****************************************************
* Dr Pete Bunting
* Reader in Remote Sensing
* Earth Observation and Ecosystem Dynamics Group
* Department of Geography and Earth Sciences
* Aberystwyth University
* Aberystwyth
* Ceredigion
* SY23 3DB
* UK
* 
* Ph: +44 (0) 1970 622615
* Mob: +44 (0) 7917 842743

* Email...@aber.ac.uk

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsub...@googlegroups.com.