Shell scripting for arcsi

[Alexandra Mates]

I'm trying to do some exciting shell scripting in Ubuntu 14.04.

I'd like to catch all the MTL files in a folder containing multiple Landsat folders and pass them through a manual aot arcsi correction, with multiple a.o.t. values. Something like the script below. I know it may be very crude, I barely have and shell scripting experience. What I ultimately want is to avoid writing 10 lines for 10 aot values, but also to run the multiple Landsat folders in parallel.
I inspired myself from https://spectraldifferences.wordpress.com/2013/08/18/gnu-parallel/#respond.
The error I get for the script below is: line 7: [: 0.05: integer expression expected
Thank you for any suggestions!



#!/bin/bash -x

cd /home/ktp/Documents/CaseStudy3/Imagery/Landsat/arcsi/01_RAW/

aot=0.05

while [ $aot -lt 1.0 ]; do
    find -name *MTL.txt | parallel arcsi.py -s ls8 -f KEA --stats -p RAD SREFSTDMDL --aeropro Maritime --atmospro MidlatitudeSummer --aot $aot --outbasename {.}_$aot.kea -o ../03_ARCSI_aot_manual -i {} > verbose2.txt
    aot=$((aot+0.1))
done

[Daniel Clewley]

Hi Alex,

I would use Python over shell scripts. Arguably for some tasks you end up writing more code but I personally think the syntax is much nicer. You can call command line tools using the subprocess module (https://docs.python.org/3/library/subprocess.html)

For example:

import subprocess

subprocess.call('find -name *MTL.txt | parallel arcsi.py -s ls8 -f KEA --stats -p RAD SREFSTDMDL --aeropro Maritime --atmospro MidlatitudeSummer --aot ' + aot + '--outbasename {.}_$aot.kea -o ../03_ARCSI_aot_manual -i {} > verbose2.txt',shell=True)

There is also a tool with ARCSI to generate a shell script with the ARCSI command for all files in a directory. You could write a Python script to call this for different AOT values (either the command line tool or Python function) and then use GNU Parallel to run everything in parallel using:

cat arcsi_commands.sh | parallel -n 4

Note the n parameter to specify a maximum of 4 cores. As ARCSI does a lot of file reading / writing you might find if you try to run to many instances at once how fast you can read and write data to the disk becomes a limiting factor. 

In response you your original question, I think you need to use integers in your while loop, so a bit hacky but something like the following should work.

aot=5

while [ $aot -lt 100 ]; do
  aotfloat=`echo "scale=2; $aot / 100" | bc`
  echo $aotfloat
  aot=$(($aot+10))
done

Thanks,

Dan

On 17 Oct 2014, at 11:26, Alexandra Mates <alexand...@envsys.co.uk> wrote:

I'm trying to do some exciting shell scripting in Ubuntu 14.04.

I'd like to catch all the MTL fles in a folder containing multiple Landsat folders and pass them trhough a manual aot arcsi correction, with multiple a.o.t. values. Something like the script below. I know it may be very crude, I barely have and shell scripting experience. What I ultimately want is to avoid writing 10 lines for 10 aot values, but also to run the multiple Landsat folders in parallel.

I inspired myself from https://spectraldifferences.wordpress.com/2013/08/18/gnu-parallel/#respond.
The error I get for the script below is: line 7: [: 0.05: integer expression expected
Thank you for any suggestions!



#!/bin/bash -x

cd /home/ktp/Documents/CaseStudy3/Imagery/Landsat/arcsi/01_RAW/

aot=0.05

while [ $aot -lt 1.0 ]; do
    find -name *MTL.txt | parallel arcsi.py -s ls8 -f KEA --stats -p RAD SREFSTDMDL --aeropro Maritime --atmospro MidlatitudeSummer --aot $aot --outbasename {.}_$aot.kea -o ../03_ARCSI_aot_manual -i {} > verbose2.txt
    aot=$((aot+0.1))
done

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at http://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.