How to set ISIS root

[Josh Williams]

Hello, I would love assistance using the Ames Stereo Pipeline Documentation 10.3.1 North Terra Meridiani page 91. The line entered is " Ames Stereo Pipeline Documentation

ISIS> mroctx2isis from=P02_001981_1823_XI_02N356W.IMG to=P02_001981_1823.cub" returns: "Please set ISISROOT before running any Isis applications". I am using anaconda3 not miniconda3. I went through the instructions line by line, but still have this error. I am clearly lost here, any help would be greatly appreciated. Thanks! 

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

> Hello, I would love assistance using the Ames Stereo Pipeline Documentation

Assistance will be offered, but it will not be cheap. 🙂

You need to set ISISROOT. The relevant blurb is here: https://stereopipeline.readthedocs.io/en/latest/installation.html?highlight=isisroot#fetching-pre-compiled-asp-with-conda

Since you say you used anaconda, not miniconda, you need to adjust appropriately the path where it installedthingss. Could be: 

export ISISROOT=$HOME/anaconda3/envs/asp

but I am guessing here.

Then you will also need to set ISISDATA, and actually download MRO data, as discussed later on that page.

---

From: ames-stereo-pi...@googlegroups.com <ames-stereo-pi...@googlegroups.com> on behalf of Josh Williams <jos...@gmail.com>
Sent: Thursday, February 3, 2022 7:48 PM
To: Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Subject: [EXTERNAL] How to set ISIS root

 

Hello, I would love assistance using the Ames Stereo Pipeline Documentation 10.3.1 North Terra Meridiani page 91. The line entered is " Ames Stereo Pipeline Documentation

ISIS> mroctx2isis from=P02_001981_1823_XI_02N356W.IMG to=P02_001981_1823.cub" returns: "Please set ISISROOT before running any Isis applications". I am using anaconda3 not miniconda3. I went through the instructions line by line, but still have this error. I am clearly lost here, any help would be greatly appreciated. Thanks! 

--
You received this message because you are subscribed to the Google Groups "Ames Stereo Pipeline Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ames-stereo-pipeline...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/3539e4f6-1d2d-4042-a405-437853513f74n%40googlegroups.com.

[Josh Williams]

Thank you! Will you accept trade? (LOL)

That suggestion worked!

How do I set the ISISDATA? I've downloaded the MRO data, but was only able to find .tiff files not ".IMG" for P02_001981_1823_XI_02N356W.tif and P03_002258_1817_XI_01N356W.tif. 

Best,

[Beyer, Ross A. (ARC-SST)[SETI INSTITUTE]]

Josh,

 

You can set ISISDATA the same way, but you will need to download the ISIS data files, detailed here: https://github.com/USGS-Astrogeology/ISIS3#the-isis-data-area

 

And instructions for how to set ISISDATA earlier in the document (and ISISROOT) so that they are set every time your “ASP” conda environment starts up.

 

However, that is just the data needed for ISIS to operate, you’ll need to download the data that you want to process on your own.  You’ve listed some filenames that appear to be MRO CTX images, but they end in .tif, which are not PDS data.  You’ll need to visit the PDS:

https://pds-imaging.jpl.nasa.gov/volumes/mro.html

 

And download the .IMG files that you’ll need to start with.

 

-- 

Ross A. Beyer

http://RossBeyer.net/science/

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/1e931744-48c8-406e-9cb5-f4c5f7f87633n%40googlegroups.com.

[Alexandrov, Oleg (ARC-TI)[KBR Wyle Services, LLC]]

To add to what Ross says, when you fetch ISIS data, you should read their doc at  https://github.com/USGS-Astrogeology/ISIS3#the-isis-data-area carefully. In particular, you need only "base" data and "mro" data. If you download the full thing, rather than just what you need, it may take a week and untold terabytes. 

Also note our doc here: https://stereopipeline.readthedocs.io/en/latest/examples.html#mars-reconnaissance-orbiter-ctx

---

From: 'Beyer, Ross A. (ARC-SST)[SETI INSTITUTE]' via Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>
Sent: Monday, February 7, 2022 8:15 AM
To: Josh Williams <jos...@gmail.com>; Ames Stereo Pipeline Support <ames-stereo-pi...@googlegroups.com>

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/86F95FCC-3BD4-4E4D-ABD5-60D9E25877EC%40nasa.gov.

[Josh Williams]

Ok I used the command: cd $ISISDATA
rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/mro . This was waaay too much data for my laptop to hold so it returned:

rsync: write failed on "/Users/joshuawilliams/mro/kernels/ck/mro_sc_psp_210727_210802.bc": No space left on device (28)

rsync: rename "/Users/joshuawilliams/mro/kernels/ck/.mro_sc_psp_210727_210802.bc.KEXArv" -> "mro/kernels/ck/mro_sc_psp_210727_210802.bc": No space left on device (28)

Did I miss something? I this actually downloading all data from MRO, because if that was the case there would definitely not be space. I would only need to download two MRO products (CTX or HiRISE stereo pairs) at a time. Then I could move the derived DTM/DEM to my other machine (windows based). Is there a way to do that without having to download all this data? Sorry for my confusion. Thank you very much for your help!

Best,

Josh

[Beyer, Ross A. (ARC-SST)[SETI INSTITUTE]]

Josh,

 

No, that’s just downloading the 195 GB of supporting data that ISIS needs to process actual images from MRO (that you still have to download separately).

 

If your laptop doesn’t have that kind of space, then it also likely does not have the horsepower to run stereo processing for HiRISE or CTX images.  Typical laptops can run stereo for small crops of these images, but they are unlikely to be able to handle stereo processing for the full images (although it is winter, so maybe you’re looking to convert your laptop into a very inefficient space heater).  You’re going to need to find a beefier system for this data processing, I’m afraid.

 

Although you *can* run ASP on a laptop, we do not recommend it.

 

You can always just download the “base” ISIS data, and then use “WEB=true” for your spiceinit step (which could keep you from having to download the “mro” support data).  However, not all ISIS processing programs can make use of the on-demand web-based SPICE data, so you may run into problems down the line.  Not sure.

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/b86ea545-24b5-449a-ad68-7d4bfc818e3fn%40googlegroups.com.

[Josh Williams]

Hello, thank you for that quick response. That is unfortunate that I cannot use the laptop. I will attempt to use it as you suggested. Just to make sure I am using the correct script:

To use the ISIS SPICE Web Service I added " --exclude='kernels' to the MRO path as follows:

rsync -azv --exclude='kernels' --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/mro .

Then use: WEB=true 

Then for the "base" ISIS data is use the following script: rsync -azv --delete --partial isisdist.astrogeology.usgs.gov::isisdata/data/base .

Does that look correct? I do have a "beefy" machine, but it is windows based. I don't mind this mac being a "heater" as I have no other use for it and don't mind it crunching overnight or days if need be.

Separate question, could I add another HardDrive to my beefy machine and instal linux on that? Or could I partition the 2 Tb SSD HD on the beefy machine to have a linux OS to run this? It is a very powerful machine with an extremely fast CPU and GPU. 

Thank you!

Best,

Josh 

[Beyer, Ross A. (ARC-SST)[SETI INSTITUTE]]

Josh,

 

You *can* use the laptop, you just won’t be real happy about it.  And I cannot stress enough how important it is to start on very small patches of an image to make sure everything is working, and only run “whole” CTX or HiRISE images when you’re reasonably certain about what the outcome will be.

 

I have a hard time following if I’m not doing it on my own machine, but that looks like a way to get what you want.  If they aren’t, you can always engage with the ISIS community to get straightened out on what your ISISDATA should look like.

 

I can’t really advise you on hardware as there are lots of ways to go about this.  We run ASP on the supercomputer here at Ames.  It can and has been run on linux clusters and various cloud instances.  I do have it installed on my laptop for some of its utilities, but I don’t do processing there, and I do have it installed on a stand-alone Linux machine.

 

You certainly can add additional hard drive(s) to an existing Windows PC and install Linux on it/them.  You’ll have to figure out how to boot into that other drive (there are bootloaders or even simple motherboard selection, that you’ll have to figure that out).  However, once you do, it is as easy as rebooting the system, picking the Linux OS, and then running processing on it as you can.  The GPU won’t do much for you for ISIS or ASP, but more CPU, memory, and HD space will serve you well.

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/d91db70f-dc4e-4a85-ad68-23cebef73182n%40googlegroups.com.

[Josh Williams]

Hello all, 

I have found a workaround! No longer using the laptop. I am using Docker in windows. I have another question for downloading CTX .IMG files from the PDS. Is there a more direct way to download MRO data from the PDS? I found this on GitHub:  edrget url=http://pdsimg.jpl.nasa.gov/data/mro/mars_reconnaissance_orbiter/ctx/mrox_0011/data/P02_001981_1823_XI_02N356W.IMG . However the link appears to be bad, do you have any suggestions on this? Best,

Josh

[Beyer, Ross A. (ARC-SST)[SETI INSTITUTE]]

Josh,

 

“edrget” was an ISIS program that has been deprecated, because tools like “curl” or “wget” do exactly the same thing (and have more options to boot).

 

If you have the URL for a PDS file, you can use curl or wget to obtain the data.

To view this discussion on the web visit https://groups.google.com/d/msgid/ames-stereo-pipeline-support/3cfdec87-9921-45d3-9883-75d94f71fa11n%40googlegroups.com.