bulk_put hundreds of thousands of rasters and filter subdirs - Eon Browser or java CLI?

[Andrew Ritchie]

Hi BlackPearl experts! I'm trying to archive several hundred thousand rasters in a directory heirarchy but NOT archive the whole heirarchy. I was hoping I could do something like this with the CLI:

./ds3 -c put_bulk -b data-archive -p sensordata/raw/ -d "/mnt/x/projects/aerialsurveys/" --filter-params contains:.NEF --filter-params contains:2023

where the two filter parameters function as AND filters on the source directory data. I've applied multiple criteria with AWS cli this way and it's been very useful. I get an error:

Error (UnrecognizedOptionException): Unrecognized option: --filter-params

and I'm wondering if there's a way I'm missing to use the java CLI to do a bulk_put where I could include only certain filetypes or directory branches OR if I can do something similar with the Eon browser. 

The goal is to avoid "shoe-shining" the tape while also not zip/tarring terabytes of files - to efficiently archive around a half-million of 40-60mb files while preserving their structure in the object heirarchy on tape storage.

Thanks for any advice/insight/criticism

Andy