HTCondor / SciTokens updates

[Brian Bockelman]

Hi all,

The HTCondor team has been cooking up a few nice improvements to the file transfer subsystem.  A few are directly applicable to SciTokens!

In particular:

1.  You can now use file transfer plugins for output files.

2.  You can now specify the credential to use for the HTTPS transfer.

The second item is precisely what we discussed in January -- prefix the URL's scheme with the token name.  So, transferring via HTTPS with a scitoken would be done using "scitokens+https://".  I was able to test this by combining HTCondor, scitokens-credmon, and xrootd.

Here's an example submit file:

Executable = test.sh

Arguments = 5

Universe = vanilla

output = test.out

error = test.err

Log = test.log

should_transfer_files = always

transfer_input_files = scitokens+https://server.example.com/xrootd_export/hello_world

transfer_output_files = hello_world.txt

transfer_output_remaps = "hello_world.txt = scitokens+https://server.example.com/xrootd_export/foo"

Queue

Enjoy!

Brian