Batching question

14 views
Skip to first unread message

Emily Lapworth

unread,
Apr 8, 2026, 4:20:31 PM (14 days ago) Apr 8
to DART User Group
Hello, I'm trying to run a batch job without uploading. Is that possible? I created a job with BagIt package format, an empty profile, and .tar serialization, with an output path specified but without an uploading step and then saved it as a workflow. I created a CSV file with 2 columns for bag-name and root-directory. I am not getting any errors from DART and it says all jobs have completed but no tar bags are created anywhere I can find. I can do the steps I want as a single job and it works successfully, but would love to be able to do the same steps as a batch if possible.

Thanks,
Emily

Nathan Tallman

unread,
Apr 8, 2026, 4:44:17 PM (14 days ago) Apr 8
to Emily Lapworth, DART User Group
Hi Emily,

Yes, this should be possible. When configuring a workflow, there isn't an output path field because it assumes you will upload bags. So when running a batch workflow without uploads, it will save the resulting bags in your default bagging location. Go to Settings --> Application Settings and see the Bagging Directory field with its path. Your bags should be there.

Let us know if you have problems!

Thanks,
Nathan



--
You received this message because you are subscribed to the Google Groups "DART User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dart-users+...@aptrust.org.
To view this discussion visit https://groups.google.com/a/aptrust.org/d/msgid/dart-users/28f07b7f-8400-4759-9afc-3a3e6ef2acd2n%40aptrust.org.

Joshua Allan Westgard

unread,
Apr 8, 2026, 5:07:12 PM (14 days ago) Apr 8
to Nathan Tallman, Emily Lapworth, DART User Group
Hi Emily,

We have a workflow where we bag content, do some analysis of the bag, and only then upload the content to APTrust. We use DART-runner for this, and I have a workflow.json file that simply has the “storageServices” key has an empty list (i.e. no services are defined).

Then we run the command like this:

> $ bin/dart-runner
> --workflow=workflows/prod-no-upload.json
> --output-dir=bags
> --delete=false < jobs/<batch>.json > tickets/<ticket dir>/dart-runner.json

Because the workflow has no storage services, no uploads will occur, and there are command line options for specifying the output location and that the local tar file should be kept (I’m not sure that is necessary since maybe no delete event would be triggered without an upload).

Once we have completed our analysis of the bag, we upload it to the receiving bucket in a separate step. I have not tried running the same workflow via the DART GUI.

Josh Westgard
> To view this discussion visit https://groups.google.com/a/aptrust.org/d/msgid/dart-users/CAFdf%3DSJfEz%2BKoKLLeFc3pr9wdcndQZ-AyT52dWNDYZq%2Be%3DJYTQ%40mail.gmail.com.



--
Joshua A. Westgard, MLS, PhD (he/him/his)
Systems Librarian | Digital Programs and Initiatives | University of Maryland Libraries
Affiliate Faculty | College of Information Studies (iSchool)
McKeldin Library | 7649 Library Ln, College Park, MD 20742-7011
www.lib.umd.edu | west...@umd.edu | +1-301-405-9136 (office)

Joshua Allan Westgard

unread,
Apr 8, 2026, 5:28:05 PM (14 days ago) Apr 8
to Emily Lapworth, DART User Group, Nathan Tallman
PS: I should have mentioned, we have used the same command plus the ‘—batch' option pointing to a batch CSV to run this workflow on a batch of jobs, though for us the local storage space became a limiting factor.

Emily A Lapworth

unread,
Apr 9, 2026, 10:28:27 AM (13 days ago) Apr 9
to Joshua Allan Westgard, DART User Group, Nathan Tallman
Thank you both! I got it to work in the DART GUI, in creating the workflow I had to select BagIt as the package format. Before I was selecting .tar and that did not work. Selecting BagIt as the package format did create tar files though (which is the outcome I wanted). 
From: Joshua Allan Westgard <west...@umd.edu>
Sent: Wednesday, April 8, 2026 5:27 PM
To: Emily A Lapworth <Emily.L...@umb.edu>
Cc: DART User Group <dart-...@aptrust.org>; Nathan Tallman <nathan....@aptrust.org>
Subject: Re: [dart-users] Batching question
 
[You don't often get email from west...@umd.edu. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

CAUTION: EXTERNAL SENDER

>>
>> --
>> You received this message because you are subscribed to the Google Groups "DART User Group" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to dart-users+...@aptrust.org.

>
>
>
> --
> Joshua A. Westgard, MLS, PhD (he/him/his)
> Systems Librarian | Digital Programs and Initiatives | University of Maryland Libraries
> Affiliate Faculty | College of Information Studies (iSchool)
> McKeldin Library | 7649 Library Ln, College Park, MD 20742-7011

Nathan Tallman

unread,
Apr 9, 2026, 2:46:50 PM (13 days ago) Apr 9
to Emily A Lapworth, Joshua Allan Westgard, DART User Group
You're right that the workflow configuration in DART 2 is a little confusing; it's also a little buggy. It's more straightforward in DART 3, which is currently available as a beta at https://aptrust.github.io/dart-docs/dart3/download/. Here's a screenshot of the improved UI in DART 3.

image.png

Glad you got it working though!

Cheers,
Nathan



Reply all
Reply to author
Forward
0 new messages