--
David Juhasz Director, AtoM Technical Services Artefactual Systems Inc. www.artefactual.com
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c51b59a0-0681-4ea9-a40e-c4cd47556df0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c51b59a0-0681-4ea9-a40e-c4cd47556df0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--Jenny Mitcham
Digital Archivist
Borthwick Institute for ArchivesUniversity of York
Heslington
York
YO10 5DD
Telephone: 01904 321170
Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/Twitter: @Jenny_MitchamSkype: jenny_mitcham
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAF8JE%3Do18q4XYmVJM5RjrSkNBs5qsTFK04Q-mBP-bm9c3AYM%3Dg%40mail.gmail.com.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c51b59a0-0681-4ea9-a40e-c4cd47556df0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
----Jenny Mitcham
Digital Archivist
Borthwick Institute for ArchivesUniversity of York
Heslington
York
YO10 5DD
Telephone: 01904 321170
Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/Twitter: @Jenny_MitchamSkype: jenny_mitcham
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAF8JE%3Do18q4XYmVJM5RjrSkNBs5qsTFK04Q-mBP-bm9c3AYM%3Dg%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZL3RZKd_Euogw4R-4_Bo4HCpybwv0AOOQv6_x0Vxfp3Sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Yes sure.Our AtoM site had 4GB but we upped it to 8GB for the purposes of this exercise...and will put it back down again tomorrow!Cheers,Jen
On Wed, 27 Jun 2018 at 15:58, Dan Gillean <d...@artefactual.com> wrote:
Hi Jen,Thanks for sharing this! Are you willing to share how much memory was allocated for this, so we have a useful reference point for others?Regards,
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/c51b59a0-0681-4ea9-a40e-c4cd47556df0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
----Jenny Mitcham
Digital Archivist
Borthwick Institute for ArchivesUniversity of York
Heslington
York
YO10 5DD
Telephone: 01904 321170
Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/Twitter: @Jenny_MitchamSkype: jenny_mitcham
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAF8JE%3Do18q4XYmVJM5RjrSkNBs5qsTFK04Q-mBP-bm9c3AYM%3Dg%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZL3RZKd_Euogw4R-4_Bo4HCpybwv0AOOQv6_x0Vxfp3Sw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--Jenny Mitcham
Digital Archivist
Borthwick Institute for ArchivesUniversity of York
Heslington
York
YO10 5DD
Telephone: 01904 321170
Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/Twitter: @Jenny_MitchamSkype: jenny_mitcham
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAF8JE%3DrFjLTAX8hJpENJLn%3D4dip9fj1ysbZW44MExcJVkUJ8MA%40mail.gmail.com.
We hacked lib/QubitInformationObjectXmlCache.class.php, in the function exportAll, after the line
$exporting++;
we added
if (isset($options['skip']) && $exporting>1000) {exit("next");}
Then we called the task using an incrementing script:
#!/bin/bash
echo "Exporting"
skip=0
while [ $(cd /path/to/your/atom && php symfony cache:xml-representations --skip="$skip" | tail -1) == "next" ]
do
(( skip+=1000 ))
echo "$skip"
done
echo "Done"
This hugely reduced the memory requirement and time to run. Archiveshub (keen as mustard) have already made a successful harvest from the result. In our case, batches of 1000 worked nicely, but this could be reduced by orders of magnitude if necessary.
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1d139fad-e5ec-477f-b8d3-eb49f7f45fc3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.