Migrating bitstream store to AWS S3
35 views
Skip to first unread message
Juan López
Aug 23, 2024, 10:40:27 AM8/23/24
to DSpace Technical Support
Hi!
I'm migrating the bitstream storage of a DSpace 7.6.2 from DSBitstore to S3Bitstore, the command "bitstore-migrate -a 0 -b 1" worked as expected.
However, I see that the S3 only got around 34 GB of data while the original assetstore had more than 200 GB of data.
Checking for what could be happening, I see that there is a lot of "orphans bitstreams" that are in the original assetstore but are missing in the database.
Is there a reason of why are this bitstreams in the assetstore but not in the bitstream table? Could it be that this DSpace never had a "/bin/dspace cleanup" executed?
Best regards,
Juan.
DSpace Technical Support
Sep 9, 2024, 12:27:38 PM9/9/24
to DSpace Technical Support
Hi Juan,
Yes, it could be that "./dspace cleanup" has not been run in a very long time. By default, deleted Bitstreams will be kept in the assetstore until the next "cleanup" is executed. So, you should schedule that cleanup to run on a semi-regular basis. For instance, our example Cron tasks have it run on a monthly basis: https://wiki.lyrasis.org/display/DSDOC7x/Scheduled+Tasks+via+Cron
Yes, it could be that "./dspace cleanup" has not been run in a very long time. By default, deleted Bitstreams will be kept in the assetstore until the next "cleanup" is executed. So, you should schedule that cleanup to run on a semi-regular basis. For instance, our example Cron tasks have it run on a monthly basis: https://wiki.lyrasis.org/display/DSDOC7x/Scheduled+Tasks+via+Cron
Tim
Reply all
Reply to author
Forward
0 new messages