Data Backup

149 views
Skip to first unread message

royabra...@gmail.com

unread,
Aug 16, 2018, 6:28:06 PM8/16/18
to AtoM Users
Hi Users,

Good Day!!

I am trying to get a solution where I can take a backup of the Access to Memory database by export from mysqlworkbench or mysqldump 

I am unable to locate the atom database in mysqlworkbench using localhost and root with the <password> that we have used. I am able to create a separate test database in the Mysql and run mysqldumps on that.

Our ATOM installation is local and we have PHP and MySQL also running. I would like to know what is the connection parameters including username, password, database name etc in order to get the mysqlworkbench to identify the ATOM db. We are also running Virtual box (Vagrant) which was part of the steps that were documented in order to setup ATOM. 

All of the following run correctly 

1) ATOM
2) PHP - Tested with localhost
3) MYSQL - tested with 

$ mysql -u root -p 
<Password> 

mysql> show databases;

This does not show any ATOM Database - Please let us know the details and how to connect the same in order to run a cron job for the local install to internally execute mysqldump every day.

Please refer screenshots attached

Thank You in advance for the help/solution

Regards,
Abraham
Screen Shot 2018-08-16 at 7.25.34 PM.png
Screen Shot 2018-08-16 at 7.25.10 PM.png
Screen Shot 2018-08-16 at 7.23.54 PM.png

Dan Gillean

unread,
Aug 17, 2018, 10:21:55 AM8/17/18
to ICA-AtoM Users
Hi Abraham, 

You can always check the credentials that AtoM is using for your database in the following configuration file: config/config.php

If you have installed our default AtoM Vagrant box without modification, you can actually use the following credentials, which our different from the default production installation ones: 
  • database name: atom
  • database user: atom-user
  • database password: ATOMPASSWORD
The following command should create a dump of your database, named atom-db.sql, in the root Vagrant installation directory - that is, where you first installed the AtoM Vagrant box on your local host computer: 
  • mysqldump -u atom-user -pATOMPASSWORD atom > /vagrant/atom-db.sql
I hope that helps! 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/ecccbee3-6ea4-490a-a3c9-c68cd5a89231%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Abraham Roy M

unread,
Aug 17, 2018, 11:49:06 AM8/17/18
to ica-ato...@googlegroups.com
Thank You so much Dan!!

Will try this when I get back to the MAC machine on Monday/Tuesday and then confirm. If we are using MySQL Work Bench to connect still the same localhost:3306 should hold good right - except instead of using root we will be using atom.

Instead of the vagrant folder we should be able to save into 

$ mysqldump -u atom -p ATOMPASSWORD --databases atom > /usr/backup 

Also please could you confirm which folder contains the images uploaded to take a backup of the same - The essential idea being since we have Atom installed on a local machine we are trying to create a cron job (crontab) to dump the sql backup and the images to an external hard drive at a frequent interval.

Thank You,
Regards,
Abraham


To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Dan Gillean

unread,
Aug 17, 2018, 12:26:33 PM8/17/18
to ICA-AtoM Users
Hi Abraham, 

I want to start with one clarification, just in case: 

The AtoM Vagrant box is not designed for production use, or for scalability. It is meant to provide users with an easy to install temporary environment for testing and development. If you intend to use AtoM in production, I suggest you install it on a server, following our full installation instructions: 
Now, regarding your post: 

I can't really speak to using MySQL Workbench with the Vagrant box, but the following slides do cover that topic a bit, in case they are of use: 
As for your suggested command for backups, a couple thoughts:  
  • You can save to /usr/, but you will have to manually create the backup subdirectory first
  • Remember that your command has to end in the specific sql file you want to create - so your path should be something like /usr/backup/my-backup.sql
  • Note that with this method you are saving the backup inside the virtual machine, so it won't be accessible when vagrant is not running, and it won't be accessible on your host computer
  • The default mySQL username in the Vagrant box is atom-user, not atom - be sure to correct this before trying your command! 
Regarding the digital object uploads, you will find these in the uploads directory, which is a subdirectory just below the root AtoM directory. In a production install, this is generally:
  • /usr/share/nginx/atom/uploads
In the AtoM Vagrant box, the full path is: 
  • /home/vagrant/atom/uploads
You'll find more information on how to back up the digital objects in the following links: 
Finally, you might find the following slides useful: 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Abraham Roy M

unread,
Aug 20, 2018, 1:26:35 PM8/20/18
to ica-ato...@googlegroups.com
Hi Dan,

Good Day!!

We are able to connect to the server now but hit the following error on mysqldump - we are using mySQL 8.x 

$ sudo /usr/local/mysql/bin/mysqldump --host 10.10.10.10 --port 3306 -u atom -pATOMPASSWORD --routines --all-databases  > ./Dump_date +%d.sql - This is the same thing from export in mysqlworkbench

14:14:34 Dumping atom (all tables)
Running: /Applications/MySQLWorkbench.app/Contents/MacOS/mysqldump --defaults-file="/var/folders/_6/1sdp7_pn1hxdmj8wgjxz06w40000gn/T/tmp1kpXf3/extraparams.cnf"  --set-gtid-purged=OFF --user=atom-user --host=10.10.10.10 --protocol=tcp --port=3306 --default-character-set=utf8 --single-transaction=TRUE --routines --events "atom"
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,                       JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')                FROM information_schema.COLUMN_STATISTICS                WHERE SCHEMA_NAME = 'atom' AND TABLE_NAME = 'access_log';': Unknown table 'COLUMN_STATISTICS' in information_schema (1109)

Thanks in advance

Thank You,
Regards,
Abraham


To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Dan Gillean

unread,
Aug 20, 2018, 1:46:15 PM8/20/18
to ICA-AtoM Users
Hi Abraham, 

When you use the default AtoM Vagrant box, or if you follow our recommended instructions for a production installation on a server, we use/recommend Percona v5.6. See: 
Did you manually uninstall Percona 5.6, and install MySQL v8.x? 

This is likely causing compatibility issues. We have not tested with the most recent versions of MySQL, and there are likely upgrade issues that would have to be analyzed and resolved before an 8.x version of MySQL could be used with AtoM. 

Please see the full recommended installation instructions (and in particular, the part on MySQL), here: 
You've made other modifications to the suggested command I provided, such as the location it is executed and some other variables. Because of all of these factors, the amount of support we can provide is unfortunately very limited - we have not yet tested AtoM with the particular modifications you have made, and provide the Vagrant environment as-is for testing and development. Because of this, I'm not sure what to suggest! But you may want to consider downgrading, and using the Percona fork of MySQL. 

If you've upgraded your AtoM installation internally as well, you need to make sure that you've run the database schema upgrade task - see: 

Best of luck, and apologies we cannot be of more assistance!

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Abraham Roy M

unread,
Oct 9, 2018, 3:49:14 PM10/9/18
to d...@artefactual.com, ica-ato...@googlegroups.com
Hi Dan,

Good Day!!

How are you?

My name is Abraham and had been in touch with you while we were playing around with the software a few weeks back to assess if this is the right solution for us.

I would request you to please answer the following queries if possible

1) Please explain the difference between the access to memory website (https://www.accesstomemory.org/en/) and the Artefactual website (https://www.artefactual.com) - Both of them are referring to the same product but they have different options - one of them is only the product downloads etc and the other one is the service where a hosting is done - Is it by the same group/company

2) As per our previous conversations/emails, we are unable to take backups from the MYSQL DB as we have MySQL 8.x as compared to the product being specified for MySQL 5.x and thus are looking at the option of having it hosted by you (from the services)

I got the following from a person whom I know who has had some interactions with AtoM earlier (https://www.artefactual.com/services/site-hosting/)

Type of planDescriptionPrice
Standard single repositoryFor single institutions with small to mid-sized holdings (tens of thousands of records; up to 20,000 digital objects). Includes 10 GB of digital object storage and severity 1 & 2 issue resolution.$1,699 per year

The Single Instance Hosting is CAD $1699/- per year - Could you clarify the following
a. Complete Hosting and Storage inclusive of the backup solution is included - in which case what is the frequency of the backup and what will happen in case there is a failure or loss of data
b. Would we have access to the DB using Remote MySQL or PHPMyAdmin
c. Are the pictures stored in the DB using BLOB - I believe that is not the case from my analysis but the DB contains a link to the File/Folder location on the server (local or hosted)
d. There are multiple screens (please bear with my ignorance on the archival records  - I am into Technology and have had limited exposure to Access to Memory) where records can be entered - I am sure each of this is for a separate purpose - if we at any point of time - have to download to an exportable format - How do we get all the records/data with the correct information and also the pictures attached - an example of what  I can think of is if the records are saved into different tables due to the functionalities/modules is there a CSV file which will collate all this information on demand to take a backup including the pictures/files. Did not notice this in the Access to Memory tool - I could be wrong.
e.  If the above-mentioned on-demand CSV is not there/available - would your team be able to do it for a specific need (not using the same data structure as AtoM but a collation since all tools may not have the same table structure or for a backup in EXCEL Spreadsheets.

We are trying to take some decisions on this and thus sorry about multiple questions/emails.
 
Thank You,
Regards,
Abraham


On Fri, Aug 17, 2018 at 11:21 AM Dan Gillean <d...@artefactual.com> wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/JuJD-wkX3Fs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Dan Gillean

unread,
Oct 9, 2018, 4:45:50 PM10/9/18
to ICA-AtoM Users

Hi Abraham, 

Regarding your first question:

1) Please explain the difference between the access to memory website (https://www.accesstomemory.org/en/) and the Artefactual website (https://www.artefactual.com) - Both of them are referring to the same product but they have different options - one of them is only the product downloads etc and the other one is the service where a hosting is done - Is it by the same group/company

Access to Memory (AtoM) is the software. AtoM is an open source web-based application for archival description, arrangement, and access. 

Artefactual is a company. We are the lead developers and maintainers of AtoM. Because we give away our software and also license all related resources (like the documentation, etc) under open licenses as well, one of the ways we stay afloat is by offering paid additional services for those who want them - including hosting, training, data migrations, custom theming, and of course application development. 

You can read more about the history of the AtoM project, the Artefactual business model, and how we maintain and develop new features for AtoM here: 
I will follow-up with you in a separate message to provide more information about Artefactual's services. In the meantime, I will answer your questions about data export here, as they may be useful to the broader community. 

AtoM supports the creation of a number of different entities - see: 
Most of these entities have their own custom exports. However, many of these entities can be linked together (for example, adding a subject term as an access point to an archival description). When this is done, some of the linked entity may be included in another entity's export. For example, if you export an archival description that has an authority record linked as a creator and a subject term linked as an access point, the archival description export will include: 
  • The creator's name, and history (from the linked authority record)
  • The authorized form of name of the linked subject term
It will not include other information that may be added to these entities directly - for example, the authority record identifier, alternative forms of name, etc. Similarly, any alternative labels, scope notes, or source notes added to the subject term will not be in the archival description export. To get this information, you would need to export the authority record and/or subjects separately. 

Here is a list of the export formats currently supported in AtoM:
  • Archival descriptions: CSV, EAD 2002 XML, DC XML, MODS XML
  • Authority records: CSV, EAC-CPF XML
  • Archival institutions: CSV
  • Terms: SKOS RDF/XML
You can read more about import and export in AtoM in this section: https://www.accesstomemory.org/docs/latest/#import-export

It's true that data for an archival description is stored across multiple different tables in the database, but it is all collated together in a consistent format on export. However, there are some entities that might be linked to AtoM that unfortunately do not currently have their own export format, and will not be included in the archival description export. One example would be the Rights module. Rights can be added to descriptions, but don't currently have a means of being exported. Adding new functionality (such as new export code) requires community support for us to be able to implement - either from development sponsorship for us to do the work, or a community code contribution following our recommended contribution guidelines

The digital objects are also not included in an archival description CSV export - instead, a URL to the master digital object is included in the digitalObjectURI column of the export. I have outlined how you can use this value to get the actual storage path of the object in this previous forum thread: 
And in this thread, I describe how objects are stored in AtoM in general: 
Currently the only way to get ALL data out of AtoM at once is via a sqldump file, and copying the uploads directory. 

As I mentioned, I will send you a separate personal email with more information on Artefactual's paid services. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

Reply all
Reply to author
Forward
0 new messages