import error - invalid filename

[marechal...@uqam.ca]

Hi !

We have a new installation of an AtoM site (2.4.1) in a Docker environment.

When importing a CSV file via the user interface, I meet this error.

I think the csv is ok because when I try to import it on a vagrant installation, it works.

Any idea what I could change or verify ?

Thanks for your help !

Vivianne Maréchal

[Dan Gillean]

Hi Vivianne, 

Strange, I don't recall seeing that issue before!

If you are able to share the CSV, I would be happy to try importing it myself to see if I can reproduce the issue. Fee free to email it to me directly off-list - dan [at] artefactual [dot] com. I will also ask one of our developers if they have seen this before, and if they have suggestions. 

In the meantime: perhaps it seems obvious, but have you tried changing the file name to see if it helps?

You might also try clearing all caches, and restarting the job scheduler to see if that helps. Assuming an installation following our recommended instructions on Ubuntu 16.04, you could use the following commands for a server installation. Note that all of these should be run from AtoM's root installation directory - if you've followed our recommended instructions, this is generally /usr/share/nginx/atom.  

Restart PHP-FPM and memcached

• sudo systemctl restart php7.0-fpm
• sudo systemctl restart memcached

Restart the atom-worker

• sudo systemctl restart atom-worker

Clear the application cache: 

• php symfony cc

Let me know if you require commands that will work on Ubuntu 14.04. 

I haven't worked with our Docker Compose recipe very much, but if those are the instructions you've followed, the documentation gives hints as to how you can run these commands in Docker: 

• https://www.accesstomemory.org/docs/latest/dev-manual/env/compose/
  

For example, the instruction given to restart the atom_worker is: 

• docker-compose restart atom_worker
  

Hopefully from this, you could determine how to restart PHP-FPM and memcached as well. 

I'm also guessing, based on some of the commands shown there, that you could clear the application cache with the following: 

• docker-compose exec atom php symfony cc

Let me know if any of the above helps! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0d698050-cfa3-4dfe-995d-87752a2c3e3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Karl Goetz]

Hi,

Further to dans suggestions, two things I would look for:

* Use the full path to the file incase the relative filename is being ‘lost in translation’

* Check permissions on the file.

The error you see ("You must specify a valid filename“) appears to turn up any time a file can’t be opened - whether its valid or not.

Karl.

On 12 Jan 2019, at 6:18 am, marechal...@uqam.ca wrote:

Hi !

We have a new installation of an AtoM site (2.4.1) in a Docker environment.

When importing a CSV file via the user interface, I meet this error.

<import_error_invalid_filename.png>

I think the csv is ok because when I try to import it on a vagrant installation, it works.

Any idea what I could change or verify ?

Thanks for your help !

Vivianne Maréchal

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0d698050-cfa3-4dfe-995d-87752a2c3e3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

<import_error_invalid_filename.png><import_error_invalid_filename.png>

-- 

Karl Goetz,  Senior Library Officer (Library Systems)

University of Tasmania, Private Bag 25, Hobart 7001

Available Tuesday, Wednesday, Thursday

University of Tasmania Electronic Communications Policy (December, 2014).
This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.

[marechal...@uqam.ca]

Thanks to both of you! We'll try these and come back with the results.

Vivianne Maréchal

Le lundi 14 janvier 2019 16:54:29 UTC-5, Karl Goetz a écrit :

Hi,

Further to dans suggestions, two things I would look for:

* Use the full path to the file incase the relative filename is being ‘lost in translation’

* Check permissions on the file.

The error you see ("You must specify a valid filename“) appears to turn up any time a file can’t be opened - whether its valid or not.

Karl.

On 12 Jan 2019, at 6:18 am, marechal...@uqam.ca wrote:

Hi !

We have a new installation of an AtoM site (2.4.1) in a Docker environment.

When importing a CSV file via the user interface, I meet this error.

<import_error_invalid_filename.png>

I think the csv is ok because when I try to import it on a vagrant installation, it works.

Any idea what I could change or verify ?

Thanks for your help !

Vivianne Maréchal

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.

To post to this group, send email to ica-at...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0d698050-cfa3-4dfe-995d-87752a2c3e3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<import_error_invalid_filename.png><import_error_invalid_filename.png>

-- 

Karl Goetz,  Senior Library Officer (Library Systems)

University of Tasmania, Private Bag 25, Hobart 7001

Available Tuesday, Wednesday, Thursday

[Dan Gillean]

Thanks for that tip, Karl!

Vivianne, I did touch base with a developer, and they suggested something similar to what Karl has pointed out. Namely, "this error is given if you specify a relative path to your CSV. Make sure none of the folders or files in the path are symlinks."

Since you are reproducing this issue while trying to import via the user interface - is your CSV stored on a shared drive, rather than the local computer where you are attempting the upload? Hopefully Karl's suggestions will set you on the right path! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To post to this group, send email to ica-ato...@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/eb26b676-2017-4d46-accb-c14e4c4c0cc1%40googlegroups.com.

[marechal...@gmail.com]

Hello and thanks for the suggestions Dan and Karl !

With the help of a nice guy in TI, we tried a lot of things, without success.

• 1-      We changed the name of the file.
  
• 2-      We imported a csv from Artefactual to test.
  
• 3-      The files were always on a local computer.
  
• 4-      We checked the permission of the file. It’s ok.
  
• 5-      We imported via user interface and CLI.
  
• 6-      We cleared the caches, restart PHP-FPM and AtoM-worker, purge everything, restart, etc.
  

7-   Dan, I send you the .csv that we used. It’s working in the vagrant installation! Just not on our Docker.

We noticed that the subjects are imported and that in advanced search, we can see the repository as a suggestion.

This is part of the check import :

bash-4.4# php symfony csv:check-import 4U_bte.csv

.............................

Analysis complete.

 

29 rows, 54 columns.

 

Empty columns:

--------------

 

qubitParentSlug appraisal accruals arrangement reproductionConditions languageNote

physicalCharacteristics locationOfCopies relatedUnitsOfDescription publicationNot

e placeAccessPoints descriptionIdentifier institutionIdentifier rules descriptionS

tatus levelOfDetail sources archivistNote publicationStatus digitalObjectURI

 

Multi-value columns (contain "|" character):

-------------------

 

genreAccessPoints(1), subjectAccessPoints(12), alternativeIdentifiers(12), alterna

tiveIdentifierLabels(12), generalNote(5), nameAccessPoints(1)

 

And this is the import :

bash-4.4# php symfony csv:import 4U_bte.csv --rows-until-update=1

WARNING: If you're importing multiple CSV files as part of the same import it's ad

visable to use the source-name CLI option to specify a source name (otherwise the

filename will be used as a source name).

.

1 rows processed in 0.01 minutes (0.36 second/row average, 14.65 MB used).

.

2 rows processed in 0.01 minutes (0.21 second/row average, 14.9 MB used).

.

3 rows processed in 0.01 minutes (0.16 second/row average, 15.15 MB used).

.

4 rows processed in 0.01 minutes (0.13 second/row average, 15.36 MB used).

.

5 rows processed in 0.01 minutes (0.12 second/row average, 15.6 MB used).

.

6 rows processed in 0.01 minutes (0.11 second/row average, 15.81 MB used).

.

7 rows processed in 0.01 minutes (0.1 second/row average, 16.06 MB used).

.

8 rows processed in 0.01 minutes (0.1 second/row average, 16.27 MB used).

.

9 rows processed in 0.01 minutes (0.09 second/row average, 16.49 MB used).

.

10 rows processed in 0.02 minutes (0.09 second/row average, 16.7 MB used).

.

11 rows processed in 0.02 minutes (0.09 second/row average, 16.95 MB used).

.

12 rows processed in 0.02 minutes (0.09 second/row average, 17.15 MB used).

.

13 rows processed in 0.02 minutes (0.08 second/row average, 17.36 MB used).

.

14 rows processed in 0.02 minutes (0.08 second/row average, 17.57 MB used).

.

15 rows processed in 0.02 minutes (0.08 second/row average, 17.82 MB used).

.

16 rows processed in 0.02 minutes (0.08 second/row average, 18.02 MB used).

.

17 rows processed in 0.02 minutes (0.08 second/row average, 18.27 MB used).

.

18 rows processed in 0.04 minutes (0.13 second/row average, 22.37 MB used).

.

19 rows processed in 0.05 minutes (0.15 second/row average, 25.13 MB used).

.

20 rows processed in 0.06 minutes (0.17 second/row average, 27.47 MB used).

.

21 rows processed in 0.06 minutes (0.17 second/row average, 29.31 MB used).

.

22 rows processed in 0.07 minutes (0.19 second/row average, 16.6 MB used).

.

23 rows processed in 0.08 minutes (0.21 second/row average, 19.12 MB used).

.

24 rows processed in 0.09 minutes (0.22 second/row average, 21.63 MB used).

.

25 rows processed in 0.1 minutes (0.23 second/row average, 24.15 MB used).

.

26 rows processed in 0.18 minutes (0.41 second/row average, 20.51 MB used).

.

27 rows processed in 0.19 minutes (0.42 second/row average, 22.96 MB used).

.

28 rows processed in 0.2 minutes (0.42 second/row average, 24.67 MB used).

.

29 rows processed in 0.21 minutes (0.43 second/row average, 28.47 MB used).

bash-4.4#

 

Do you have a suggestion for us ?

 

Thanks a lot !

Vivianne

[Dan Gillean]

Hi Vivianne, 

Thank you for sending me a copy of the CSV file. I don't see any obvious issues with it (personally, I don't recommend you include spaces in your legacyID values as you did in row 27, but it still imported successfully), and I was able to import it into a couple different AtoM test sites without issue. 

The log output you sent makes it seem as if these records are in fact importing - is the log from your Docker environment, or is this from when you testing importing it in a Vagrant box? If it's from Docker, did you remember to re-index after to see if your records were in fact imported? By default, CSV imports from the command-line are not indexed (so they consume less memory and import more quickly), unless you use the --index flag when running the CSV import task. This means you will need to re-run the search:populate task after the import to see your changes. 

Can you tell me more about your Docker environment? Are you using the Docker compose env we provide in the documentation? Or is this your own custom Docker environment?

If it is Artefactual's Docker Compose recipe, have you tried prefacing your import commands with docker-compose exec, similar to the commands we run in the documentation during installation and configuration? For example: 

• docker-compose exec php symfony csv:import 4U_bte.csv --rows-until-update=1

I might also suggest providing the full absolute path to the CSV in the command, if you haven't already. 

If this is your own custom Docker environment, anything further you can tell us about how it's created, what versions of the dependencies you have installed (e.g. AtoM version, PHP version, MySQL version, etc) will be useful for our team to provide further suggestions. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1af82eb6-388b-49f8-bcaa-8010a51f1752%40googlegroups.com.

[marechal...@uqam.ca]

Hi Dan,

thanks a lot for the answer. Re-indexing worked ! Wow !

Will we be able to import CSV in the user interface in a Docker environnement ?

We understand that we'll have to "translate" the command in line that you give in your documentation to fit the Docker environnement. Do you have a document or a ressource to recommand us about CLI in Docker for AtoM ?

I'm not able to tell you more about the custom Docker environnement the IT departement gave us, sorry, but I'll collect the informations for the next bug ;-)

Thanks a lot and have a good day !

Vivianne

[Dan Gillean]

Hi Vivianne, 

Glad to hear that your import was successful and that you can see the records after re-indexing! Regarding your questions: 

Will we be able to import CSV in the user interface in a Docker environnement ?

Yes, I don't see why not. If your Docker instance is correctly configured, then imports via the user interface should be fine. 

We understand that we'll have to "translate" the command in line that you give in your documentation to fit the Docker environnement. Do you have a document or a ressource to recommand us about CLI in Docker for AtoM ? 

Unfortunately, I don't know enough about your Docker environment to be able to advise. However, the search:populate command is one of AtoM's command-line tools, so if you've successfully run this, then you should be able to run other AtoM CLI tools! 

In AtoM's Docker Compose environment we outline in our docs, I showed in my previous reply how to use AtoM command line tasks - essentially, I think they need to be preceded by docker-compose exec - otherwise they are executed the same way. I don't know about your environment, but like I said, if you run them the same way you ran the search:populate task, then it should work! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/516c6ec5-1d2d-4839-bd6b-1b07bd22aef1%40googlegroups.com.