Latest Docker Build - breaking changes

[fga...@greenchef.com]

Hello we have a breaking change to our jsreport reports Docker build and we cannot continue. Can you help us? We were just about to buy a license and cannot proceed until we get this fixed.

2016-03-23T17:31:50.033Z - info: Initializing embedded storage at /home/jsreport/data

2016-03-23T17:31:50.075Z - error: Failed to load collection templates Error: EISDIR: illegal operation on a directory, read

    at Error (native)

2016-03-23T17:31:50.078Z - error: Error occured during reporter init Error: EISDIR: illegal operation on a directory, read

    at Error (native)

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

2016-03-23T17:31:50.079Z - error: Failed to load collection images Error: EISDIR: illegal operation on a directory, read

    at Error (native)

2016-03-23T17:31:50.080Z - error: Failed to load collection data Error: EISDIR: illegal operation on a directory, read

    at Error (native)

Background: We are building a Docker container to run our reports in production. To do this, we copy the /data file to our container from our local drive. When we start the jsreport server, it errors out. This was previously running fine until a day or so ago.

Did the latest release invalidate some of the folders underneath the /data folder? Is there a new structure? or... It seems that the new docker build cannot use the files that we have been creating using an older version of jsreports maybe?

Here is our DockerFile:

FROM node:4.4.0

MAINTAINER -----

EXPOSE 2945

RUN apt-get update && apt-get install -y sudo

RUN adduser --disabled-password --gecos "" jsreport

RUN echo "jsreport ALL=(root) NOPASSWD: /usr/local/bin/node" >> /etc/sudoers

RUN echo "jsreport ALL=(root) NOPASSWD: /usr/local/bin/npm" >> /etc/sudoers

VOLUME ["/jsreport"]

WORKDIR /home/jsreport

ADD run.sh /home/jsreport/run.sh

# Install current version of jsreports

RUN sudo npm install jsreport --production

# Install 3rd Party plugin (ours) that saves report output to S3

RUN sudo npm install jsreport-s3-storage

# Initialize jsreports

RUN node node_modules/jsreport --init

# Copy over all of the report code. Make sure you are building the Docker container

# from the root of your report folder (ex: $HOME/local-reports)

COPY /data /home/jsreport/data

# Copy over all of our environment's config files. Note: these are not kept in

# source control. See Production Ops staff for information.

COPY dev.config.json /home/jsreport/

COPY prod.config.json /home/jsreport/

ENV NODE_ENV production

# Start the jsreports server

CMD ["bash", "/home/jsreport/run.sh"]

[Jan Blaha]

Hi,

it happened after rebuilding docker image? Because if the image was not rebuilt, then you should have there still the same jsreport version as previously I guess....

According to the error message. I see log entry "Initializing embedded storage at /home/jsreport/data"

This could happen if you created your templates with the newer fs-store (configuration value connectionString: { name: 'fs' }, but your current configuration file specifies the older embedded implementation (connectionString: { name: 'nedb' }. Is this possible? You can find out if your templates were created with fs-store by checking /data/templates. If the sub-directories are having the same name as templates' names, then it were created with fs-store.

Maybe you are creating your local templates with the fs-store but docker image runs on the older version with nedb store?

Can you check which jsreport version you have actually in the image?

Maybe also paste the /home/jsreport/run.sh

[Jan Blaha]

Please post and review also the connectionString values from both config files

[fga...@greenchef.com]

Hi Jan - I am trying the suggestions you sent via email. I will post back with more information shortly.

[fga...@greenchef.com]

It looks like I have some directories and files that are the same name (ex: schedules is both a file and a folder). For 'fs' storage, should I have a file or a folder?

[Jan Blaha]

[fga...@greenchef.com]

I'm in a state where i don't know how to recover. If I set things to 'fs', I get this error.

error: Failed to load collection templates Error: Unable to read config.json, you are likely trying to use fs jsreport store provider on the data set created with nedb provider. Please switch connection string name to nedb. 

Error: ENOTDIR: not a directory, open '/home/jsreport/data/templates/.DS_Store/config.json'

    at ReadFileContext.<anonymous> (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/lib/persistence.js:258:17)

    at ReadFileContext.callback (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:78:16)

    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:303:13)

2016-03-23T19:31:14.387Z - error: Error occured during reporter init Error: Unable to read config.json, you are likely trying to use fs jsreport store provider on the data set created with nedb provider. Please switch connection string name to nedb. 

Error: ENOTDIR: not a directory, open '/home/jsreport/data/templates/.DS_Store/config.json'

    at ReadFileContext.<anonymous> (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/lib/persistence.js:258:17)

    at ReadFileContext.callback (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:78:16)

    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:303:13)

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

2016-03-23T19:31:14.389Z - error: Failed to load collection data Error: Unable to read config.json, you are likely trying to use fs jsreport store provider on the data set created with nedb provider. Please switch connection string name to nedb. 

Error: ENOTDIR: not a directory, open '/home/jsreport/data/data/.DS_Store/config.json'

    at ReadFileContext.<anonymous> (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/lib/persistence.js:258:17)

    at ReadFileContext.callback (/home/jsreport/node_modules/jsreport/node_modules/jsreport-fs-store/node_modules/fs-extra/node_modules/graceful-fs/graceful-fs.js:78:16)

    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:303:13)

2016-03-23T19:31:14.428Z - error: Failed to load collection schedules Error: EISDIR: illegal operation on a directory, read

    at Error (native)

If I set things to 'nedb' - I get the following: 

2016-03-23T19:39:37.337Z - info: Initializing embedded storage at /home/jsreport/data

2016-03-23T19:39:37.379Z - error: Failed to load collection templates Error: EISDIR: illegal operation on a directory, read

    at Error (native)

2016-03-23T19:39:37.383Z - error: Error occured during reporter init Error: EISDIR: illegal operation on a directory, read

    at Error (native)

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

2016-03-23T19:39:37.385Z - error: Failed to load collection data Error: EISDIR: illegal operation on a directory, read

    at Error (native)

2016-03-23T19:39:37.387Z - error: Failed to load collection images Error: EISDIR: illegal operation on a directory, read

    at Error (native)

Do have any suggestions on how I can get the server up and running and my report templates to load? I can't seem to put the filesystem back

in a working order...or don't know what the right way to do it is. The main thing is, I don't want to lose my work...

[fga...@greenchef.com]

I also get this as well. I think one of the issues is, we save the entire /data folder in source control, and some of the supporting files maybe don't get saved (ex: .DS_Store...). I'm not sure how to get this to work and recover my report templates.

2016-03-23T19:53:03.054Z - info: Initializing embedded storage at /home/jsreport/data

2016-03-23T19:53:03.071Z - error: Error occured during reporter init Error: EEXIST, file already exists '/home/jsreport/data/schedules'

    at Object.fs.mkdirSync (fs.js:653:18)

    at Function.sync (/home/jsreport/node_modules/jsreport/node_modules/mkdirp/index.js:71:13)

    at Function.Persistence.ensureDirectoryExists (/home/jsreport/node_modules/jsreport/node_modules/jsreport-embedded-store/lib/persistence.js:63:12)

    at new Persistence (/home/jsreport/node_modules/jsreport/node_modules/jsreport-embedded-store/lib/persistence.js:55:17)

    at EmbeddedCollection.load (/home/jsreport/node_modules/jsreport/node_modules/jsreport-embedded-store/lib/embeddedProvider.js:107:32)

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-embedded-store/lib/embeddedProvider.js:43:20

    at Array.map (native)

    at EmbeddedProvider.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-embedded-store/lib/embeddedProvider.js:39:56)

    at DocumentStore.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/store/documentStore.js:43:24)

    at q.ninvoke.then.then.then.then.then.then.self.initializeListener.fire.then.self.extensionsManager.recipes.push.name (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:75:33)

    at node.js:906:3

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

2016-03-23T19:53:03.083Z - error: Failed to load collection templates Error: EISDIR, read

2016-03-23T19:53:03.085Z - error: Failed to load collection data Error: EISDIR, read

2016-03-23T19:53:03.087Z - error: Failed to load collection images Error: EISDIR, read

GC-MAC-002:scripts franklingauer$ 

[fga...@greenchef.com]

I have rebuilt the jsreport environment using 'fs' and I can get everything to work locally. I have all my report templates, and everything works fine. Now I want to build a Docker container that includes my report templates so we can run it in production. I have had this running for weeks now, and something is not right anymore. It no longer builds and works.

The Dockerfile does 3 basic things:

1) Installs jsreports and our plugin

2) Copies in our report template files (/data)

3) Runs jsreport server 

Here is the Dockerfile commands:

VOLUME ["/jsreport"]

WORKDIR /home/jsreport

ADD run.sh /home/jsreport/run.sh

RUN sudo npm install jsreport --production

RUN sudo npm install jsreport-s3-storage

RUN sudo node node_modules/jsreport --init

# This was the original line that worked 

COPY /data/ /home/jsreport/data/

# I've tried this thinking that the location of the /data folder moved, but it does not work

COPY /data/ /home/jsreport/

COPY prod.config.json /home/jsreport/

COPY dev.config.json /home/jsreport/

CMD ["bash", "/home/jsreport/run.sh"]

This is the error when I do: COPY /data /home/jsreport/data/  (which leads me to believe that the /data folder has moved somehow)

2016-03-23T21:20:22.344Z - info: Initializing jsreport in development mode using configuration file dev.config.json

2016-03-23T21:20:22.348Z - info: Setting process based strategy for rendering. Please visit http://jsreport.net/learn/configuration for information how to get more performance.

2016-03-23T21:20:22.353Z - info: Searching for available extensions in /home/jsreport/

2016-03-23T21:20:22.356Z - error: Error occured during reporter init Error: ENOENT: no such file or directory, stat '/home/jsreport/data/data'

    at Error (native)

    at Object.fs.statSync (fs.js:844:18)

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/util/util.js:46:21

    at Array.forEach (native)

    at Object.exports.walkSync (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/util/util.js:44:10)

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/extensionsManager.js:154:25

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

GC-MAC-002:greenchef-reports franklingauer$ 

This is the error when I do: COPY /data /home/jsreport/

2016-03-23T21:12:23.198Z - info: Initializing jsreport in development mode using configuration file dev.config.json

2016-03-23T21:12:23.203Z - info: Setting process based strategy for rendering. Please visit http://jsreport.net/learn/configuration for information how to get more performance.

2016-03-23T21:12:23.210Z - error: Error occured during reporter init Error: EEXIST: file already exists, mkdir '/home/jsreport/data'

    at Error (native)

    at Object.fs.mkdirSync (fs.js:794:18)

    at sync (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/node_modules/mkdirp/index.js:70:13)

    at Function.sync (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/node_modules/mkdirp/index.js:76:24)

    at new module.exports (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/blobStorage/fileSystemBlobStorage.js:14:12)

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:68:28

From previous event:

    at /home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:78:8

From previous event:

    at Reporter.init (/home/jsreport/node_modules/jsreport/node_modules/jsreport-core/lib/reporter.js:57:30)

    at Reporter.reporter.init (/home/jsreport/node_modules/jsreport/lib/extendInit.js:8:21)

    at Object.<anonymous> (/home/jsreport/server.js:1:85)

G

So here is what I am thinking:

1) I used to be able to copy in my report templates, and build a Docker container to run in production, but I can't anymore for some reason.

2) Did the server used to startup and not error out if these folders/files already exist? (i.e. 2016-03-23T21:12:23.210Z - error: Error occured during reporter init Error: EEXIST: file already exists, mkdir '/home/jsreport/data') because that is what is happening now. Bear in mind i'm copying an 'existing' /data folder, with all of its contents when building a Docker container for production. When the server starts up now, it sees that those folders/files exist, and gives an error. Previously, it would not do this... 

[fga...@greenchef.com]

I've managed to work though this. Part of this was due to the main NPM site having issues and builds not completing fully. Somehow I got everything put back together and it is somewhat functional at this time. Thank you for your help!