QEWD docker DB persistence

[Noel da Costa]

Hi Rob,

I've been successful in trying out the baseline tutorials. However if I stop my docker container all my data vanishes.

I don't seem to be able to find the instructions to make this persist?

Tx,

Noel

[Noel da Costa]

I found this: https://github.com/robtweed/qewd-jsdb/blob/master/REPL.md#starting-qewd--qewd-jsdb

... which talks about persistence but it doesn't work for me. If I type in `./start` or `./start_p` it's not a recognised command. The way I start the docker container is by using the `docker run` commands described in the `QEWD-up` documentation. I.e. this: https://github.com/robtweed/qewd/tree/master/up#starting-your-qewd-container

So I'm confused as to what the `./start` thing is vs. `docker run` thing for starting containers and also about how to persist the data if the container stops.

Tx,

Noel

[Rob Tweed]

Yes the generated start_p script is one way to do it - and all done for you. If you want to do things the “proper way”, just list the start_p script file and you’ll see that it’s just starting the container with another volume mapping to pre-initialised YottaDB files that reside on the host instead. That’s the proper way to persist a database on the host, bu feel free to reimplement what start_p is doing for you in whatever way you think is best for you.

I’ve just tried to make life super easy for you...

Rob

Sent from my iPhone

On 3 Apr 2021, at 22:32, Noel da Costa <noeld...@gmail.com> wrote:

I found this: https://github.com/robtweed/qewd-jsdb/blob/master/REPL.md#starting-qewd--qewd-jsdb

--
You received this message because you are subscribed to the Google Groups "Enterprise Web Developer Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to enterprise-web-develope...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/enterprise-web-developer-community/f7f5c2c0-662f-4c25-bd30-8b2d516b23d8n%40googlegroups.com.

[Rob Tweed]

Just a follow up to clarify: the start and start_p scripts are generated wrappers around the standard docker run command, the idea being to save you the bother of having to type the very long docker run command needed to start QEWD, and, in doing so, reducing the chance of you getting something wrong. 

By all means copy the docker run command in the script and use whatever other means you want to invoke it. You could, of course, use Docker compose and create a YAML file using the info in the start or start_p scripts. At the end of the day it’s just docker run behind the scenes...

Rob

Sent from my iPhone

On 4 Apr 2021, at 00:55, Rob Tweed <rob....@gmail.com> wrote:

Yes the generated start_p script is one way to do it - and all done for you. If you want to do things the “proper way”, just list the start_p script file and you’ll see that it’s just starting the container with another volume mapping to pre-initialised YottaDB files that reside on the host instead. That’s the proper way to persist a database on the host, bu feel free to reimplement what start_p is doing for you in whatever way you think is best for you.

[Noel da Costa]

Hi Rob,

Ok but where do I find these `start` and `start_p` scripts though? 

I’m not seeing them anywhere.

Tx,

Noel

You received this message because you are subscribed to a topic in the Google Groups "Enterprise Web Developer Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/enterprise-web-developer-community/oa35nLnIPW0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to enterprise-web-develope...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/enterprise-web-developer-community/2BA37D74-2A81-4962-9761-130F42FEB994%40gmail.com.

[Noel da Costa]

I found them – they get installed by the `install.sh` file that's included with `qewd-jsdb`.

Incidentally that `install.sh` script does assume an Debian distribution.

What is `install.js` for? (It's not described in the README).

Tx,

Noel

On Sunday, 4 April 2021 at 11:21:16 UTC+1 Noel da Costa wrote:

Hi Rob,

[Rob Tweed]

Yes It helps to follow the installation instructions. And yes what I wrote is for Debian installations. I’m happy for anyone else to figure out the equivalent instructions for other Linux flavours and donate it if they’re so minded. 

Install.js is invoked within the node-runner container that is fired up by install.sh. Node-runner creates a temporary node runtime where I then do all the installation and configuration work. Study the code and my node-runner repo on GitHub 

Rob

Sent from my iPhone

On 4 Apr 2021, at 11:59, Noel da Costa <noeld...@gmail.com> wrote:

I found them – they get installed by the `install.sh` file that's included with `qewd-jsdb`.

--
You received this message because you are subscribed to the Google Groups "Enterprise Web Developer Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to enterprise-web-develope...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/enterprise-web-developer-community/09d8baca-64af-4073-ad35-318290a9e5a5n%40googlegroups.com.

[Rob Tweed]

So again, to try to clarify some issues: everything QEWD-up needs in order to start up is in the config.json file. If you know what you’re doing, the idea is that you’d normally maintain that file manually. However, there are so many potential configuration options such as what database you’re using, what OS you’re using, how many child processes you want, what port you want to listen on etc etc, it can be confusing for anyone just wanting to kick the tyres of QEWD and quickly try it out. Also until you’re familiar with these parameters and the moving parts of QEWD, if you get something wrong in the config.json, it can be difficult to figure out why it’s going wrong.

So, in order to resolve that, I created the automated install/configure script that you see recommended for use in the QEWD-baseline and QEWD-jsdb repos. All they actually do is build the config.json for you in a foolproof way (provided you read and follow the instructions in the readme), and create a wrapper around the docker run command needed to start the dockerised version of QEWD. The script also pulls in pre-Initialised YottaDB database files so you can maintain persistence on the host. 

None of that is essential. If you know how QEWD and QEWD-up works, and know how to initialise yottadb files and run Docker containers with the correct volume mapped for QEWD, then you can do it all yourself, as documented in all the other QEWD-related repos.

For anyone not 100% familiar with QEWD, I recommend you use my automation to get it working. The idea of open source is that all the source code is there for you to study and figure out what I’ve done and how what I’ve done works and why. See what my scripts create, study their code to see how they create what they create, and you’ll quickly learn how to control and operate QEWD for yourself in your own way that suits you. Nothing I’ve done is intended to force you to work in a particular way, other than the core QEWD-up files and syntax and the dockerised version’s mapped volume internal path 

Hope this helps to explain the rationale of what I’ve created

Rob

Sent from my iPhone

On 4 Apr 2021, at 12:24, Rob Tweed <rob....@gmail.com> wrote:

Yes It helps to follow the installation instructions. And yes what I wrote is for Debian installations. I’m happy for anyone else to figure out the equivalent instructions for other Linux flavours and donate it if they’re so minded.