New User: Blog - Media-graph Project (AuraDB free)

[John White]

Watched the video series on building a blogging engine. I'd had an idea bouncing around my mind of a hybrid blog and knowledge graph to map media mentions on a topic, so decided to try Structr and blog about the process as well. 

Just jumped in to using Structr and had a bunch of trouble trying to set up a local container, so just punted and spun up a Google Cloud VM and installed the debian package. 

I immediately realized that the videos must have included the export-import cycle because they were avoiding the setup of a neo4j database server.

For whatever reason, I had assumed that the process would involve installing the neo4j debian package with some sane setting guidance. Absent that, I decided to spin up a free-tier AuraDB instance.

Are there any tips on using AuraDB? I got a URI to use with the hosted graph, but it's protocol is listed as:

neo4j+s://[host]

When I use neorj+s:// in the Connection URL field of the Database Connections tab, it looks like it just .... spins on the "Connection is being established - Please wait" notification.

When I use neo4j://  I got a no routing service error

Could not perform discovery for database ''. No routing server available.

bolt connections don't seem to work:

Connection to the database terminated. Please ensure that your database is listening on the correct host and port and that you have compatible encryption settings both on Neo4j server and driver. Note that the default encryption setting has changed in Neo4j 4.0.

[Christian Morgner]

Hi John,

can you share some more details on the problems with Docker? Did you try https://github.com/structr/docker-setup?

You can run Structr without a Neo4j database, just make sure to do a deployment export of your application frequently. (See https://docs.structr.com/docs/fundamental-concepts#deployment)

In my opinion, the Structr Debian package on a Linux system is the best option for a quick setup. If you install Neo4j Community Edition, you can configure a connection to your local Neo4j database when you access the Structr Backend Ui for the first time. (See https://docs.structr.com/docs/configuration-settings#database-connections)

Or you can try a sandbox at https://structr.com (See https://docs.structr.com/docs/running-structr#sandbox-service)

I don't recommend to connect to a Neo4j AuraDB instance, because the connection will be very slow. Structr and Neo4j need to run on the same machine (or at least in the same network) to run smoothly, because Structr makes a lot of requests to the database, and the latency adds up.

It should however absolutely be possible to use AuraDB with Structr 4.1.2, see https://docs.structr.com/docs/configuration#add-a-connection.

Does that help?

Please don't hesitate to contact me if you have any questions.

Best,
Christian



Am 05.07.22 um 22:32 schrieb John White:

> --
> You received this message because you are subscribed to the Google Groups "structr" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to structr+u...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/structr/af6329d0-9d84-4274-8aee-ca837ff39b31n%40googlegroups.com <https://groups.google.com/d/msgid/structr/af6329d0-9d84-4274-8aee-ca837ff39b31n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Christian Morgner
Structr GmbH · Hanauer Landstr. 291a · 60314 Frankfurt · Germany
Phone: +49 163 2536088
Twitter: @cmor_

http://structr.com

[John White]

> You can run Structr without a Neo4j database, just make sure to do a deployment export of your application frequently. (See https://docs.structr.com/docs/fundamental-concepts#deployment)

Oh sure, for development, but for production, I'd need persistence in the face of a service reset. So part of my testing the platform is testing whether I can actually deploy a production-style environment. I'm definitely still following along with the video in an in-memory environment.

Unless it's possible to script the reload of app and data upon service restart?

> In my opinion, the Structr Debian package on a Linux system is the best option for a quick setup. If you install Neo4j Community Edition, you can configure a connection to your local Neo4j database when you access the Structr Backend Ui for the first time. (See https://docs.structr.com/docs/configuration-settings#database-connections)

Is there a recommended VM size to host both neo4j and structr? I've noticed a WARN message in the logs to configure at least 8GB of heap memory. My understanding is that a good neo4j configuration would have 1GB just for neo4j. Is that the ballpark I should be looking at? 9GB with some overhead for the host?

To view this discussion on the web, visit https://groups.google.com/d/msgid/structr/99f984e2-67df-f39c-7816-4ee6368e2f37%40structr.com.

--

John White

[Christian Morgner]

> Oh sure, for development, but for production, I'd need persistence in the face of a service reset. So part of my testing the platform is testing whether I can actually deploy a production-style environment. I'm definitely still following along with the video in an in-memory environment.
> Unless it's possible to script the reload of app and data upon service restart?

That is possible, but not convenient, so I would recommend using the in-memory mode only for testing. Even during development, there is a risk of losing a whole day's work if you accidentally stop Structr (which has happened to me...).

> Is there a recommended VM size to host both neo4j and structr? I've noticed a WARN message in the logs to configure at least 8GB of heap memory. My understanding is that a good neo4j configuration would have 1GB just for neo4j. Is that the ballpark I should be looking at? 9GB with some overhead for the host?

The amount of memory you need depends on the size of your application and the amount of data in the database. Structr makes heavy use of caching to speed things up, and memory is very cheap (at least it should be...), so I'd recommend 8 GB for Structr and the same for Neo4j. The more the better, if you have lots of data and high load. Our Structr installations typically run with 8-32 GB for Structr and 4-8 GB for Neo4j.

> > To unsubscribe from this group and stop receiving emails from it, send an email to structr+u...@googlegroups.com <mailto:structr%2Bunsu...@googlegroups.com>.
> > To view this discussion on the web, visit https://groups.google.com/d/msgid/structr/af6329d0-9d84-4274-8aee-ca837ff39b31n%40googlegroups.com <https://groups.google.com/d/msgid/structr/af6329d0-9d84-4274-8aee-ca837ff39b31n%40googlegroups.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/structr/af6329d0-9d84-4274-8aee-ca837ff39b31n%40googlegroups.com?utm_medium=email&utm_source=footer>>.

>
> --
> Christian Morgner
> Structr GmbH · Hanauer Landstr. 291a · 60314 Frankfurt · Germany
> Phone:   +49 163 2536088
> Twitter: @cmor_
>
> http://structr.com
>
> --
> You received this message because you are subscribed to the Google Groups "structr" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to structr+u...@googlegroups.com <mailto:structr%2Bunsu...@googlegroups.com>.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/structr/99f984e2-67df-f39c-7816-4ee6368e2f37%40structr.com.
>
>
>
> --
> John White

> --
> You received this message because you are subscribed to the Google Groups "structr" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to structr+u...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/structr/CADNO%2BRxjT7kBYk_ZQA7ETwVuHDQCqfZJknYaziA-HsosYgny6Q%40mail.gmail.com <https://groups.google.com/d/msgid/structr/CADNO%2BRxjT7kBYk_ZQA7ETwVuHDQCqfZJknYaziA-HsosYgny6Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[John White]

On Thursday, July 7, 2022 at 12:09:33 AM UTC-7 christian.morgner wrote:

> Oh sure, for development, but for production, I'd need persistence in the face of a service reset. So part of my testing the platform is testing whether I can actually deploy a production-style environment. I'm definitely still following along with the video in an in-memory environment.
> Unless it's possible to script the reload of app and data upon service restart?

That is possible, but not convenient, so I would recommend using the in-memory mode only for testing. Even during development, there is a risk of losing a whole day's work if you accidentally stop Structr (which has happened to me...).

> Is there a recommended VM size to host both neo4j and structr? I've noticed a WARN message in the logs to configure at least 8GB of heap memory. My understanding is that a good neo4j configuration would have 1GB just for neo4j. Is that the ballpark I should be looking at? 9GB with some overhead for the host?

The amount of memory you need depends on the size of your application and the amount of data in the database. Structr makes heavy use of caching to speed things up, and memory is very cheap (at least it should be...), so I'd recommend 8 GB for Structr and the same for Neo4j. The more the better, if you have lots of data and high load. Our Structr installations typically run with 8-32 GB for Structr and 4-8 GB for Neo4j.

Thanks for the guidance. For anyone else searching this archive for Google Cloud machine sizes, I used a g2-standard-4 to get 16 GB of RAM (which comes with 4 CPUs). I might later test a g2-standard-2 with 8GB of RAM for testing/development.

The debian package of neo4j doesn't automatically configure the neo4j service to start with the VM, so I needed to use systemctl to fix that.

sudo systemctl enable neo4j

Maybe another question I should have asked earlier:

Is there documentation on the differences in features from commercial license to Community Edition?