Internal Server Error? Help Packaging Wagtail CRX (Python CMS)

[Jackson Couse]

Hi, I'm learning about packaging and I've run into an internal server error. Anyone able to help me out?

Here the lowdown:

I installed the terrific Wagtail CRX content management system locally. Everything seems OK and works as expected: https://docs.coderedcorp.com/wagtail-crx/getting_started/install.html

Then I followed the packaging instructions in the Sandstorm tutorial, using uwsgi, and that all seems fine.

When I load the app into my dev-mode, local sandstorm, it uploads, and the size of the grain is 104 MB, which I guess is about right for an empty CMS, but the page gives the message "Internal Server Error." Hmmm.

So, what's next?

• I saw the note about the Python HOME glitch, maybe this applies? https://docs.sandstorm.io/en/latest/developing/raw-python/
  
• Debugging the running grain? Maybe it's a typo or conflict somewhere? I see there are some tips here, but not sure where to start. https://docs.sandstorm.io/en/latest/developing/debugging/
  
• Maybe this isn't a good app for Sandstorm or a different approach is needed? Appreciate feedback about evaluating apps because there are a half dozen I'd like to try adding to my server but I'm new to virtual environments, vagrant, and packing for Sandstorm (though I have been following from the start), so I'm very much still feeling my way around. The other apps I'm looking at trying are Invoice Ninja (accounts, PHP), Loomio (decisions, Ruby), Cobudget (shared budgets, Node.js), Matomo (was Piwik stats, PHP), Zammad (help desk, Ruby), and maybe, once I really get a good handle on this, Vaultwarden (passwords, Rust).
  
• Any python-based apps or similar projects I should look at as an example?
  
• Maybe I need to do this as raw packaging? https://docs.sandstorm.io/en/latest/developing/raw-packaging-guide/ 
  

Here is the log. Note that it says "failed to open python file /opt/app/main.py" and

"no python application found" near the end.

Debug log: Untitled Wagtail CRX publishing system

...** SANDSTORM SUPERVISOR: Starting up grain. Sandbox type: userns
2023-03-16T14:50:21.401985Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161)
2023-03-16T14:50:21.401990Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000)
2023-03-16T14:50:21.409698Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.32) initializing of server in progress as process 15
2023-03-16T14:50:21.421961Z 0 [Warning] [MY-000081] [Server] option 'innodb-log-file-size': unsigned value 1048576 adjusted to 4194304.
2023-03-16T14:50:21.423056Z 0 [Warning] [MY-013907] [InnoDB] Deprecated configuration parameters innodb_log_file_size and/or innodb_log_files_in_group have been used to compute innodb_redo_log_capacity=8388608. Please use innodb_redo_log_capacity instead.
2023-03-16T14:50:21.428256Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-03-16T14:50:21.433308Z 1 [ERROR] [MY-012585] [InnoDB] Linux Native AIO interface is not supported on this platform. Please check your OS documentation and install appropriate binary of InnoDB.
2023-03-16T14:50:21.433580Z 1 [Warning] [MY-012654] [InnoDB] Linux Native AIO disabled.
2023-03-16T14:50:21.729150Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
2023-03-16T14:50:23.198127Z 6 [Note] [MY-010454] [Server] A temporary password is generated for root@localhost: f(6F?pcF_g%!
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
2023-03-16T14:50:26.231057Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161)
2023-03-16T14:50:26.231063Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000)
2023-03-16T14:50:26.444473Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.32) starting as process 58
2023-03-16T14:50:26.456637Z 0 [Warning] [MY-000081] [Server] option 'innodb-log-file-size': unsigned value 1048576 adjusted to 4194304.
2023-03-16T14:50:26.458440Z 0 [Warning] [MY-013907] [InnoDB] Deprecated configuration parameters innodb_log_file_size and/or innodb_log_files_in_group have been used to compute innodb_redo_log_capacity=8388608. Please use innodb_redo_log_capacity instead.
2023-03-16T14:50:26.464896Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
2023-03-16T14:50:26.470658Z 1 [ERROR] [MY-012585] [InnoDB] Linux Native AIO interface is not supported on this platform. Please check your OS documentation and install appropriate binary of InnoDB.
2023-03-16T14:50:26.470819Z 1 [Warning] [MY-012654] [InnoDB] Linux Native AIO disabled.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
2023-03-16T14:50:26.595486Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
waiting for mysql to be available at /var/run/mysqld/mysqld.sock.
2023-03-16T14:50:26.961182Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2023-03-16T14:50:26.961287Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2023-03-16T14:50:26.973158Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: /var/run/mysqld/mysqlx.sock
2023-03-16T14:50:26.973248Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.32'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server - GPL.
waiting for uwsgi to be available at /var/run/uwsgi.sock.
*** Starting uWSGI 2.0.19.1-debian (64bit) on [Thu Mar 16 14:50:27 2023] ***
compiled with version: 10.2.1 20210110 on 11 June 2021 09:08:33
os: Linux-5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13)
nodename: sandbox
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /
detected binary path: uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 7459
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /var/run/uwsgi.sock fd 3
Python version: 3.9.2 (default, Feb 28 2021, 17:03:44)  [GCC 10.2.1 20210110]
PEP 405 virtualenv detected: /opt/app-venv
Set PythonHome to /opt/app-venv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x564153008470
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72936 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
failed to open python file /opt/app/main.py
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 113, cores: 1)
--- no python application found, check your startup logs for errors ---
[pid: 113|app: -1|req: -1/1] 127.0.0.1 () {58 vars in 1224 bytes} [Thu Mar 16 14:50:27 2023] GET / => generated 21 bytes in 0 msecs (HTTP/1.1 500) 2 headers in 83 bytes (0 switches on core 0)
** SANDSTORM SUPERVISOR: Grain still in use; staying up for now.
*** lost connection to grain (probably because it shut down) ***

Anyway, thanks! Excited to learn about Sandstorm and to work with my first server. :)

--

[Jacob Weisz]

Can you put your packaging files somewhere online so we can take a look?

--

  Jacob Weisz

  goo...@jacobweisz.com

--

You received this message because you are subscribed to the Google Groups "Sandstorm Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sandstorm-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/cbbad2e4-ae46-4a85-8593-cb357b3d5978%40app.fastmail.com.

[Jackson Couse]

OK great, thanks Jacob. Here's the zipped project: https://user.fm/files/v2-62b39843f4c5c50a47b4cd59cb0186f8/WagtailCRX.tar.gz

[Jacob Weisz]

Sorry about the delay getting back. So there's a few things that stand out from the above messages and examining the project so far:

The plain spk packaging tool was designed to grab a locally-installed application, and make it into a Sandstorm package, but vagrant-spk is a framework where you programmatically set up the environment for the application.

It doesn't look like you've done anything with the default setup.sh, build.sh, and launcher.sh files in .sandstorm yet. These are kinda the core to building a Sandstorm app with vagrant-spk. vagrant-spk creates a virtual machine to install Sandstorm into, and the project folder (generally the folder above the .sandstorm folder) is mounted at /opt/app. So, for example, build.sh will try to install dependencies from /opt/app/requirements.txt, but your requirements.txt would actually be at /opt/app/mysite/requirements.txt because you have that extra subfolder in there. Similarly, launcher.sh is trying to launch /opt/app/main.py and our Python package builds it's venv at /opt/app-venv (all within the Vagrant VM, of course).

The 104 MB grain size you are seeing is actually something you may not be intending to use in your package at all, considering I see an sqlite file: The default scripts in the uwsgi stack install and start up MySQL, which generates about 100 MB of *stuff* when you start it. And as a final note, the application directory for a Sandstorm app is read-only (i.e. part of the app package, not a grain package), so presuming you need to be able to edit that db.sqlite3 file, it needs to be created in /var at runtime.

This is an existing Django app package for Sandstorm that is relatively recently crafted: https://github.com/babybuddy/babybuddy-sandstorm You'll notice setup.sh sets up the environment for Python and SQLite, build.sh gets and sets up the actual application code from the main BabyBuddy repo, and launcher.sh, which runs inside the grain sandbox, creates the grain's storage (/var contents) and launches the application.

A bunch of the apps you mention, I would personally love to see on Sandstorm, and I'm happy to help as much as I can with any of these efforts. I hope the above helps, and again, sorry it took me four days to get here.

--

  Jacob Weisz

  goo...@jacobweisz.com

To view this discussion on the web visit https://groups.google.com/d/msgid/sandstorm-dev/f560ad77-bb00-45c3-b3b2-09397bd3a68cn%40googlegroups.com.