File upload issue in Apache2.4 running in singularity (3.7.x) instance

[Puthick Hok]

Hi,

I'm running Apache2.4 inside singularity instance with ptp network. The image of the singularity instance is based on OpenSUSE Leap 15.1. I have 2 hosts running OpenSUSE 15.2 and Centos 7 that run the same setup of the singularity instance with Apache2.4 inside the instance. On Centos 7, everything is fine. On OpenSUSE 15.2 hosting environment, I have a problem with a file upload. The application that I'm working on requires a big upload file, in this case 650MB. During the curl upload, the process is terminated by something that I don't know. If the upload file is 450MB went through the network and got to the Apache. If the file size is 650MB, the file didn't reach the apache and there is connection terminated and partial data received at apache log.

Regards,

Puthick

[v]

I'm more familiar with nginx, but usually when something like this happens it's because a parameter for the server needs to be tweaked. Have you looked at those related to max upload sizes for Apache? https://www.arclab.com/en/kb/php/how-to-increase-max-file-upload-size-apache-mysql.html. I'm not sure why there would be differences between the OS's, but it's worth giving a shot anyway. If that doesn't work I would try looking at permissions, and ownership of the uploaded file. If that doesn't work I'd look at the issues outlined here https://stackoverflow.com/questions/39308316/cant-upload-file-on-my-own-apache-server and maybe see if you can figure out why there isnt anything in the error logs.

--
You received this message because you are subscribed to the Google Groups "Singularity Community Edition" group.
To unsubscribe from this group and stop receiving emails from it, send an email to singularity-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/singularity-ce/d9b0b190-04c6-44bb-8940-3bac9ab60789n%40googlegroups.com.

[Puthick Hok]

I'm developing an analytic platform for agriculture that will use slurm to manage computing resources and singularity to manage the run time environment. In this platform, an authorised developer can upload a package in the form of a tar.gz file containing a singularity image and how the image is used to analyse data. Because of the complex software dependency like R, it is hard to build a singularity image under 500MB. I like singularity because of its simplicity and there is no need to have an image depends on other image.

We knew and encountered the apache file upload before. When I lowered the upload limit to only 50MB, and my upload is 300MB, at my curl client side I got a 500 internal server error message delivered in the HTML format and at the server error log I got an message saying the upload is exceeding the limit. With this 50MB upload limit, if the upload is 600MB, there is the same error message. When the upload limit in the apache is raised to 800MB, and the upload is 600MB, on my curl client side I got empty reply from the server and there is a segmentation fault at the apache log. If the upload is only below 500MB with upload limit of 800MB, the upload is successful.

I just realised that on openSUSE I have singularity 3.7.2 while on Centos I have 3.8.0.

[David Trudgian]

Hi,

The fact that the uploads are working when the host is CentOS 7 indicates that this is probably not an issue with SingularityCE, but might be something with the openSUSE host, or networking configuration relevant to it.

How is the application written, and what memory (RAM) is available on the CentOS vs openSUSE host?

There are bugs that come up in searches for relevant keywords where apache will segfault if e.g. mod_fcgid is used and there isn't enough space for the entire upload to fit in RAM: https://bugzilla.redhat.com/show_bug.cgi?id=1486653 - it could be something similar?

Some more information about the resource of the CentOS and openSUSE environments, and any other differences would be useful.

DT

[Puthick Hok]

Hi David,

The application is written in Perl using Dancer framework.

The CentOS host has 4GB of RAM and it is an LXC container machine while the OpenSUSE has 16GB of RAM and it is a bare metal host.

As it is so easy to deploy my application with the help from singularity, it would be interesting to see what would happen if the OpenSUSE is also an LXC machine instead of the bare metal.

There is more evidence now that this is more of a mod_fcgid issue.

Regards,

Puthick

You received this message because you are subscribed to a topic in the Google Groups "Singularity Community Edition" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/singularity-ce/8TB1jowJtXA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to singularity-c...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/singularity-ce/644f9fd5-89b1-458d-9456-a95b0034b6c1n%40googlegroups.com.