Error uploading SKOS

[jesus san miguel]

I get this error while trying to import a 5MB SKOS file:

These are my values on bootstap.php

$CONFIG = array(
  'atom.development_mode'   => filter_var(getenv_default('ATOM_DEVELOPMENT_MODE', false), FILTER_VALIDATE_BOOLEAN),
  'atom.elasticsearch_host' => getenv_or_fail('ATOM_ELASTICSEARCH_HOST'),
  'atom.memcached_host'     => getenv_or_fail('ATOM_MEMCACHED_HOST'),
  'atom.gearmand_host'      => getenv_or_fail('ATOM_GEARMAND_HOST'),
  'atom.mysql_dsn'          => getenv_or_fail('ATOM_MYSQL_DSN'),
  'atom.mysql_username'     => getenv_or_fail('ATOM_MYSQL_USERNAME'),
  'atom.mysql_password'     => getenv_or_fail('ATOM_MYSQL_PASSWORD'),
  'atom.debug_ip'           => getenv_default('ATOM_DEBUG_IP', ''),
  'php.max_execution_time'  => getenv_default('ATOM_PHP_MAX_EXECUTION_TIME', '120'),
  'php.max_input_time'      => getenv_default('ATOM_PHP_MAX_INPUT_TIME', '120'),
  'php.memory_limit'        => getenv_default('ATOM_PHP_MEMORY_LIMIT', '4G'),
  'php.post_max_size'       => getenv_default('ATOM_PHP_POST_MAX_SIZE', '720M'),
  'php.upload_max_filesize' => getenv_default('ATOM_PHP_UPLOAD_MAX_FILESIZE', '640M'),
  'php.max_file_uploads'    => getenv_default('ATOM_PHP_MAX_FILE_UPLOADS', '20'),
  'php.date.timezone'       => getenv_default('ATOM_PHP_DATE_TIMEZONE', 'Europe/Madrid')
);

[José Raddaoui]

Hola Jesús,

Changing the value of 'php.upload_max_filesize' in 'bootstap.php' and recreating the 'atom' container works fine for me ...

  ---> 

Could it be that the host has the 'ATOM_PHP_UPLOAD_MAX_FILESIZE' env. var. set to 2M somewhere? You could check if that's the case with 'env | grep ATOM' (using the same user and session than docker-compose).

Best regards,

Radda.

[jesus san miguel]

Env. var is not present, and recreating the container does not help in my case:

[José Raddaoui]

Hi Jesús,

Sorry, it's not in the host where you should look for that env. var. It should be set inside the container, so maybe it's set in the environment or docker-compose files?

Best,

Radda.

[jesus san miguel]

I do not see the env var inside the containers:

or in the compose files...

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/GMbUjR95EVY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/18fc65ee-e395-4309-99d9-3a62e0e54272n%40googlegroups.com.

[José Raddaoui]

Hi Jesús,

Quite puzzling ... I tried with the development branch (qa/2.x using php:7.3-fpm-alpine) and the stable/2.6.x branch (using php:7.2-fpm-alpine), in both cases updating the base image to the latest version and rebuilding the containers ...

docker pull php:7.2-fpm-alpine

docker-compose up -d --build

Without changes in those branches, in both environments the SKOS import shows the default 64 MB. If "git diff" doesn't show any modification that may be causing this mismatch, you could try checking the actual configuration in place in the container. For example, you could modify the "index.php" file, leaving only ...

<?php

phpinfo();

Then, going to the AtoM homepage in the browser will show the configuration in that environment. You can search for "upload" to check the values and also look for the configuration files:

Loaded Configuration File

Scan this dir for additional .ini files

Additional .ini files parsed

Then check the values in those .ini files and the PHP configuration:

docker-compose exec atom cat /usr/local/etc/php/php.ini

docker-compose exec atom cat /usr/local/etc/php-fpm.d/atom.conf

I hope this brings some light to the issue.

Best,

Radda.

[jesus san miguel]

Hi Radda,

I do not see any related differences in git diff (just in case I attached the result)

These are the values reported by phpinfo:

Configuration File (php.ini) Path	/usr/local/etc/php
Loaded Configuration File	/usr/local/etc/php/php.ini

Scan this dir for additional .ini files

/usr/local/etc/php/conf.d

Additional .ini files parsed

/usr/local/etc/php/conf.d/docker-php-ext-calendar.ini, /usr/local/etc/php/conf.d/docker-php-ext-gettext.ini, /usr/local/etc/php/conf.d/docker-php-ext-memcache.ini, /usr/local/etc/php/conf.d/docker-php-ext-mysqli.ini, /usr/local/etc/php/conf.d/docker-php-ext-opcache.ini, /usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini, /usr/local/etc/php/conf.d/docker-php-ext-sockets.ini, /usr/local/etc/php/conf.d/docker-php-ext-sodium.ini, /usr/local/etc/php/conf.d/docker-php-ext-xsl.ini, /usr/local/etc/php/conf.d/docker-php-ext-zip.ini

And these are the values that the container is using: Shouldn't php.ini include some values like upload_max_filesize instead of being empty?

[atomadminpre@svatomi atom]$ docker-compose exec atom cat /usr/local/etc/php/php.ini
[atomadminpre@svatomi atom]$ docker-compose exec atom cat /usr/local/etc/php-fpm.d/atom.conf
[global]
error_log = /proc/self/fd/2
daemonize = no

[atom]
access.log = /proc/self/fd/2
clear_env = no
catch_workers_output = yes
user = root
group = root
listen = [::]:9000
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3
env[ATOM_DEBUG_IP] = 172.22.0.1
[atomadminpre@svatomi atom]$

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/e939726b-3caf-4d70-9929-8b45e8c7680cn%40googlegroups.com.

[José Raddaoui]

Hola Jesús,

As you noticed, it's really rare that the php.ini file is empty in the container. I tested with your values from bootstrap.php and it worked well on my instance ...

$ docker-compose exec atom cat /usr/local/etc/php/php.ini

[PHP]

output_buffering = 4096

expose_php = off

log_errors = on

error_reporting = E_ALL

display_errors = stderr

display_startup_errors = on

max_execution_time = 120

max_input_time = 120

memory_limit = 2048M

log_errors = on

post_max_size = 720M

default_charset = UTF-8

cgi.fix_pathinfo = off

upload_max_filesize = 640M

max_file_uploads = 20

date.timezone = Europe/Madrid

session.use_only_cookies = off

opcache.fast_shutdown = on

opcache.max_accelerated_files = 10000

opcache.validate_timestamps = off

# Development-specific configuration

expose_php = on

opcache.validate_timestamps = on

I also changed the ATOM_DEVELOPMENT_MODE to "off", ran "docker-compose up -d" again and the config was modified:

$ docker-compose exec atom cat /usr/local/etc/php/php.ini

[PHP]

output_buffering = 4096

expose_php = off

log_errors = on

error_reporting = E_ALL

display_errors = stderr

display_startup_errors = on

max_execution_time = 120

max_input_time = 120

memory_limit = 2048M

log_errors = on

post_max_size = 720M

default_charset = UTF-8

cgi.fix_pathinfo = off

upload_max_filesize = 640M

max_file_uploads = 20

date.timezone = Europe/Madrid

session.use_only_cookies = off

opcache.fast_shutdown = on

opcache.max_accelerated_files = 10000

opcache.validate_timestamps = off

This is the part of the bootstrap process that creates the php.ini based in the config, but I don't know what may be going on, sorry I can't be more helpful.

https://github.com/artefactual/atom/blob/stable/2.6.x/docker/bootstrap.php#L234-L275

Best regards,

Radda.

[jesus san miguel]

Hi Radda,

I'm no programmer but I tried to isolate the issue creating a "kk.php" file to write the php.ini and ran it into the container. In this way I was trying to simulate the bootstap.php that is run in entrypoint.sh, but got an unexpected end of file error:

bash-5.0# cat kk.php
<?php                                                                                                              
                                                                         
define('_ATOM_DIR', '/atom/src');
define('_ETC_DIR', '/usr/local/etc');

$CONFIG = array(
  'atom.development_mode'   => filter_var(getenv_default('ATOM_DEVELOPMENT_MODE', false), FILTER_VALIDATE_BOOLEAN),
  'atom.elasticsearch_host' => getenv_or_fail('ATOM_ELASTICSEARCH_HOST'),
  'atom.memcached_host'     => getenv_or_fail('ATOM_MEMCACHED_HOST'),
  'atom.gearmand_host'      => getenv_or_fail('ATOM_GEARMAND_HOST'),
  'atom.mysql_dsn'          => getenv_or_fail('ATOM_MYSQL_DSN'),
  'atom.mysql_username'     => getenv_or_fail('ATOM_MYSQL_USERNAME'),
  'atom.mysql_password'     => getenv_or_fail('ATOM_MYSQL_PASSWORD'),
  'atom.debug_ip'           => getenv_default('ATOM_DEBUG_IP', ''),
  'php.max_execution_time'  => getenv_default('ATOM_PHP_MAX_EXECUTION_TIME', '120'),
  'php.max_input_time'      => getenv_default('ATOM_PHP_MAX_INPUT_TIME', '120'),

  'php.memory_limit'        => getenv_default('ATOM_PHP_MEMORY_LIMIT', '512M'),
  'php.post_max_size'       => getenv_default('ATOM_PHP_POST_MAX_SIZE', '72M'),
  'php.upload_max_filesize' => getenv_default('ATOM_PHP_UPLOAD_MAX_FILESIZE', '64M'),

  'php.max_file_uploads'    => getenv_default('ATOM_PHP_MAX_FILE_UPLOADS', '20'),

  'php.date.timezone'       => getenv_default('ATOM_PHP_DATE_TIMEZONE', 'America/Vancouver')
);

$php_ini = <<<EOT

[PHP]                                                                
output_buffering = 4096                                            
expose_php = off                                                
log_errors = on                                                      
error_reporting = E_ALL                                              
display_errors = stderr                                            
display_startup_errors = on                                                        

max_execution_time = ${CONFIG['php.max_execution_time']}                        
max_input_time = ${CONFIG['php.max_input_time']}                                
memory_limit = ${CONFIG['php.memory_limit']}                                    
log_errors = on                                                                      
post_max_size = ${CONFIG['php.post_max_size']}                                  

default_charset = UTF-8                                                                
cgi.fix_pathinfo = off                                  

upload_max_filesize = ${CONFIG['php.upload_max_filesize']}
max_file_uploads = ${CONFIG['php.max_file_uploads']}      
date.timezone = ${CONFIG['php.date.timezone']}            

session.use_only_cookies = off                            
opcache.fast_shutdown = on                                
opcache.max_accelerated_files = 10000                    
opcache.validate_timestamps = off                            

EOT;                                                                                                  
                                                                                             
@unlink(_ETC_DIR.'/php/php.ini');
file_put_contents(_ETC_DIR.'/php/php.ini', $php_ini);
bash-5.0# php kk.php

Parse error: syntax error, unexpected end of file in /atom/src/docker/kk.php on line 50
bash-5.0# 

The php syntax seems correct to me, and it checked fine in an online parser...

The environment vars in the container are:

bash-5.0# env
xpack.security.enabled=false
cluster.routing.allocation.disk.threshold_enabled=false
bootstrap.memory_lock=true
ATOM_MEMCACHED_HOST=memcached
ATOM_DEVELOPMENT_MODE=off
HOSTNAME=4f47f9f932c8
PHP_VERSION=7.2.33
PHP_MD5=
PHP_INI_DIR=/usr/local/etc/php
GPG_KEYS=1729F83938DA44E27BA0F4D3DBDB397470D12172 B1B44D8F021E4E2D6021E995DC9FF8D3EE5AF27F
PHP_LDFLAGS=-Wl,-O1 -pie
ATOM_MYSQL_PASSWORD=atom_12345
PWD=/atom/src/docker
MYSQL_ROOT_PASSWORD=my-secret-pw
ATOM_DEBUG_IP=172.22.0.1
ES_JAVA_OPTS=-Xms640m -Xmx640m
MYSQL_PASSWORD=atom_12345
LD_PRELOAD=/usr/lib/preloadable_libiconv.so:/opt/dynatrace/oneagent/agent/bin/current/linux-x86-64/liboneagentproc.so
MYSQL_USER=atom
HOME=/root
ATOM_ELASTICSEARCH_HOST=elasticsearch
PHP_SHA256=0f160a3483ffce36be5962fab7bcf09d605ee66c5707df83e4195cb796bbb03a
ATOM_MYSQL_DSN=mysql:host=percona;port=3306;dbname=atom;charset=utf8mb4
PHPIZE_DEPS=autoconf dpkg-dev dpkg file g++ gcc libc-dev make pkgconf re2c
TERM=xterm
ATOM_GEARMAND_HOST=gearmand
PHP_URL=https://www.php.net/distributions/php-7.2.33.tar.xz
FOP_HOME=/usr/share/fop-2.1
PHP_EXTRA_CONFIGURE_ARGS=--enable-fpm --with-fpm-user=www-data --with-fpm-group=www-data --disable-cgi
SHLVL=1
COMPOSER_ALLOW_SUPERUSER=1
PHP_CFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
MYSQL_DATABASE=atom
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
ATOM_MYSQL_USERNAME=atom
PHP_ASC_URL=https://www.php.net/distributions/php-7.2.33.tar.xz.asc
PHP_CPPFLAGS=-fstack-protector-strong -fpic -fpie -O2 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
_=/usr/bin/env
OLDPWD=/atom/src
bash-5.0# 

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/997ee908-fb2e-402e-93be-481994052ae8n%40googlegroups.com.

[José Raddaoui]

Hi Jesús,

Syntax looks good to me too, but I ran it locally and it complained about missing functions...

$ php test.php

PHP Fatal error:  Uncaught Error: Call to undefined function getenv_default() in /home/radda/test.php:7

Stack trace:

#0 {main}

  thrown in /home/radda/test.php on line 7

After adding the functions bellow at the beginning of the script, it worked as expected ...

function getenv_default($name, $default)

{

  $value = getenv($name);

  if (false === $value)

  {

    return $default;

  }

  return $value;

}

function getenv_or_fail($name)

{

  $value = getenv($name);

  if (false === $value)

  {

    echo "Environment variable ${name} is not defined!";

    exit(1);

  }

  return $value;

}

I don't really know what may be going on, but maybe it's obscuring the real error somehow. Maybe a full recreation of the container fixes the PHP weirdness you're seeing ...

docker-compose up -d --build --force-recreate

Best regards,

Radda.

[jesus san miguel]

--force-recreate was my first option, but ended in the same dark place.

Adding the functions did no good to me:

I can only think that the host RH 7.8 does not have php installed, but I am stumped as why the creation script gets fails with the syntax error and a simple <?php phpinfo(); works fine:

bash-5.0# php kk.php

Parse error: syntax error, unexpected end of file in /atom/src/docker/kk.php on line 76

bash-5.0# cat kk.php
<?php                                                                                                              
                                                                         
define('_ATOM_DIR', '/atom/src');
define('_ETC_DIR', '/usr/local/etc');

function getenv_default($name, $default)
{
  $value = getenv($name);

  if (false === $value)
  {
    return $default;
  }

  return $value;
}


function getenv_or_fail($name)
{
  $value = getenv($name);

  if (false === $value)
  {
    echo "Environment variable ${name} is not defined!";
    exit(1);
  }

  return $value;
}

bash-5.0# cd ..
bash-5.0# ls
CONTRIBUTING.md     README.md           cache               css                 ead.dtd             index.php           lib                 qubit_dev.php       uploads
COPYRIGHT           SECURITY.md         composer.json       data                favicon.ico         index.php.original  log                 robots.txt          vendor
Dockerfile          apps                composer.lock       docker              fotos.csv           init                phpunit.xml         symfony
LICENSE             build.xml           config              downloads           images              js                  plugins             test
bash-5.0# cat index.php
<?php
phpinfo();
bash-5.0# php index.php
phpinfo()
PHP Version => 7.2.33

System => Linux 4f47f9f932c8 3.10.0-1127.8.2.el7.x86_64 #1 SMP Thu May 7 19:30:37 EDT 2020 x86_64
Build Date => Sep  1 2020 06:37:19
Configure Command =>  './configure'  '--build=x86_64-linux-musl' '--with-config-file-path=/usr/local/etc/php' '--with-config-file-scan-dir=/usr/local/etc/php/conf.d' '--enable-option-checking=fatal' '--with-mhash' '--enable-ftp' '--enable-mbstring' '--enable-mysqlnd' '--with-password-argon2' '--with-sodium=shared' '--with-pdo-sqlite=/usr' '--with-sqlite3=/usr' '--with-curl' '--with-libedit' '--with-openssl' '--with-zlib' '--enable-fpm' '--with-fpm-user=www-data' '--with-fpm-group=www-data' '--disable-cgi' 'build_alias=x86_64-linux-musl'
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /usr/local/etc/php
Loaded Configuration File => /usr/local/etc/php/php.ini
Scan this dir for additional .ini files => /usr/local/etc/php/conf.d
Additional .ini files parsed => /usr/local/etc/php/conf.d/docker-php-ext-calendar.ini,

/usr/local/etc/php/conf.d/docker-php-ext-gettext.ini,
/usr/local/etc/php/conf.d/docker-php-ext-memcache.ini,
/usr/local/etc/php/conf.d/docker-php-ext-mysqli.ini,
/usr/local/etc/php/conf.d/docker-php-ext-opcache.ini,
/usr/local/etc/php/conf.d/docker-php-ext-pdo_mysql.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sockets.ini,
/usr/local/etc/php/conf.d/docker-php-ext-sodium.ini,
/usr/local/etc/php/conf.d/docker-php-ext-xsl.ini,
/usr/local/etc/php/conf.d/docker-php-ext-zip.ini

PHP API => 20170718
PHP Extension => 20170718
Zend Extension => 320170718
...

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1ac4866c-ea33-4669-8cac-1360c1a2f77cn%40googlegroups.com.

[José Raddaoui]

Hi Jesús,

Are you running the PHP command and checking the php.ini in the host? I only did it to check the script but you should check always inside the container, where RH 7.8 won't be an issue.

Best,

Radda.