File permission issues when running script as scheduler

[lamest.e...@gmail.com]

I have a simple application whose scheduler automatically backs up rolling log files in a database. 

There is an option in the controller to manually update whenever you want (rather than waiting for the log to roll).

The scheduler task has to handle such cases by comparing two versions of the log file to avoid duplication (the instance at the time of manual update, and the instance at the time of roll-over)

The source of the files looks like this

I'm seeing this error in the database logs for the failed scheduled task:

Traceback (most recent call last):

   File "/home/www-data/web2py/gluon/scheduler.py", line 315, in executor

     result = dumps(_function(*args, **vars))

   File "applications/viewer/models/scheduler.py", line 127, in wrapper_monitor

     old_file = open(os.path.join(request.folder, 'temp', 'wrapper.log.old'), 'w+')

 IOError: [Errno 13] Permission denied: 'applications/viewer/temp/wrapper.log.old'

I believe it is failing on line 121 because the log.new is not blank ( and the script fails again at 127 due to same permission error)

This is what is left over after the error (it would normally clean up after itself, had it completed successfully)

So I restored the log files to a state that the scheduler is designed to handle, copy/pasted the scheduler task logic into the controller, and manually executed it (via url in browser), and everything works great.

This is what I see afterwards (commented out the code that cleans up after itself).

This seems like a user/permissions 101 issue to me but I am pretty new to this whole Linux thing. I am running Linux Mint 17.3 and used the included script for Ubuntu to deploy web2py.

To start the scheduler I am executing nohup python web2py.py -K <appname> &

The resulting Python process shows up as my username. The apache service shows up as www-data. The service that is generating the logs is also under my username, if that matters.

Any help appreciated in getting this thing to run in the model via scheduler.

[Niphlod]

if your files are created by the www-data process and are not for anyone else to touch, the problem will rise. 

you can't expect an app running in the same permission env on www-data to behave (permission-wise) as one running on your user ("fivestar", I guess?)

study your permission (start with 777, and acknowledge 777 is not safe) or start the scheduler process in the same security env (su - www-data -c ...)

[lamest.e...@gmail.com]

Thank you for your response. I did run the scheduler process as www-data and it was able to open the file, however, new error on the database insert.

scheduler_run table shows status of TIMEOUT on the task, but the timeout is set to zero, which should be unlimited?

This script takes about 5 minutes to execute due to the large number of inserts (I have been playing around with different ways to insert to MySQL but so far 5 minutes is the shortest I can manage).

Is there some timeout setting at the system level that is causing this error?

Traceback (most recent call last):

   File "/home/www-data/web2py/gluon/scheduler.py", line 315, in executor

   result = dumps(_function(*args, **vars))

   File "applications/viewer/models/scheduler.py", line 139, in wrapper_monitor

   db3.IGN_wrapper.insert(log_ts = int(time.mktime(time.strptime(item[0], '%Y/%m/%d %H:%M:%S'))), entry_type = item[1], entry = item[2])

   File "/home/www-data/web2py/gluon/packages/dal/pydal/objects.py", line 726, in insert

   ret = self._db._adapter.insert(self, self._listify(fields))

   File "/home/www-data/web2py/gluon/packages/dal/pydal/adapters/base.py", line 746, in insert

   raise e Error: (<type 'exceptions.SystemExit'>, SystemExit(1,)) 

Once again, this part works just fine when executed from a controller.

[Niphlod]

0 is not allowed, so it's impossible you queued it. allowed ranges are positive from 1 to - possibly - 2^64 . Anyway, set 600 (10 minutes) to avoid timeout issues.

[lamest.e...@gmail.com]

correct Niphlod, I was viewing through the appadmin and the colum names were shifted. It was not the timeout column that had a zero value. I increased the timeout (it was default value of something like 60 seconds) it is working now. Thanks again.