scheduler gives and error about datetime

[Pierre]

Gooday,

In the controler the function works correctly.  But as a task, it gives an error about Json not beeing able to deal with datetime.

any suggestion would be appreciated :)

here is the error :  TypeError: datetime.datetime(2020, 6, 16, 5, 17, 49) is not JSON serializable

here is the code:

#reload(sys)
#sys.setdefaultencoding('utf-8')
import sys
import os
import time
import datetime
import sqlite3
#
#------------------------------------------------------------------------------
def pump_most_recent():
    state = 'unknown'
    try:
        conn=sqlite3.connect('z:/alarm.sqlite')
    except sqlite3.Error as err:
        state = err # ='connexion impossible'
    else:
        state = 'connection ok'
    #   last entry in the local storage is kept in the acue table, have to get this in order to query the prod databases
        k = db(db.acue).select(db.acue.dstd2) # get the row
        l = k[0].dstd2                        # get the date of interest
        curs=conn.cursor()
        rows = curs.execute("select * from log_txt where date > ?", (l,)).fetchall()
        curs.close()
        conn.close()
    #
        for row in rows:
            a = datetime.datetime.strptime(row[1],'%Y-%m-%d : %H:%M:%S') # date avec le temps
            b = row[2]
            c = row[3]
            d = row[4]
            e = row[5]
            f = row[6]
            g = row[7]
            h = row[8]
            i = row[9]
            j = row[10]
    #
            db.alogs.insert(quand=a,etat=b,zone1=c,zone2=d,zone3=e,zone4=f,zone5=g,zone6=h,zone7=i,note=j)
    finally:
        return locals()
#
#-------------------------------------------------------------------------------
#
def acue_src_dst_update():
#
    z = acue_src_dst_show()
    a = z.get('a')
    b = z.get('b')
    c = z.get('c')
    d = z.get('d')
#
    db.acue.truncate()
    db.acue.insert(srcd1=a,srcd2=b,dstd1=c, dstd2=d)
    acue_after_update = db(db.acue).select()
#
    return locals()
#------------------------------------------------------------------------------------------------------
from gluon.scheduler import Scheduler
Scheduler(db,dict(sync_data=pump_most_recent,sync_dates=acue_src_dst_update))

[AGRogers]

Hi

I use this code to solve this problem:

result = json.dumps(result, indent=4, sort_keys=True, default=str)

Don't know what it does but it solves my problem.

I found it here:

https://stackoverflow.com/questions/11875770/how-to-overcome-datetime-datetime-not-json-serializable  

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/ba7a30ef-1e82-430e-8295-8fd4be74c806o%40googlegroups.com.

[Pierre]

thanks AG ............ I will look into your suggestion.

Fiddling with the error, found out that it's when reading back from sqlite that the error arise.  It's like something from web2py doesn't like the format of my date in the field of the table.

But at this time, i do not know how to tell "web2py" that the string representing the datetime in the field is formated like this  '%Y-%m-%d : %H:%M:%S'

wonder if the datetime format used in my sqlite table field is the problem.   So curius that everything works fine in the controler but not as a task.

To unsubscribe from this group and stop receiving emails from it, send an email to web...@googlegroups.com.

[AGRogers]

Yeah, i dont really understand how web2py sends locals() etc back to the view and how that differs from JSON data. I just keep trying random things until it works :)

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/b9332712-6352-4a98-a051-71149d83f134o%40googlegroups.com.

[Pierre]

I advance like you by trying tons of things :)

I did the most simple thing with the task ............ just putting a = datetime.datetime.now() ............ and it gives the exact same error.

Tasks doesn't like datetime period :)

Not yet figured out how to jsonify the task yet based on your suggestion, but still bumping my head on it :)

To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/b9332712-6352-4a98-a051-71149d83f134o%40googlegroups.com.

[AGRogers]

I haven't done anything in the scheduler yet. It's on my to do list though. I will come back here if I have any good suggestions. 

To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/0de5b6aa-5500-48eb-88b4-5cecf7a26005o%40googlegroups.com.

[Dave S]

On Wednesday, June 17, 2020 at 6:41:32 AM UTC-7, Pierre wrote:

Gooday,

In the controler the function works correctly.  But as a task, it gives an error about Json not beeing able to deal with datetime.

any suggestion would be appreciated :)

here is the error :  TypeError: datetime.datetime(2020, 6, 16, 5, 17, 49) is not JSON serializable

This does not look right:

 

from gluon.scheduler import Scheduler
Scheduler(db,dict(sync_data=pump_most_recent,sync_dates=acue_src_dst_update))

 

In a models file called scheduler.py, I have:

from gluon.scheduler import Scheduler

scheduler = Scheduler(db)


def taskfunc(arg1, arg2):
      var1 = do_something(arg1)
      if var1 in arg2:
           return "success"

In a controller, I kick off the task with 

    chunk1 = dostuff(request.args[0])
    string1 =  targets[rand()]
    tid = scheduler.queue_task('taskfunc', pvars={'arg1': chunk1, 'arg2': string2}, timeout=1200)

(the controller knows scheduler from the models)

Sending a date as an argument would be done by using a pvars dictionary in the same way.   For one of my tasks, I send a datetime via pvars.   The task also has a request.now.  (It has the whole request, but you can tell it's a scheduler request).

Do you find the documentation on Scheduler unclear?

/dps

[Pierre]

For a temporary solution, i made an external python prg and used cron on my pi to sync the databases everynight and works find with sqlite syntax.

It's way more extensive work but seems to be reliable for many days now.

I have read again the doc, and cannot figure out at all my mistake.

At this time i scheduled manualy the task via the admin interface to do tests, so i can follow the stage, queued, running, etc.

thanks

[AGRogers]

Thanks for the update. Hopefully I won't need it 😏

--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups "web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to web2py+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/web2py/38baa4e9-9942-4faf-bc57-4e12d59051beo%40googlegroups.com.