Using of Model. __post_setup__

[edbo....@gmail.com]

Hello,

I think I'm using the Model. __post_setup__ class not correctly, but how I'm using it is working in Tryton-3.6.5 but not in Tryton-4.0.0 and maybe there are other ways to do it.

What I'm trying to do, is to build an IOT-platform into Tryton where agents can connect to and update several values they allowed to change. To minimize database-queries and updating database-records or inserting new ones, I build a dict which lives in memory. The agents send the new values in and Tryton will update the values in the dict, and a cron-job will archive those values into the database at certain moments. The new values are also used to trigger events like sending mail, executing scripts etc.

To create the dict in memory I'm using the __post_setup__ class to populate the dict with the several values.

    @classmethod
    def __post_setup__(cls):
        super(Trigger, cls).__post_setup__()

        # get the triggers
        triggers = cls.search( [('id', '>', 0 )] )
       
        if triggers:
            # Build the datastructure in memory
            for trigger in cls.browse( triggers ):

                sensors = []
                on_activate = []
                on_deactivate = []
               
                # get the sensors and conditions
                for sensor in trigger.sensor:
                    if len(sensor.condition) > 2:
                        sensors.append({
                            "id"        : sensor.id,
                            "sensor"    : sensor.sensor.id,
                            "condition" : sensor.condition,
                            })
                                   
                # only add the trigger to the list when there are
                # sensors and conditions
                if len(sensors):
                    MEM_TRIGGERS.append({
                        "id"         : trigger.id,
                        "name"       : trigger.name,
                        "sequence"   : trigger.sequence,
                        "min_delay"  : trigger.minimum_time_delay,
                        "min_active" : trigger.minimum_time_active,
                        "when"       : datetime.datetime.utcnow(),
                        "activated"  : trigger.activated,
                        "enabled"    : trigger.enabled,
                        "sensors"    : sensors
                    })

I have extended the ir.trigger with
sensor = fields.One2Many('iot.triggersensors', 'trigger', 'Sensor')

And the iot.triggersensors

class TriggerSensor(ModelSQL, ModelView):
    "TriggerSensor"
    __name__ = 'iot.triggersensors'

    name = fields.Char('Name')
    trigger = fields.Many2One('ir.trigger', 'Trigger', domain=[('on_sensor','=',True)], ondelete='CASCADE')
    condition = fields.Char('Condition')


This is working in Tryton version 3.65, but in 4.0.0 I get

  File "/home/ed/ERP/iot/trytond-4.0.0/trytond/pool.py", line 229, in setup
    cls.__post_setup__()
  File "/home/ed/ERP/iot/trytond-4.0.0/trytond/modules/iot/iot.py", line 1173, in __post_setup__
    for sensor in trigger.sensor:
  File "/home/ed/ERP/iot/trytond-4.0.0/trytond/model/modelstorage.py", line 1352, in __getattr__
    read_data = self.read(list(ids), ffields.keys())
  File "/home/ed/ERP/iot/trytond-4.0.0/trytond/model/modelsql.py", line 744, in read
    getter_result = field.get(ids, cls, fname, values=result)
  File "/home/ed/ERP/iot/trytond-4.0.0/trytond/model/fields/one2many.py", line 86, in get
    field = Relation._fields[self.field]
AttributeError: type object 'iot.triggersensors' has no attribute '_fields'

So my question is, are there other methods to accomplish this (after the Tryton-server is started create a dict into memory)? or is this a bug in Tryton-4.0.0?



Regards,

Ed

[Cédric Krier]

It is definitely not a bug in 4.0. Your code was working before by
chance. The ORM should never be used during the initialization of the
Pool. Your code doesn't work any more probably because of the
optimization of the pool initialization.

Also you should be careful with global variable, your structure is
probably not thread-safe.

Finally, are you sure you need this? Of course database query is
expensive but included in all the stack of a request, I don't think it
has so much effect.

--
Cédric Krier - B2CK SPRL
Email/Jabber: cedric...@b2ck.com
Tel: +32 472 54 46 59
Website: http://www.b2ck.com/

[edbo....@gmail.com]

Hmm, ok, but the documentation says "Setup the class AFTER added into the trytond.pool.Pool". So IMO this means that the class is added into the pool and then __post_setup__ will be called.

 

Also you should be careful with global variable, your structure is
probably not thread-safe.

I don't use global. In my iot.py I have
MEM_SENSORS = {}      # store sensor values in memory

just below values like
__metaclass__ = PoolMeta
logger = logging.getLogger(__name__)
STATES = {
    'readonly': ~Eval('active', True),
    }
DEPENDS = ['active']

So I think this is not global and I have written an alternative write-function for the agents, so the agents are just like clients but very restricted in their possibilities. But you are right about not being thread-safe. That's why I'm thinking to put Tryton-celery in between. But I have to dig deeper into that and maybe I can solve this issue with that.

 

Finally, are you sure you need this? Of course database query is
expensive but included in all the stack of a request, I don't think it
has so much effect.

I'm running the system on a BeagleBone Black and it's running surprisingly well! even better then I have thought of. I have one agent which also runs on the BB and collects data from 8 sensors at several intervals (minimal interval 2 sec, maximum interval 5 minutes). The memory consumption of postgresql, tryton-server and agent is around 30MB and the CPU is running around 30% load. Connecting with the client to get the archived values will take some seconds, but that's not a problem.

I'm switched to the memory-part because the BB couldn't keep up with updating the records in the database and starts to swap. I also tried to use sqlite in memory, but the same happened. So I need this kind of behavior and it's just working well.