restore items state after an openhab restart

[gilles.l...@gmail.com]

Hi,

I'm using some"virtual" items  to allow some automation for others item by rules.

An example is my pool pump.

in the item file I put:
Switch  Switch_Automatisation_Piscine   "Automatisation Piscine"        (G_Piscine,G_Virtual)

in persitence file:

Switch_Automatisation_Piscine : strategy = everyChange, restoreOnStartup


If my switch "Switch_Automatisation_Piscine" is "ON" and if I restart Openhab, my switch is not restored to "ON".


My persistence service is mysql and openhab is 1.51


Is there something else to do ?

[gilles.l...@gmail.com]

I'm not alone,

As in this topic:

https://groups.google.com/forum/#!topic/openhab/vHKlICF8LAQ

[gilles.l...@gmail.com]

I check mysql database and the everychange instruction is ok.

I can see each entry with state and date in mysql

So why restore at startup isn't working. I try a debug start but can't see an error about persitence. I don't see any  DEBUG o.o.c.p.i.PersistenceManager[:377]- Restored .......


Some one can help me ? I don't know what to do.

[Hans Jørgensen]

I have the same problem... haven't looked into it much though, but it is annoying..

[Thomas E.-E.]

Hi,

have you tried the RRD4J Service just to narrow down the issue?

Thanks, Thomas E.-E.

[gilles.l...@gmail.com]

Hi Thomas,

Thank you listening.

I try to use rrd4j persitence.

here is my persistence settings:

// Persistence strategies have a name and a definition and are referred to in the "Items" section
Strategies {
        everyMinute      : "0 * * * * ?"
        everyHour       : "0 * * * * ?"
        everyDay        : "0 * * * * ?"

        // If no strategy is specified for an item entry below, the default list will be used.
        default = everyChange
}

/*
 * Each line in this section defines for which item(s) which strategy(ies) should be applied.
 * You can list single items, use "*" for all items or "groupitem*" for all members of a group
 * item (excl. the group item itself).
 */
Items {

        Switch_Arrosage_Automatique : strategy = everyChange, restoreOnStartup

}

I haven't any rrd file in "etc/rrd4j" whereas mysql has values.

My default persitence service is mysql. even if I try to change default to rrd4j, there is no changing.


In Conclusion, if rrd4j or mysql is used restore is'nt working. I can see mysql value, but rrd files are not created.

Do you have an idea ? Does a sart_debug can be usefull ?

Gilles

[gilles.l...@gmail.com]

Arf I'm stupid.

It's working very well when rrd4j persitence jar is in addon directory.

After a restart of openhab,  my virtual switch is restored with rrd4j persistence. So it"s working with rrd4j.

But with mysql persistence it's not working even if the data are stored.

Gilles.

[Thomas E.-E.]

Hi Gilles,

thanks for this update and good to know rrd4j is working fine for you!

Anybody of our mySQL specialists in to help out?

Best, Thomas E.-E.

[Chris Jackson]

I checked my system (using mySQL) and it seems to be working ok. I’ve not checked every item, but for sure it is restoring as I have values for items where the sensor has been removed and therefore hasn’t updated for months.

Chris

[aaron...@gmail.com]

MySql persistence is not restoring for me either. 

[Chris Jackson]

I don’t really understand why it wouldn’t work. The mySQL persistence service doesn’t know anything about restore on startup or any of the other strategies - this is all done by the openhab core. The mySQL service just knows about store and retrieve services, and clearly these work fine - at least for me, and I think Gilles said it was working fine for him as well? If you run HABmin, or use on of the history functions in a rule, do they work?

If so, then I think the issue must be in the core. As I said, restore on startup is working fine here, so it can work. Maybe this is another issue with startup order or something? Startup order of bindings is asynchronous I believe - maybe Thomas or Kai can comment on how the system ensures the persistence providers are running when the persistence core runs the startup, and if all the items are loaded at that time?

[Chris Jackson]

One other thought - what platform are you using, and are you also using other persistence services other than mySQL? Maybe this has an impact?

Chris

[gilles.l...@gmail.com]

Hi Chris, Thomas,

I'm on ubuntu 14.04 lts with lxc virtualization. My openhab is in 1.51 version and I've only rrd4j and mysql for persistence. I prefer only use mysql but the problem is there.

I use rrd4j for virtuals switchs and it's working.  If I want to use mysql for restoring a switch position it doesn't work.

I use phpmyadmin to see my values in database and I can see that values are stored (on , off etc ...)

If it's working for Chris perpahps we can compare mysql setup in mysql and openhab.

For example in my openhab.cfg for mysql I only have the connector, the user name and password nothing else.

Chris what are your parameters ? And what about mysql collation UTF8_general_ci or UTF8_unicode_ci ?

Gilles.

[Chris Jackson]

>
> I use rrd4j for virtuals switchs and it's working. If I want to use mysql for restoring a switch position it doesn't work.

Do you mean that it doesn’t work at all, or doesn’t work on startup? I don’t store any switches, but I do store door contacts, and this works fine - at least I’m using it to graph out the door open/close in HABdroid, and it’s ok - likewise in HABmin. I don’t know if I’ve checked this works with restore on startup (I don’t have this configured to restore on startup).

> Chris what are your parameters ? And what about mysql collation UTF8_general_ci or UTF8_unicode_ci ?

I have the following configuration
# the database url like 'jdbc:mysql://<host>:<port>/<user>'
mysql:url=jdbc:mysql://127.0.0.1/openhab

# the database user
mysql:user=xxxxx

# the database password
mysql:password=xxxxx

# reconnect counter
mysql:reconnectCnt=1

The server is set up for utf8_general_ci.

Chris

[Chris Jackson]

>>
>> I use rrd4j for virtuals switchs and it's working. If I want to use mysql for restoring a switch position it doesn't work.
> Do you mean that it doesn’t work at all, or doesn’t work on startup? I don’t store any switches, but I do store door contacts, and this works fine - at least I’m using it to graph out the door open/close in HABdroid, and it’s ok - likewise in HABmin. I don’t know if I’ve checked this works with restore on startup (I don’t have this configured to restore on startup).

I just checked the ‘restore on startup’ for my front door sensor and it works fine.

Chris

[Chris Jackson]

Having spent some more time looking at the restore on startup with mysql, I can’t see any problem. All I can think of is that this is a system startup issue - one scenario I can think of is this -:

The mysql binding starts, at this time OSGi registers the persistence service with the persistence manager, and the persistence manager reads the .persist file and runs through the items, doing the restore. However, over in the mysql binding, the connection to the database might not yet be complete. If the startup queries come in when the connection hasn’t been made, there is no debug message - it just silently fails.

I’ve just created a PR to add some debug to warn of this. I’ll have a bit more of a look to see if there’s a way to speed up initialisation, however I wouldn’t really want to block the system while the connection to the server is made as this could take some time in some cases (e.g. remote servers).

Chris

[gilles.l...@gmail.com]

Chris,

My problem is only at openhab startup.

I've got almost same setup for mysql in openhab. The only thing is mysql:reconnectCnt=1

Is this important to have this option ?

I got utf__general_ci too.

So it's not that.

Gilles

[gilles.l...@gmail.com]

It's clear in debug mode there is no information about a restore problem.

I will do a test with removing all my bindings and rules, to be sure there is no problem with that sort of things.

Gilles.

[Chris Jackson]

I’m reasonably sure that the problem is just a startup timing issue. It’s not a bug as such, but there’s no way to make sure that the persistence store is working when openhab calls the ‘restore on startup’ query. Once Thomas merges the changes I’ve made, this should be clear as I’ve added debug. However, I’m not sure that there’s a way to solve it since the system doesn’t provide any synchronisation, and I don’t think we’ve got control over when the store is registered with the core.

Let’s see what the debug shows….

Chris

[Chris Jackson]

>
> I've got almost same setup for mysql in openhab. The only thing is mysql:reconnectCnt=1
>
> Is this important to have this option ?

No - this isn’t important. It’s something that allows the binding to reconnect to mysql if there’s a timeout. A use a while back had a lot of problems with disconnects so I added some code to automatically reconnect on error.

Chris

[aaron...@gmail.com]

Interesting that there's no startup ordering control. If it's of any worth, I am running OSX and MySQL is just used for OpenHAB.

[Chris Jackson]

I think the system will ensure that the startup restore is run after the items are loaded, and once the persistence store is registered. So, for persistence stores that don’t take any time to start up (e.g. db4o and rrd4j), it’s not a problem. However, the persistence store is registered automatically with the persistence manager automatically when the bundle starts (at least that’s my understanding). At some later time, other things happen, and I don’t think there’s any ordering (maybe Kai or Thomas can comment). For example, the mysql system needs to read the user and password, and server config string out of the config file (actually, openhab will pass this information to the binding). Once this is read, then the binding can connect to mysql, but if the persistence manager tries the restore before this, then it’s not going to work……..

I’ve put some debug in so we’ll be able to see if this is what’s happening…

Chris

[Dimitar Valchev]

The same behavior with some configuration details
https://groups.google.com/forum/#!topic/openhab/898Y6GL_BC0

I've changed rrd4 to mysql, removed Z-Wave binding because of it - no improvement.
Seems that the items gets the proper values but after some time they are receiving ON command in the event bus.

[Chris Jackson]

Hi Gilles,

Thomas has merged the debug I added, so you should be able to grab the new snapshot mysql binding (I'm not sure if it will be in todays or tomorrows) and it would be interesting to see what this shows...

Cheers

Chris

[gilles.l...@gmail.com]

Hi Chris,

Ok I will test and give a feeback.

I just confirm a debug session with old mysql binding.

The mysql connection arrived later than the persistence is seen.

Cheers,

Gilles.

[Dimitar Valchev]

Just tested it with the latest available SNAPSHOT #739 - no difference.
Some items are uninitialized while other (all that has gpio binding) are switched to ON on system start.
openhab.log attached

Cheers
Dimitar

[Markus Rathgeb]

Perhaps this line of your log is relevant:

 

22:18:34.192 WARN  o.o.c.p.e.PersistenceExtensions[:446]- There is no queryable persistence service registered with the name 'mysql'

 

[Chris Jackson]

On Tuesday, 7 October 2014 20:33:32 UTC+1, Dimitar Valchev wrote:

Just tested it with the latest available SNAPSHOT #739 - no difference.

Please can you provide a debug log?

Thanks

Chris 

[Dimitar Valchev]

Hi Chris,

I'm receiving "There is no queryable persistence service registered with the name 'mysql" each time when I reboot the host (in my case Beaglebone Black rev. C) but after restarting openhab service or after some period of time everything works well with mysql so you can ignore the error message at least for the "restoring items state" issue.

Anyway I can upload the debug log (when I get back at home) if you think there will be something which might help. Unfortunately I didn't discover any message related to that.

Thanks for the reply
Dimitar

[Chris Jackson]

I'm receiving "There is no queryable persistence service registered with the name 'mysql" .....

I don't think this message matters with respect to the problem of not restoring data on startup. If I remember correctly, each time a new persistence service is registered, it will run the startup restore. In your case, I guess you have a script or something that is running before the mysql service is registered (remembering that the binding startup is asynchronous). I would suggest that this is outside of the restore...

If it's all working, then no need for a debug log, however if (when) it stops working, a log would be useful.

Cheers

Chris

[Dimitar]

Hi Chris,

Attached you'll find the debug logs.

Note:
"I guess you have a script or something that is running before the mysql service is registered" this happens even before I have configured any rules.

Hope the attachments will help with the troubleshooting.

Cheers

Dimitar

[gilles.l...@gmail.com]

Hi Chris,

From my side here is the log extract:

20:00:11.331 DEBUG o.o.p.m.i.MysqlPersistenceServiceActivator[:32]- mySQL persistence bundle has been started.
20:00:11.620 DEBUG o.o.p.m.i.MysqlPersistenceService[:434]- mySQL configuration starting
20:00:11.623 DEBUG o.o.p.m.i.MysqlPersistenceService[:359]- mySQL: Attempting to connect to database jdbc:mysql://localhost:3306/openhab
20:00:11.687 DEBUG o.o.c.p.i.PersistenceManager[:146]- Initializing mysql persistence service.
20:00:11.985 DEBUG o.o.p.m.i.MysqlPersistenceService[:503]- Query aborted on item Switch_Arrosage_Automatique - mySQL not initialised!
20:00:12.008 DEBUG o.o.c.p.i.PersistenceManager[:428]- Scheduled strategy mysql.everyHour with cron expression 0 0 * * * ?
20:00:12.045 DEBUG o.o.c.p.i.PersistenceManager[:428]- Scheduled strategy mysql.everyDay with cron expression 0 0 0 * * ?

Hope that helps.

Cheers ,

Gilles.

[Chris Jackson]

Thanks Gilles. It’s interesting - I’ll have a look at this ‘cause something doesn’t look quite right…

Chris

[Yannick Hein]

Good to see that someone is working on this. I have the same problem, running with mySQL on a Synology NAS.

I can also confirm that this seems to be a timing issue and not a general bug, because when I change the mysql.persist and it is reloaded, then the restoring works.

My workaround for now: also store the data in rrd4j and use that to restore (which is really annoying, because it cannot store datetime, so storage of the timestamp of the highest power output of my photovoltaic generator does not work).

It has to be mentioned that the NAS is working hard to get OpenHAB running. It is not as bad as a RasPi, but it takes up to 4 minutes I would guess. Maybe this timing issue appears more in low-power devices?

Also, I sometimes have rules running on "restoreOnStartup" items and they report an UnDef error - is it somehow possible to make sure that the restoreOnStartup is executed before the very rule (System Started or any other cron / item change)? Is there some sort of sequence diagram somewhere that explains how OH is started? Maybe it would be useful for developers and users as a manual.

Thanks for all the work!
Yannick

[Martin Klimke]

I have also noticed using rddj2 that some items are not restored when the rule engine is already executing and requesting their content.

The result are "Undefined" or "uninitialized errors"

The workaround I am using is to delay the use of some items by a trigger that is triggered 20 seconds after startup.

This requires careful inspection of every rule and a trial an error.

Pretty annoying. Same is true for the Astrobinding which is not finished when the rules engine is already executing.

I guess this is a general flaw of the overall architecture of OPENHAB.

[Wez Hunter]

I've had a look into this and need help/feedback from probably Chris or Kai with regards to the way configuration is loaded via the ConfigAdmin asynchronous "updated" [ConfigDispatcher > MysqlPersistenceService] method during startup. Its a lazy method by design according to OSGi spec.

Most of the persistence services that rely on a connection being established (MySQL etc) prior to being queryable need their configuration loading from ConfigAdmin first - ConfigAdmin is obviously required for loading server connection details.

OH is wiring the events up for the "restoreOnStartup" during initalize (item) of PersistenceManager which is called once, prior to the config being loaded via the "updated" method.

[addPersistenceService > startEventHandling > initialize (item)]  PersistenceManager.

As its not possible to load the config synchronously from ConfigAdmin (according to OSGi specs) then the restoreOnStartup for this particular type of service fails as config is not loaded and thus connection not established and restore fails.

I don't have enough experience or knowledge at this point to resolve - there seems to be a fair few theoretical solutions but none that I can identify that are practical or require minimal changes and cause minimal impact - that's my very slightly informed guess anyway.

My guess (based on zero experience!) - 

Don't wire up the events during initalize of PersistenceManager for restoreOnStartup and wait for ConfigAdmin to load prior to doing so (events etc) ??

ConfigAdmin loads before PersistenceManager ??

Something else ??

I don't wish to use other persistence stores like rrd4j, db4o - although good and technically a "better" solution for restoration of states from local file system. I simply don't want to have to manage the additional persist config nor the extra, although marginal, memory overhead with another persistence binding purely for restoration of item states. I'd trust my MySQL service will (mostly) always be available as its locally hosted.

Any help/suggestions would be appreciated!

[Chris Jackson]

I'm not an expert on the OSGi framework inner workings, but the OSGi services get defined in the XML and if I remember correctly, the framework registers these services itself when the binding starts. I don't think we can influence this, or the ConfigAdmin registration, and therefore the synchronisation between the two.

ie -:

The OSGi registers the (mySQL) persistence service with the persistence manager - When the persistence manager sees a new registration, it runs the startup restore.

The OSGi registers the (mySQL) admin service - When the admin service gets this new registration, it will call the 'update' method

These two are asynchronous, and there's no way to change them. The best option is probably for the persistence service to send a 'ready' message to the persistence manager before it performs the startup restore, but this is not insignificant and probably not worth it for OH1.

Kai probably (certainly!) has a better understanding of this :)

Chris

[Wez Hunter]

Thanks for your quick feedback Chris - really appreciated!

Thought that might be a case but needed confirmation from someone else who would be more informed than I :)

Kai is most certainly best to answer this then - maybe even to check if this particular issue exists in OH v2 - which I'm not up-to speed with yet...

Cheers,

Wez

[Markus Rathgeb]

So, if someone would like to use the restoreOnStartup persistence function, what is the best that he could do?

Is the db4o persistence better for this purpose?

[gilles.l...@gmail.com]

So no way for OH1, I'm sad.....

My actual solution is mysql for storing informations and rd4j for restoring value at start-up.

It's a working solution that I use for less than one month but it's stable.

Thank you for what you do.

Cheers,

Gilles

[dkos...@gmail.com]

Hi all,

I have similar problem. It's working on restart.

But, it wasn't working when I put new *.items file.

when I override the existing items file, the server said, refreshing 'a.items', then when I open my HABDroid, the switch was back to off.

Can someone please point me to the right direction?

Thank you!!!

dK

On Tuesday, 9 September 2014 23:08:42 UTC+10, gilles.l...@gmail.com wrote:

Hi,

I'm using some"virtual" items  to allow some automation for others item by rules.

An example is my pool pump.

in the item file I put:
Switch  Switch_Automatisation_Piscine   "Automatisation Piscine"        (G_Piscine,G_Virtual)

in persitence file:

Switch_Automatisation_Piscine : strategy = everyChange, restoreOnStartup


If my switch "Switch_Automatisation_Piscine" is "ON" and if I restart Openhab, my switch is not restored to "ON".


My persistence service is mysql and openhab is 1.51


Is there something else to do ?

[Matthew Denecke]

I am getting the same issues as Deddy KOSASIH.
When I restart OpenHab my switch does not restore to what it was before. I do see the last settings in the mysql database.
Has there been any resolution on this?

[Fabio Quarti]

Hi Matthew,

do you solve the issue?

I've the same issue and I don't found how to solve it.

Thanks

[Matthew Denecke]

Yes just use rrd4j. MYSQL is not working. It saves data to the database but would not restore. Once I setup rrd4j.persist using below in the file it worked and works great!!
 
Strategies {
    everyMinute : "0 * * * * ?"                                            // rrd4j needs persistence at minute level for graphing  
    default = everyChange
}

Items {
   put your item name here : strategy = everyChange, everyMinute, restoreOnStartup
}

Example of what I have..
Strategies {
    everyMinute : "0 * * * * ?"        // update every Minute                                   
    default = everyChange
}

Items {
    hvac_target : strategy = everyChange, everyMinute, restoreOnStartup
    hvac_current_sensor : strategy = everyChange, everyMinute, restoreOnStartup
    itm_securiy_arm_all : strategy = everyChange, everyMinute, restoreOnStartup
    hvac_setpoint : strategy = everyChange, everyMinute, restoreOnStartup
    itm_sethvac_mode_heater : strategy = everyChange, everyMinute, restoreOnStartup
}

Let me know if you if you need help with this on your setup. I will be more then happy to take a look at your files.

[Deddy KOSASIH]

Sorry for the late reply, Matthew.

I did not end up using any persistance method. My solution was that I code the device, in this case the arduino to tell openhab every few minutes and openhab update the state accordingly.

I was thinking to setup a system starup to ping all nodes for state update, but did not work for my case because I dont think there is such rule that trigger when items is updated.

[Matthew Denecke]

Using the rrd4j was very easy and worked right away. Just add the rrd4j.persistanc file and boom... Worked.. Well need to stop the openhab server and restart.
I have been using it for a few weeks and love it.