item states missing in db4o or habmin - does somebody have the same issue

[Lars Bretschneider]

as it look like some records are missing on habmin or database itself ... can somebody test and see where it is missing

root@openhab:~# tail -f /var/log/openhab.log | grep WS444_Sensor9_temperature
08:08:00.012 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.6'
08:08:00.929 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.6' in rrd4j database
08:09:00.012 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.6'
08:09:00.839 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.6' in rrd4j database
08:10:00.010 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.6'
08:10:00.900 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.6' in rrd4j database
08:10:06.085 INFO  runtime.busevents[:26] - WS444_Sensor9_temperature state updated to 9.9
08:10:06.099 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.9'
08:10:06.100 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.9'
08:10:06.103 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.9' in rrd4j database
08:11:00.012 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.9'
08:11:00.339 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.9' in rrd4j database
08:12:00.011 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.9'
08:12:00.788 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.9' in rrd4j database
08:13:00.012 DEBUG o.o.p.d.i.Db4oPersistenceService[:117] - Stored item state 'WS444_Sensor9_temperature' -> '9.9'
08:13:00.582 DEBUG o.o.p.r.internal.RRD4jService[:121] - Stored 'WS444_Sensor9_temperature' with state '9.9' in rrd4j database

as you may the stored item at 8:11 is mssing in habmin

[Chris Jackson]

HABmin currently subsamples data. If there are more than 1000 samples, then it will not display some samples. This was put in to reduce the amount of data to be sent to the charting library, which can otherwise slow down. For charting, it makes no difference, but obviously when you look at the table, it shows up. When I get a chance I intend to rewrite the table display to work differently as the current display was a quick hack and has it’s own issues.

This is probably why you’re missing data - although I can’t be 100% sure of course.

Chris

[Lars Bretschneider]

for me is quite enough, if service request returns all values, that are in database

[Lars Bretschneider]

i mean, with url you sent me ;-)

[Chris Jackson]

Yes - that’s fine. All records are returned through this interface.

[Dave Hock]

I might be seeing this issue. I have been trying to persist "virtual" items, ie a switch that is only used during scripting or rules, and they never restore on restart. I also get this error every restart

07:59:04.517 WARN  o.o.c.p.e.PersistenceExtensions[:404]- There is no queryable persistence service registered with the name 'rrd4j'

I wonder if the system startup rules are running before the persistance bindings are loaded???

[Chris Jackson]

Interestingly, there’s the same bug showing up here as in the email I sent earlier (in Nolans thread). This is a bit of a nasty bug as it’s changing the received frames, although I don’t know that it’s causing any really major problems. The thing I couldn’t correlate when Iooked earlier is the following part -

2014-04-22 07:59:34 DEBUG o.o.b.z.i.p.ZWaveController$ZWaveReceiveThread[:920]- Receive Message = 01 05 00 13 10 01 F8 

2014-04-22 07:59:34 DEBUG o.o.b.z.i.p.SerialMessage[:216]- Assembled message buffer = 01 07 00 13 10 01 00 00 FA 

2014-04-22 07:59:34 DEBUG o.o.b.z.i.p.ZWaveController[:142]- Message: class = SendData (0x13), type = Request (0x00), buffer = 01 07 00 13 10 01 00 00 FA 

When we receive the message, it’s7 bytes long, but just a couple of log messages later, and it’s strangely mutated into 9 bytes! This is caused by the same bug as earlier, but it’s hidden in the depths of some seemingly harmless classes.

The offending line is -:

logger.debug(incomingMessage.toString());

which looks like this -:

public String toString() {

return String.format("Message: class = %s (0x%02X), type = %s (0x%02X), buffer = %s", 

new Object[] { messageClass, messageClass.key, messageType, messageType.ordinal(),

SerialMessage.bb2hex(this.getMessageBuffer()) });

};

and it’s the getMessageBuffer call that’s causing the problem again. I’m slightly surprised I didn’t spot this earlier since I did a search on the uses of this call… Anyway, in this call one thing it does is -:

int messageLength = messagePayload.length + 

(this.messageClass == SerialMessageClass.SendData && 

this.messageType == SerialMessageType.Request ? 5 : 3); // calculate and set length

This increases the size of the received packet by 2 bytes. Also, we again calculate the checksum, and perform a bunch more processing, so it’s wasting more of Nolans valuable RPi processor time :)

Since the message buffer gets dissected into its individual components, to print it out again, the binding has to totally recreate it (and it seems due to some assumptions, its not transparent, and we end up with these issues). I will change this so that after the frame has been received (and printed), we only print the payload - which is stored whole.  

This will fix some issues (we won’t go and add some extra bytes to our received frames!), however, it probably isn’t the answer to your question of what the heck is screwing up the network on startup…  I’ll take another look at that, and my previous email about the timeouts is the next mystery. For this, I’m less sure, but my previous comment about the timeout needs some investigation as I don’t understand the response there.

Let me fix this issue first (since I’ve largely worked it through) and then I’m happy to take a look at more logs to see if we can understand this some more :)

Cheers

Chris

[lbl...@googlemail.com]

i spent some time and rrd is working as expected, how do you switch between the archives in rrd-file ? if i have self configured rrd-files could it possible to give an parameter to url/query which archive to use ?