Not able to retrieve samples from Mongo db

[Balakrishnan Narendran]

Hi,

   I have integrated rrd4j(3.1) with mongodb (3.2). But the sample data doesn't seem to be persistent on restart. I see the data getting updated in mongo db when the rrdDb.close() is called. And on open the data is being queried and updated in the byte buffer. But on dumping the data after reconnect, all the sample data is lost. Could someone please help me on this.? 

   

   Adding the code for RrdDb initializing and Mongo Factory creation.

   

   

      RrdDb initialization  

        RrdDef rrdDef = new RrdDef(rrdPath,(System.currentTimeMillis() - TimeUnit.DAYS.toMillis(300))/1000,10);

rrdDef.addDatasource(dsName, DsType.GAUGE ,  20, 1d, Double.MAX_VALUE);

int secondUnitStep = (int)(TimeUnit.MINUTES.toSeconds(1)/10);

int secondUnitRows = (int)(TimeUnit.HOURS.toSeconds(1)/TimeUnit.MINUTES.toSeconds(1));

rrdDef.addArchive(ConsolFun.TOTAL, 0.5, 6, 60); 

RrdDb rrdDB = new RrdDb(rrdDef,rrdMongoFactory);

Mongo Factory class

package mx.july.jmx.proximity.util;

public class RrdMongoFactory extends RrdBackendFactory {

private String name;

private final DBCollection rrdCollection;

public RrdMongoFactory(String name, DBCollection rrdCollection) {

this.name = name;

this.rrdCollection = rrdCollection;

this.rrdCollection.createIndex(new BasicDBObject("path", 1), "path_idx");

RrdBackendFactory.registerAndSetAsDefaultFactory(this);

}

  

  public void setName(String name) {

  this.name = name;

  }

public String getName() {

return name;

}

/** {@inheritDoc} */

@Override

protected RrdBackend open(String path, boolean readOnly) throws IOException {

return new RrdMongoDBBackend(path, rrdCollection);

}

 

/** {@inheritDoc} */

@Override

protected boolean exists(String path) throws IOException {

BasicDBObject query = new BasicDBObject();

query.put("path", path);

return rrdCollection.findOne(query) != null;

}

/** {@inheritDoc} */

@Override

protected boolean shouldValidateHeader(String path) throws IOException {

return false;

}

}

    RrdDb dump before close

== HEADER ==

signature:RRD4J, version 0.2 lastUpdateTime:1488536144 step:10 dsCount:1 arcCount:3

== DATASOURCE ==

DS:320:GAUGE:20:1.0:1.7976931348623157E308

lastValue:8.0 nanSeconds:0 accumValue:32.0

== ARCHIVE ==

RRA:TOTAL:0.5:6:60

interval [1488532560, 1488536100]

accumValue:117.0 nanSteps:0

Robin 5/60: NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN +1.7860000000E02 NaN +1.7130000000E02 +1.6600000000E02 

RrdDb dump after reconnect

== HEADER ==

signature:RRD4J, version 0.2 lastUpdateTime:1462616268 step:10 dsCount:1 arcCount:3

== DATASOURCE ==

DS:320:GAUGE:20:1.0:1.7976931348623157E308

lastValue:NaN nanSeconds:8 accumValue:0.0

== ARCHIVE ==

RRA:TOTAL:0.5:6:60

interval [1462612680, 1462616220]

accumValue:NaN nanSteps:4

Robin 0/60: NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN 

[Fabrice Bacchella]

Did you have a look at existing mongo code ?

https://github.com/rrd4j/rrd4j/blob/master/src/main/java/org/rrd4j/core/RrdMongoDBBackend.java

https://github.com/rrd4j/rrd4j/blob/master/src/main/java/org/rrd4j/core/RrdMongoDBBackendFactory.java

--
You received this message because you are subscribed to the Google Groups "rrd4j-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rrd4j-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Balakrishnan Narendran]

Fabrice,

  Thank you for responding. I am using RrdMongoDBBackend itself for my operations and both open and close methods are getting executed. For factory class I have used is pretty much same as the RrdMongoDBBackend, except for the index creation. Since the mongo db version i am using is 3.2, I have modified accordingly in RrdMongoFactory. 

[Fabrice Bacchella]

I see no problems. It looks like more a mongo problem, but I never used it, so I'm not sure.

Le 3 mars 2017 à 11:32, Balakrishnan Narendran <balu....@gmail.com> a écrit :

[Fabrice Bacchella]

Every time you run

RrdDb rrdDB = new RrdDb(rrdDef,rrdMongoFactory);

You rebuild the rrd, whatever the backend is.

[Balakrishnan Narendran]

you mean for each run at the start initialize RrdDb  and at the end close it right..? when i do this, samples from the previous runs are lost. 

[Fabrice Bacchella]

Exactly. You need to first create the rrd with

RrdDef rrdDef = new RrdDef(rrdPath, ...

RrdDb rrdDB = new RrdDb(rrdDef,rrdMongoFactory);

And latter open it with:

RrdDb rrdDB = new RrdDb(rrdPath,rrdMongoFactory);

[Balakrishnan Narendran]

Thanks Fabrice ,

  With this approach the data sample are getting retained even after restart. Thanks a lot