[rrd-users] changing step time?
Gregory Guthrie
1) What is the actual meaning of "step" - specifically what happens if values are fed in more often?
The documentation says that if updates are less frequent, they are interpolated (up to heartbeat), but I didn't see this converse case.
2) Can I change step after the database is created?
- I setup one value for daily step size, but now want to do twice-daily updates.
- Can I change step size, and just start inserting updates more often?
3) If I were to re-create the database, is there any (easy) way to preserve the data already collected?
4) If I have a few known initial values, can I insert them into a live rrd database using the rrdedit tool?
I tried to do this using rrdtool, but got this error:
illegal attempt to update using time 1262354400 when last update time is 1270818003 (minimum one second step)
The database started on 1/1/2010, and I tried inserting some values all after that. Looking at the database (dump) it looks like there are lots of empty slots to patch these into.
Is there somewhere where these sort of usage details are described?
Thanks.
-------------------------------------------
Gregory Guthrie
-------------------------------------------
_______________________________________________
rrd-users mailing list
rrd-...@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users
Simon Hobson
>1) What is the actual meaning of "step" - specifically what happens
>if values are fed in more often?
> The documentation says that if updates are less frequent, they
>are interpolated (up to heartbeat), but I didn't see this converse
>case.
Step is the size of each bin in the database - ie the length of each
time slot for which it can store data. Anything you feed it more
often that the step time is accumulated until the period is complete
and then it's all used to work out the single value stored for that
bin.
Eg, if you feed in values of 0,0,5,0,0 which all fit within one step
time, then the min, max, and average are all 1.
Alex has some excellent tutorials at
http://www.vandenbogaerdt.nl/rrdtool/ - they explain a lot.
>2) Can I change step after the database is created?
> - I setup one value for daily step size, but now want to do
>twice-daily updates.
> - Can I change step size, and just start inserting updates more often?
Don't think so. I think you have to dump the data then do one of two things :
1) create a new database and feed the old data in
2) edit the data and re-import it
>3) If I were to re-create the database, is there any (easy) way to
>preserve the data already collected?
>
>4) If I have a few known initial values, can I insert them into a
>live rrd database using the rrdedit tool?
> I tried to do this using rrdtool, but got this error:
> illegal attempt to update using time 1262354400 when last
>update time is 1270818003 (minimum one second step)
You cannot go backwards in time. So to put historical data in, you
must doing before putting current data in. So create a new database,
craft some insert statements to enter the historical values, and then
start inserting new data.
--
Simon Hobson
Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
Gregory Guthrie
> > "step" - specifically what happens if values are fed in more often?
> time slot for which it can store data. Anything you feed it more
> often that the step time is accumulated until the period is complete
> and then it's all used to work out the single value stored for that
> bin.
>
> Eg, if you feed in values of 0,0,5,0,0 which all fit within one step
> time, then the min, max, and average are all 1.
>
> Alex has some excellent tutorials at
> http://www.vandenbogaerdt.nl/rrdtool/ - they explain a lot.
> >2) Can I change step after the database is created?
> > - I setup one value for daily step size, but now want to do twice-daily updates.
> > - Can I change step size, and just start inserting updates more often?
>
> Don't think so. I think you have to dump the data then do one of two things :
> 1) create a new database and feed the old data in
> 2) edit the data and re-import it
My data comes from a live feed source, is there any standard way to export from rrd in a manner that allows re-import?
I suppose one could write something that would do fetches from the old, and updates into the new.
> You cannot go backwards in time. So to put historical data in, you
> must doing before putting current data in.
Ah; so update can be backwards in time, (negative time), but all inserts must be in a forward sequence.
(Seems like these things should be in the main documentation and tutorial.)
Alex van den Bogaerdt
>> time, then the min, max, and average are all 1.
> This is a surprise! It would certainly appear at first glance that the
> max/min were 5 & 1.
5 & 0, but yes, I can see why you would expect that.
Min, avg and max are defined for R(ound)R(obin)A(rchive)s which contain
C(onsolidated)D(ata)P(oint)s, not for P(rimary)D(ata)P(oint)s. PDPs are just
a rate during a fixed interval.
What is described above is how PDPs are built.
Nothing stops you from having step==1 second and no RRA where each CDP gets
only one PDP. For instance: step==1, RRA each CDP==300PDPs,
C(onsolidation)F(unction) MIN, same with AVERAGE, same with MAX.
You can now update anywhere in the 5-minute interval and still get MIN and
MAX as you expect them.
>> Alex has some excellent tutorials at
>> http://www.vandenbogaerdt.nl/rrdtool/ - they explain a lot.
> I'll certainly go there now, thanks.
Old, but still relevant.
>> >2) Can I change step after the database is created?
>> > - I setup one value for daily step size, but now want to do
>> > twice-daily updates.
>> > - Can I change step size, and just start inserting updates more
>> > often?
>>
>> Don't think so.
Correct. The amount of time per CDP is the amount of PDPs per CDP multiplied
by the amount of time per PDP. And the amount of time per PDP is what "step"
defines. Altering "step" (which can be done) means altering your data.
Example: if you would change step from 3600 into 1800, then suddenly each
CDP is valid for only half the time it used to be.
>> I think you have to dump the data then do one of two things :
>> 1) create a new database and feed the old data in
>> 2) edit the data and re-import it
> My data comes from a live feed source, is there any standard way to export
> from rrd in a manner that allows re-import?
>
> I suppose one could write something that would do fetches from the old,
> and updates into the new.
The biggest challenge is to get MIN and MAX correct.
>> You cannot go backwards in time. So to put historical data in, you
>> must doing before putting current data in.
> Ah; so update can be backwards in time, (negative time), but all inserts
> must be in a forward sequence.
>
> (Seems like these things should be in the main documentation and
> tutorial.)
It sort of follows from the way RRDtool works. If you think you can make the
manual and/or tutorials more clear on this, I'm sure Tobi will accept
changes from you.
cheers,
Alex
Simon Hobson
> > > "step" - specifically what happens if values are fed in more often?
>> Step is the size of each bin in the database - ie the length of each
>> time slot for which it can store data. Anything you feed it more
>> often that the step time is accumulated until the period is complete
>> and then it's all used to work out the single value stored for that
>> bin.
>>
>> Eg, if you feed in values of 0,0,5,0,0 which all fit within one step
>> time, then the min, max, and average are all 1.
>This is a surprise! It would certainly appear at first glance that
>the max/min were 5 & 1.
RRD does not store anything but an accumulator within a step - so if
you feed in those 5 values, the accumulator will hold the values 0,
0, 5, 5, 5 immediately after each update. At the end of the step,
we've told it about 5 <somethings> in total, and if the step were 5
then the overall rate during the step is 1. Consolidate that with
min, max, or average, and you still get 1.
I think it will be clearer when you've read Alex's tutorial on normalisation.
Alex van den Bogaerdt wrote:
> >> I think you have to dump the data then do one of two things :
>>> 1) create a new database and feed the old data in
>>> 2) edit the data and re-import it
>> My data comes from a live feed source, is there any standard way to export
>> from rrd in a manner that allows re-import?
>>
>> I suppose one could write something that would do fetches from the old,
>> and updates into the new.
>
>The biggest challenge is to get MIN and MAX correct.
In a situation like that, you would have to accept that the data
doesn't exist. In changing from a step of 1 (day, hour, whatever) to
half then the only sensible way I could see would be to simply
duplicate each set of values - so at time 0.5 you insert the values
you had at time 1, and insert them again at time 1. Or if the rrd
definition allows, just insert at time 1 and let rrdtool fill in the
same value at time 0.5.
The end effect is that the rrd database will hold the same
rates/consolidated values from time 0 to 1 as it did before. But
going forward, the new data will be processed at the higher
resolution.
--
Simon Hobson
Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.
_______________________________________________
Tobias Oetiker
Yesterday Gregory Guthrie wrote:
> I started a new database a few weeks ago, and have a few questions;
>
> 1) What is the actual meaning of "step" - specifically what
> happens if values are fed in more often? The documentation
> says that if updates are less frequent, they are interpolated
> (up to heartbeat), but I didn't see this converse case.
the step is the 'base' interval at which an rrd handles data. This
means that if you feed data more often, rrdtool will average the
data into a 1 step interval (Primary Data Point) before handing it
to the different RRAs
>
> 2) Can I change step after the database is created?
> - I setup one value for daily step size, but now want to do twice-daily updates.
> - Can I change step size, and just start inserting updates more often?
you can not chnage the step size of an existing rrd file, but you
can feed updates as often as you have them.
> 3) If I were to re-create the database, is there any (easy) way
> to preserve the data already collected?
I have recently written a tool called rrdjig
https://svn.oetiker.ch/rrdtool/trunk/contrib/rrdjig/
which might be interesting for you. It lets you 'replay' updates
from an existing rrd into a new one. rrdjig will try to recreate
the update instructions that were fed into the original rrd file.
In your case, this means it would generate 1 update instruction per
day. If you setup a new rrd to store two updates per day you must
make sure the mrhb of the new rrd is sufficient large.
cheers
tobi
--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch to...@oetiker.ch ++41 62 775 9902 / sb: -9900
Tobias Oetiker
Today Gregory Guthrie wrote:
> Thanks, interesting..
> > > "step" - specifically what happens if values are fed in more often?
> > Step is the size of each bin in the database - ie the length of each
> > time slot for which it can store data. Anything you feed it more
> > often that the step time is accumulated until the period is complete
> > and then it's all used to work out the single value stored for that
> > bin.
> >
> > Eg, if you feed in values of 0,0,5,0,0 which all fit within one step
> > time, then the min, max, and average are all 1.
> This is a surprise! It would certainly appear at first glance that the max/min were 5 & 1.
min and max are the minimal and maximal 1 step value ... otherwhise
this would be pretty arbitrary ...
cheers
tobi
--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
http://it.oetiker.ch to...@oetiker.ch ++41 62 775 9902 / sb: -9900
_______________________________________________