OpenTSDB v1.1.0-rc1 is available for testing
479 views
Skip to first unread message
tsuna
Jan 21, 2013, 6:10:59 AM1/21/13
to OpenTSDB
Hi all,
Here is finally the much belated release candidate of OpenTSDB v1.1.0.
It's cut off from master @ 3b14ccd + 2 patches in my branch (one
updates NEWS, the other adds a workaround for a small annoyance of the
web UI). Please help test it out and report any bugs or feedback. I
would like to cut the release within a week or two, and start working
on the next release, which should incorporate many of the improvements
available out there, in various forks on GitHub.
This release contains significant performance improvements and bug
fixes. Some have been seen to gain up to 5-10x more write throughput.
Self-contained pre-bootstrapped tarball:
https://code.google.com/p/opentsdb/downloads/detail?name=opentsdb-1.1.0-rc1.tar.gz
Here is the relevant excerpt of the NEWS file:
Noteworthy changes:
- Licensing adjustment: allow LGPLv2.1+ in addition to LGPLv3+.
- Various fixes used when customizing size of UniqueId. The default size
is 3 bytes and is a compile-time constant rarely changed in practice.
- New a new standard deviation aggregator, `dev'.
- New `fgcolor', `bgcolor' and `smooth' query parameters to /q.
- New `tz' query string parameter to allow specifying a custom time zone.
- Stop accepting connections when shutting down.
- A new `dropcaches' administrative command allows discarding in-memory
caches. Right now these are UID mappings.
- Browser history support in the web UI.
- Allow "1d-ago" style input in date boxes.
- Fix the 30d integer overflow in the web UI.
- Add the ability to use mouse for drag-to-zoom on graphs.
- Integration with Maven.
- Work around a Netty performance bug, increasing write throughput by 10x.
- Properly parse floating point values in scientific notations.
- Allow tuning the number of worker threads or using OIO.
- Fix auto-completion bug causing partial results to show in the web UI.
- Various internal bug fixes.
Issues closed: https://github.com/OpenTSDB/opentsdb/issues?milestone=1&page=1&state=closed
Thank you to all of those who contributed to this release! I'm hoping
we can release more often this year, maybe once every quarter or two.
$ git diff --stat v1.0.0.. | tail -n 1
114 files changed, 4079 insertions(+), 2063 deletions(-)
$ git shortlog v1.0.0..
Adrien Mogenet (1):
Add styling options to smooth curves.
Alex Ioffe (1):
Standard deviation aggregator.
Andrey Stepachev (1):
Allow tuning the number of worker threads or using OIO.
Aravind Gottipati (1):
Allow -w and -c to accept floats.
Benoit Sigoure (55):
Start version 1.1.0.
Licensing adjustment: allow LGPLv2.1+ instead of LGPLv3+.
Fix helper functions when using varying UniqueId sizes.
Add one more unit test for stddev aggregator.
Simplify computation of standard deviations.
Don't throw unnecessary exceptions when parsing timestamps.
Add parameters to change the foreground and background colors.
Upgrade to asynchbase 1.2.0.
Fix some year-2038 bugs.
Add a `tz' query string parameter to allow specifying a timezone.
Force upper-case the compression codec and warn on unknown codecs.
Compression codec issue was only a regression in HBase 0.92.1.
Fix a tyop in a variable name.
Add a helper class to parse query strings.
Suggest potential tags in the first unused row.
Add helper methods to set/clear both tag name and value.
Upgrade to Netty 3.4.3.
Update to async 1.2.0.
Upgrade to asynchbase 1.3.0.
Add new stats from HBaseClient as of 1.3.0.
Stop accepting new connections when shutting down.
Handle failure to flush the compaction queue when shutting down.
Add a new "dropcaches" command.
Update the browser history each time we refresh the graph.
Parse the start and end dates, and `wxh' on page load.
Fill in the metric and tags forms from history.
Properly clear out all metric forms.
Add a method to get the first value of a query string parameter.
Redirect graph requests with no type to the home page.
Allow "1d-ago" style input in date boxes.
Don't rely on some non-portable shell substitutions.
Quote some shell variable values.
Upgrade to asynchbase 1.3.2.
Upgrade to Netty 3.5.2.
Fix yet another sign extension bug.
Fix regression that prevents reading pre-1.0 floating point values.
Update asynchbase, Netty, and suasync.
Don't log an "Unexpected exception" when a client connection times out.
Additional accessors to BuildData to avoid cross-jar inlining.
Stop tracking mouse movements when not using the zoom box.
Don't block mouse click events on the image.
Fix make distcheck by adding a missing file to Makefile.am.
Finish integration with Maven.
Improve TSD write throughput by 10x.
Fix check_tsd -v when IPv6 is used.
Fix coding style a bit in tsd_check.
Properly parse floating point values in scientific notations.
Make sure the .git directory exists before writing HEAD.
Cache auto-completion results even if we don't use them.
Don't leak scanners when fetching suggestions from HBase.
Fetch the correct number of suggestions from HBase.
Add a small helper script to clean the --cachedir from cron.
Ship the misc tools in the tarball.
Update NEWS with changes made for v1.1.0.
Add a workaround to prevent the graph from being dragged around.
Christophe Furmaniak (3):
Upgrade suasync to 1.3.1.
Update javassist's version to match what's in Maven.
Provide a pom.xml for Maven users.
Dave Barr (1):
Add -W (downsample window) flag.
Geoffrey Anderson (1):
Add the ability to use mouse for drag-to-zoom on graphs.
Jacek Masiulaniec (1):
check_tsd: add a --rate flag.
Ron (1):
Javadoc fixes.
Simon Matic Langford (1):
Add secure option to options for check_tsd.
Slawek Ligus (2):
Fix for 30d integer overflow in DateTimeBox.
Properly allow use of both axis with a log scale.
Tay Ray Chuan (1):
Drop redundant call to AC_PROG_MKDIR_P.
Will Moss (7):
Always clear the last line when recompacting the tag table.
Listen to history events to handle forward / back.
Factor out a function to add metric forms.
Handle the "Right Axis" checkbox.
Parse the `Axis' box from history.
Parse the key side box.
Add autoreload to the URL.
--
Benoit "tsuna" Sigoure
Here is finally the much belated release candidate of OpenTSDB v1.1.0.
It's cut off from master @ 3b14ccd + 2 patches in my branch (one
updates NEWS, the other adds a workaround for a small annoyance of the
web UI). Please help test it out and report any bugs or feedback. I
would like to cut the release within a week or two, and start working
on the next release, which should incorporate many of the improvements
available out there, in various forks on GitHub.
This release contains significant performance improvements and bug
fixes. Some have been seen to gain up to 5-10x more write throughput.
Self-contained pre-bootstrapped tarball:
https://code.google.com/p/opentsdb/downloads/detail?name=opentsdb-1.1.0-rc1.tar.gz
Here is the relevant excerpt of the NEWS file:
Noteworthy changes:
- Licensing adjustment: allow LGPLv2.1+ in addition to LGPLv3+.
- Various fixes used when customizing size of UniqueId. The default size
is 3 bytes and is a compile-time constant rarely changed in practice.
- New a new standard deviation aggregator, `dev'.
- New `fgcolor', `bgcolor' and `smooth' query parameters to /q.
- New `tz' query string parameter to allow specifying a custom time zone.
- Stop accepting connections when shutting down.
- A new `dropcaches' administrative command allows discarding in-memory
caches. Right now these are UID mappings.
- Browser history support in the web UI.
- Allow "1d-ago" style input in date boxes.
- Fix the 30d integer overflow in the web UI.
- Add the ability to use mouse for drag-to-zoom on graphs.
- Integration with Maven.
- Work around a Netty performance bug, increasing write throughput by 10x.
- Properly parse floating point values in scientific notations.
- Allow tuning the number of worker threads or using OIO.
- Fix auto-completion bug causing partial results to show in the web UI.
- Various internal bug fixes.
Issues closed: https://github.com/OpenTSDB/opentsdb/issues?milestone=1&page=1&state=closed
Thank you to all of those who contributed to this release! I'm hoping
we can release more often this year, maybe once every quarter or two.
$ git diff --stat v1.0.0.. | tail -n 1
114 files changed, 4079 insertions(+), 2063 deletions(-)
$ git shortlog v1.0.0..
Adrien Mogenet (1):
Add styling options to smooth curves.
Alex Ioffe (1):
Standard deviation aggregator.
Andrey Stepachev (1):
Allow tuning the number of worker threads or using OIO.
Aravind Gottipati (1):
Allow -w and -c to accept floats.
Benoit Sigoure (55):
Start version 1.1.0.
Licensing adjustment: allow LGPLv2.1+ instead of LGPLv3+.
Fix helper functions when using varying UniqueId sizes.
Add one more unit test for stddev aggregator.
Simplify computation of standard deviations.
Don't throw unnecessary exceptions when parsing timestamps.
Add parameters to change the foreground and background colors.
Upgrade to asynchbase 1.2.0.
Fix some year-2038 bugs.
Add a `tz' query string parameter to allow specifying a timezone.
Force upper-case the compression codec and warn on unknown codecs.
Compression codec issue was only a regression in HBase 0.92.1.
Fix a tyop in a variable name.
Add a helper class to parse query strings.
Suggest potential tags in the first unused row.
Add helper methods to set/clear both tag name and value.
Upgrade to Netty 3.4.3.
Update to async 1.2.0.
Upgrade to asynchbase 1.3.0.
Add new stats from HBaseClient as of 1.3.0.
Stop accepting new connections when shutting down.
Handle failure to flush the compaction queue when shutting down.
Add a new "dropcaches" command.
Update the browser history each time we refresh the graph.
Parse the start and end dates, and `wxh' on page load.
Fill in the metric and tags forms from history.
Properly clear out all metric forms.
Add a method to get the first value of a query string parameter.
Redirect graph requests with no type to the home page.
Allow "1d-ago" style input in date boxes.
Don't rely on some non-portable shell substitutions.
Quote some shell variable values.
Upgrade to asynchbase 1.3.2.
Upgrade to Netty 3.5.2.
Fix yet another sign extension bug.
Fix regression that prevents reading pre-1.0 floating point values.
Update asynchbase, Netty, and suasync.
Don't log an "Unexpected exception" when a client connection times out.
Additional accessors to BuildData to avoid cross-jar inlining.
Stop tracking mouse movements when not using the zoom box.
Don't block mouse click events on the image.
Fix make distcheck by adding a missing file to Makefile.am.
Finish integration with Maven.
Improve TSD write throughput by 10x.
Fix check_tsd -v when IPv6 is used.
Fix coding style a bit in tsd_check.
Properly parse floating point values in scientific notations.
Make sure the .git directory exists before writing HEAD.
Cache auto-completion results even if we don't use them.
Don't leak scanners when fetching suggestions from HBase.
Fetch the correct number of suggestions from HBase.
Add a small helper script to clean the --cachedir from cron.
Ship the misc tools in the tarball.
Update NEWS with changes made for v1.1.0.
Add a workaround to prevent the graph from being dragged around.
Christophe Furmaniak (3):
Upgrade suasync to 1.3.1.
Update javassist's version to match what's in Maven.
Provide a pom.xml for Maven users.
Dave Barr (1):
Add -W (downsample window) flag.
Geoffrey Anderson (1):
Add the ability to use mouse for drag-to-zoom on graphs.
Jacek Masiulaniec (1):
check_tsd: add a --rate flag.
Ron (1):
Javadoc fixes.
Simon Matic Langford (1):
Add secure option to options for check_tsd.
Slawek Ligus (2):
Fix for 30d integer overflow in DateTimeBox.
Properly allow use of both axis with a log scale.
Tay Ray Chuan (1):
Drop redundant call to AC_PROG_MKDIR_P.
Will Moss (7):
Always clear the last line when recompacting the tag table.
Listen to history events to handle forward / back.
Factor out a function to add metric forms.
Handle the "Right Axis" checkbox.
Parse the `Axis' box from history.
Parse the key side box.
Add autoreload to the URL.
--
Benoit "tsuna" Sigoure
Ion Savin
Jan 21, 2013, 7:20:21 AM1/21/13
to open...@googlegroups.com
Hi tsuna,
> Noteworthy changes:
> [...]
> - Integration with Maven.
>[...]
It this a step towards replacing the current build system with maven?
Regards,
Ion Savin
> Noteworthy changes:
> [...]
> - Integration with Maven.
>[...]
It this a step towards replacing the current build system with maven?
Regards,
Ion Savin
tsuna
Jan 21, 2013, 8:15:13 PM1/21/13
to Ion Savin, open...@googlegroups.com
On Mon, Jan 21, 2013 at 4:20 AM, Ion Savin <co...@gmx.net> wrote:
> It this a step towards replacing the current build system with maven?
No, it's here to help those who want to integrate with various IDEs
> It this a step towards replacing the current build system with maven?
that need/want a pom.xml and a specific directory structure.
--
Benoit "tsuna" Sigoure
tsuna
Jan 25, 2013, 2:39:37 AM1/25/13
to OpenTSDB
On Mon, Jan 21, 2013 at 3:10 AM, tsuna <tsun...@gmail.com> wrote:
> Please help test it out and report any bugs or feedback. I
> would like to cut the release within a week or two, and start working
> on the next release, which should incorporate many of the improvements
> available out there, in various forks on GitHub.
Any feedback? Everyone's OK cutting this release as-is? Should we
> Please help test it out and report any bugs or feedback. I
> would like to cut the release within a week or two, and start working
> on the next release, which should incorporate many of the improvements
> available out there, in various forks on GitHub.
wait some more to give people time to deploy? I know Box and Tumblr
said they were going to upgrade sooner than later (and with Tumblr
pushing a whopping 1 million data points per second, I'd be curious to
see how the performance improvements on the write path are helping
them reclaim capacity).
Can we get a quick show of hand of who's running the RC or something
built from master as of November 20 (where the performance bug got
fixed) or January 11 (when various annoying UI glitches got fixed)?
--
Benoit "tsuna" Sigoure
Peter Speybrouck
Jan 25, 2013, 4:16:30 AM1/25/13
to open...@googlegroups.com
I'm not using it in any production environment, but I will build it tonight and do some simple tests.
Ion Savin
Jan 25, 2013, 6:05:44 AM1/25/13
to OpenTSDB
> Any feedback? Everyone's OK cutting this release as-is? Should we
> wait some more to give people time to deploy?
Started testing this week (not using the RC in production yet). Will do
> wait some more to give people time to deploy?
some performance testing over the weekend. So far it looks good.
Regards,
Ion Savin
Richard Hesse
Jan 25, 2013, 11:50:55 AM1/25/13
to tsuna, OpenTSDB
We're using it in production and haven't noticed anything odd so far (about 3 days).
-richard
ManOLamancha
Jan 25, 2013, 11:59:08 AM1/25/13
to open...@googlegroups.com
On Friday, January 25, 2013 2:39:37 AM UTC-5, tsuna wrote:
Any feedback? Everyone's OK cutting this release as-is? Should we
wait some more to give people time to deploy? I know Box and Tumblr
said they were going to upgrade sooner than later (and with Tumblr
pushing a whopping 1 million data points per second, I'd be curious to
see how the performance improvements on the write path are helping
them reclaim capacity).
Can we get a quick show of hand of who's running the RC or something
built from master as of November 20 (where the performance bug got
fixed) or January 11 (when various annoying UI glitches got fixed)?
Could you give me until Sunday? I may want to submit a tiny patch to filter out annotation columns in the 'tsdb' table so that if someone upgrades to whatever 2.0 turns out to be, they could go back to 1.1.0 without issues. I.e. if we store annotations in 'tsdb' with a special cell qualifier, I want to make sure the query doesn't barf when it scans over that name.
tsuna
Jan 26, 2013, 3:41:06 AM1/26/13
to ManOLamancha, open...@googlegroups.com
On Fri, Jan 25, 2013 at 8:59 AM, ManOLamancha <clars...@gmail.com> wrote:
> Could you give me until Sunday?
Sure, let's wait a few more days. I will ping the guys at StumbleUpon
> Could you give me until Sunday?
also to see whether they tested the new version.
--
Benoit "tsuna" Sigoure
Ion Savin
Feb 1, 2013, 4:35:08 AM2/1/13
to OpenTSDB
Regards,
Ion Savin
tsuna
Feb 4, 2013, 4:22:38 AM2/4/13
to OpenTSDB
Update: the release is on hold for
https://github.com/OpenTSDB/opentsdb/pull/150 to be integrated. This
was brought up by Chris and it's pretty important to ensure 1.1.0 will
be forward compatible with future releases. This way, once you're on
1.1.0, you will be able to upgrade to 2.0, and downgrade back to 1.1.0
if you want to, and the 1.1.0 code will not choke on unexpected
KeyValues that are 2.0-specific, such as annotations.
I'm wondering whether we can also sneak in a quick fix for
https://github.com/OpenTSDB/opentsdb/issues/152 – this is an annoying
glitch in the UI and should be relatively straightforward to fix.
--
Benoit "tsuna" Sigoure
https://github.com/OpenTSDB/opentsdb/pull/150 to be integrated. This
was brought up by Chris and it's pretty important to ensure 1.1.0 will
be forward compatible with future releases. This way, once you're on
1.1.0, you will be able to upgrade to 2.0, and downgrade back to 1.1.0
if you want to, and the 1.1.0 code will not choke on unexpected
KeyValues that are 2.0-specific, such as annotations.
I'm wondering whether we can also sneak in a quick fix for
https://github.com/OpenTSDB/opentsdb/issues/152 – this is an annoying
glitch in the UI and should be relatively straightforward to fix.
--
Benoit "tsuna" Sigoure
Adam Steffes
Feb 18, 2013, 2:28:43 AM2/18/13
to open...@googlegroups.com
Any feedback? Everyone's OK cutting this release as-is? Should we
wait some more to give people time to deploy? I know Box and Tumblr
said they were going to upgrade sooner than later (and with Tumblr
pushing a whopping 1 million data points per second, I'd be curious to
see how the performance improvements on the write path are helping
them reclaim capacity).
Are there any docs or other info on how Tumblr achieved this amazing data rate? I want to do something similar at Jive. :)
-Adam
tsuna
Feb 18, 2013, 2:30:10 AM2/18/13
to Adam Steffes, open...@googlegroups.com
On Sun, Feb 17, 2013 at 11:28 PM, Adam Steffes <adams...@gmail.com> wrote:
> Are there any docs or other info on how Tumblr achieved this amazing data
> rate? I want to do something similar at Jive. :)
I think they just have a 20 or 25-node cluster, nothing special. 1M
> Are there any docs or other info on how Tumblr achieved this amazing data
> rate? I want to do something similar at Jive. :)
QPS shouldn't be too hard to achieve with this number of nodes.
--
Benoit "tsuna" Sigoure
Reply all
Reply to author
Forward
0 new messages