AsyncHBase 1.5.0-rc1 available for download and testing (HBase 0.96 compatibility inside)

392 views
Skip to first unread message

tsuna

unread,
Oct 29, 2013, 12:57:13 AM10/29/13
to AsyncHBase, HBase users
Hi all,
The first release candidate of AsyncHBase "post-singularity" is now
available for download. AsyncHBase remains true to its initial
promise: the API is still backward compatible, but under the hood it
continues to work with all production releases of HBase of the past
few years.

This release was tested against HBase 0.89, 0.90, 0.92, 0.94, and
0.96. While 0.2x probably still works, I didn't take the time to test
it, because… well, really, you shouldn't be using such ancient version
of HBase. Really.

The Maven build was broken by the addition of protobufs in the build
process. Any Maven fans out there who would like to help fix it?
Without it I can't easily publish new artifacts to Maven repo.


Here is the relevant excerpt of the NEWS file:

This release introduces compatibility with HBase 0.96 and up, and adds
a dependency on Google's protobuf-java library. Note that HBase 0.95.x,
which was a "developer preview" release train, is NOT supported.

Please note that support for explicit row locks has been removed from
HBase 0.95 and up. While the classes and functionality remain usable
when using earlier versions of HBase, an `UnsupportedOperationException'
will be raised if one attempt to send a `RowLockRequest' to a newer
version of HBase.

Please note that while AsyncHBase never made any guarantees about the
exact order in which multiple edits are applied within a batch, the order
is now different when talking to HBase 0.96 and up.

New public APIs:
- Scanners can now use a variety of different filters via the new
`ScanFilter' interfaces and its various implementations.
- It's possible to specify specific families to scan via `setFamilies'.
- Scanners can put an upper bound on the amount of data fetched by RPC
via the new `setMaxNumKeyValues' (works with HBase 0.96 and up only).
- HBaseRpc now has a `failfast()' and a `setFailfast(boolean)' pair
of methods to allow RPCs to fail as soon as their encounter an
issue out of the ordinary (e.g. not just a `NotSuchRegionException').
- `GetRequest' has additional constructor overloads that make its API
more uniform with that of other RPCs.

Noteworthy bug fixes:
- DeleteRequest wasn't honoring its timestamp if one was given (#58).
- When a connection attempt fails, buffered RPCs weren't cleaned up
or retried properly.
- When one RPC fails because of another one (e.g. we fail to send an
RPC because a META lookup failed), the asynchronous exception that
is given to the callback now properly carries the original RPC that
failed.
- There was an unlikely race condition that could cause an NPE while
trying to retrieve the ROOT region from ZooKeeper.


Pre-compiled JAR: http://tsunanet.net/~tsuna/asynchbase/asynchbase-1.5.0-rc1.jar
Source: https://github.com/tsuna/asynchbase
Javadoc: http://tsunanet.net/~tsuna/asynchbase/1.5.0/org/hbase/async/HBaseClient.html

$ git diff --stat v1.4.1.. | tail -n 1
70 files changed, 4824 insertions(+), 487 deletions(-)

$ git shortlog v1.4.1..
Andrey Stepachev (1):
Add support for multiple families/qualifiers in scanners.

Benoit Sigoure (65):
Start v1.5.0.
Add Viral to AUTHORS for his work on ScanFilter.
Document ScanFilter and prevent it from being subclassed externally.
Convert the regexp key filtering mechanism to the ScanFilter.
Document how to run integration tests.
Enhance filters a bit and add integration tests.
Add a new helper function to produce better errors during tests.
Mention new scanner filters in NEWS.
Allow RPCs to fail-fast.
Update NEWS / THANKS.
Update suasync to 1.3.2.
Properly clean up when connection fails before being opened.
Properly report which RPC has failed in HasFailedRpcException.
Fix a small race condition when looking up the ROOT region.
Add HBase protocol buffers to the compilation process.
De-serialize the HBase 0.95+ znode that points to META.
Fix the process of META lookups for HBase 0.95 and up.
Send the proper "hello" message to HBase 0.95 and up.
Remove some unused helper code to create buffers.
Update the per-RPC header for HBase 0.95+.
Refactor how RPC objects know what RPC method they're for.
Compile Java code generated from protobuf files separately.
Kill some trailing whitespaces.
Add a helper function to de-serialize Protocol Buffer VarInt32.
Add a semi-evil helper class to avoid copying byte arrays from protobufs.
De-serialize RPC responses from HBase 0.95+.
Add a helper function to de-serialize protocol buffers.
Handle META lookups with 0.95+.
Dedup byte arrays when deserializing KeyValues from 0.95+.
Make sure we have all the data we need before de-serializing.
Convert GetRequest to HBase 0.95+.
Convert AtomicIncrementRequest to HBase 0.95+.
Make the run target depend on the jar.
Sync protobuf files with changes made in the HBase 0.95 branch.
Add some make variables for the compilers used.
Add support for "cell blocks".
Avoid unnecessary string copies on each RPC.
Convert scanners and filters to HBase 0.95+.
Add a missing accessor to retrieve the max number of KVs on a Scanner.
Expose HBase 0.95's ability to limit the number of bytes
returned by a scanner.
Log the channel when logging outgoing RPCs.
Convert single-put RPCs to HBase 0.95+.
Ensure that RPCs that shouldn't get cell blocks don't.
Convert the CAS RPC to HBase 0.95+.
Convert single-Delete RPCs to HBase 0.95+.
Catch up with the protobuf changes made in 0.96
Explicit row locks are no longer supported in HBase 0.95 and up.
The 'getProtocolVersion' RPC no longer exists in 0.95+.
Add support for batched edits with HBase 0.96+.
Optimize multi-action a bit with HBase 0.95+ by sorting less.
Remove some warnings that come up when compiling with JDK 7.
Fix de-serialization of multi-packet responses.
Note that ColumnRangeFilter was added in HBase 0.92.
Fix up a logging message.
Upgrade to logback 1.0.13.
Upgrade to SLF4J 1.7.5.
Make sure to not schedule timers with a negative timeout.
Add some constructor overloads for GetRequest.
Upgrade to async 1.4.0.
Add a couple integration tests covering NoSuchColumnFamilyException.
Fix distclean: remove com/google/protobuf from the build directory.
Make sure we compile for Java 6.
Update NEWS for the v1.5.0 release.
Prevent warnings due to generated protobuf classes.
Upgrade to Netty 3.7.0.

Viral Bajaria (2):
Initial commit for ScanFilter.
Add more scanner filters.

Xun Liu (1):
Properly honor timestamps in DeleteRequest.

--
Benoit "tsuna" Sigoure

tsuna

unread,
Nov 5, 2013, 3:56:41 AM11/5/13
to AsyncHBase, HBase users
Hi all,
RC1 contained a few bugs that managed to escape, so we're cutting a
second RC. As a bonus, a few new things were added:
- Add support for deleting one specific KV at a specific timestamp
(instead of all previous KVs) – thanks Xun Liu.
- Maven compilation is fixed – thanks Stack.
- Support for prefetching META entries for a given table and a given
key range – thanks Brandon Forehand.

Bugs fixed:
- Compilation of protobufs on case sensitive filesystems now works (doh!)
- A race subtle / unlikely condition if we got disconnected from a
RS right after connecting.
- A couple bugs prevented AsyncHBase from being able to scan -ROOT-
or hbase:meta.
- With 0.96, no user-generated RPC could be directed at hbase:meta


Pre-compiled JAR: http://tsunanet.net/~tsuna/asynchbase/asynchbase-1.5.0-rc2.jar
Also available for Maven users as 1.5.0-SNAPSHOT on OSS Snapshots.
Diffstat between rc1 and rc2:
12 files changed, 390 insertions(+), 61 deletions(-)

Shortlog between rc1 ad rc2:

Benoit Sigoure (11):
Make sure we compile for Java 6.
Add a note regarding some infuriating limitations of the JRE.
Fix an edge case in HBase 0.96 one-shot scanners.
Add/improve a few toString() methods.
Fix accesses to "hbase:meta".
Make Scanner usable with -ROOT- / hbase:meta.
Handle an edge case with `prefetchMeta' related to 0.96 compatibility.
Update NEWS/THANKS/AUTHORS.
Fix the distclean rule so we can distclean twice in a row.
Have pom.xml cleaned during `distclean'.

Brandon Forehand (1):
Add support for prefetching the meta region.

Phil Smith (1):
Here's some one-liners to compile and run tests.

St.Ack (1):
Make mvn build accomodate protobuf files

Xun Liu (1):
Add support to delete a value at the specified timestamp.

--
Benoit "tsuna" Sigoure

Ted Yu

unread,
Nov 30, 2013, 12:26:49 AM11/30/13
to tsuna, AsyncHBase, HBase users
Benoit:
In HACKING, a sample command is given:
$ HBASE_HOME=~/src/hbase make integration ARGS='test f'

This means the integration tests need to be run on one of the servers where HBase is deployed, right ?

Cheers

tsuna

unread,
Dec 2, 2013, 1:33:13 PM12/2/13
to Ted Yu, AsyncHBase, HBase users
On Fri, Nov 29, 2013 at 9:26 PM, Ted Yu <yuzh...@gmail.com> wrote:
> In HACKING, a sample command is given:
> $ HBASE_HOME=~/src/hbase make integration ARGS='test f'
>
> This means the integration tests need to be run on one of the servers where
> HBase is deployed, right ?

It defaults to using "localhost" as the ZK quorum specification, but
you can specify whatever else you want:

$ HBASE_HOME=~/src/hbase make integration ARGS='table family
quorumSpec znodePath'

I'm thinking of cutting the 1.5 release today, please let me know if
you want a bit more time to test it before the release is cut.

--
Benoit "tsuna" Sigoure

Ted Yu

unread,
Dec 2, 2013, 1:34:58 PM12/2/13
to tsuna, AsyncHBase, HBase users
bq. I'm thinking of cutting the 1.5 release today

Please go ahead - my testing would focus on backward compatibility.

Cheers

ripsacCTO

unread,
Dec 3, 2013, 12:04:39 AM12/3/13
to async...@googlegroups.com, HBase users
Hi Tsuna,

Just wanted to know , if there is any way of Scanning using column value and with filters on column value.

Eg. Suppose I have column values against qualifier waterlevel as 40,20,30,10 etc.

If my requirement is like that I want to fetch data with requirement like give me data with water level > 20.

Is such a query possible. Please suggest some examples.

Does the latest version of asyncbase support the above ? If yes , please suggest with some example code base.

Thanks in advance.

Viral Bajaria

unread,
Dec 3, 2013, 1:36:38 AM12/3/13
to ripsacCTO, async...@googlegroups.com, HBase users
Depends on what version of HBase you are using. If you are using HBase 0.95+, with 1.5.0 asynchbase you will be able to use any filters that exist in HBase. Though you might need to add the required classes since asynchbase needs them to serialize to the old RPC protocol for < 0.95 and uses protobufs for versions > 0.95.

Thanks,
Viral

Michael Stack

unread,
Dec 3, 2013, 10:22:00 PM12/3/13
to async...@googlegroups.com, HBase users
+1

I've been using this RC with what will be hbase 0.96.1 on a small cluster under light loading and all seems to be basically work.

St.Ack

S. Zhou

unread,
Dec 24, 2013, 11:34:05 AM12/24/13
to async...@googlegroups.com, HBase users
I am trying the new version and run into some problem: details here: https://groups.google.com/forum/#!topic/asynchbase/zsIsLOZgiVc
Could u please help? We are trying to migrate to Hadoop 2.2 with HBase 0.96. But this issue blocks the migration of one application.
Reply all
Reply to author
Forward
0 new messages