An update regarding hamsterdb 2.1.0

[Christoph Rupp]

Hi,

hamsterdb 2.1.0 is nearly ready. In fact, it's stable and you can already download the sources. But there's a small known issue (the hamserver MSVC Project files are not linking correctly) which prevents me from releasing the new version already now. And since i'll be on vacation for the next 3 weeks I won't be able to release it till beginning of March.

I already made announcements that 2.1.0 will have an API cleanup. I was able to reduce the code size by more than 15%, and the API changes are small, but there are many and you definitely will have to adjust your sources. I tried to pack all changes into this release, therefore you do not have to expect any changes for the near future.

The README describes all changes:
https://github.com/cruppstahl/hamsterdb/blob/v2/README

A lot of deprecated macros and functions were removed; the function names are now more consistent, and duplicate functionality was removed. AES and zlib-compression were also dropped.

I also rewrote the memory mapped I/O handling, which improves performance when opening an existing database.

If you want to have a look then you can pull the sources and check out the "v2" branch:
https://github.com/cruppstahl/hamsterdb

bye
Christoph

[Raul]

Why is encryption being dropped?

I'm just getting started with hamsterdb, and exactly that's the feature that pushed me to try it in the first place, getting other embedded databases to do encryption was a pain in the neck, and the only one that had it out of the box is hamsterdb.

Couldn't you just leave the encryption functions in? drop compression but still leave encryption? even as optional if you want, have a package with and another without.

Thanks

[Christoph Rupp]

Hi Raul,

it was dropped for a couple of reasons.
- I wasn't aware that anybody uses encryption
- maintaining it (in the current implementation) always cost me a lot of time
- it was insecure because the header page, the journal and the log were not encrypted
- it was slow because the AES code did not use hardware support (which is available in most CPUs nowadays)
- it was also insecure because each page was CRC encrypted, but the CRC engine was always restarted for each page (the CRC state was not transferred from one page to the next)

I will try to think of solutions, but if the ratio of implementation effort vs added value is too high then adding it doesnt make sense for me. Creating a separate version with AES would increase my maintenance efforts even more.

Best regards
Christoph

2013/2/8 Raul <raul.guerre...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "hamsterdb User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hamsterdb-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Raul]

Ok,

But what about instead of just forgetting about encryption and compression altogether, as these are done at the lowlevel I/O part, why don't you instead implement a hook so anyone can actually plug logic right before the read and write I/O so people can preprocess the raw data before any of this actions can be executed?

That way, if someone needs encryption, compression or whatever preprocessing right before data is written, then they can create hookups, so, if let's say, someone uses hamsterdb for iOS, they can create their own optimized encryption using the integrated openSSL, or whatever.

Doing that even opens up the doors to more creative actions, like logging, I/O replication and whatnot.

What do you think?

[Christoph Rupp]

Actually that was how it was implemented; encryption used a "page level filter" and compression used a "record level filter", These APIs used callbacks that were invoked whenever you read or write a page (resp. a record). This turned out to be overkill and caused a lot of maintenance headaches.

But i got your message and i will think about a solution.

bye

Christoph

2013/2/8 Raul <raul.guerre...@gmail.com>