Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Need a file based database library

0 views
Skip to first unread message

Andreas Otto

unread,
Nov 24, 2009, 3:44:12 AM11/24/09
to
Hello,

I'm looking for a database library able to be used as a local application
storage.

The following requirements are necessary:

1. No TCP/IP or UDP interface
2. available as shared/static library
3. able to use multiple INDEX files to access DATA
-> the INDEX self have to be persistent
4. SQL not required
-> better without SQL interface
5. as simple as possible
6. source-code available
7. GPL/Apache or something equal as LICENSE
8. C/C++ interface


mfg

Andreas Otto

jacob navia

unread,
Nov 24, 2009, 4:12:11 AM11/24/09
to
Andreas Otto a �crit :

Translation:

I want a database system for my product. I do not want to pay a penny for this,
and I want to profit from the work of other people to allow me to earn money
without working too much.

Berkeley DB would fill your requirements. It fills (1) (2) (3) (4) (6)
and (8) Happily it is surely NOT free, and the last time I asked they wanted
at least US$ 25 000.

Andreas Otto

unread,
Nov 24, 2009, 4:54:49 AM11/24/09
to
jacob navia wrote:

> Andreas Otto a écrit :


>> Hello,
>>
>> I'm looking for a database library able to be used as a local
>> application storage.
>>
>> The following requirements are necessary:
>>
>> 1. No TCP/IP or UDP interface
>> 2. available as shared/static library
>> 3. able to use multiple INDEX files to access DATA
>> -> the INDEX self have to be persistent
>> 4. SQL not required
>> -> better without SQL interface
>> 5. as simple as possible
>> 6. source-code available
>> 7. GPL/Apache or something equal as LICENSE
>> 8. C/C++ interface
>>
>>
>> mfg
>>
>> Andreas Otto
>>
>
> Translation:
>
> I want a database system for my product.

-> right


> I do not want to pay a penny for this,

-> right


> and I want to profit from the work of other people

-> right


> to allow me to earn money without working too much.

-> wrong, I have enough money ... I don't need more

> Berkeley DB would fill your requirements. It fills (1) (2) (3) (4) (6)
> and (8) Happily it is surely NOT free, and the last time I asked they
> wanted at least US$ 25 000.

-> thanks for your help,,, I already check for Berkeley DB
and as I read the name "Oracle" I know that this is not I'm
looking for

currently I check "qdbm" the only thing I'm missing is that I don't get
back a pointer to the internal database memory after a key look-up ...
this mean every data access is either a malloc or a memcopy operation
-> not good !!!!

mfg

Andreas Otto (aotto1968)

Nick Hounsome

unread,
Nov 24, 2009, 4:57:49 AM11/24/09
to
On 24 Nov, 08:44, Andreas Otto <aotto1...@users.sourceforge.net>
wrote:

sqlite3

Andreas Otto

unread,
Nov 24, 2009, 5:32:29 AM11/24/09
to
Nick Hounsome wrote:

thanks,

the problem is that sqlite uses SQL this mean that a query have to through
some kind analyse to build a query-plan ....

To be more precise I looking for a simple Key/Value based interface with
Value be an arbitrage C/C++ struct and a lookup just return the pointer
to this struct.


mfg

Andreas Otto

Andreas Otto

unread,
Nov 24, 2009, 6:13:46 AM11/24/09
to

Hello,

I want to be more precise ...

I looking for an "get" style function to return the Value "pointer" without
doing memory action like "malloc" or "memcpy".
Reason, I want to use this SW to store a C "struct" of my own data format.
This struct sould be returned directly from the DB
memory (usually the cache and not the harddrive). To use this pointer some
kind of transaction is necessary

1. Get INTERNAL pointer and LOCK data cache entry
2. do some operation on the "pointer"
3. after finish, UNLOCK data cache entry


mfg

Andreas Otto

ddd

unread,
Nov 24, 2009, 6:20:09 AM11/24/09
to

you can try http://tokyocabinet.sourceforge.net/

seems quite nice

Uwe Klein

unread,
Nov 24, 2009, 6:28:00 AM11/24/09
to

sounds like memory mapped file access?

uwe

Andreas Otto

unread,
Nov 24, 2009, 6:52:33 AM11/24/09
to
ddd wrote:

thanks, but the restriction ...

>>>>>>>>>>>>>>>>>>
I looking for an "get" style function to return the Value "pointer" without
doing memory action like "malloc" or "memcpy".
Reason, I want to use this SW to store a C "struct" of my own data format.

This struct should be returned directly from the DB


memory (usually the cache and not the harddrive). To use this pointer some
kind of transaction is necessary

1. Get INTERNAL pointer and LOCK data cache entry
2. do some operation on the "pointer"
3. after finish, UNLOCK data cache entry
<<<<<<<<<<<<<<<<<<<<

is still alive ....

does you have a solution?


mfg

Andreas Otto


Neil Madden

unread,
Nov 24, 2009, 6:22:59 AM11/24/09
to

Metakit/Vlerq, SQLite, maybe Berkeley DB. There are a few others that
you can Google.

Is there a Tcl aspect to this question?

-- Neil

Andreas Otto

unread,
Nov 24, 2009, 6:59:27 AM11/24/09
to
Uwe Klein wrote:

mmap(2) give a hint but is far to less to work with....
the "database" layer is missing ... no key/value search
I just want to have that the Value from the Key/Value pair
is of my "own" format.

mfg

Andreas Otto (aotto1968)

Andreas Otto

unread,
Nov 24, 2009, 7:01:08 AM11/24/09
to
Neil Madden wrote:

>>
>
> Metakit/Vlerq, SQLite, maybe Berkeley DB. There are a few others that
> you can Google.
>
> Is there a Tcl aspect to this question?
>

Tcl will be one of the data consumer targets


mfg

Andreas Otto (aotto1968)


Uwe Klein

unread,
Nov 24, 2009, 7:19:56 AM11/24/09
to
Andreas Otto wrote:
> Neil Madden wrote:
>
>
>>Metakit/Vlerq, SQLite, maybe Berkeley DB. There are a few others that
>>you can Google.
>>
>>Is there a Tcl aspect to this question?
>>
>
>
> Tcl will be one of the data consumer targets

what about XoTcl persistent storage?

uwe

Uwe Klein

unread,
Nov 24, 2009, 7:18:59 AM11/24/09
to

Robert Heller

unread,
Nov 24, 2009, 8:32:04 AM11/24/09
to


Since you are asking in the Tcl newsgroup...

How about using SWIG to implement a Tcl interface to your C/C++ structs
and then using a Tcl Hash table (aka Tcl arrays) as your database? If
all of the structures are declared in .h files, you can include these .h
files *as is* and SWIG will generate Tcl accessor methods for them.

>
>
> mfg
>
> Andreas Otto
>
>

--
Robert Heller -- 978-544-6933
Deepwoods Software -- Download the Model Railroad System
http://www.deepsoft.com/ -- Binaries for Linux and MS-Windows
hel...@deepsoft.com -- http://www.deepsoft.com/ModelRailroadSystem/

Andreas Otto

unread,
Nov 24, 2009, 9:06:41 AM11/24/09
to
Robert Heller wrote:


>
>
> Since you are asking in the Tcl newsgroup...
>
> How about using SWIG to implement a Tcl interface to your C/C++ structs
> and then using a Tcl Hash table (aka Tcl arrays) as your database? If
> all of the structures are declared in .h files, you can include these .h
> files *as is* and SWIG will generate Tcl accessor methods for them.
>

Yes, this will work but I'll use a libmsgque interface ....
the problem is that I have to program all the management stuff
by my own -> In fact I have to program the entire database by
my own and this is what I don't want ....

I think my restrictions and my usage is quite simple but it seems,
until now, all databases (I checked) use the same technology
for getting data
1. do a malloc for every item returned and the end-use have to
free the data returned
2. end-user provide storage and the database do a mencpy to
fill the storage
this all is ok for small size usage or for a low access time requirement
but not for me.

It is the same problem that I solved with libmsgque for the communication
layer. for "normal" operation "no" malloc and "no" memcpy is performed
this mean close to zero IO overhead.

I search a database with the same design specification
-> close to zero IO overhead

And I do not want to program everything by my own


mfg

Andreas Otto

Lew Pitcher

unread,
Nov 24, 2009, 9:22:15 AM11/24/09
to
On November 24, 2009 03:44, in comp.lang.c, Andreas Otto
(aott...@users.sourceforge.net) wrote:

> Hello,
>
> I'm looking for a database library able to be used as a local
> application storage.
>
> The following requirements are necessary:

[snip]

Try the Berkeley DB (open source C embeddable database, now distributed by
Oracle)
http://www.oracle.com/technology/software/products/berkeley-db/index.html


--
Lew Pitcher
Master Codewright & JOAT-in-training | Registered Linux User #112576
Me: http://pitcher.digitalfreehold.ca/ | Just Linux: http://justlinux.ca/
---------- Slackware - Because I know what I'm doing. ------


Igmar Palsenberg

unread,
Nov 24, 2009, 10:48:55 AM11/24/09
to

sqlite.

Igmar

tom.rmadilo

unread,
Nov 24, 2009, 3:06:18 PM11/24/09
to
On Nov 24, 12:44 am, Andreas Otto <aotto1...@users.sourceforge.net>
wrote:

DJB's cdb (constant database) is very fast and has other desirable
properties:

http://cr.yp.to/cdb.html

This database is used in his dns package (among many others), so it
should support multiple indexes in one file (but this is an
application feature built with the library).

License: good question, free to use however you want, not sure about
redistribution.

One problem is it doesn't run on windows.

Gene

unread,
Nov 24, 2009, 8:18:07 PM11/24/09
to
On Nov 24, 3:44 am, Andreas Otto <aotto1...@users.sourceforge.net>
wrote:
I've used Berkeley DB in a couple of projects, and it works very
well. Sleepy Cat, the company that built the original. It's been
bought by Oracle. I don't know what the current license requirement
is.
0 new messages