Q: Embedded Database Recommendations?
Peter K.
I have a requirement for a database on an embedded platform.
Currently, I have looked at SQLite (http://www.sqlite.org/) and MySQL
(http://www.mysql.com/).
Does anyone have any other suggestions?
Does anyone have any comments on either of these two regarding
performance (specifically SELECT speeds and memory footprint)?
Any pointers appreciated!
Ciao,
Peter K.
--
"And he sees the vision splendid
of the sunlit plains extended
And at night the wondrous glory of the everlasting stars."
FreeRTOS.org
...but you do not mention what these requirements are so how can be offer
advice?
--
Regards,
Richard.
+ http://www.FreeRTOS.org & http://www.FreeRTOS.org/shop
17 official architecture ports, more than 6000 downloads per month.
+ http://www.SafeRTOS.com
Certified by TÜV as meeting the requirements for safety related systems.
Dale Harris
"Peter K." <p.koot...@remove.ieee.org> schreef in bericht
news:uvdzbn...@remove.ieee.org...
> Hi All,
>
> I have a requirement for a database on an embedded platform.
>
> Currently, I have looked at SQLite (http://www.sqlite.org/) and MySQL
> (http://www.mysql.com/).
>
> Does anyone have any other suggestions?
>
> Does anyone have any comments on either of these two regarding
> performance (specifically SELECT speeds and memory footprint)?
MySQL is not an embedded database IMHO, way too big. The licensing is also
very commercial since you're only really entitled to use it if for free you
are an open-source programmer.
I'm using SQLite in many projects and the performance is excellent and the
memory footprint very, very small (several KBytes). There's also an
in-memory version of the database in development (or maybe already
available).
** Posted from http://www.teranews.com **
Peter K.
> > I have a requirement for a database on an embedded platform.
>
> ...but you do not mention what these requirements are so how can be offer
> advice?
Repeated from my original post:
>>Does anyone have any comments on either of these two regarding
>>performance (specifically SELECT speeds and memory footprint)?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Peter K.
FreeRTOS.org
news:uzlonq...@remove.ieee.org...
> "FreeRTOS.org" <noe...@given.com> writes:
>
>> > I have a requirement for a database on an embedded platform.
>>
>> ...but you do not mention what these requirements are so how can be offer
>> advice?
>
> Repeated from my original post:
>
>>>Does anyone have any comments on either of these two regarding
>>>performance (specifically SELECT speeds and memory footprint)?
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Well the SELECT speed and memory footprint of mySQL are both superb.
[caveat - you need a 2GHz Pentium and 1GByte of RAM]
CBFalconer
>
> I have a requirement for a database on an embedded platform.
>
> Currently, I have looked at SQLite (http://www.sqlite.org/) and
> MySQL (http://www.mysql.com/).
>
> Does anyone have any other suggestions?
>
> Does anyone have any comments on either of these two regarding
> performance (specifically SELECT speeds and memory footprint)?
>
> Any pointers appreciated!
If an 'in memory' database is satisfactory, take a look at my GPLd
hashlib. It is written in pure standard C, so extremely portable.
I own it, so arrangements can be made if the GPL requirement to
release your code is a problem. See:
<http://cbfalconer.home.att.net/download/hashlib.zip>
--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
Peter K.
> If an 'in memory' database is satisfactory, take a look at my GPLd
> hashlib. It is written in pure standard C, so extremely portable.
> I own it, so arrangements can be made if the GPL requirement to
> release your code is a problem. See:
>
> <http://cbfalconer.home.att.net/download/hashlib.zip>
>
Thanks, Chuck, I'll take a look.
Paul Carpenter
noe...@given.com says...
> "Peter K." <p.koot...@remove.ieee.org> wrote in message
> news:uzlonq...@remove.ieee.org...
> > "FreeRTOS.org" <noe...@given.com> writes:
> >
> >> > I have a requirement for a database on an embedded platform.
> >>
> >> ...but you do not mention what these requirements are so how can be offer
> >> advice?
> >
> > Repeated from my original post:
> >
> >>>Does anyone have any comments on either of these two regarding
> >>>performance (specifically SELECT speeds and memory footprint)?
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
>
> Well the SELECT speed and memory footprint of mySQL are both superb.
>
> [caveat - you need a 2GHz Pentium and 1GByte of RAM]
Hmm my working gateway email server uses mySQL, originally on a sub GHz
processor and 256MB of RAM, Got upgraded to a newer nano-ATX board
1GHz and 1GB RAM. System runs Apache and email server and several other
apps, some of them using the half dozen databases.
However the system is so lightly loaded, the real question on SELECT
speeds and memory footprint depends on
Size and complexity of database
Number of concurrent users (access to database)
Number of SELECT requests per min/per hour...
What forms of caching are being used
etc....
--
Paul Carpenter | pa...@pcserviceselectronics.co.uk
<http://www.pcserviceselectronics.co.uk/> PC Services
<http://www.pcserviceselectronics.co.uk/fonts/> Timing Diagram Font
<http://www.gnuh8.org.uk/> GNU H8 - compiler & Renesas H8/H8S/H8 Tiny
<http://www.badweb.org.uk/> For those web sites you hate
Peter K.
> Hmm my working gateway email server uses mySQL, originally on a sub GHz
> processor and 256MB of RAM, Got upgraded to a newer nano-ATX board
> 1GHz and 1GB RAM. System runs Apache and email server and several other
> apps, some of them using the half dozen databases.
Thanks, all good information.
> However the system is so lightly loaded, the real question on SELECT
> speeds and memory footprint depends on
>
> Size and complexity of database
> Number of concurrent users (access to database)
> Number of SELECT requests per min/per hour...
> What forms of caching are being used
> etc....
The database will be large (e.g. tables of 1,000,000 or more entries),
but not particularly complex.
There weill only be one application accessing the database at a time
(or at all).
Assuming only one (concurrent) connection, I wonder why the number of
select requests per time period is a factor (it clearly will be for
multiple connections).
I suppose what I'm looking for is how long it takes to do:
SELECT * FROM SomeTable WHERE SomeField = 'SomeValue'
and SomeField is the primary key. By "how long" I mean mean, standard
deviation, and [if possible] worst case times.
Everett M. Greene
> "Peter K." <p.koot...@remove.ieee.org> wrote
> > "FreeRTOS.org" <noe...@given.com> writes:
> >
> >> > I have a requirement for a database on an embedded platform.
> >>
> >> ...but you do not mention what these requirements are so how can be offer
> >> advice?
> >
> > Repeated from my original post:
> >
> >>>Does anyone have any comments on either of these two regarding
> >>>performance (specifically SELECT speeds and memory footprint)?
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> [caveat - you need a 2GHz Pentium and 1GByte of RAM]
This is an embedded newsgroup. How can 1Gbyte of RAM be
considered "superb"?
The caveat doesn't seem like much of an endorsement. It
seems more a statement that a brick can be made to fly
if enough brute force is applied.
Paul Carpenter
says...
> Paul Carpenter <pa...@pcserviceselectronics.co.uk> writes:
>
> > Hmm my working gateway email server uses mySQL, originally on a sub GHz
> > processor and 256MB of RAM, Got upgraded to a newer nano-ATX board
> > 1GHz and 1GB RAM. System runs Apache and email server and several other
> > apps, some of them using the half dozen databases.
>
> Thanks, all good information.
>
> > However the system is so lightly loaded, the real question on SELECT
> > speeds and memory footprint depends on
> >
> > Size and complexity of database
> > Number of concurrent users (access to database)
> > Number of SELECT requests per min/per hour...
> > What forms of caching are being used
> > etc....
>
> The database will be large (e.g. tables of 1,000,000 or more entries),
> but not particularly complex.
>
> There weill only be one application accessing the database at a time
> (or at all).
>
> Assuming only one (concurrent) connection, I wonder why the number of
> select requests per time period is a factor (it clearly will be for
> multiple connections).
This will determine
how much data is moving around,
so determines how fast a storage unit (and interface),
how much needs to be buffered in memory, to keep up.
Could the serving be better served by being done from a RAM copy.
Can the processor/OS/database software keep up with the expected
number of requests.
Are the vast majority of database operations ONLY Read, and the
database updated once a day/hour/week/etc...
What sort of response time is a MUST have, would be nice, or would
be great! In other words how fast *must* the data be returned.
If you only make 1 request a second, there are a hell of a lot of
solutions, whereby cost of processor/RAM and size become more of the
issue in determining what software to use.
If you make 10,000 requests a second, you better have a really fast
processor with lots of RAM, possibly with RAM copy (or RAMdisk) for
the database in live use. Here the software becomes more of an issue
that determines the host hardware/software.
> I suppose what I'm looking for is how long it takes to do:
>
> SELECT * FROM SomeTable WHERE SomeField = 'SomeValue'
So simple requests, but what is the dataset size?
Number of fields in a record?
Fixed or variable length length fields in the records?
> and SomeField is the primary key. By "how long" I mean mean, standard
> deviation, and [if possible] worst case times.
To read a 10 byte or 10MB record? (yes some people have fields in the
SAME database that is a large picture or other document!)
Standard deviation on a system doing NOTHING else?
What type of system are you using to benchmark this on?
The other important issue is
What else is running
How much processor time available
How much storage access time available?
(i.e. does this application have near total
allocation of storage unit or is somebody
streaming video to/from the storage unit
at the same time!)
There is NO simple answer to questions like this.
Consider this analogy
Take a deck of cards
Shuffle the deck (our database)
Now consider these requests
Find the 15th card down - very fast and easy to find
Find a heart of any size - slower but easy.
Find All the Three's - More complicated
Find the 5 of hearts - more complicated
A lot depends on how your data is organised and normalised and
many other issues, you have not given the details on.
FreeRTOS.org
news:20080712.7...@mojaveg.lsan.mdsg-pacwest.com...
It was pure sarcasm (its a British thing) and not meant to be taken
literally or in any way as fact. The OP had a 'requirement' for an embedded
database, but without giving any clue what that requirement was wants advice
about memory footprint. I don't know if it is to run on an 8bit device or a
dual core Pentium - hence the sarcasm.
Peter K.
of string?" type question. Yours (and other) responses have been
interesting.
Peter K.
> It was pure sarcasm (its a British thing) and not meant to be taken
> literally or in any way as fact. The OP had a 'requirement' for an embedded
> database, but without giving any clue what that requirement was wants advice
> about memory footprint. I don't know if it is to run on an 8bit device or a
> dual core Pentium - hence the sarcasm.
Richard,
It came across to me that you hadn't read the question. Your
'sarcasm' comes across as something quite else.
Thankfully there have been more useful, and thoughtful, responses from
others.
Paul Keinanen
wrote:
>Paul Carpenter <pa...@pcserviceselectronics.co.uk> writes:
>
>> Hmm my working gateway email server uses mySQL, originally on a sub GHz
>> processor and 256MB of RAM, Got upgraded to a newer nano-ATX board
>> 1GHz and 1GB RAM. System runs Apache and email server and several other
>> apps, some of them using the half dozen databases.
>
>Thanks, all good information.
>
>> However the system is so lightly loaded, the real question on SELECT
>> speeds and memory footprint depends on
>>
>> Size and complexity of database
>> Number of concurrent users (access to database)
>> Number of SELECT requests per min/per hour...
>> What forms of caching are being used
>> etc....
>
>The database will be large (e.g. tables of 1,000,000 or more entries),
>but not particularly complex.
Apart from RAM data bases, on a virtual memory machine, the most
primitive data base would be just mapping the tables into virtual
memory and let the OS do the data loading with page faults as well as
the caching of frequently used data.
>There weill only be one application accessing the database at a time
>(or at all).
>
>Assuming only one (concurrent) connection, I wonder why the number of
>select requests per time period is a factor (it clearly will be for
>multiple connections).
>
>I suppose what I'm looking for is how long it takes to do:
>
>SELECT * FROM SomeTable WHERE SomeField = 'SomeValue'
>
>and SomeField is the primary key. By "how long" I mean mean, standard
>deviation, and [if possible] worst case times.
If you are only interested in accessing through the primary (and even
secondary keys) any COBOL style ISAM (Indexed Sequential Access
Method) file system should do. With current amount of memory and only
a very small (one million) amount of records, the whole key tree
could be in memory, so the SELECT time is not an issue. Retrieving the
actual data from disk will of course cost some extra. Unfortunately,
none of the popular (Windows and Unix variants) operating systems
support ISAMs at the OS level, so some drivers are required.
Paul
Peter K.
John Devereux
> Hi All,
>
> I have a requirement for a database on an embedded platform.
>
> Currently, I have looked at SQLite (http://www.sqlite.org/) and MySQL
> (http://www.mysql.com/).
>
> Does anyone have any other suggestions?
>
> Does anyone have any comments on either of these two regarding
> performance (specifically SELECT speeds and memory footprint)?
>
> Any pointers appreciated!
>
> Ciao,
>
> Peter K.
Datadraw looks interesting.
--
John Devereux
toby
> "Peter K." <p.kootsoo...@remove.ieee.org> wrote in message
>
> news:uzlonq...@remove.ieee.org...
>
> > "FreeRTOS.org" <noem...@given.com> writes:
>
> >> > I have a requirement for a database on an embedded platform.
>
> >> ...but you do not mention what these requirements are so how can be offer
> >> advice?
>
> > Repeated from my original post:
>
> >>>Does anyone have any comments on either of these two regarding
> >>>performance (specifically SELECT speeds and memory footprint)?
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Well the SELECT speed and memory footprint of mySQL are both superb.
>
> [caveat - you need a 2GHz Pentium and 1GByte of RAM]
You need nothing like that. A recent poster on c.d.mysql is
approaching a 45MB memory footprint after basic tuning achieved a
footprint of 80MB. Certainly it will run just fine on a 486/66 with
256MB or less. But I am speaking about modern versions of the full
networked server; the embedded version or 3.23.x are probably less
demanding.
>
> --
> Regards,
> Richard.
>
> +http://www.FreeRTOS.org&http://www.FreeRTOS.org/shop
> 17 official architecture ports, more than 6000 downloads per month.
>
Peter K.
> Datadraw looks interesting.
That it does! Thanks, John.
Boudewijn Dijkstra
<p.koot...@remove.ieee.org>:
> I have a requirement for a database on an embedded platform.
>
> Currently, I have looked at SQLite (http://www.sqlite.org/) and MySQL
> (http://www.mysql.com/).
>
> Does anyone have any other suggestions?
>
> Does anyone have any comments on either of these two regarding
> performance (specifically SELECT speeds and memory footprint)?
>
> Any pointers appreciated!
How about Polyhedra?
http://www.enea.com/polyhedra
--
Gemaakt met Opera's revolutionaire e-mailprogramma:
http://www.opera.com/mail/
Peter K.
>
> How about Polyhedra?
> http://www.enea.com/polyhedra
Interesting, Boudewijn! Thanks for the tip.
Ignacio G.T.
> Hi All,
>
> I have a requirement for a database on an embedded platform.
>
> Currently, I have looked at SQLite (http://www.sqlite.org/) and MySQL
> (http://www.mysql.com/).
>
> Does anyone have any other suggestions?
>
> Does anyone have any comments on either of these two regarding
> performance (specifically SELECT speeds and memory footprint)?
>
> Any pointers appreciated!
You might consider:
eXtremeDB, form McObject
Berkeley DB, from Oracle
These are considered "truely" embedded DBs.
CBFalconer
> CBFalconer <cbfal...@yahoo.com> writes:
>
>> If an 'in memory' database is satisfactory, take a look at my GPLd
>> hashlib. It is written in pure standard C, so extremely portable.
>> I own it, so arrangements can be made if the GPL requirement to
>> release your code is a problem. See:
>>
>> <http://cbfalconer.home.att.net/download/hashlib.zip>
>
> Thanks, Chuck, I'll take a look.
For your purposes one advantage is that the code is very compact.
The system still gobbles memory to store the data, but that is
inherent for anything. The code is sharable, so only one code
instance can handle any number of completely independent data
tables. The code module will occupy about 1.5k when compiled for
the 80386.
Peter K.
> You might consider:
>
> eXtremeDB, form McObject
> Berkeley DB, from Oracle
>
> These are considered "truely" embedded DBs.
Thanks, Ignacio!
Both look interesting.
Regards,
Peter K.
> For your purposes one advantage is that the code is very compact.
> The system still gobbles memory to store the data, but that is
> inherent for anything. The code is sharable, so only one code
> instance can handle any number of completely independent data
> tables. The code module will occupy about 1.5k when compiled for
> the 80386.
Thanks. I've set myself up with a test case, and I've run MS SQL
Server (yes, not really embedded, but what the hey), MySQL and SQLite
through it.
I hope to do the same now with various other potential solutions.
I'll let you know how things go.
Regards,