DBISAM vs. NexusDB
Haralabos Michael (Creation Power)
> Anybody have any preliminary comparisons on the current betas of
> DBISAM 4 vs NexusDB?
I'm trying to figure out this <g> but the latest Nexus DB is causing
an AV into my Delphi 7 IDE =(
--
Haralabos Michael
peppy...@emailias.com
FlashFiler 3) is soon to be released (they are selling betas right now
I believe).
Over the years, I've toyed with BDE alternatives. I looked at
FlashFiler when it was by TurboPower--but never bought. I looked at
DBISAM; even Apollo and CodeBase. But, I wound up never using
anything except the BDE, and then Borland's ADO support. That's all
I've used, plus kbmMemTable.
I did just the other day download the FF 2.13 Open Source version. I
have a small project that is primarily for my own purposes, but I also
will probably be giving it away as freeware. Since I'm not planning
on making any money on it, I'm not anxious to spend any... <g> But I
do want a simple, single EXE deployment, so I figure that the Open
Source FF seems like it would be a perfect fit for my little project.
But, even so, I'll be keeping an eye on the upcoming releases of
DBISAM and NexusDB just for reference. If I get an actual paying
project, I'll want to know which is better... <g>
Zoran
<peppy...@emailias.com> wrote in message
news:viiaivoi1soqba4au...@4ax.com...
> I hear that DBISAM 4 is coming out soon. Also NexusDB (formerly
> FlashFiler 3) is soon to be released (they are selling betas right now
> I believe).
NexusDB has nothing to do with FF. It is completely new design, new code,
and new features. They both are client/server, and that is where the
similarity ends.
Eivind Bakkestuen
yet on standby. We don't sell betas. :)
I don't believe DBISAM has released a public beta of v4 yet, it is therefore
unlikely that you'll find somebody willing to break their NDA. Once there is
a public beta, you are welcome to compare.
--
Eivind Bakkestuen
Nexus Database Systems
<peppy...@emailias.com> wrote in message
news:viiaivoi1soqba4au...@4ax.com...
Eivind Bakkestuen
we would appreciate it if you would report bugs to us. That would
significantly increase the chances that it will be fixed. Please use either
the group below, or our online bugreport system (go to main page below, then
Support on left side) to enter your bug report. (I can't find your name
anywhere in messages on our newsgroup.)
news://news.nexusdb.com/nexusdb.public.support
Regards,
Eivind Bakkestuen
Nexus Database Systems
"Haralabos Michael (Creation Power)" <spac...@nospam.otenet.gr> wrote in
message news:3f25...@newsgroups.borland.com...
David Marcus
> NexusDB has nothing to do with FF. It is completely new design, new code,
> and new features. They both are client/server, and that is where the
> similarity ends.
Not true. While the NexusDB code is new, the design is based on FF. The
intention is to make upgrading very easy.
--
David Marcus
Brian Moelk
> DBISAM and NexusDB just for reference. If I get an actual paying
> project, I'll want to know which is better... <g>
That is the best strategy IMO: evaluate it for yourself.
I think with regards to these products, it's a matter of what you value that
will determine which is "best".
> Anybody have any preliminary comparisons on the current betas of
> DBISAM 4 vs NexusDB?
Not likely since I believe that DBISAM 4 is a closed beta at this time.
--
Brian Moelk
bmo...@NObrainendeavorSPAM.FORcomME
http://www.brainendeavor.com
Eric Hill
is
> yet on standby. We don't sell betas. :)
>
> I don't believe DBISAM has released a public beta of v4 yet, it is
therefore
> unlikely that you'll find somebody willing to break their NDA. Once there
is
> a public beta, you are welcome to compare.
Does NEXUSDB use a single data file or multiple data files?
Eric
Brian Moelk
Multiple.
Feel free to ask questions anywhere, however you'll more likely get better,
faster and more detailed answers over on the NexusDB newsgroups:
news://news.NexusDB.com
Eivind Bakkestuen
FF2 was in large part designed by Thorsten Engler (who is now the NexusDB
Architect). But he was not responsible for implementing the design, and many
things didn't turn out like they were designed.
NexusDB shares much of the same design, but this time Thorsten has been the
main implementer as well. So there is no doubt about that, you will
recognize parts of the feature set. However, the main intention is not to
make upgrading easy for FlashFiler users (although its a nice sideeffect),
but simply to create the best database design possible.
--
Eivind Bakkestuen [NDD]
Nexus Database Systems
"David Marcus" <david...@alum.mit.edu> wrote in message
Mark Steiner
peppy...@emailias.com wrote:
> I hear that DBISAM 4 is coming out soon. Also NexusDB (formerly
> FlashFiler 3) is soon to be released (they are selling betas right now
> I believe).
And I hear that AidAim Software is developing Accuracer Database System
which will allow both file-server and client-server access and will have
single-file database.
So we will have a wide choice of client-server BDE alternative systems
soon.
Brian Moelk
> which will allow both file-server and client-server access and will have
> single-file database.
They've been doing that for quite some time. Haven't seen any movement on
it, so I wouldn't count on it unless the AidAim guys say it's being actively
worked on and can provide some beta's, feature lists, etc.
> So we will have a wide choice of client-server BDE alternative systems
> soon.
BTW, File-Based vs. Client/Server is an important distinction between DBISAM
and NexusDB. AFAIK, DBISAM still uses a File-Based locking scheme even in
C/S mode.
--
Brian Moelk
bmoe...@SPAMbrainendeavorFOR.MEcom
http://www.brainendeavor.com
Brian Moelk
> > C/S mode.
>
From: http://www.elevatesoft.com/manual_Locking_Concurrency.htm
"All locks in DBISAM are performed using calls to the operating system,
which is either Windows 95/98/ME, or Windows NT/2000. If running on a
network, these calls are then routed by the operating system to the file
server's operating system, which could be Netware, Windows NT, Windows
95/98/ME, LANtastic, etc. The benefits of this approach are that there are
no lock files required and dangling locks left from an improper shutdown can
be cleaned up by the operating system in most cases.
DBISAM takes advantage of the fact that Windows 95/98/ME/NT/2000 all allow
an application to lock portions of a file beyond the actual size of the
file. This process is known as virtual byte offset locking."
IIRC, this is exactly how MS Access works as well.
David Marcus
> AFAIK, DBISAM still uses a File-Based locking scheme even in
> C/S mode.
What does this mean? How does the locking scheme work?
--
David Marcus
Trev
DBISAM
>>and NexusDB.
Interesting. What locking method does Nexus use and how, in your opinion, is
it better?
Trev
"Brian Moelk" <bmo...@NObrainSPAMendeavorFOR.MEcom> wrote in message
news:3f26593e$1...@newsgroups.borland.com...
Brian Moelk
is
> it better?
Nexus is a true C/S engine that uses internal locks (critical sections and
other OS provided synchronization primitives). The engine itself acquires
exclusive access to the physical file and therefore the file isn't shared.
You can read about it at: http://www.nexusdb.com/ under Features In
Detail->Client/Server
Related to locking strategies, one of the unique things that Nexus offers is
snapshot transactions. These transactions require no locks but provide a
consistent (serializable) view of the database. The end result is that
readers using snapshot transactions do not block writers.
From what I understand about the internal architecture, it's different than
Interbase/Firebird's multi-versioning in that it only creates new
versions/copies of the blocks necessary based on the active snapshot
transactions rather than creating new record versions for each write.
Thorsten has explained the exact details to me, but it hurts my head. <g>
IMO, NexusDB is an incredible piece of engineering that deserves to be
looked at seriously.
Brian Moelk
controlled
> by a central server.
It is controlled by a central server: the file server.
<SNIP>
> But I wouldn't jump to the
> conclusion that NexusDB will have better transaction performance just
> because DBISAM locks all tables during a transaction.
No need to jump to any conclusions, you can check it out for yourself.
> Maybe, maybe not, and
> even if so it may depend on how transactions are used within the
> application.
Certainly you can write poor code using any DB.
The really, really nice thing about NexusDB is that it also supports Nested
Transactions which allows you to write transactional code in a very natural
and clean way.
Thorsten Engler [NexusDB]
is
> it better?
NexusDB uses it's own highly optimized in memory structures for locking
(content locks on record and table level, transaction locks and some other
types of short lived locks.). As a true C/S database the server process
opens the table files in exclusive mode. All access to the tables is through
the server process. Calls into the OS file locking functions require a
context switch into kernel mode through a call gate. Using structures in
user mode memory is significantly faster.
Another huge advantage of a true C/S architecture is that the server process
can aggressively cache database pages in memory without any chance that the
cache will be out of data.
--
Thorsten Engler [NexusDB Architect]
Dan Palley
news:3f26a87a$1...@newsgroups.borland.com...
> >>BTW, File-Based vs. Client/Server is an important distinction between
> DBISAM
> >>and NexusDB.
>
> Interesting. What locking method does Nexus use and how, in your opinion,
is
> it better?
>
Client/Server implies that clients access the database through a db server
and not by accessing the db files directly. The db server handles locking
internally since it's the only process that actually reads/writes to the db
files.
Dan
Herbert Sitz
> DBISAM
> >>and NexusDB.
>
> Interesting. What locking method does Nexus use and how, in your opinion,
is
> it better?
>
> Trev
>
The real limitation with DBISAM arises (I think) because the data files can
be accessed by multiple engines at one time. That is, even if you have a
DBISAM Server for a database, the files for that database can be accessed
directly by a client that's using the fileserver engine. This was a design
decision made by the DBISAM developers, as I understand it.
The result is that with DBISAM it's not possible to have locking controlled
by a central server.
The end result is that to avoid possible deadlocks during an explicit DBISAM
transaction all tables in a database are locked by DBISAM. Thus, you must
make sure your transactions are short-lived because other clients will be
unable to make any deletions or updates while a transaction is in progress.
In practice I think this system seems to work pretty well, and I think the
DBISAM guys are pretty happy with it as a design tradeoff for allowing both
F/S and C/S access to the same database. For people who have no use for F/S
at all it's obviously not the ideal solution. But I wouldn't jump to the
conclusion that NexusDB will have better transaction performance just
because DBISAM locks all tables during a transaction. Maybe, maybe not, and
even if so it may depend on how transactions are used within the
application.
-- Herbert Sitz
Herbert Sitz
news:3f26bcca$1...@newsgroups.borland.com...
> > The result is that with DBISAM it's not possible to have locking
> controlled
> > by a central server.
>
> It is controlled by a central server: the file server.
>
Ah, yes, of course. I should have said, "not controlled by a central
database server program."
-- Herb
Tim Young
<< The real limitation with DBISAM arises (I think) because the data files
can be accessed by multiple engines at one time. That is, even if you have
a DBISAM Server for a database, the files for that database can be accessed
directly by a client that's using the fileserver engine. This was a design
decision made by the DBISAM developers, as I understand it.
The result is that with DBISAM it's not possible to have locking controlled
by a central server. >>
Actually it is - there's a SINGLE_USER define in DBISAM (been in there for
years) that you can turn on to bypass OS locks and have the engine open up
tables exclusively. It's the same locking used for in-memory tables. The
only downside to this is that we don't take advantage of this at higher
levels in terms of caching because we didn't go any further with it then
that. However, it's not particularly that difficult to do, and actually
makes the design of the engine *much* simpler because we don't have to deal
with cache refresh issues like change detection.
--
Tim Young
Elevate Software
www.elevatesoft.com
Joe Real
says...
|BTW, File-Based vs. Client/Server is an important distinction between DBISAM
|and NexusDB. AFAIK, DBISAM still uses a File-Based locking scheme even in
|C/S mode.
I think you have been misinformed about DBISAM. You can use DBISAM to force the
server lock only and not a File-Based locking scheme just like the inflexible
designs of most database engines today.
There is definitely a HUGE ADVANTAGE of having a C/S database engine do both. And I
hope NexusDB will have this feature in order for us to at least try it out. This is
a very important feature at least for us. Here's the quote of Eryk from ElevateSoft
newsgroup:
"Yes. DBISAM does that so you can have a DBISAM server process accessing the
same tables as other processes operating in 'file share' mode. A typical use
for this is when you have both normal users and a webserver CGI accessing
the same database at the same time. Forcing the CGI to connect in C/S mode
is inefficient compared to using direct file access and adds nothing to
system reliability in most cases. There is a $DEFINE in the code that will
switch DBISAM to maintaining locks in a server memory structure only and
bypassing OS locking calls however it buys you next to nothing in
performance terms which is why we don't bother talking about it often.
Basically, this isn't a significant issue performance wise.
Eryk"
and from Tim Young:
"Yep, that's why you can have multiple DBISAM servers access the same data
alongside of CGI/ISAPI apps without having to go through one server."
Brian Moelk
DBISAM
> |and NexusDB. AFAIK, DBISAM still uses a File-Based locking scheme even
in
> |C/S mode.
>
> I think you have been misinformed about DBISAM.
Perhaps I am misinformed, but nothing in the quotes you mentioned from Eryk
or Tim indicate that I am misinformed.
> You can use DBISAM to force the
> server lock only and not a File-Based locking scheme just like the
inflexible
> designs of most database engines today.
It is one thing to lock the file exclusively just because you can and quite
another to leverage that architectural choice for the benefits it yields.
> There is definitely a HUGE ADVANTAGE of having a C/S database engine do
both.
Disagree. I don't see this capability as an advantage, in fact I see it as
a disadvantage. Again, this depends on what you value.
> And I
> hope NexusDB will have this feature in order for us to at least try it
out.
It doesn't and probably never will.
> "Yes. DBISAM does that so you can have a DBISAM server process accessing
the
> same tables as other processes operating in 'file share' mode. A typical
use
> for this is when you have both normal users and a webserver CGI accessing
> the same database at the same time.
I wouldn't use CGI, would you? And if I did, I would use a server process
and put the client in the CGI application. IMO, there is no point in
supporting technological architectures that don't make sense.
> Forcing the CGI to connect in C/S mode
> is inefficient compared to using direct file access and adds nothing to
> system reliability in most cases.
Disagree. What happens if you have a network failure when you are writing
to the file?
> There is a $DEFINE in the code that will
> switch DBISAM to maintaining locks in a server memory structure only and
> bypassing OS locking calls however it buys you next to nothing in
> performance terms which is why we don't bother talking about it often.
> Basically, this isn't a significant issue performance wise.
Perhaps for DBISAM it doesn't make a performance difference, but take a
specific example: caching.
The simple fact that it is impossible to keep a data cache in synch with a
file-based architecture should provide some indication that a true C/S
engine can provide better performance through intelligent caching.
> "Yep, that's why you can have multiple DBISAM servers access the same data
> alongside of CGI/ISAPI apps without having to go through one server."
Yes, that is why they *can* do that. But, what's wrong with all the
requests going through one server?
Let me ask you a question: Why do Oracle, SQL Server, DB2 and Interbase all
use a Client/Server model for their databases?
Joe Real
Rather than arguing about the technical details of why is one better than another,
help me how to go about efficiently using the NexusDB, let us say that I am going to
use ASTA middle-tier servers for NexusDB.
Our current design is that we are embedding database engines into the ASTA Servers,
in which case NexusDB has a small footprint to be embedded as such. We have been
running several instances of our ASTA Servers as there are processors and we create
processor affininites for each instance. Thus we literally have the multiple numbers
of NexusDB engines as there are ASTA servers running against one physical database
location only during any database write operation, but we point the ASTA servers on
several mirrored database engines for read only database operations optimized with
all necessary indices. The indices are maintained and created during the mirroring
event all brokered by ASTA. We can support more than 1500 users at a time with this
setup with no hiccups on our system. Will this be possible with NexusDB? To the end
user, as they go through our load balancing server, there is only one server.
Rob
> both.
>
> Disagree. I don't see this capability as an advantage, in fact I see it
as
> a disadvantage. Again, this depends on what you value.
It's a HUGE advantage. That's why I'm switching from FF to DBIsam. I've
lost a lot of customers over the inability to put the files on a novell
server and/or use a simple file share system for small networks. The server
is way overkill and too complicated for most my customers with small nets.
That said, Nexus is looking good and has some great people working on it.
Herbert Sitz
news:3f271dfa$1...@newsgroups.borland.com...
Are you guys talking about the same thing? I think everyone would agree
that, all other things being equal, you'd much rather have an engine that
can work as F/S or C/S, depending on the setup you want. F/S is easier to
embed, deploy, and support. C/S is faster and more robust for situations
where lots of users will be accessing.
NexusDB does only C/S. DBISAM does both F/S and C/S. Advantage to DBISAM.
The disadvantage to DBISAM's setup is that it's configured to use O/S
locking so that both F/S and C/S engines can access the same database at the
same time. This prevents the C/S engine from taking full advantage of more
sophisticated locking schemes and better caching. DBISAM does have a
setting so that you can run the C/S version with exclusive access, but (as
Brian says) this isn't as much of an improvement as it sounds like since the
locking and caching are still no more sophisticated than with the F/S
version. Advantage to NexusDB.
Depending on what your needs are, one or the other of the these advantages
may carry more weight for you. Is that an accurate description of the
differences we're talking about?
-- Herb
Brian Moelk
> lost a lot of customers over the inability to put the files on a novell
> server and/or use a simple file share system for small networks. The
server
> is way overkill and too complicated for most my customers with small nets.
For small networks in which you need to absolutely minimize complexity,
DBISAM's architecture is more advantageous. But there are certainly
drawbacks to this approach mainly dealing with scalability and things of
that nature.
IMO, it's not that much more complex to have a server running a FF/NexusDB
server/service somewhere on the network. It's exactly like running an email
server and many small businesses have one of those processes running. FWIW,
DBISAM's C/S engine is exactly the same, it requires a server process.
Therefore the advantage is there for DBISAM, but IMO it is an exaggeration
to say that it is a "HUGE advantage", especially in all caps. <g>
> That said, Nexus is looking good and has some great people working on it.
Indeed. Again, it really depends on what you value.
Brian Moelk
another,
> help me how to go about efficiently using the NexusDB, let us say that I
am going to
> use ASTA middle-tier servers for NexusDB.
I am not an ASTA user and therefore can only provide general advice.
> Our current design is that we are embedding database engines into the ASTA
Servers,
> in which case NexusDB has a small footprint to be embedded as such. We
have been
> running several instances of our ASTA Servers as there are processors and
we create
> processor affininites for each instance.
Why do you set processor affinity? Does ASTA not thread well or use SMP to
the fullest extent? Or does DBISAM not?
> Thus we literally have the multiple numbers
> of NexusDB engines as there are ASTA servers running against one physical
database
> location only during any database write operation, but we point the ASTA
servers on
> several mirrored database engines for read only database operations
optimized with
> all necessary indices.
This architecture will not work with NexusDB. From my understanding of what
app servers in n-tier architectures are supposed to do is to hook into a
centralized DB server (or cluster) and cache for reading which is
essentially what you are doing except you have multiple DB engines pointed
to centralized files for writing.
In this case, I would change the architecture to cache in the app server,
write a NexusDB monitor/extender to coordinate and notify the app servers
when their cache is invalidated and use a single NexusDB database server.
> The indices are maintained and created during the mirroring
> event all brokered by ASTA.
I'm not sure why this is necessary, but that's fine, it seems to work well
for you.
> We can support more than 1500 users at a time with this
> setup with no hiccups on our system.
Sounds like you've got a winner...no need to change anything. I'd stay with
DBISAM if it works for you.
> Will this be possible with NexusDB? To the end
> user, as they go through our load balancing server, there is only one
server.
Anything is possible. <g> It just depends on how much time, effort and
resources you put into things; sounds like you've spent a lot of time on
this project to make it work.
Trev
1. We use DBISAM because of the scaleability offered by the F/S-C/S systems.
We can write a stand alone demo app that is really easy to install in F/S
mode. Then without altering files or anything else, our clients can upgrade
to the C/S version. I also like the DBSys utility which means that we can be
maintaining the data in F/S mode while users are accessing the data in C/S.
2. The DBISAM locking method is subject to OS quirks. This is going back a
while, but, we lost a HUGE amount of time trying to understand why certain
network configurations gave different results in terms of speed. This would
not be the case with NexusDB.
Trev
"Herbert Sitz" <hs...@nwlink.com> wrote in message
news:3f272234$1...@newsgroups.borland.com...
Trev
all
use a Client/Server model for their databases?>>
Now you're talking!
I am not neccessarily looking at NexusDB as an alternative to DBISAM. I am
more interested in how NexusDB compares with the above 'industrial strength'
engines.
Presently, if we need to go bigger than DBISAM, we use Firebird for two main
reasons:
1. License fees
2. We can embed it ito our kbmMW servers for easy installation/maint.
I'm very impressed with what I've read so far about NexusDB. Guess it's time
to download a copy a play a while :)
Trev
"Brian Moelk" <bmo...@NObrainendeavorSPAM.FORcomME> wrote in message
news:3f26da90$1...@newsgroups.borland.com...
Robert Kaplan
> may carry more weight for you. Is that an accurate description of the
> differences we're talking about?
>
There is another factor, application bloat. Since DBISAM does not know if
the program will be run in C/S of FS mode, the same EXE needs to have the
support for both. Of course, that gives you the choice of switching the same
identical program form FS to C/S via a couple of changes in an INI file.
I think this is the old argument of specialization vs generalization (if
those are the right terms). If you specialize, other things being equal you
will always be more efficient. By supporting with the same components both
FS and C/S, DBISAM can't be as efficient and sophisticated as a pure C/S
engine, and DBISAM will tend to lose in C/S vs C/S evaluations.
Robert (a happy DBISAM user, btw)
Brian Moelk
> the program will be run in C/S of FS mode, the same EXE needs to have the
> support for both. Of course, that gives you the choice of switching the
same
> identical program form FS to C/S via a couple of changes in an INI file.
There is no bloat because DBISAM's C/S is only C/S because it supports the
process separation of Client and Server. The internal concurrency
architecture does not change simply because you make it SINGLE_USER and lock
the data files exclusively or expose a network/remoting layer.
> I think this is the old argument of specialization vs generalization (if
> those are the right terms). If you specialize, other things being equal
you
> will always be more efficient. By supporting with the same components both
> FS and C/S, DBISAM can't be as efficient and sophisticated as a pure C/S
> engine, and DBISAM will tend to lose in C/S vs C/S evaluations.
That's right. Internally DBISAM does not have the concurrency architecture
that pure (or in your words "specialized") C/S architectures employ and
therefore reaps no performance, scalability or robustness benefits.
From my understanding, which could be incorrect, DBISAM C/S is a network
enabled wrapper for a F/S database.
Robert Kaplan
"Brian Moelk" <bmo...@NObrainendeavorSPAM.FORcomME> wrote in message
> > There is another factor, application bloat. Since DBISAM does not know
if
> > the program will be run in C/S of FS mode, the same EXE needs to have
the
> > support for both. Of course, that gives you the choice of switching the
> same
> > identical program form FS to C/S via a couple of changes in an INI file.
>
> There is no bloat because DBISAM's C/S is only C/S because it supports the
> process separation of Client and Server. The internal concurrency
It seems to me that even if I ONLY use C/S in DBISAM, my program still
contains (redundantly) the code to do all the low level file access datail
processing being done by the server. From figuring out based on the index
what disk segments to read, parsing disk reads into fields, to determining
if a record has changed before I post, etc etc etc. All of that exists in my
program just in case I decide to use FS. You would not need that type of
funcionality in a pure client.
>
> That's right. Internally DBISAM does not have the concurrency
architecture
> that pure (or in your words "specialized") C/S architectures employ and
> therefore reaps no performance, scalability or robustness benefits.
>
It's not just concurrency. A "pure" C/S program can handle transactions much
better, can use memory more efficiently, etc. If the server is the only
program accessing the database, you have a lot more freedom in the
implementation techniques.
Robert
Brian Moelk
> that, all other things being equal, you'd much rather have an engine that
> can work as F/S or C/S, depending on the setup you want.
IMO, all things cannot be equal in that F/S and C/S internal architectures
are fundamentally different.
The differences are much more than just acquiring the data files shared or
exclusively and more than having separate client and server processes; it
has to do with how concurrency is implemented and who enforces those
constraints. The choices made have far reaching repercussions throughout
the implementation of the DB engine and thus all things are not equal.
> F/S is easier to
> embed, deploy, and support. C/S is faster and more robust for situations
> where lots of users will be accessing.
Embedding is slightly easier with a F/S database. In this specific case, it
is easy to embed NexusDB, but easier to embed DBISAM if you measure ease by
number of components.
Deployment is easier only in one case: sharing the data files with multiple
users. In this case, a F/S database is easier in that there is no central
server process.
Support is a very application specific issue that I do not think one can say
is easier with a F/S or a C/S database.
FMPOV, the ease of use arguments for a F/S database do not outweigh the
arguments for a C/S database. But again, this comes down to what you value.
> NexusDB does only C/S. DBISAM does both F/S and C/S. Advantage to
DBISAM.
DBISAM has a C/S process/service, which from my understanding, means that it
exposes an remote/networked API that DBISAM clients can retrieve data, store
data, delete data, etc. This does not mean that DBISAM has the concurrency
architecture that C/S-only databases have.
IMO, it is this internal concurrency architecture that makes the difference
in performance and robustness between C/S and F/S databases.
Therefore even using DBISAM C/S, where you have Clients and Servers, it
still functions as a F/S database internally.
Bottom line, you can't be all things to all people.
If you want a F/S database, DBISAM is a good choice. If you want a C/S
database, Nexus is a good choice. Both are flexible in their deployment
capabilities in that you can embed them or use them as separate server
processes. The fundamental differences lie in their architectural
underpinnings.
Eryk Bottomley [Elevate Software]
> There is no bloat because DBISAM's C/S is only C/S because it supports the
> process separation of Client and Server. The internal concurrency
> architecture does not change simply because you make it SINGLE_USER and
lock
> the data files exclusively or expose a network/remoting layer.
<snip>
> From my understanding, which could be incorrect, DBISAM C/S is a network
> enabled wrapper for a F/S database.
Concurrency isn't the issue. What Robert means is DBISAM links a complete
SQL engine (for example) into the client as well as all the other stuff
needed for direct file operations as well as the transport layer needed to
communicate with the server and execute SQL etc. on the server side. If
DBISAM were as you suggest it would be executing SQL and filters etc. on the
client side only and just using the server to marshal raw data pages back
and forth (which would obviously be silly).
Eryk
---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.498 / Virus Database: 297 - Release Date: 08/07/2003
Brian Moelk
> contains (redundantly) the code to do all the low level file access datail
> processing being done by the server. From figuring out based on the index
> what disk segments to read, parsing disk reads into fields, to determining
> if a record has changed before I post, etc etc etc. All of that exists in
my
> program just in case I decide to use FS. You would not need that type of
> funcionality in a pure client.
Sorry...yes that makes sense. There is bloat in this case.
> It's not just concurrency. A "pure" C/S program can handle transactions
much
> better, can use memory more efficiently, etc. If the server is the only
> program accessing the database, you have a lot more freedom in the
> implementation techniques.
Agreed.
Brian Moelk
> SQL engine (for example) into the client as well as all the other stuff
> needed for direct file operations as well as the transport layer needed to
> communicate with the server and execute SQL etc. on the server side.
Ahh...ok. Understood.
> If
> DBISAM were as you suggest it would be executing SQL and filters etc. on
the
> client side only and just using the server to marshal raw data pages back
> and forth (which would obviously be silly).
Sorry, I was a bit unclear, that's not what I meant. There is an
intelligent network/remoting layer in DBISAM's C/S setup, however the core
engine is the same as what is used if you embedded it.
Joe Real
My main point is that there are distinct advantages to embedding a database engine
unto several server applications such as ASTA and IntraWeb Servers and have several
instances running, all of them accessing the same physical database on the same
multi-processor machine with a lot of memory for caching. The responses of your
IntraWeb and ASTA Servers are almost instantaneous for query operations, the fact
that all of these applications share the same physical tables which are cached in
the same memory in the same machine. We have not achieved this kind of instantaneous
performance with having our middleware and our IntraWeb server configured as pure
clients of a true C/S system, simply because of of the overhead of our servers
acting as pure clients of a true C/S database engine. We have tried using MS-SQL,
Oracle client, DB2-Client, Interbase client via our IntraWeb servers and ASTA
servers, there is always a very annoying lag when pumping out data. On the other
hand, embedding the engine itself to the IntraWeb and ASTA servers have always
produced instantaneous responses. The only way to have this kind of performance is
to have mutli-application shared access to the database on the same physical
machine. Make that machine robust in hardware and your IntraWeb and Asta servers
coded properly and you will have a very stable system. In one of our clients, we
never had any downtime except for maintenance and hardware upgrades, and that has
been two years since. So there really is a tremendous advantage for us to go the
route where our embedded database engine can file share the files on the same
machine. Thus this is the hybrid solution that we are truly after and we have proven
it to work in our case.
Both ASTA and DBISAM are excellent when used in threads. ASTA has several built-in
threading features. But as for utilizing various processors when spawning several
threads, that is not automatic with DBISAM, you will have to code it yourself, and
there are plans for building this into the DBISAM engine.
Assigning affinities of various applications, ie, including several instances of
applications, to various processors is our NON-programming approach to increasing
our overall throughput, balancing load and fully utilizing all of the processors
power within a single machine. Yes it would be nice if we can code our servers to do
this, or have the embedded database engine do this. For the moment, this is the
cheapest way for us to do it. Besides, we still have another layer of scaling, and
that is we use our ASTA acnhor servers to span several physical machines, so far, a
single machine is doing its job well. When we will have more clients, then we can
test more.
So perhaps the NexusDB will be tested with respect to pure C/S engines, in which
case will have very limited application for us at the moment.
Tim Young
<< It seems to me that even if I ONLY use C/S in DBISAM, my program still
contains (redundantly) the code to do all the low level file access datail
processing being done by the server. From figuring out based on the index
what disk segments to read, parsing disk reads into fields, to determining
if a record has changed before I post, etc etc etc. All of that exists in my
program just in case I decide to use FS. You would not need that type of
funcionality in a pure client. >>
Sure, but you wouldn't have the ability to have local temporary disk or
in-memory table processing either, which is a necessity in all but the
thinnest of clients.
<< It's not just concurrency. A "pure" C/S program can handle transactions
much better, can use memory more efficiently, etc. If the server is the only
program accessing the database, you have a lot more freedom in the
implementation techniques. >>
Absolutely. And DBISAM as an engine was never "forked" in this direction
specifically because it wasn't wise to do so given its architectural roots.
As indicated earlier, we played around with some things like the SINGLE_USER
define but decided that such deviations would best be left for something
else.
Brian Moelk
> a database engine
There are some advantages, however they are not HUGE in my opinion.
As I've said before, if you had sufficient caching technology in your ASTA
or IntraWeb servers my guess would be that you could acheive the same
performance. That's the real "problem" that running multiple instances of
DBISAM as you do "solves".
> We have not achieved this kind of instantaneous
> performance with having our middleware and our IntraWeb server
> configured as pure clients of a true C/S system, simply because of of
> the overhead of our servers acting as pure clients of a true C/S database
> engine. We have tried using MS-SQL,
> Oracle client, DB2-Client, Interbase client via our IntraWeb servers
> and ASTA servers, there is always a very annoying lag when pumping out
data.
Did you try caching anything in the middle layer?
>On the other hand, embedding the engine itself to the IntraWeb and
> ASTA servers have always produced instantaneous responses.
An embedded database should outperform a database engine that runs in a
separate process yes.
> The only
> way to have this kind of performance is
> to have mutli-application shared access to the database on the same
> physical machine.
I disagree that that is the *only* way to acheive that performance. Perhaps
that is the only way you have acheived that level of performance.
> Make that machine robust in hardware and your IntraWeb and Asta
> servers coded properly and you will have a very stable system. In one of
our
> clients, we never had any downtime except for maintenance and hardware
> upgrades, and that has been two years since.
Like I said before, DBISAM seems to be working for you and you have put
forth a lot of effort to make sure your application is stable. I do believe
that the same level of performance and stability can be achieved using a C/S
database. We can agree to disagree.
>So there really is a tremendous advantage for us to go the
> route where our embedded database engine can file share the files on the
> same machine. Thus this is the hybrid solution that we are truly after and
we
> have proven it to work in our case.
Yes, you have mentioned that before.
> Both ASTA and DBISAM are excellent when used in threads. ASTA has
> several built-in threading features. But as for utilizing various
processors
> when spawning several threads, that is not automatic with DBISAM, you will
> have to code it yourself, and there are plans for building this into the
> DBISAM engine.
I don't understand how DBISAM can be "excellent when used in threads" yet
still have problems utilizing SMP systems? What additional code would you
have to write?
> So perhaps the NexusDB will be tested with respect to pure C/S engines, in
> which case will have very limited application for us at the moment.
In your current architecture it won't work. You'll need to provide some
kind of caching mechanism in the middle tier to enable that throughput.
David Marcus
> The really, really nice thing about NexusDB is that it also supports Nested
> Transactions which allows you to write transactional code in a very natural
> and clean way.
Would you explain? Or, provide an example?
--
David Marcus
Brian Moelk
Nested
> > Transactions which allows you to write transactional code in a very
natural
> > and clean way.
>
> Would you explain? Or, provide an example?
Sure.
db.StartTransaction;
try
//update some data here
db.StartTransaction;
try
//update more data here
db.Commit;
except
db.Rollback;
end;
//here the inner transaction could commit or rollback
db.Commit; { the outer transaction will commit; the result of the inner
transaction is preserved. }
except
db.Rollback; { the outer transaction will rollback the inner transaction
even if the inner one has committed. }
end;
Imagine using nested transactions within a call stack. IMO, it's very easy
and natural to use transactions in this way.
Chad Z. Hower aka Kudzu
> can be accessed by multiple engines at one time. That is, even if you
> have a DBISAM Server for a database, the files for that database can be
> accessed directly by a client that's using the fileserver engine. This
> was a design decision made by the DBISAM developers, as I understand it.
This isnt a limitation - this is a major feature. We run apps that locally us
file access, but sometimes we connect remotely for maintainance.
> The end result is that to avoid possible deadlocks during an explicit
> DBISAM transaction all tables in a database are locked by DBISAM. Thus,
Why would DBISAM ever lock all part of every table?
--
Chad Z. Hower (a.k.a. Kudzu) - http://www.hower.org/Kudzu/
"Programming is an art form that fights back"
ELKNews - Get your free copy at http://www.atozedsoftware.com
Stefan Paege
To quote the changes.txt published for the DBISAM 4 Beta:
--- snip----------------
[...] Previously [referring to v3] table locking was done at the individual
table level, so if you started a transaction on a database with 50 physical
tables opened for that database, DBISAM would have to place a transaction
lock on all 50 open tables before starting the transaction. It would also
have to subsequently write lock them during a commit and then unlock
everything for each table after the transaction was committed or rolled
back.
--- snip----------------
Regards
--
elektronik-labor Carls GmbH & Co. KG
Stefan Paege
Martijn Tonies
"Brian Moelk" <bmo...@NObrainendeavorSPAM.FORcomME> wrote in message
> > Interesting. What locking method does Nexus use and how, in your opinion,
> is
> > it better?
>
> Nexus is a true C/S engine that uses internal locks (critical sections and
> other OS provided synchronization primitives). The engine itself acquires
> exclusive access to the physical file and therefore the file isn't shared.
>
> You can read about it at: http://www.nexusdb.com/ under Features In
> Detail->Client/Server
>
> Related to locking strategies, one of the unique things that Nexus offers is
> snapshot transactions. These transactions require no locks but provide a
> consistent (serializable) view of the database. The end result is that
> readers using snapshot transactions do not block writers.
Right, but a snapshot transaction - by itself - isn't unique to NexusDB, right?
--
With regards,
Martijn Tonies
Database Workbench - the developer tool for InterBase & Firebird
Upscene Productions
http://www.upscene.com
Brian Moelk
right?
Depends on what you define as a snapshot transaction. I don't know of
another database that does what Nexus calls a snapshot transaction in the
same way. I admit not knowing a whole lot about the internals of various
DB's. However there are two general methodologies: locks and versioning.
In reference to NexusDB's snapshot transactions; it is an
algorithm/methodology to acheive certain transactional isolation levels
while maximizing throughput/minimizing locks. In some ways it is closer to
versioning, but it is still different in its implementation. Certainly
other databases could implement a similar strategy, so in that sense it is
not unique.
In the context of databases written in Delphi it is very unique. In the
context of the database market, FMPOV, it is still unique.
David Marcus
> > Would you explain? Or, provide an example?
>
> Sure.
I meant an example of why I would want to do that. I've never felt the
need.
--
David Marcus
Brian Moelk
> need.
Oh...ok. I'll make up something based on a bank; I hope this example will
illustrate the usefulness.
Say you wanted to transfer money from one account to another and wanted to
create a record of that transfer. If the transfer fails, you deduct a small
penalty from the senders account in addition to the originating fee.
so something like this:
db.StartTransaction;
try
TransferTable.append;
TransferTableTransferFee.Value = 5; //originating fee
TransferTableStatus.Value = 'START';
db.StartTransaction;
try
//transfer the money from the accounts here...
db.Commit;
TransferTableStatus.Value = 'OK';
except
db.RollBack;
TransferTableStatus.Value = 'FAIL';
TransferTableTransferFee.Value = TransferTableTransferFee.Value +
20; //penalty
end;
TransferTable.post;
//deduct transfer fee from the sender's account
db.Commit;
except
db.Rollback;
end;
Without nested transactions, if the transfer fails for some reason, rolling
back the partial changes that occured within the transfer process gets
hairy.
Also, nested transactions allow you to organize your code naturally. You
could take the transfer process and put it in a separate method which would
include transactional logic and re-raise any exception caught. This way,
you could use that transfer method without any record of that transfer being
created (I suppose for money laundering..) or completely different transfer
policies based on different account types, etc. all without any code changes
for transactional logic.
Thomas Miller
FB 1.5. Not so unique.
Brian Moelk wrote:
--
Thomas Miller
Delphi Client/Server Certified Developer
BSS Accounting & Distribution Software
BSS Enterprise Accounting FrameWork
http://www.bss-software.com
http://sourceforge.net/projects/dbexpressplus
Brian Moelk
> FB 1.5. Not so unique.
No, it's not the same. Oracle may call them "snapshot transactions", but
that doesn't mean the same thing as what NexusDB calls "snapshot
transactions".
My understanding of a SELECT FOR UPDATE is that it acquires exclusive row
locks so that you can ensure that when you update those rows it will go
through. This statement is actually a pessimistic locking mechanism based
on the assumption that you will update those rows you have selected. If I
am mistaken please explain.
AFAIK, in the case of Oracle, reads will read the rollback segment. Oracle
locks these rows from being included in another SELECT FOR UPDATE statement.
I'm not sure what Firebird does, but I assume that the overall functionality
is similar in that it prevents these rows from being written. I presume
that reads will read the version of the relevant rows before the SELECT FOR
UPDATE was issued; again enforcing a pessimistic updating mechanism.
AFAIK, this is not the same as a snapshot transaction as far as NexusDB is
concerned. Snapshot transactions are read-only transactions. The whole
idea behind them is to provide a consistent view of the DB so they do not
block write transactions. The internal implementation is what I was
refering to as far as how snapshot transactions are actually implemented and
how the goal of "readers don't block writers and writers don't block
readers" is accomplished.
Thomas Miller
>
> AFAIK, this is not the same as a snapshot transaction as far as NexusDB is
> concerned. Snapshot transactions are read-only transactions. The whole
> idea behind them is to provide a consistent view of the DB so they do not
> block write transactions. The internal implementation is what I was
> refering to as far as how snapshot transactions are actually implemented and
> how the goal of "readers don't block writers and writers don't block
> readers" is accomplished.
>
Then I fail to see the significance of this feature. Oracle and FB
already do this without any coding. Are you saying that you can
start a snap shoot transaction and it will do everthing in
the transaction as the data was at that time? So if I do 20
different SELECTS over a 1 minute time, the last SELECT will
read the data as it was 1 minute ago, regardless of the COMMITS
that may have happened during the running of SELECT #10. What
is the benefit of this?
no...@noone.com
"Brian Moelk" <bmo...@NObrainendeavorSPAM.FORcomME> wrote in message
>
> concerned. Snapshot transactions are read-only transactions. The whole
> idea behind them is to provide a consistent view of the DB so they do not
> block write transactions.
Maybe you need to explain this a little better. I assure you that FB
already does this without any effort. Your definition simply sounds like a
"snapshot" on any SQL server.
> The internal implementation is what I was
> refering to as far as how snapshot transactions are actually implemented
and
> how the goal of "readers don't block writers and writers don't block
> readers" is accomplished.
It seems to me that all SQL servers have a "snopshot" transaction defined in
the same way. Updates on FB are not visible until commit, but the original
record is not locked for reads, only updates. Most (if not all) SQL servers
work in this manner. True pessimistic locking is all but gone.
Brian Moelk
It's as much of a feature as row-level locking or multi-versioning is.
> Oracle and FB
> already do this without any coding. Are you saying that you can
> start a snap shoot transaction and it will do everthing in
> the transaction as the data was at that time?
The technical answer to your question is "yes" however remember that
SnapShot transactions, as far as NexusDB is concerned, are read-only
transactions.
> So if I do 20
> different SELECTS over a 1 minute time, the last SELECT will
> read the data as it was 1 minute ago, regardless of the COMMITS
> that may have happened during the running of SELECT #10. What
> is the benefit of this?
If all of those 20 selects are in the same snapshot transaction, yes. There
is huge benefit. An example: online backup. Essentially, snapshot
transactions, as defined by NexusDB, provides serializable transaction
isolation for reads while writers are not blocked.
Brian Moelk
is
> > concerned. Snapshot transactions are read-only transactions. The whole
> > idea behind them is to provide a consistent view of the DB so they do
not
> > block write transactions.
>
> Maybe you need to explain this a little better. I assure you that FB
> already does this without any effort.
FB does accomplish the same goal, however the way in which it accomplishes
it is very different internally than how NexusDB accomplishes it. The
algorithms are different. That is my point.
NexusDB requires no extra effort on the part of the end-user either: SQL
Select statements use Snapshot Transactions.
> Your definition simply sounds like a
> "snapshot" on any SQL server.
It's not what it is, it's how it's done. You can get a snapshot of a
database that only has a single exclusive table lock; just lock the table
and read your records. The issue is how it's done.
> It seems to me that all SQL servers have a "snopshot" transaction defined
in
> the same way.
Not in this context; for NexusDB it is a specific form of transaction that
has a unique underlying algortithm/technique.
> Updates on FB are not visible until commit, but the original
> record is not locked for reads, only updates.
That's right. Again, the ends are the same; the means are different.
> Most (if not all) SQL servers
> work in this manner. True pessimistic locking is all but gone.
That's not true. SQL Servers that issue read locks block writes.
Eric Hill
Oracle
> locks these rows from being included in another SELECT FOR UPDATE
statement.
Just FYI:
Oracle locks rows either updated or selected for an update so successive
updates on the same rows will lock until the first transaction has been
committed or rolled back. Any data changed during a transaction is moved
to an undo segment so other transactions can query the pre-transaction
state of the update. The new data is written into the tables (again, the
undo segments have the old data for queries) with a seqeuence number. When
an update is committed, the data in the tables (already updated) becomes
current (global sequence is incremented), and the undo segment is released
as free space as it is usually unneeded anymore.
What this boils down to is updates allow one session to update one row at a
time, but many readers (queries) can happen over the same set of rows
without data issues. "old" data is fetched from the undo segments as
needed in the query - this is handled by the engine transparently.
Not that any of this matters to the discussion at hand <g> but it's good to
know...
Eric
David Marcus
> Say you wanted to transfer money from one account to another and wanted to
> create a record of that transfer. If the transfer fails, you deduct a small
> penalty from the senders account in addition to the originating fee.
Got it. Thanks.
--
David Marcus