kbmMemTable vs. TClientDataset

[LCI]

I've been playing around with kbmMemTable. It sorts records much faster than
TClientDataset.
How do they compare in memory usage storing the same data?

Thks.

[Alessandro Federici - RemObjects Software]

"LCI" <i...@lcitoys.com> wrote in message
news:3f2689bc$1...@newsgroups.borland.com...

From what I hear the KBM table also supports compression so it should also
manage memory much better in top of being much faster.

--
Regards,
Alessandro Federici

RemObjects Software, Inc.
http://www.remobjects.com

Best Communications Tool In 2003 Delphi Informant Readers Choice Awards

[Sterling Bates]

"LCI" <i...@lcitoys.com> wrote in message
news:3f2689bc$1...@newsgroups.borland.com...

In my experience, anything is faster than Borland's data components. I had
an ISAPI application that relied heavily on database interaction, and
realized through perf tuning that Borland's stuff dragged the app to a
crawl. Taking them out made the app 30x faster (literally).

My subjective, probably atypical, but hands-on experience :)

Sterling

[Stewart Bourke]

I am very interested in these comments.

Which specific components are you talkiing about - i.e. which components did
you remove, and what did you replace them with?

Thanks,

Stewart Bourke

"Sterling Bates" <ster...@bigbangmapsonco.com> wrote in message
news:3f2695d4$1...@newsgroups.borland.com...

[John Wester [Group W]]

In article <3f26...@newsgroups.borland.com>, stewart...@infosol.ie
says...

>I am very interested in these comments.
>
>Which specific components are you talkiing about - i.e. which components did
>you remove, and what did you replace them with?
>
>Thanks,
>

For me, it was using:
- IBObjects for Interbase/Firebird access,
- Direct Oracle Access for Oracle,
- BetterADO for ADO access
- kbmMemTable for any temporary in memory data storage
- DeCAL (http://sourceforge.net/projects/decal/) for containers

These are a standard part of my palette.
--
John
Life is complex. It has real and imaginary components.

[Captain Jake]

In borland.public.delphi.thirdpartytools.general, Alessandro Federici -
RemObjects Software <alef@[nospam]remobjects.com> wrote in message
<3f268df2$1...@newsgroups.borland.com>...

>Best Communications Tool In 2003 Delphi Informant Readers Choice Awards

Didn't take you very long to add that to your sig!

(Congratulations, by the way.)

--
***Posted by Jake's Custom Newsgroup Reader***

Posted using Jake's Super Newsreader 0.9.2.953

[Alessandro Federici - RemObjects Software]

"Captain Jake" <johnjac76[nospam]@comcast.net> wrote in message
news:3f26...@newsgroups.borland.com...

> Didn't take you very long to add that to your sig!

:-)))

> (Congratulations, by the way.)

Thanks!

[Sterling Bates]

"Stewart Bourke" <stewart...@infosol.ie> wrote in message
news:3f26...@newsgroups.borland.com...

> I am very interested in these comments.
>
> Which specific components are you talkiing about - i.e. which components
did
> you remove, and what did you replace them with?
>

Basically I didn't replace them with components :) From TADOConnection and
TADORecordset I moved to Variants and direct instantiations of
ADO.Connection and ADO.Recordset. It requires a little more debugging, but
in the end it was well worth it.

Sterling

[LCI]

Sterling,

Thanks for your comment. As traffic increases, my ISAPI app is slowing down
to a crawl. I suspect that TClientDataset is one of the main reasons. From
reading in news archives this clearly is an old issue that Borland for some
reason refuses to address.

"Sterling Bates" <ster...@bigbangmapsonco.com> wrote in message
news:3f2695d4$1...@newsgroups.borland.com...
>

[Sterling Bates]

"Dan Palley" <d...@trams.com> wrote in message
news:3f26ba40$1...@newsgroups.borland.com...
> Can you be more specific as to what performance issue exists in
> TClientDataset that Borland refuses to address? Is there a QC entry you
can
> reference?
>
> We use CDS a lot and periodically have instances with tens of thousands of
> records and performance is excellent.

The problem isn't the number of records, it's the processor time required to
execute & access a single query. When multiple users are accessing the DB
at the same time, with additional processing done (in this case by an
ISAPI), and when every millisecond counts, Borland's dataset components are
simply too fat and heavy.

They do far more work per call than is ever needed by my applications (I
would argue that the same is true for many applications). They're great for
beginners or for desktop apps where # of users is 1, or where usability is
the key.

To give you an idea (this is from memory): My average database call would
take 30 ms, including my own processing of the data. Once I removed the
Borland components and replaced them with Variants and direct instantiations
of the ADO COM objects, my queries (including processing the data) took less
than 1 ms. Suddenly my script pages were compiling in 10 ms instead of 1
second with numerous queries.

Sterling

[Dan Palley]

Can you be more specific as to what performance issue exists in
TClientDataset that Borland refuses to address? Is there a QC entry you can
reference?

We use CDS a lot and periodically have instances with tens of thousands of
records and performance is excellent.

Dan

"LCI" <i...@lcitoys.com> wrote in message

news:3f26...@newsgroups.borland.com...

[LCI]

I don't know about QC entry, but plenty of newsgroups posts describing
performance issues, such as slow sorting, memory leaks, memory usage, etc.

"Dan Palley" <d...@trams.com> wrote in message
news:3f26ba40$1...@newsgroups.borland.com...

[Dmitry Arefiev [gs-soft.ru]]

Hello !

A bit yearlier, in this conference was a thread
"ANN: kbmMemTable v. 4.00 released!....". It has
a link to performance tests. See it for details - when
and how much CDS is slower, than anything other <g>

PS: Borland made good design, but not optimal
implementation. Which was never optimized and,
IMHO, will never ...

Regards,
Dmitry
--
===========================================
Dmitry L. Arefiev, director of gs-soft.ru ltd.
Solutions for successful companies

Author of NCOCI8 - Freeware Delphi to Oracle8i direct access

ICQ: 50741007
EMail: dare...@gs-soft.ru
Company: http://www.gs-soft.ru
NCOCI8: http://www.da-soft.com

[Ed Dressel]

> We use CDS a lot and periodically have instances with tens of thousands of
> records and performance is excellent.

could be--but kbm and other in-mem datasets will show remarkable increases
in speed over CDS. Just give them a spin--kbm is free.

Ed Dressel

[Dan Palley]

Ok, I've reported two bugs with CDS to QC and they're both fixed. If you're
having specific issues with CDS, I'd suggest opening a QC entry. They're
working great for me, though.

Dan

"LCI" <i...@lcitoys.com> wrote in message

news:3f26c4cd$1...@newsgroups.borland.com...

[Dan Palley]

"Ed Dressel" <none> wrote in message
news:3f26c810$1...@newsgroups.borland.com...

Is kbm a drop-in replacement for CDS in a multi-tier environment and does it
have all CDS functionality like nested datasets? Like I said, I'm not
having any CDS-related performance problems at the moment, but if it was
simply a matter of substituting a kbm for CDS, then I'd be willing to give
it a try.

Dan

[Dan Palley]

"Sterling Bates" <ster...@bigbangmapsonco.com> wrote in message

news:3f26bdff$1...@newsgroups.borland.com...

> "Dan Palley" <d...@trams.com> wrote in message
> news:3f26ba40$1...@newsgroups.borland.com...
> > Can you be more specific as to what performance issue exists in
> > TClientDataset that Borland refuses to address? Is there a QC entry you
> can
> > reference?
> >

> The problem isn't the number of records, it's the processor time required
to
> execute & access a single query. When multiple users are accessing the DB
> at the same time, with additional processing done (in this case by an
> ISAPI), and when every millisecond counts, Borland's dataset components
are
> simply too fat and heavy.

Ok, it sounds like your issue is with the DSP mechanism that is used in
conjunction with a CDS. I assume you had a lot more work to do than simply
replace CDS with an alternative. You also had to redesign your middle tier.

Dan

[Sterling Bates]

> Ok, it sounds like your issue is with the DSP mechanism that is used in
> conjunction with a CDS. I assume you had a lot more work to do than
simply
> replace CDS with an alternative. You also had to redesign your middle
tier.

Not at all... Since most everything was object-oriented I only had to
modify part of the data query object.

While the testing took some time, the code change itself was pretty much
that simple.

[Herbert Sitz]

"Dan Palley" <d...@trams.com> wrote in message

news:3f26ceb8$1...@newsgroups.borland.com...

> Is kbm a drop-in replacement for CDS in a multi-tier environment and does
it
> have all CDS functionality like nested datasets? Like I said, I'm not
> having any CDS-related performance problems at the moment, but if it was
> simply a matter of substituting a kbm for CDS, then I'd be willing to give
> it a try.
>
> Dan
>

I believe it can be a drop-in replacement depending on what you're doing
with the CDS. Unfortunately, I'm pretty sure kbmMemTable doesn't support
nested datasets.

-- Herbert Sitz

[Jim McKay]

Dan Palley wrote:
> Is kbm a drop-in replacement for CDS in a multi-tier environment and
> does it have all CDS functionality like nested datasets?

It achieves same functionality using one of several different means...
slightly more work, but far from onerous. And it's effecient.

Kim has newsserver: news.components4developers.com: ask your
questions on kbmMemTable group... it's *very* well supported.

--
Regards:
Jim McKay

"Doing my part to piss off the right wing scoundrels!"

"Who are you going to believe... us or your own two eyes?"
--Groucho Marx

Posted with: XanaNews 1.15.2.7

[Eivind Bakkestuen]

The CDS is slower than the others on large datasets (1.000.000 records +),
we're talking orders of magnitude here.

Even so, the test alluded to isn't very well made. For instance, no matter
how many records you test it with, the locates only locates within the first
1.000 records inserted, and it spends more time updating the screen than
doing database operations. <g>

--

Eivind

"Dmitry Arefiev [gs-soft.ru]" <dare...@gs-soft.ru> wrote in message

[Dmitry Arefiev]

> The CDS is slower than the others on large datasets (1.000.000 records +),
> we're talking orders of magnitude here.

Could disagree. CDS is slower on any amount fo data.
On small amount of records difference is not so big. But
on big amounts - ~10 times slower.

> Even so, the test alluded to isn't very well made. For instance, no matter
> how many records you test it with, the locates only locates within the first
> 1.000 records inserted, and it spends more time updating the screen than
> doing database operations. <g>

Some time ago i made own performance tests, and result
was the same <g> CDS is simply slooooow.

--

[Thomas Miller]

Unfortunately, that is a big issue for us. Does it also support
cloned datasets? We use both extensively.

Herbert Sitz wrote:

>>
>
> I believe it can be a drop-in replacement depending on what you're doing
> with the CDS. Unfortunately, I'm pretty sure kbmMemTable doesn't support
> nested datasets.
>
> -- Herbert Sitz
>
>

--
Thomas Miller
Delphi Client/Server Certified Developer
BSS Accounting & Distribution Software
BSS Enterprise Accounting FrameWork

http://www.bss-software.com
http://sourceforge.net/projects/dbexpressplus

[Iman L Crawford]

Thomas Miller <tmi...@bss-software.com> wrote in
news:3f2ab111$1...@newsgroups.borland.com:

> Does it also support cloned datasets? We use both extensively.

Yes, and they're thread safe.

--
Iman

"If I had to live my life all over again," said Woody Allen, "I'd do it all
exactly the same ­ only I wouldn't read Beowulf."

[Thomas Miller]

I guess I have to hope Version 5 has nested datasets.

Iman L Crawford wrote:

> Thomas Miller <tmi...@bss-software.com> wrote in
> news:3f2ab111$1...@newsgroups.borland.com:
>
>>Does it also support cloned datasets? We use both extensively.
>
>
> Yes, and they're thread safe.
>

--

[Jason Chapman (JAC2)]

I had cause to retest some code I had written for batch updates, it had 38K
header records being imported from a text file and it had to map the
MX/SEDOL code to our internal primary key for the header before importing.
Because we have no control over the text file and its order it means that
the first draft of code went through the text file and did a locate on a CDS
that had been loaded up with the header file.

This meant 38K locates on the CDS - I found that this was significantly
slower with the CDS than with the mem table. Can't remember exact figures,
but it would have been significant if the user had to watch it. As it was,
it happens overnight, so no great shakes.

Just my 2c

JAC

"Dan Palley" <d...@trams.com> wrote in message

news:3f26cf88$1...@newsgroups.borland.com...