OLAP Council Benchmarks
Dave Menninger
For some reason I can't see Dan Druker's November 15th posting regarding
"Datawarehousing Products Evaluation + Benchmarks" via my news host. So
I've started a new thread and pasted in the relevant portions of Dan's
message. (I obtained a copy of Dan's message from one of our developers
who accessed the newsgroup from home.)
Dan: Since Arbor administers this newsgroup, can you explain why I can
see messages before and after yours, but I can't see your message?
Dan Druker wrote:
>For instance, since Oracle came out with your benchmark one year after
>the OLAP council issued it, and 6 months after Arbor, I must assume you
>spent quite a bit of time working on it.
I hope Essbase does better math than you. By my calculation, June 16th
to October 6th is less than four months (not six). We had no reason to
run the benchmark until there was a target to beat, so we did not begin
running it until you laid down the gauntlet.
>At six billion dollars, Oralce's
>clearly has unlimited benchmarking resources and I can't imagine you were
>sitting on your hands for a year.
So let me see if I have this straight, you are telling your customers
they should buy from Oracle because we have more resources? Or are you
telling them they shouldn't buy from us because we have more resources?
Or are you telling the customers that we don't have ANY resources
because everyone at Oracle works on benchmarks not products?
>In Arbor's case since we were the first ever vendor to run the benchmark, we
>had to work out a large number of bugs in the OLAP council's benchmark
>generation programs, wait for fixes from the author of the programs, etc -
Unfortunately you did not work out all the bugs nor resolve all the
ambiguities in the benchmark so we were left to clean up more of the
same.
>...Oracle used a number of
>benchmark special tricks that are not realistic for real world customers
>applications.
No "tricks" were used. We implemented the benchmark using techniques
which are not only common in Express applications, but are documented
and recommended to our customers. See other specific comments below.
>For instance, the OLAP report at http://www.olapreport.com refers to the
>precompiled static queries Oracle used in the OLAP benchmark and states
>"Clearly, this must improve the query performance, but it is unlikely
>that real customer applications could use such extreme performance
>tweaks without losing all flexibility."
This argument orginated from Arbor marketing propoganda, not Nigel
Pendse. First of all, the queries are not "static". They are
parameterized. Looking at the Essbase calc script for query 1 you have
done the same thing: P1 is the product parameter, P2 is the customer
parameter, P3 is the channel parameter, etc. As your comments embedded
within the code indicate these parameters are "filled in at query run
time". So I guess we have to say you used "static" queries also.
Let me explain how stored procedures are used in every application we
build. The graphical componenets of our products, including the
Selector, call a pre-built library of stored procedures for most of
their operations. No coding on the part of the user is required. The
user does not even know they are calling stored procedures. The
following call was generated using the Selector (from within Express
Analyzer) operating against our standard demo database:
->CALL XP_SLLIMIT('GEOGRAPHY', 'VIEWDRILL', 'STANDARD', 'UK', '1',
'ADD', '1', 'N')
The XP_SLLIMIT stored procedure takes various arguments and is the heart
of the selector. In this example, the arguments indicate the dimension
on which to perform the operation (GEOGRAPHY), the type of operation
(DRILL), which hierarchy to use (STANDARD), the relative starting point
for the drill operation (UK), the direction to drill (Down vs. up),
whether to add the results of the drill operation to the current display
or to replace the contents of the current display, etc. To be honest I
forget which argument is which. However, it doesn't matter! I didn't
have to know anything about this stored procedure. I just used our
standard point and click environment. Our software engineers figured
out the best split between client and server operations and created
parameterized stored procedures to implement that split. Similar calls
could be made for top/bottom operations, text matching, date ranges,
comparisons, etc.
Since Arbor does not have stored procedures you have no choice but to
try to trash their use. Clever marketing if you can get someone to
believe your argument. However, all the major relational database
vendors have seen the value of stored procedures and have chosen to
support them. Or are you saying that those vendors are just engineering
benchmark tricks into their products?
>To understand this stuff, people really need to download the full
>disclosures of both benchmarks - available on both Arbor and Oracle's web
>sites. It's painful to look through the 1,300 lines of complex code
>required to implement the benchmark in Express, but for readers that do
>there are a number of insights that can be gleaned.
I agree that people need to download the full disclosures. Then they
can see the myths you are propogating. By our count here are the
statistics:
Express 28 program objects
673 total Lines Of Code excluding comments:
418 lines - Data Access & Manipulation
80 lines - Database Object Definition
175 lines - Programming Construct
Essbase 16 program objects
406 total Lines Of excluding comments:
175 lines - Data Access & Manipulation
194 lines - Database Object Definition
37 lines - Programming Construct
By the way I find it very painful to look through all the Essbase code.
Perhaps I should have included a count of "\n" character sequences in
the statistics above.
>Page 4 line 85 contains the code used to precompile the static queries
>Pages 19 to 21 contain the static query programs
See comments above about static queries and stored procedures
>Page 21 line 63 is contains an example of a query hardcoded to not even
>access the Express database if the benchmark harness requests data not
>generated by the benchmark generator.
This is just one example of the advantages of having a powerful stored
procedure language.
>Page 4 line 30 shows INPLACE commands which override the normal data
>integrity / rollback features of Express.
It is accurate to say we used "INPLACE", but it is patently false to say
it compromises data integrity. You might even call the INPLACE setting
our "Arbor compatibility mode". If this compromises data integrity,
then you must say that all Arbor's benchmarks (and applications) also
compromise data integrity. The behavior is exactly the same as Essbase
-- abort in the middle of a data load and you must restart the data
load. It is not a "trick" and it is documented and recommended to our
customers on page 12 & 13 of our performance tuning tech note which is
distributed with our product.
--
Dave
+-------------------------------+
/ /|
+-------------------------------+ |
|Dave Menninger | |
|Oracle Corporation | |
|Sr Director, Product Marketing | |
|OLAP Products Division | |
|Voice:781-684-5797 | |
|Fax: 781-684-5895 | +
|EMail: dmen...@us.oracle.com |/
+-------------------------------+
The statements and opinions expressed here are my own and do
not necessarily represent those of Oracle Corporation.
Daniel Druker
Dave Menninger <dmen...@us.oracle.com> writes:
>>...Oracle used a number of
>>benchmark special tricks that are not realistic for real world customers
>>applications.
>>For instance, the OLAP report at http://www.olapreport.com refers to the
>>precompiled static queries Oracle used in the OLAP benchmark and states
>>"Clearly, this must improve the query performance, but it is unlikely
>>that real customer applications could use such extreme performance
>>tweaks without losing all flexibility."
>This argument orginated from Arbor marketing propoganda, not Nigel
>Pendse.
I don't understand what you are implying by this. Nigel Pendse is
pretty careful to evaluate what the vendors claim before he posts
information to his web site. You seem to think Nigel's opinion is
pretty important- Oracle issued a press releases earlier this year
that stated "The OLAP Report was compiled by OLAP industry veterans,
Nigel Pendse and Richard Creeth, ... Their extensive experience
enables them to deeply interview vendors and analyze information
to develop thorough, incisive product profiles and reviews."
I don't think its very appropriate for you to quote Nigel when he
says nice things about Express and attack him when he doesn't.
If Nigel agrees with some of Arbor's issues with artificial techniques
used in the Express benchmark, that seems to me to be entirely
appropriate given his role as an industry analyst.
>However, all the major relational database
>vendors have seen the value of stored procedures and have chosen to
>support them. Or are you saying that those vendors are just engineering
>benchmark tricks into their products?
Interesting you bring this up. The TPC-D, which is the relational world's
decision support benchmark equivalent to the OLAP council benchmark,
specifically bans using Stored Procedures or Static queries in running
the TPC-D benchmark, and specificaly states that vendors must use only Dynamic
queries. Why - because using ONLY static precompiled queries is not
realistic for real world decision support applications which are
supposed to be ad-hoc and dynamic.
I never said stored procedures were bad, of course there is a use for
them. But not for hard coding every query in advance to cook a benchmark.
The TPC-D specification which states this is at http://www.tpc.org
The TPC have already figured out many the tricks that vendors use
in benchmarking, and have taken steps to prohibit them.
It's clear the OLAP council needs to do the same for the APB-1.
>Looking at the Essbase calc script for query 1 you have
>done the same thing: P1 is the product parameter, P2 is the customer
>parameter, P3 is the channel parameter, etc. As your comments embedded
>within the code indicate these parameters are "filled in at query run
>time". So I guess we have to say you used "static" queries also.
Dave - I think you are confusing the client query execution programs
which are used to generate multiple concurrent query streams coming from
simulated users vs with calculation scripts which are used to perform
some of the more complex calculations inside the Arbor Essbase
OLAP Server.
In both Arbor and Oracle's benchmark implementation the client query
programs simulate multiple users concurrently accessing the OLAP server-
they start multiple threads on the client computers, read in the files
generated by the OLAP council's benchmark program, and then submit queries
to the OLAP servers. In Arbor's full disclosure these are the .c and .h
files (example phase3.c) found in the Arbor APB-1 Report.doc full
disclosure.
In Oracle's implementation you have the same client query programs, found on
your website as http://www.oracle.com/products/olap/collatrl/client.zip
The difference is that your client query programs only send parameters to
precompiled static queries stored in advance on the Express server,
while Arbor's client query programs send only dynamic queries to the
Arbor Essbase OLAP server.
>Let me explain how stored procedures are used in every application we
>build.
This is the heart of the matter - Oracle's OLAP benchmark implementation
uses ONLY precompiled static queries and NO dynamic queries. If every
application you and your customers build uses ONLY static queries then
and only then are your results are relevant to what your customers
can expect using Express.
Since you and I will argue about this forever and never agree, I think a
simple solution is to ask Express users whether they write precompiled
static programs for every query that their users might ever issue in
real sales analysis applications.
>>To understand this stuff, people really need to download the full
>>disclosures of both benchmarks - available on both Arbor and Oracle's web
>>sites. It's painful to look through the 1,300 lines of complex code
>>required to implement the benchmark in Express, but for readers that do
>>there are a number of insights that can be gleaned.
>I agree that people need to download the full disclosures. Then they
>can see the myths you are propogating. By our count here are the
>statistics:
>Express 28 program objects
> 673 total Lines Of Code excluding comments:
>Essbase 16 program objects
> 406 total Lines Of excluding comments:
Dave - It appears you are including in these counts the C source code
for the client query programs which are used to simulate concurrent users
to run the benchmark. This is really misleading, since no real customers
need to write C programs to simulate multiple concurrent users for a
benchmark using either Express or Essbase.
Note that 1,300 lines of code I counted for Express does NOT include
the C source code you used to implement the client query programs-
the 1,300 lines is ONLY the proprietary Express 4GL code required to
implement the Express OLAP server application itself, found in the
files codedsc.pdf and datadsc.pdf on your website.
The 1,300 lines of Express code compares directly to the 79 line
calculation script (including comments) called newfcst2.csc in
Arbor's implementation. If you want, you can also include an
additional 63 lines of scripts that Arbor used to automate the
loading of the application so it can be done lights out in batch,
called dbbuild6.scr and histload.scr. All these scripts are in the
file Arbor APB-1 Report.doc which is in the zipped full disclosure
on Arbor's web site.
Oracle has an additional 1,950 lines of code in five programs
called APBdrv.c, Dump.c, Snapi.h, Dump.h, and sntypes.h found in
the file client.zip on your web site. You used these programs
to generate concurrent client query streams. Note that I did not
count these 1,950 lines of benchmark driver C programs in my analysis
of your benchmark because no customer would ever have to implement
them.
However, you seem to include Arbor's C programs that generate
concurrent client query streams in your figures above.
The .C and .H files (like Phase3.c) in Arbor APB-1 Report.doc
are comparable to your five .c and .h programs I mention above.
Like I said above, I think this is misleading because no customer
would every have to write this stuff for either Oracle Express
or Arbor Essbase - the relevant comparison is the code used to
implement the OLAP application in Arbor Essbase and in Oracle
Express, which every customer would have to do.
But don't take my word for it. I refer you again to Nigel Pendse's
analysis on http://www.olapreport.com "Under the OLAP Council
benchmark rules, both Oracle and Arbor were obliged to publish all
the code required to create the application and run the benchmark.
It is instructive to download and compare them: the Express code is
extremely lengthy and forbiddingly complex compared to that
required for Essbase. This is often characteristic of
real-world applications."
This is a really easy one for people to figure out for themselves -
just download the full disclosures from Arbor and Oracle's web sites,
paste the code required to implement the application in each OLAP Server
into your favorite word processor, and run a line count utility.
The files in question are codedsc.pdf and datadsc.pdf in the
OLAP benchmark section of Oracle's web site, and the scripts ending
in .csc and .scr in the word document Arbor APB-1 Report.doc
(part of the full disclosure zip file on Arbor's web site.)
Perhaps some independant third party will do this for us so you and
I won't have to keep arguing about it.
>>Page 21 line 63 is contains an example of a query hardcoded to not even
>>access the Express database if the benchmark harness requests data not
>>generated by the benchmark generator.
>This is just one example of the advantages of having a powerful stored
>procedure language.
As above, these kind of tricks are specifically banned by the TPC-D
benchmark. It's clear that the OLAP council needs to tighten up the
APB-1 OLAP benchmark to eliminate loopholes. While not technically
prohibited by the OLAP benchmark, Oracle's use of static query
programs clearly violates the spirit of the benchmark specification,
which states "OLAP queries are ad hoc and very dynamic."
Like I said earlier, I don't think you and I will ever agree on this
forum so I put it to the readers of the newsgroup - what do you think ?
Is it realistic to use only precompiled static queries in running the
OLAP benchmark ? Do Express sites out there write stored procedure
programs for every query that users ever issue ? Should the OLAP
council ban static precompiled queries from the OLAP benchmark like the
Transaction Processing Council has from the TPC-D ?
For those of you who have downloaded the full disclosures, do you
see a difference in the amount of code and complexity to implement
the benchmark application in Oracle Express vs Arbor Essbase or
do the two look about the same as Dave implies ?
Hope everyone out there is finding this debate interesting...
Regards,
--
- Dan
Daniel Druker (408) 541-4027 or
Arbor Software Corporation 1-800-510-3197 x4027
1344 Crossman Avenue ddr...@arborsoft.com (work)
Sunnyvale, CA, 94089 ddr...@netcom.com (home)
Daniel Druker
Dave Menninger <dmen...@us.oracle.com> writes:
>Dan: Since Arbor administers this newsgroup, can you explain why I can
>see messages before and after yours, but I can't see your message?
Hi Dave,
comp.databases.olap is a unmoderated newsgroup, which means
it is not administered by Arbor Software or anyone else. You may or may
not have noticed that I post from my personal account at netcom, if you
did not see my posting there may be a problem with Oracle's news feed or
with your news server or client. If you periodically miss messages I
suggest you talk to your data center, or get an account on AOL or another
internet server provider with a more reliable internet connection.
The only connection between Arbor and the comp.databases.olap newsgroup
is that back in 1994 I generated the proposal for creating the group in the
first place, took it through the usenet voting process, put together the
initial FAQ for the group, etc. The goal of this was to create a place on
the internet to discuss OLAP and data warehousing in a public forum open
to all. There is no other link between Arbor and comp.databases.olap
other than I and other Arbor employees occasionally post messages to the
group.
So there is no conspiracy here.
Ory Adler
Hi there Dave....
I couldn't help but notice you, trying to slaughter a message posted by Dan
Druker. Personally I wouldn't want to get involved in this mass between you
and Druker regarding the benchmarking, I don't believe Arbor would post
benchmarking which prove Express is the better product, nor would ORACLE
post a benchmarking proving Essbase is the superior product. Leaving the
marketing tactics aside I would like to comment to you something about
ORACLE's customer sales/support tactics. In the past little while I have
been involved with several projects involving ORACLE's RDBMS feeding Essbase
cubes (A great match if you ask me as you get the best of both worlds). I
believe ORACLE's Relational database is a great product and bottom line it
is and has been the best relational database in the market in the last
while, what does bother me are comments from colleagues(some are/were ORACLE
fanatics) and personal experiences with your support people. I don't
understand why after I placed a call for tech support it took a week for
them to get back to me??? I also don't see why ORACLE wouldn't bother with
any clients not willing to spend at least a quarter million on an
implemenation, the list goes on and on but maybe you should forward this
message to the right people because it would be a shame to see a product as
great as ORACLE would fall because of poor technical support and clients
marketing/targeting.
Regards,
Ory Adler
Clarity Systems
Nigel Pendse
Dave Menninger (of Oracle) and Dan Drucker (of Arbor Software) have both
quoted comments from me in their discussion of their respective benchmark
results, so it's probably time I joined in.
First, let me say that both products have demonstrated such excellent
query performance in this particular benchmark that the differences in
query speeds are unlikely to be important for most companies. The
benchmark clearly shows that well-optimized multidimensional servers can
perform at an extraordinary rate. However, I've no doubt that we'll see
better figures than either of the sets of results so far published.
Indeed, I've already heard from another vendor claiming a further
ten-fold improvement.
More interestingly, while demonstrating their excellent performance, both
Arbor and Oracle have done an excellent (if unintended) job of
demonstrating the major weakness in each of their respective products.
First, Essbase demonstrated database explosion in action, growing some
30Mb of input data to an amazing 7.2Gb. This 240-fold growth is what
happens when you fully pre-calculate spare, multidimensional databases.
It has nothing to do with the storage of sparse data, something Essbase
does well. See http://www.olapreport.com/DatabaseExplosion/htm for a
full explanation of database explosion, which is a problem that afflicts
all versions of Essbase up to and including the current 4.1. Arbor says
that Essbase 5.0 will not suffer from the problem, and will presumably be
producing figures to demonstrate this.
Oracle's excellent benchmark figures were about five times faster, at
least in part because of Express' multicube architecture and because
Express doesn't have to pre-calculate everything. But Oracle also
succeeded in confirming the great complexity of Express applications. It
also appears to have been rather aggressive in creating highly optimized,
but fixed, hard-coded queries which aren't representative of what could
be implemented with typical real-world applications. Again, this
demonstrates that Express is very programmable, but also that a great
deal of it is needed for best performance. It also demonstrates that
startling scores in artificial benchmarks shouldn't be accepted too
literally.
Nigel Pendse
OLAP Solutions
http://www.olapreport.com
-------------------==== Posted via Deja News ====-----------------------
http://www.dejanews.com/ Search, Read, Post to Usenet
Tfidd
Nigel,
Thank you for clearing the marketing noise up.
Tom Fidd
Nigel Pendse
In article <8802279...@dejanews.com>,
Nigel Pendse <nig...@compuserve.com> wrote:
>
....
I gave the wrong URL. This is the correct one:
>
> First, Essbase demonstrated database explosion in action, growing some
> 30Mb of input data to an amazing 7.2Gb. This 240-fold growth is what
> happens when you fully pre-calculate spare, multidimensional databases.
> It has nothing to do with the storage of sparse data, something Essbase
> does well. See http://www.olapreport.com/DatabaseExplosion.htm for a
> full explanation of database explosion, which is a problem that afflicts
> all versions of Essbase up to and including the current 4.1. Arbor says
> that Essbase 5.0 will not suffer from the problem, and will presumably be
> producing figures to demonstrate this.
....
Dave Menninger
You are right, we are not going to agree on all the details.
Nevertheless, the
bottom line is that Express completed the tasks, as specified by the
OLAP
Council Benchmark and officially audited, five times faster than
Essbase. The
magnitude of the difference is remarkable and is to my knowledge
unprecedented. You can quibble about little nits here and there, but
you are
in no position to argue over the validity of the benchmark. Arbor
participated
in the development of the benchmark and voted in favor of adopting it
along
with the rest of the OLAP Council. It was also a wonderful benchmark
when you
first published results.
If you recall, we used a third party with real world experience to help
develop the benchmark and arbitrate differences between different types
of
OLAP implementations to make sure we had a well rounded, vendor neutral
specification. To quote Nigel Pendse author of the OLAP Report at
http://www.olapreport.com, "the benchmark has been explicitly designed
not to
favor any single product." In fact, the third party who worked with the
Council to define the benchmark was familiar with Essbase as well as
Express.
You approved the selection of the third party and in fact chose the same
third
party to audit your benchmark results. You couldn't have been too upset
with
the work that he produced. To come back now and raise objections is
nothing
but sour grapes.
You bring up issues of stored procedures. Express has them, Essbase
doesn't.
The performance advantage of stored procedures is a good thing not a bad
thing. That's why every successful relational database has them. To say
that
we used them "for hard coding every query in advance" is ridiculous.
That
would suggest we have 250,000 precompiled stored procedures. We had
10. How
do those ten procedures handle so many variations? The data selection
is
totally dynamic. You should know that the essence of a query in a
multidimensional engine is selecting the dimension members that identify
the
data to retrieve, locating that data, and returning the results. All of
these
activities take place in our execution of the benchmark. In addition, we
even
did many on-the-fly calcs. You did none. In the real world all calcs
cannot be
anticipated in advance as you well know.
Lets face it, you were beaten soundly. Naturally you must attempt to put
the
best spin on it that you can, but most of your "spin" has been totally
disingenuous. You complained about network speed when it was obvious
from
your own published information that this was irrelevant(using less than
5% of
the network capacity). Well, we have completed our audit of APB-1
results
using 10baseT, the same network configuration that you used, and we are
still
five times faster. We will make the results available on our website as
soon
as we return from the holiday.
Another example of exaggerated spin control is your code comparisons.
To say
that the calc script is the only code in the Essbase implementation is
ridiculous. When we download your disclosure documents and look at the
output
of your outline definition, we see hundreds of lines of code. In
addition,
you repeatedly include our comments in your line counts and exclude them
from
yours so you can present a distorted picture. I stand by the figures I
provided previously.
Finally, I made no disparaging remarks about Nigel Pendse as your
posting
implies. I simply pointed out that he was quoting Arbor's arguments, so
you
were in fact quoting yourself. The way to counter our benchmark is with
results. We welcome your next round and are ready to respond with ours.
Daniel Druker
>You are right, we are not going to agree on all the details.
>Nevertheless, the bottom line is that Express completed the
>tasks, as specified by the OLAP Council Benchmark
Dave - there is no arguing this. The question is
whether Oracle's results are relevant for real customers
evaluating OLAP products, or they are so artificially
tweaked they are meaningless.
>You approved the selection of the third party and in fact chose
>the same third party to audit your benchmark results. You couldn't
>have been too upset with the work that he produced. To come back
>now and raise objections is nothing but sour grapes.
And in the benchmark definition the author of the benchmark
talks about how important Dynamic and Ad-hoc queries are to OLAP.
I think the benchmark itself is actually quite well done and
I think that Dan Bulos who authored the benchmark did a
great job. It's version 1.0 of the benchmark after all.
The area the OLAP council needs to work on is that there
are too many loopholes and gray areas in Version 1.0
of the benchmark which Oracle clearly exploited.
The OLAP benchmark spec is only a few pages, while the TPC-D
spec runs to over 100. This is because the Transaction
Processing Council has been doing benchmarking long enough
to know that you have to be excruciatingly explicit about
what is allowed and what isn't allowed in a benchmark or
the vendors will use unrealistic techniques and benchmark
specials to cook their numbers. Just like Oracle did
in the OLAP benchmark using static precompiled queries
in a benchmark that goes out of its way to talk about how
dynamic and ad-hoc OLAP is.
The really nice thing about the OLAP benchmark is it makes
vendors publish all of their code so people can figure out
what is really going on.
>The performance advantage of stored procedures is a good thing not a bad
>thing. That's why every successful relational database has them.
The TPC-D benchmark, which is admittedly more mature than the
APB-1 OLAP benchmark and is the relational database industry
standard for decision support benchmarks, bans using stored
procedures for queries because it isn't realistic to expect
real customers to write stored procedures for every query
every user ever asks.
Just as you did in your posting last week, you are not
separating the availablity of a feature to the validity of
its use by real customers. Remember when Oracle RDBMS
discrete transactions were banned in benchmarks years ago -
they make benchmarks run really really fast but no real
customer uses the same technique for real applications.
And discrete transactions were technically legal at the
time, but the RDBMS benchmarking police eventually banned
them. This is an exact parallel to Oracle's use of static
precompiled queries in the APB-1 OLAP benchmark.
>To say that we used [stored procedures] "for hard coding every
>query in advance" is ridiculous. That would suggest we have 250,000
>precompiled stored procedures. We had 10. How do those ten
>procedures handle so many variations?
You need to read your own code and the OLAP benchmark definition.
Every single query that is executed by Express in the benchmark
goes through one of your 10 pre-compiled static stored procedure
programs. I refer you to pages 19-21 of your full disclosure
and to your client query programs.
The OLAP council benchmark generator creates 250,000 queries, each
of which specifies a particular set of input parameters for one
of 10 standard query types. The only thing that changes from
query to query is the input parameters, which is why you were able
to cook the benchmark by precompiling static programs which
take advantage of the fact there are only 10 query types. The
worst is the Express static query program (number 10) which
doesn't even hit the Express server at all if certain known
combinations of data are queried.
In the Arbor Essbase benchmark, our client query harness
generates 250,000 fresh dynamic queries, so the OLAP server
sees and does exactly the same thing and performs exactly the same
as if real users were doing live ad-hoc dynamic queries using
any standard front-end query tool.
>Another example of exaggerated spin control is your code comparisons.
>When we download your disclosure documents and look at the output
>of your outline definition, we see hundreds of lines of code. In
>addition, you repeatedly include our comments in your line counts
>and exclude them from yours so you can present a distorted picture.
Ok - now I understand what is going on here.
What you are calling a "hundreds of lines of code" outline
definition is not code at all. As it says in the full
disclosure, NO code is required to create the outline, it is
created using graphical tools. The auditor chose to export
the outline structure to a text file to document it for the
full disclosure document rather than print out a bunch of
screenshots of Arbor's graphical tools.
Similary in our last round of postings you seemed to think
our client C query harness was part of the code required to
implement the APB-1 OLAP application in the Essbase server.
This was also not correct. It's probably our fault for
originally including everything in a single document.
Last weekend we placed an updated version of Arbor's full
disclosure documents on our web site, which makes it much
easier to compare how you go about implementing the OLAP benchmark
application in Arbor Essbase vs Oracle Express. In the new version
we broke out the client C code programs used to implement the
benchmark harness into a separate file just like Oracle did.
It's now much easier for people to compare the amount of code
required to implement the exact same application in the
two products. Here is the URL:
http://www.arborsoft.com/reports/apb1/disclosure.zip
>I stand by the figures I provided previously.
So now you know that both the C client query harness and the
outline definitions that you previously counted are not part of
the code required to implement the APB-1 OLAP application in
Arbor Essbase. Coupled with Nigel Pendse's posting earlier
this week about how complex it is to build applications in
Express I think we are reaching closure on this subject.
By the way, when I count lines of code I include comments in
both Oracle and Arbor's code. I don't think it's realistic to
strip comments from either in counts since real customers always
comment their code.
I am a bit surprised there is very little posting on this
thread by anyone other than Dave and myself. Do people out there
care or not care about benchmarks ? Are you sick of Dave and I
going back and forth or do you find this debate
interesting/helpful/amusing ?
I hope we are not boring everyone to death or horrifying
everyone with our competitive zeal.
Sherwin Shao
> I am a bit surprised there is very little posting on this
> thread by anyone other than Dave and myself. Do people out there
> care or not care about benchmarks ? Are you sick of Dave and I
> going back and forth or do you find this debate
> interesting/helpful/amusing ?
Well I care. Not about those results specifically, but how
it impacts my job. I do a lot of programming in Essbase. I
think it's a great tool for multidimensional modeling.
However, it is the ONLY tool I have used. (so far)
Although the scripting language is great for simple tasks, I
find to accomplish complex tasks that the amount of
workarounds required is painful.
Here are my views of Essbase limitations:
1. Scripts cannot call other scripts.
2. No parameter passing into scripts.
3. Loop counters must be incremented inside member blocks.
4. There is no way to access data from other cubes.
5. You cannot FIX on a part of a dimension, and then
calculate along the same dimension.
(ie, FIX( @IDESCENDANTS( F1997 )) F1997 = F1997Q1 + F1997Q2
... ENDFIX
(This is a real performance killer for modeling!)
I sincerely hope that these questions will lead to a better
version of Essbase, or other better products.
Daniel Druker
I've moved this discussion into a new thread...
shao@interlog$.com (Sherwin Shao) writes:
>I do a lot of programming in Essbase. I
>think it's a great tool for multidimensional modeling.
>Although the scripting language is great for simple tasks, I
>find to accomplish complex tasks that the amount of
>workarounds required is painful.
>Here are my views of Essbase limitations:
>1. Scripts cannot call other scripts.
>2. No parameter passing into scripts.
>3. Loop counters must be incremented inside member blocks.
>4. There is no way to access data from other cubes.
>5. You cannot FIX on a part of a dimension, and then
>calculate along the same dimension.
>(ie, FIX( @IDESCENDANTS( F1997 )) F1997 = F1997Q1 + F1997Q2
>... ENDFIX
> (This is a real performance killer for modeling!)
Several of these things are fixed or are better in Arbor Essbase 5.
You can now pass variables into scripts using the Substitution Variables
feature of Arbor Essbase 5. For instance a script can reference a variable
called ¤t_month, and you can have a substitution variable that sets
current_month to "November" Note that substitution variable work with
calculation scripts, report scripts, and in queries - so you can also
have a spreadsheet template that has ¤t_month in it, and will
automatically adapt to the current month when users do queries.
You can access data from other cubes using transparent partitions. The
cubes can be on the same machine or distributed across the network.
You should find that there are fewer needs to use calcultion scripts.
Using dynamic calculations means that you often don't need to
calculate specific regions of the cube, which is often done
in version 4.1 for performance reasons. This might solve your final issue
above.
I've passed your note onto product management, though I know that some
of your things like calc scripts calling other calc scripts are already
pretty high on their enhancement list. They may also have some better
answers for you than I do off the top of my head above.
Kevin L
Hi Dan
Daniel Druker <ddr...@netcom.com> wrote in article
<ddrukerE...@netcom.com>...
>.......I think we are reaching closure on this subject.
You Wish !! ;-)
You asked :
>I am a bit surprised there is very little posting on this
>thread by anyone other than Dave and myself.
Me too - so here's someone else having their say...
> Do people out there
>care or not care about benchmarks ? Are you sick of Dave and I
>going back and forth or do you find this debate
>interesting/helpful/amusing ?
>I hope we are not boring everyone to death or horrifying
>everyone with our competitive zeal.
Not atall. I for one find the debate interesting, helpful and amusing.
Interesting, cos I like trying to see if I guessed right what your next
excuse, for what must have been shattering news, will be.
Helpful, because the longer the thread lasts the more people discover what
Arbor don't want them to know - Express is not only more flexible, scalable
and complete than Essbase, but also a staggering 5 times faster than
Essbase !! (And more efficient at data storage too - the Express data was a
third of the size of the Essbase data don't forget).
Amusing because you quite often come up with an even more outrageous excuse
than I had guessed !
For too long Oracle (and IRI in the past) has sat back and let Arbor Sales
people get away with spreading misinformation and innuendo about the
relative merits of Express and Essbase. So I think it is good that Dave
has decided to set the record straight on a few things..
For several years Arbor have been explaining (sometimes explicitly, often
by implication) to the OLAP world how SLOW Express is, how its COMPLEX
Multi-cube model is BAD, how its formulae are SLOW and irrelevent compared
to Essbases lightning fast data loading capabilities.
Now the benchmarks are published. And your marketing spin has been
proven wrong :
Express is FAST (5 times faster than Essbase overall)
Express formulae are not just responsible for extremely FAST data load
times, but also contribute to the stunning 8 times FASTER than Essbases
pre-calculated hypercubes !!. This incedentally is one of the reasons the
Express database was a 3rd of the size as the Essbase one was. Perhaps
that is why after all these years of slagging off Express' 'calc on the
fly' formulae you seem to be introducing your first version of this in
Essbase.
The multicube model is not only much more FLEXIBLE, but contributes greatly
to Express' speed advantage (3 TIMES faster than Essbase data load/calc
times, a startling 8 TIMES faster than Essbase query times). It is another
one of the reasons the Express database was a 3rd of the size as the
Essbase one was. You seem to finally have come around to this too since
you have begun to implement multi-cubes in the latest announcements about
Essbase.
The early response from Arbor regarding the humiliating news, as Nigel
Pendse reported on the www.olapreport.com home page a few weeks ago, was
to complain about the network and client workstations used by Oracle .
It may have bought you some time while you dreamt up the next line of
defence, but as the world now can see, neither factor had the slightest
bearing on either Oracle or Arbors benchmark statistics. You know and I
know and Oracle know that if you had been able to improve your benchmark
results by installing a faster network or swisher PCs you would have done.
As soon as Dave exposed this bunch of sour grapes, you moved onto the
good old 'everything in Express is complex and you have to write lots of
code' arguement.
Yes there is a programming environment in Express. The Express SPL (4GL if
you like) is incredibly powerful. And if you choose to, you can use it to
build very sophisticated applications. This is exactly why many Express
users choose Express - because there is a robust multi-dimensional stored
procedure language in Express Server it can go places that other servers
can only dream about.
But of course, the Arbor claim is (mostly by innuendo now, since you know
it to be a bit of a porky pie) that EVERYTHING you do in Express involves
you writing 'thousands of lines of complex code'. This is rubbish.
Almost all Express applications do indeed make heavy use of parameterised
stored procedures. But they are, in the vast majority of systems, either
delivered as part of an application (eg Oracle Sales Analyzer is
implemented in 99% of cases without the need to write a single line of
code) , or generated automatically via the graphical building tools
supplied with Express. Essbase as you confirm in your latest posting does
exactly the same thing as the latter.
Explaining away hundreds of lines of Essbase code in your last posting you
said :
>'NO code is required to create the outline, it is
>created using graphical tools. The auditor chose to export
>the outline structure to a text file to document it for the
>full disclosure document rather than print out a bunch of
>screenshots of Arbor's graphical tools.'
So you would not have started this ridiculous arguement if Oracle had just
shown screen shots of a client application that generated the code ??
Would you have said 'Well done Oracle, no code in Express either' ?? Of
course not. You would have said 'Hey, Oracle are cheating, there is code
there and they haven't disclosed it'. So it really is a little lame
claiming that the Arbor code to create an outline is not code atall but the
Express code to create the Express structures is .
One last question. This surrounds your protesting about the so called
'static' queries you accuse Oracle of pre-implementing. I agree entirely
with Dave - it really is ridiculous to claim the moral high ground on this
when, at the very same time, you precalculated every possible numeric
result in the benchmark, while Oracle's Express formulae dynamically
calculated many of the query results completely on the fly. Anyhow, my
question is this : If the ability to pass arguements to a piece of
pre-compiled code is such a bad thing, why did you explain the following in
a response to a recent question elsewhere on this very NewsGroup : ?
The question :
>shao@interlog$.com (Sherwin Shao) writes:
>I do a lot of programming in Essbase. I
>think it's a great tool for multidimensional modeling.
>Although the scripting language is great for simple tasks, I
>find to accomplish complex tasks that the amount of
>workarounds required is painful.
.......Incidentally - what he needs is a stored procedure language.....
>Here are my views of Essbase limitations:
>
>1. Scripts cannot call other scripts.
>2. No parameter passing into scripts.
>3. Loop counters must be incremented inside member blocks.
>4. There is no way to access data from other cubes.
........etc
Your Answer :
>Several of these things are fixed or are better in Arbor Essbase 5.
>You can now pass variables into scripts using the Substitution Variables
>feature of Arbor Essbase 5. For instance a script can reference a variable
>called ¤t_month, and you can have a substitution variable that sets
>current_month to "November" Note that substitution variable work with
>calculation scripts, report scripts, and in queries - so you can also
>have a spreadsheet template that has ¤t_month in it, and will
>automatically adapt to the current month when users do queries.
Sound spookily like another Arbor U-turn is coming....
First Arbor argued that hypercube was the (only) answer. Now Essbase is
being re-written to play catch up and support Express style multi-cubes.
Second Arbor argued that the pre-calculate-everything approach was the
(only) answer. Now Essbase is being re-written to play catch up and
support Express style formulea.
Arbor used to say that the only way to do OLAP was exclusively in a MDDB.
Now Essbase is being re-written to work with data stored in DB2.
Following Express again. Express has had relational support for many years
(and not just DB2 either).
NOW it looks like the latest argument (having 'static pre-comiled queries
that accept arguements is an illegal ruse dreamed up by techies at Oracle
to do fast benchmarks, but of no use to users' ) is set to fall by the
way-side...... What price Essbase 6 or Essbase 7 having a proper
server-based stored procedure language ?!
If I was considering buying an OLAP Server, I think I'd see this, along
with the performance advantages , as a sure sign that Oracle knows OLAP
best.
Anyway, I truly do wish you good luck with your next attempt at the
benchmark - you owe it to your loyal customers. Perhaps now your
developers have adopted some of the Express features it will be easier for
Essbase to respond to the challenge.....
>.......I think we are reaching closure on this subject
The best way to do that would be to post better results. I'm sure Dave
(and all the Arbor users who must now wish they had bought Express) would
relax a bit if Arbor was to even get close.
Regards, Kevin
_ _ _ _
/_/_/_/_/\
/_/_/_/_/\/\
/_/_/_/_/\/\/\
\_\_\_\_\/\/\/
\_\_\_\_\/\/
\_\_\_\_\/
=from :====================================================================
Kevin Lancaster ROCKPORT Software Limited
EMail : KevinLa...@NOSPAMcompuservePLEASE.com
===========================================================================
The statements and opinions expressed here are my own and do not
necessarily represent those of Rockport Software Limited.
Daniel Druker <ddr...@netcom.com> wrote in article
<ddrukerE...@netcom.com>...
ElkinsEcon
In article <ddrukerE...@netcom.com>, ddr...@netcom.com (Daniel Druker)
writes:
>I am a bit surprised there is very little posting on this
>thread by anyone other than Dave and myself. Do people out there
>care or not care about benchmarks ? Are you sick of Dave and I
>going back and forth or do you find this debate
>interesting/helpful/amusing ?
>
>
It's interesting, helpful and amusing! There was no way I was going to try to
read those benchmarks myself! !
Steve Elkins
Elkins Analytics
Lindsay Gillies (Digital)
Daniel Druker wrote in message ...
>
>
[snip snip]
>I am a bit surprised there is very little posting on this
>thread by anyone other than Dave and myself. Do people out there
>care or not care about benchmarks ? Are you sick of Dave and I
>going back and forth or do you find this debate
>interesting/helpful/amusing ?
>
>I hope we are not boring everyone to death or horrifying
>everyone with our competitive zeal.
>
It's all I can do to absorb all the information being hurled about!
I think the real difficulty is that all M/D products
(ROLAP/OLAP/MOLAP/DOLAP/Whatever) have very particular characteristics,
which inhere deeply in the details. As Nigel points out, once the data's
there, performance is excellent for both Express and Essbase. I also think
its true that more work is done in Essbase in the server itself, while more
is required via external code to use Express effectively. I personally
prefer the server-based approach, and I'll use VBA/Excel if I want to get at
exposed functionality at a more discrete level---but some might find the
script-level exposure of that functionality a la Express more appropriate
for the application.
What is really confusing to many users is the details that seperate products
and in effect make many if not most comparisons meaningless. I mean that
much choice must be driven by the actual requirements, that these tools are
not commodity implementations and are much more sensitive to the
application, its implementation and maintenance costs.
So you customers have to do the really hard work required to differentiate
all the products in order to select the right one for your application---or
you can pay Nigel, which is money well spent. Its hard to say, at this
point (as much as I prefer myself to use Essbase in my architectures) that
there is a clear pan-domain solution. Each tool has its advantage profile.
Daniel Druker
"Kevin L" <KevinLa...@NOSPAMcompuserve.com> writes:
>Me too - so here's someone else having their say...
Hi Kevin,
Your name sounds familiar- are you the same person as
klanc...@oracle.co.uk Kevin Lancaster, OLAP Consulting, of
Oracle Corporation UK Limited ?
If so the tone of some parts of this note make a lot of
sense. If you aren't this same person I aplogize in advance.
My guess that you are indeed the same person and you've
recently left Oracle.
A lot of the comments you make in your posting are
pretty emotional, and thus very hard for me to give you
a meaningful technical response to, like this:
>For too long Oracle (and IRI in the past) has sat back and let Arbor Sales
>people get away with spreading misinformation and innuendo about the
>relative merits of Express and Essbase.
However some of your points are technically based so I will try to
respond to them with some more concrete information...
>Almost all Express applications do indeed make heavy use of parameterised
>stored procedures. But they are, in the vast majority of systems, either
>delivered as part of an application
The point about stored procedures in the context of the OLAP Benchmark
is whether it is realistic for all possible QUERIES to be processed
ONLY through stored procedures in real-world applications. This is the
thing that is banned by the other industry standard benchmarks like
the TPC-D, and is what analysts like Nigel Pendse refer to as
unrealistic benchmark tweaks. In the Oracle implementation of the
OLAP benchmark, EVERY query was sent through a static, precompiled
stored procedure.
Kevin - I think this could be a really easy issue to clear up if you
would just be kind enough to answer the following question:
You've implemented lots of Oracle Express systems. In real implementations
that you work on, do you recommend that your Oracle Express customers
should program stored procedures in advance for every query every user
ever asks ?
>Explaining away hundreds of lines of Essbase code in your last posting you
>said :
>>'NO code is required to create the outline, it is
>>created using graphical tools.
>So you would not have started this ridiculous arguement if Oracle had just
>shown screen shots of a client application that generated the code ??
Absolutely correct. If Oracle had been able to use Oracle Express
Administrator, the OLAP application development tool that ships with
Express, to generate all or part the OLAP benchmark application this
would have been much less of an issue for Oracle. However the full
disclosure contains 1,300 lines of complex code, so it is apparent
that Oracle Express Administrator could not be used to implement the
application.
It's pretty clear from this that Oracle Express Administrator either
a) couldn't be used for an application with the complexity of
the OLAP benchmark application, or
b) produced an application that performed poorly so the application
had to be re-writen by hand.
The requirement to fully disclose all code used to implement the
OLAP benchmark application was put in the benchmark for exactly this
reason. If a customer has an application that is similar to that defined in
the OLAP benchmark, they now know exactly what can they look forward to
if they try to implement it in either Oracle Express or Arbor Essbase.
And from the full disclosures, the code required is around 1,300 commented
lines of proprietary programming in Express vs only around 75 commented
lines in Arbor Essbase.
>Sound spookily like another Arbor U-turn is coming....
This next section is largely a rehash of Oracle internal competitive
marketing slams about Arbor Essbase.
>First Arbor argued that hypercube was the (only) answer. Now Essbase is
>being re-written to play catch up and support Express style multi-cubes.
Not at all. The partitioning features of Arbor Essbase 5 are completely
different than those of Express multicubes. For instance,
Arbor Essbase 5 partitions
o Are logical views of one or more physical or logical cubes
o Are used to join together, link across or replicate between cubes
o Can appear to the user as a single uniform cube
o Can cascade, so one partition can refer to another
o Can combine single cells, portions of cubes or entire cubes
o Can have different dimensional structures, hierarchies, member names,
and sparsity patterns in each partition
o Include any combination of precalculated or dynamically calculated data
o Can be distributed across multiple computers on a network
o Can be loaded and calculated in parallel
o Can combine both relational and multdimensional data
o Are transparent to the application developer and to the end-user
o Support synchronization and replication of metadata and data
o Can be used to navigate between applications that have different
dimensions and business rules
o Are created and managed using graphical tools without programming
From my understanding, Express multicubes exist to let you define
each variable in a database as having different dimensionality,
and thus manage sparsity.
Note that I'm not an Express expert by any means but I've run all
this stuff by quite a few Arbor employees who are - we've have quite
a number of ex-Oracle Express folks who work here. I'm sure you'll
let me know if Oracle Express multicubes have all the features
I describe of Arbor Essbase Partitions.
>Second Arbor argued that the pre-calculate-everything approach was the
>(only) answer. Now Essbase is being re-written to play catch up and
>support Express style formulea.
Also not correct. Arbor has actually been very clear about the advantages
and disadvantages of precalculation vs dynamic calculation. You need
only search the archives of this newsgroup to see many postings by me on
this subject.
In terms of Arbor Essbase 5 dynamic calculations being similar to
Express-style formulae, that is only superficially true. For example,
Arbor Essbase 5 dynamic calculations support both calc on the fly,
where calculated data are thrown away after they are used, and calc
on the fly and store, where dynamically calculated data are stored
for re-use by later users. You can choose calculation options
on a per-member basis across any dimension, and you can change
the calculation option for any member dynamically, without unloading
data or rebuilding applications. And which calculation options are
used are totally transparent to all front-end tools, applications,
and users.
There are also graphical tools to manage the calculations - it just takes
a click of a button to set a calculation type, or you can automatically
set the calculation types during data and structure loading.
>Arbor used to say that the only way to do OLAP was exclusively in a MDDB.
>Now Essbase is being re-written to work with data stored in DB2.
>Following Express again. Express has had relational support for many years
>(and not just DB2 either).
Actually the relational storage interface in Arbor Essbase 5 will work
with Oracle, Informix and Sybase in addition to DB2. The DB2 version
will be available first, but the others will follow shortly after. And
Essbase has had relational support for a long time too - what the
relational storage interface does is let Arbor's customers choose
whether they wish to store their OLAP database in an optimized
multidimensional data store or directly in Oracle, Informix, Sybase,
or DB2. Or in a hybrid combination.
The key thing that is different about the relational storage interface than
other products on the market is that it is functionaly identical to the
multidimensional data store including support for all OLAP calculations
and multi-user read/write support. It also automatically generates, manages,
and populate the tables and indices in the underlying RDBMS, so you don't
have to have an army of DBAs managing lots of summary tables.
>Regards, Kevin
> _ _ _ _
> /_/_/_/_/\
> /_/_/_/_/\/\
> /_/_/_/_/\/\/\
> \_\_\_\_\/\/\/
> \_\_\_\_\/\/
> \_\_\_\_\/
>=from :====================================================================
>
>Kevin Lancaster ROCKPORT Software Limited
>
>EMail : KevinLa...@NOSPAMcompuservePLEASE.com
>
>===========================================================================
>
> The statements and opinions expressed here are my own and do not
> necessarily represent those of Rockport Software Limited.
Former Kevin Lancaster signature from previous comp.databases.olap
postings:
>Kevin Lancaster, OLAP Consulting
>
>Oracle Corporation UK Limited
>The Ring, Bracknell
>Berkshire
>RG12 1BW
>UK
>
>Check Out The OLAP Web Page :
>http://www.uk.oracle.com/products/olap/html/
>
> \\__o_\____.
>-*-*-*-(@)_____(@)\
>_________________________________________________________________________
> ** The statements and opinions expressed here are my own and **
> ** do not necessarily represent those of Oracle Corporation. **
>_________________________________________________________________________
There's a lot of stuff here for follow-on discussions.
So what kind of company is ROCKPORT software UK ? What do your
do for them ? How long ago did you leave Oracle, assuming you are
the same Kevin Lancaster ?
John Durkin
I did an evaluation of Express a couple of years ago (during the days
of IRI), and as I remember the big turn-off to Express was the
difficulting in deploying an application due to the heavy coding
requirements and the use of kluges like "conjoins" to get around
limitations in Express's scriping language. Frankly, our selection
decision was driven by 1) the graphical development interface, which
allowed our non-I.T. finance and other line people to develop their
own applications, and 2) Essbase's superior client-server
architecture, with all the security bells and whistles. Although calc
performance was a factor, it was not our determining factor.
Oracle invited me to a OLAP seminar last year, and I noticed that
conjoins were still alive and well, althought the interface was
cleaned up and OLAP objects and other gimmicks were added.
WIth the explosion in OLAP over the past few years and constant
introduction of new OLAP products I am constantly bombarded with
vendors or users who are entranced by the latest OLAP
application-which is good. However, for all the OLAP evaluations I
have been through, we always go back to Essbase, due to its ease of
development and superior Excel interface. Don't underestimate the
importace to the interface: in the end, users want their data or
reports in Excel, and any other tool, OLAP or other, just gets in the
way. This is one area where Express has always lagged.
My personal opinion is that Express will cease to exist as a product
as soon as its functionality can be reasonably put directly into
Oracle's RDB. This could be good or bad, depending on your
perspective. At this time I don't believe that Oracle the Corporation
is fully behind the independance of Express as a product.
As for on-the-fly calcs, this functionality has been in TM1 for years.
I suspect a comparison of TM1 versus Essbase would reveal a large
performance difference as well. The achilles heel of on-the-fly-calc
has always been drill down to deep layers of data. This is where the
pre-calculation method of Essbase excels. However, on-the-fly calcs
are fully adequate for many applications, which is why I suspect Arbor
is including this functionality in version 5.0.
Just my two cents....
John Durkin
Tokyo, Japan
Fred Hale
On Fri, 28 Nov 1997 20:56:37 GMT, ddr...@netcom.com (Daniel Druker)
wrote:
>I am a bit surprised there is very little posting on this
>thread by anyone other than Dave and myself. Do people out there
>care or not care about benchmarks ? Are you sick of Dave and I
>going back and forth or do you find this debate
>interesting/helpful/amusing ?
>
>I hope we are not boring everyone to death or horrifying
>everyone with our competitive zeal.
No. I, like Steve and a few others, find this exchange very
interesting. I have to concur with John Durkin on his point..
>However, for all the OLAP evaluations I
>have been through, we always go back to Essbase, due to its ease of
>development and superior Excel interface. Don't underestimate the
>importace to the interface: in the end, users want their data or
>reports in Excel, and any other tool, OLAP or other, just gets in the
>way. This is one area where Express has always lagged.
This was one of the main reasons I chose Essbase along with its ease
of model building and server focus. Response time is important, but
end-users/business managers do not care whether they have to wait 1
sec or 5 secs when they are used to waiting minutes with legacy RDBs.
Point being, does a 5 times slower response time really matter?
My two cents...the OLAP benchmark is fine. Many people will be
interested to see which companies faired well. These will be the
companies they will evaluate. However, the software product that
meets a companies needs, in the end, will win. I only hope more
companies are smart enough to do their "homework" regarding OLAP
technologies.
In my case, realizing that each company's requirements are different,
I asked Oracle and Arbor to build a prototype model to prove that
their software would work as advertised. Oracle declined. Arbor did
one in a matter of one and a half days. This is what sells software,
not figures on a piece of paper.
Fred Hale
Sea-Land Service, Inc.
Charlotte, NC
Greg Hutchinson
> >thread by anyone other than Dave and myself. Do people out there
> >care or not care about benchmarks ? Are you sick of Dave and I
> >going back and forth or do you find this debate
> >interesting/helpful/amusing ?
> >However, for all the OLAP evaluations I
> >have been through, we always go back to Essbase, due to its ease of
> >development and superior Excel interface. Don't underestimate the
> >importace to the interface: in the end, users want their data or
> >reports in Excel, and any other tool, OLAP or other, just gets in the
> >way. This is one area where Express has always lagged.
> Fred Hale
> Sea-Land Service, Inc.
> Charlotte, NC
I found it interesting because it validated my findings exactly from 2
project over the last year - always nice to know you're correct even if
some of it is a gut response to go along with the analytic.
As far as the user interface and tool set, I went with the Hyperion OLAP
product...TM1 on steroids. Better Excel interface that Arbor IMHO,
report writer, graphical building, etc., and superior performance in a
near real time business environment.
I can go down the street and talk to friends about an example of Arbor
data explosion from a LARGE computer manufacturer and look out my back
door to the "round towers" to see an example of a complex building tool
and maintenance nightmare app. I know because I sold it to them.
It's a shame Hyperion doesn't advertise better to showcase what effort
tehy put into the TM1 effort.
Greg Hutchinson
Hired Gun
Foster City, CA
Allen Worthington
Greg Hutchinson <ghut...@pacbell.net> wrote in article
<349008B7...@pacbell.net>...
> I can go down the street and talk to friends about an example of Arbor
> data explosion from a LARGE computer manufacturer and look out my back
> door to the "round towers" to see an example of a complex building tool
> and maintenance nightmare app. I know because I sold it to them.
In what respects is the Essbase application you mentioned a "maintenance
nightmare app"? Also, what do you mean by "complex building tool" (I
suppose you are referring to the Essbase Application Manager)?
Allen Worthington
John Durkin
100% concur with Fred.
fred...@sealand.com (Fred Hale) wrote:
>On Fri, 28 Nov 1997 20:56:37 GMT, ddr...@netcom.com (Daniel Druker)
>wrote:
>
>>I am a bit surprised there is very little posting on this
>>thread by anyone other than Dave and myself. Do people out there
>>care or not care about benchmarks ? Are you sick of Dave and I
>>going back and forth or do you find this debate
>>interesting/helpful/amusing ?
>>
>>I hope we are not boring everyone to death or horrifying
>>everyone with our competitive zeal.
>
>No. I, like Steve and a few others, find this exchange very
>interesting. I have to concur with John Durkin on his point..
>
>>However, for all the OLAP evaluations I
>>have been through, we always go back to Essbase, due to its ease of
>>development and superior Excel interface. Don't underestimate the
>>importace to the interface: in the end, users want their data or
>>reports in Excel, and any other tool, OLAP or other, just gets in the
>>way. This is one area where Express has always lagged.
>
>This was one of the main reasons I chose Essbase along with its ease
>of model building and server focus. Response time is important, but
>end-users/business managers do not care whether they have to wait 1
>sec or 5 secs when they are used to waiting minutes with legacy RDBs.
>Point being, does a 5 times slower response time really matter?
>
>My two cents...the OLAP benchmark is fine. Many people will be
>interested to see which companies faired well. These will be the
>companies they will evaluate. However, the software product that
>meets a companies needs, in the end, will win. I only hope more
>companies are smart enough to do their "homework" regarding OLAP
>technologies.
>
>In my case, realizing that each company's requirements are different,
>I asked Oracle and Arbor to build a prototype model to prove that
>their software would work as advertised. Oracle declined. Arbor did
>one in a matter of one and a half days. This is what sells software,
>not figures on a piece of paper.
>
ElkinsEcon
It appears that the combatants have finally tired and this debate is winding
down, so let me offer a few final observations. First, the main things we have
learned from this discussion should not be of any great surprise to serious
students of OLAP.
First, the trade-off between precalculation and query response time has long
been well understood, though never so well quantified. Thus no-one should be
surprised that TM/1 (which performs no pre-aggregation of data), requires less
DASD than Express (which allows partial pre-aggregation), and much less than
Essbase (which, until v 5.0, pre-aggregates everything); or that it stages it's
cube for analysis much more quickly; or that, on the other hand, it's query
response time is much slower (that's the tradeoff).
Second, it is clear that, if one has the opportunity, the aptitude, the
inclination and the time to use a 4GL to tune the design of an OLAP application
(as one can with Express), one can coax improved performance from it. The
tradeoff is a steeper learning curve and, as Fred Hale points out, a slower
speed of deployment.
In my experience, satisfied Express users tend to be programmer types who have
previous experience with 4GLs. Because they are already comfortable working in
a 4GL environment, their learning curve to take effective advantage of the
Express stored procedural language is reasonably short. They are able to make
effective use of these capabilities and they appreciate their value.
Satisfied TM/1 uers tend to be power spreadsheet user types who have grown up
developing complex spreadsheet applications for relatively small user
communities and are grateful for the capability provided by TM/1 to extend the
spreadsheet analysis paradigm to new heights.
Satisfied Essbase users are neither of the above. Like Fred, they just want to
obtain the benefits of OLAP quickly. They appreciate Essbase's ease and speed
of deployment and the ability to quickly and easily extend its analytical
capabilites by just snapping on any of a large selection of Essbase compatible,
complementary third party OLAP client front ends and other DSS products.
To large degree, beauty is in the eye of the beholder: you're going to be
comfortable in a familiar environment.
Steve Elkins
Elkins Analytics
Michael Bowen
In article <19971218191...@ladder02.news.aol.com>, elkin...@aol.com (ElkinsEcon) wrote:
[..]
>Satisfied Essbase users are neither of the above. Like Fred, they just want to
>obtain the benefits of OLAP quickly. They appreciate Essbase's ease and speed
>of deployment and the ability to quickly and easily extend its analytical
>capabilites by just snapping on any of a large selection of Essbase compatible,
>complementary third party OLAP client front ends and other DSS products.
>
>To large degree, beauty is in the eye of the beholder: you're going to be
>comfortable in a familiar environment.
i agree in general, which poses a curious question to me as one who used to be
deep into both 4GLs and spreadsheets. where have all the API programmers in IS
departments gone? independent? commercial front-end tool building?
it's a kind of generic question. right now, being focussed on data
architecture, i haven't had much opportunity to investigate what regular IS
staffs are looking like. if you ignore java and the intranet, is anybody
really building scratch client server for business intelligence any longer? is
cognos, wired and b.o. the answer?
do IS staffs write anything that looks at ERP or DW data from scratch using
Powerbuilder, JYACC, SQLWindows, Delphi, VB? how much market share does oracle
financials have? and what do holos 4GL stuff look like at the end of its
lifespan?
for some reason, i keep getting the impression, that all the world's IS staffs
would rather either 4GL themselves into oblivion or just buy turnkey. am i
hanging out in the wrong cities?
---------------------------------------------------------------------------------------------
michael bowen
standard disclaimers apply
---------------------------------------------------------------------------------------------
John Durkin
I would add that many of the initial Essbase customers were former System W
users who migrated to "client server."
Remember System W?????? OLAP before there was OLAP!
ElkinsEcon wrote in message
<19971218191...@ladder02.news.aol.com>...
>Satisfied Essbase users are neither of the above. Like Fred, they just
want to
>obtain the benefits of OLAP quickly. They appreciate Essbase's ease and
speed
>of deployment and the ability to quickly and easily extend its analytical
>capabilites by just snapping on any of a large selection of Essbase
compatible,
>complementary third party OLAP client front ends and other DSS products.
>
>To large degree, beauty is in the eye of the beholder: you're going to be
>comfortable in a familiar environment.
>
>Steve Elkins
>Elkins Analytics
Allen Worthington
John Durkin <yama...@gol.com> wrote in article
<67ja3c$rdc$1...@godzilla.gol.com>...
> I would add that many of the initial Essbase customers were former System
W
> users who migrated to "client server."
>
>
> Remember System W?????? OLAP before there was OLAP!
Do you mean the good 'ol days when you could actually do goal-seeking in
your "OLAP" environment? In a few cases current OLAP offerings are a step
back. Personally, I wouldn't mind trading a couple of million queries per
day against solid, W-like, goal-seeking functionality.
Allen Worthington
Martin Richmond-Coggan
Steve Elkins writes:
"Thus no-one should be
surprised that TM/1 (which performs no pre-aggregation of data),
requires less
DASD than Express (which allows partial pre-aggregation), and
much less than
Essbase (which, until v 5.0, pre-aggregates everything); or that
it stages it's
cube for analysis much more quickly; or that, on the other hand,
it's query
response time is much slower (that's the tradeoff). "
delivers its responses twice as fast as Essbase - even AFTER
Essbase has been fully pre-calculated?
Martin Richmond-Coggan
Applix TM1
mart...@applix.co.uk
--
Martin Richmond-Coggan Tel: +44(0)117 923 8584
TM1 Software Europe Ltd Fax: +44(0)117 923 7598
http://www.tm1.com 10044...@compuserve.com
************* TM1 - 'INSIGHT ON DEMAND' *****************