APIs make programming easier. Right?

355 views
Skip to first unread message

DrQ

unread,
Apr 19, 2012, 1:57:40 AM4/19/12
to guerrilla-cap...@googlegroups.com
What Is An API & What Are They Good For? 
http://www.makeuseof.com/tag/api-good-technology-explained/


Larry Ellison
(in court this week)
“Arguably, its the most difficult thing we do at Oracle,” Ellison said when asked how hard it is to build an API.

M. Edward (Ed) Borasky

unread,
Apr 19, 2012, 2:42:21 AM4/19/12
to guerrilla-cap...@googlegroups.com

An API is a fancy name for a library, right? ;-)


--
Twitter: http://twitter.com/znmeb Computational Journalism Server
http://j.mp/compjournoserver

Data is the new coal - abundant, dirty and difficult to mine.

James Newsom

unread,
Apr 19, 2012, 8:15:40 AM4/19/12
to guerrilla-cap...@googlegroups.com
On 4/19/2012 1:42 AM, M. Edward (Ed) Borasky wrote:
On Wed, Apr 18, 2012 at 10:57 PM, DrQ <red...@yahoo.com> wrote:
What Is An API & What Are They Good For?
http://www.makeuseof.com/tag/api-good-technology-explained/


Larry Ellison (in court this week)
�Arguably, its the most difficult thing we do at Oracle,� Ellison said when
asked how hard it is to build an API.
http://www.wired.com/wiredenterprise/2012/04/ellison-page/


Charlie Kindel

"Don�t build APIs"
http://ceklog.kindel.com/2012/04/18/dont-build-apis/
An API is a fancy name for a library, right? ;-)


I think what Larry was trying to say is that the hardest thing they do is write API's that allow them to maximize revenue with F.U. licensing schemes. :-)

Mario François Jauvin

unread,
Apr 19, 2012, 7:08:01 AM4/19/12
to <guerrilla-capacity-planning@googlegroups.com>
> Data is the new coal - abundant, dirty and difficult to mine.

Information is like nuclear fusion - hard to create and easy to consume

Mario François Jauvin
MFJ Associates, (613) 686-5130, option 1
Sent from my iPhone

> --
> You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
> To post to this group, send email to guerrilla-cap...@googlegroups.com.
> To unsubscribe from this group, send email to guerrilla-capacity-...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/guerrilla-capacity-planning?hl=en.
>

Mario François Jauvin

unread,
Apr 19, 2012, 7:03:45 AM4/19/12
to <guerrilla-capacity-planning@googlegroups.com>
An API make a component (library, web service) easier to use. That in itself will help us write programs that use these components.

Mario François Jauvin
MFJ Associates
Sent from my iPhone

On 2012-04-19, at 2:44 AM, "M. Edward (Ed) Borasky" <zn...@znmeb.net> wrote:

rml...@gmail.com

unread,
Apr 19, 2012, 5:42:54 PM4/19/12
to Guerrilla Capacity Planning
Since we are concerned with performance, I will interject that there
are several considerations that should be documented in any API.
1. APIs should publish performance characteristics, especially if they
are intended for server side use:
a. throughput, service times, response times, and utilizations
under increasing load for hardware components [CPU, disk, NIC, etc.]
on defined hardware assets without stubbing out services.
b. document the tested commands
c. plot of numer of threads (x-axis) and throughput, response time,
and utilization (y-axis)
d. plot of throughput (x-axis) and response time (y-axis)
2. Users of APIs need to know functional and logical aspects of the
API:
a. intra or inter process/thread/host/etc. communications?
b. synchronous or asyncronous with respect to what?
c. limitations of the API (e.g., 32 bit-ness limits memory address
space)
d. threadsafe?
e. side effects and responsibilities?
f. inputs, output on success/failure, faults and error conditions.
g. are operations expected to be idempotent/ACID? What is the
consistency model?
h. stateful or stateless?
i. what security issues exist? (e.g., password exchange, possible
sources of data loss, etc.)
j. what are the downstream dependencies and infrastructure?

Regards,
Bob

On Apr 18, 9:57 pm, DrQ <redr...@yahoo.com> wrote:
> What Is An API & What Are They Good For?<http://www.makeuseof.com/tag/api-good-technology-explained/>
>   <http://www.makeuseof.com/tag/api-good-technology-explained/>http://www.makeuseof.com/tag/api-good-technology-explained/
> *
>
> Larry Ellison *(in court this week)
> “Arguably, its the most difficult thing we do at Oracle,” Ellison said when
> asked how hard it is to build an API.http://www.wired.com/wiredenterprise/2012/04/ellison-page/
>
> *Charlie Kindel*

Scott Johnson

unread,
Apr 19, 2012, 7:38:00 PM4/19/12
to guerrilla-cap...@googlegroups.com
There should always be two sets of APIs.  One high performing set for internal organizational use.  A second, poorly performing set for external users and customers.  These should also have undocumented features, poorly documented features, and inconsistent approaches in specifying the calls.

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.
To post to this group, send email to guerrilla-cap...@googlegroups.com.
To unsubscribe from this group, send email to guerrilla-capacity-...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/guerrilla-capacity-planning?hl=en.




--
Regards,

Scott Johnson

706-414-1564 Work
scott....@wwt.com
j...@ieee.org

Greg Hunt

unread,
Apr 20, 2012, 8:02:59 AM4/20/12
to guerrilla-cap...@googlegroups.com
Bob,
I suspect your item 1 will be difficult to usefully generalise.  It is very hard comparing, for example, a Sun multi-threaded and machine with a randomly chosen Intel server, let alone comparing one random Intel server with another.  How many combinations of models of i3, i5, i7 and Xeon processors are there with combinations of chipsets, motherboard designs and memory?  Then we move on to the network and the disk.  I have been torturing myself with disk characteristics lately: OS-level IO scheduler, SCSI interface provider, SAN fabric, storage array characteristics.  When all that is specified how much do we actually know?  

Greg

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

Andrew Sliwkowski

unread,
Apr 20, 2012, 7:16:26 AM4/20/12
to guerrilla-cap...@googlegroups.com
DrQ,
Great question...I read the pros and cons ... a  had a LOLm with from the developer from microsoft...
The powerpoint slide deck from the google principle engineer (all due respect) .... that the reasons API's are so hard ... is there are so many rules...(based on this powerpoint) ...
but I guess they have to ..especially when an API is for communicating from one language to another.... Remote API ....  

Lately I have been impressed with windows powershell ... it is kind of an api to get the information I want.... The tab completion and good IDE makes a world of difference...

A good API is like a musical instrument ... there is no limit to how fluent you can get...

PDQ is a great example of great API .... There are so many good working examples ...that it allows the API user to give feedback quickly to API designer .... My intuition says tells me a fusion of lego component(drag and drop) approach with cross-cutting concerns of AOP.... grounded in test driven design....   But most importantly ... the ability to take a Natural language requirement and convert into an API called that delivers.....   wolfram alfa comes to mind ...

My top 5 API's would be:
1. Loadrunner web API
2. PDQ
3. QTP visual basic API
4. R module API
5. Weblogic JMX Api

Least liked API:
.... hmmm

--
You received this message because you are subscribed to the Google Groups "Guerrilla Capacity Planning" group.

rml...@gmail.com

unread,
Apr 20, 2012, 4:21:09 PM4/20/12
to Guerrilla Capacity Planning
Greg---

There are well understood back of envelope calculations that one can
make to approximate workload performance from one architecture to
another. Depending on the methodology used (e.g., using published
benchmarks in the same technology domain as the API [e.g., Java) using
source and target architectures, operating systems, et cetera) a
knowledgeable performance analyst can create a conversion factor that
captures first order effects (and probably more accuracy). The details
of the conversion factor can even include such things as CPU:bus ratio
but for most purposes this is unnecessary. There are well known
factors that can be applied to changes in L1 and L2 caches sizes, etc.
The process of creating conversion factors is widely used among system
vendors to compare current and project future performance. This
subject is probably beyond the scope of this discussion. Dr. Q can
probably include a chapter on the approaches, methods, and caveats
with worked examples in his next book that we are all probably eager
to buy.

Bob

Alexander Podelko

unread,
Apr 20, 2012, 4:24:26 PM4/20/12
to guerrilla-cap...@googlegroups.com
Actually it is a smaller issue with 1. At least theoretically it is possible to normalize - and most commercial vendors have some mechanisms of cross-platform performance analysis (what-if scenarios). How well - that is another story, it is indeed challenging. More speculations about the subject http://applicationperformanceengineeringhub.com/how-do-we-measure-computer-resources/

A more challenging issue is that API performance depends on arguments. For example, you have a call returning all people working in a department - and it may be one person, or the whole corporation. Or a call executing an sql query (jdbc) - it may be a simple query or multiple joins of huge tables. And if in the first case you may try to bind performance to the number of people returned (set performance for specific numbers of people returned - but the question is for what levels), in the second case you can't bind it to anything - whatever formal parameters of sql query you choose (number of tables to join, number of record in the tables, number of records returned), they don't quite define the query performance. And this is a huge challenge if you are trying to define performance requirements.

Regards,

Alex


From: Greg Hunt <gr...@firmansyah.com>
To: guerrilla-cap...@googlegroups.com
Sent: Friday, April 20, 2012 8:02 AM
Subject: Re: APIs make programming easier. Right?

Greg Hunt

unread,
Apr 20, 2012, 5:27:47 PM4/20/12
to guerrilla-cap...@googlegroups.com
Bob,
Thanks but I have actually done it.  My point was that there are an awful lot of factors to consider and an awful lot of effort to get a very rough conversion.  For example you refer to using Java benchmarks for scaling, but Java benchmarks are usually of limited use for comparison if the application's workload characteristics, heap usage, JDK version and GC strategy are not roughly comparable with the benchmark.  This week I have seen a 75% reduction in the elapsed time of a process from switching between two apparently production ready SCSI interfaces from different vendors (I knew there would be an effect but the vendor's published material suggested that it would not be that big).   Its not that its impossible, its that it takes a lot of effort to get a useful and reliable conversion.  

Greg

steve jenkin

unread,
Apr 20, 2012, 10:32:46 PM4/20/12
to guerrilla-cap...@googlegroups.com
Bob,

Recently I went looking for good references on CPU cache-sizing and
didn't find anything I could really use [in the context of 3 decades of
a ~20%/yr increase in CPU cycle times vs DRAM latency].

Do you have references, or more usefully, search terms?

My intuition is, with a increase of CPU/DRAM latency ratio of 'n':
- to maintain the same "cache hit-ratio", last-level cache has to scale
by 'n', and
- to maintain the same effective CPU performance (% of all-in-cache),
the cache hit-ratio needs to increase, but I failed at identifying that
formula [probably staring me in the face :-(].

My thought is that as the latency differential between CPU/DRAM
increases, the proportion of on-chip area dedicated to cache for CPU's
is growing super-linearly... But without having any formulas, that's a
very weak statement.

Thanks in advance.

steve jenkin

rml...@gmail.com wrote on 21/04/12 6:21 AM:


> There are well known
> factors that can be applied to changes in L1 and L2 caches sizes, etc


--
Steve Jenkin, Info Tech, Systems and Design Specialist.
0412 786 915 (+61 412 786 915)
PO Box 48, Kippax ACT 2615, AUSTRALIA

stev...@gmail.com http://members.tip.net.au/~sjenkin

Dr Carlo Kopp, PEng

unread,
Apr 21, 2012, 4:49:00 AM4/21/12
to guerrilla-cap...@googlegroups.com

Steve,

There are standard formulas for cache performance, but you need specific knowledge of the L1 and L2 architecture to use these, ie split / common, size, write-through vs write-back etc. This is not always disclosed by manufacturers in its entirety. Moreover, in a multicore CPU you usually have consistency mechanisms in operation to resolve potential inconsistency, and these may also impact cache performance.

Most CPUs lacked a cache trace mechanism so you needed a CPU specific emulator which you run with the application of interest and this logs the cache behaviour for that application, so that you could look at cache occupancy and perform optimisations.

Usually you will find that increasing cache size beyond typical loop span in an applications yields no further gains, and much the same happens with increasing set associativity beyond 4X.

When I taught undergraduate machine architecture, before the courses got dumbed down, we used to have the students run lab simulations of cache behaviour to determine the best choices. This involved trace runs captured for a specific application.

Without intimate knowledge of the behaviour (spatial/temporal locality) of the target application and the cache architecture itself this can be an intractable problem.

Cordially,

Carlo

PS This is a lab exercise I designed in 2001-2002 - http://www.csse.monash.edu.au/~davida/teaching/cse2324/Pracs/prac5.html

On 21/04/12 2:32 AM, steve jenkin wrote:
Bob,

Recently I went looking for good references on CPU cache-sizing and
didn't find anything I could really use [in the context of 3 decades of
a ~20%/yr increase in CPU cycle times vs DRAM latency].

Do you have references, or more usefully, search terms?

My intuition is, with a increase of CPU/DRAM latency ratio of 'n':
 - to maintain the same "cache hit-ratio", last-level cache has to scale
by 'n', and
 - to maintain the same effective CPU performance (% of all-in-cache),
the cache hit-ratio needs to increase, but I failed at identifying that
formula [probably staring me in the face :-(].

My thought is that as the latency differential between CPU/DRAM
increases, the proportion of on-chip area dedicated to cache for CPU's
is growing super-linearly... But without having any formulas, that's a
very weak statement.

Thanks in advance.

steve jenkin

rml...@gmail.com wrote on 21/04/12 6:21 AM:
There are well known
factors that can be applied to changes in L1 and L2 caches sizes, etc


--


Dr Carlo Kopp, Associate Fellow AIAA, Senior Member IEEE, PEng
Computer Scientist
Email:   Carlo...@monash.edu
W3:      
http://www.csse.monash.edu.au/~carlo
Ph:       +61-3-9905-5229
Cell:     +61-437-478-224

Clayton School of Information Technology, Faculty of Information Technology
Monash University, Clayton, 3800, AUSTRALIA
Monash Provider No. 00008C
PRIVACY, CONFIDENTIALITY, DISCLAIMER AND COPYRIGHT NOTICE © 2007 - 2010
This communication is copyright and intended for the named recipient/s only. Being an addressee on this Email does not imply and should not be inferred, in any way, as meaning the addressee endorses or agrees with its contents. The contents of this  document, including any attachments, should not be copied, distributed, or disclosed to any third party person or organisation without written permission. If received in error or incorrectly onforwarded to you, kindly notify the sender by reply E-mail and permanently delete this message and its attachments.

steve jenkin

unread,
Apr 21, 2012, 11:50:44 AM4/21/12
to guerrilla-cap...@googlegroups.com
Dr Carlo Kopp, PEng wrote on 21/04/12 6:49 PM:

Without intimate knowledge of the behaviour (spatial/temporal locality) of the target application and the cache architecture itself this can be an intractable problem.
Carlo,

Thanks very much. I think that says what I already knew...

cheers
steve

Darryl Gove

unread,
Apr 21, 2012, 2:28:57 PM4/21/12
to guerrilla-cap...@googlegroups.com, steve jenkin
You can probably extract the cache size data you need from wikipedia.
https://en.wikipedia.org/wiki/Intel_processor

For x86 you'll probably find the trend you're looking for - early x86
processors had little cache more recent ones have much larger cache. You
also need to look at cache/thread or /core.

If you look at server processors (SPARC, Power etc.) I think you'll find
the trend is less clear. Server processors have had significant cache
for longer, so I don't think you'll find the same growth.

I suspect that the limitation is more to do with process technology than
any equation that governs the ideal amount of cache. I would imagine
that the argument goes "We need to fit two cores onto this die, and then
what ever is left we'll turn into cache.".

Regards,

Darryl.


On 4/20/2012 7:32 PM, steve jenkin wrote:
> Bob,
>
> Recently I went looking for good references on CPU cache-sizing and
> didn't find anything I could really use [in the context of 3 decades of
> a ~20%/yr increase in CPU cycle times vs DRAM latency].
>
> Do you have references, or more usefully, search terms?
>
> My intuition is, with a increase of CPU/DRAM latency ratio of 'n':
> - to maintain the same "cache hit-ratio", last-level cache has to scale
> by 'n', and
> - to maintain the same effective CPU performance (% of all-in-cache),
> the cache hit-ratio needs to increase, but I failed at identifying that
> formula [probably staring me in the face :-(].
>
> My thought is that as the latency differential between CPU/DRAM
> increases, the proportion of on-chip area dedicated to cache for CPU's
> is growing super-linearly... But without having any formulas, that's a
> very weak statement.
>
> Thanks in advance.
>
> steve jenkin
>
> rml...@gmail.com wrote on 21/04/12 6:21 AM:
>> There are well known
>> factors that can be applied to changes in L1 and L2 caches sizes, etc
>
>


--
http://www.darrylgove.com/

Darryl Gove

unread,
Apr 21, 2012, 2:38:19 PM4/21/12
to guerrilla-cap...@googlegroups.com
On 4/21/2012 1:49 AM, Dr Carlo Kopp, PEng wrote:
>
> Steve,
>
> There are standard formulas for cache performance, but you need specific
> knowledge of the L1 and L2 architecture to use these, ie split / common,
> size, write-through vs write-back etc. This is not always disclosed by
> manufacturers in its entirety. Moreover, in a multicore CPU you usually
> have consistency mechanisms in operation to resolve potential
> inconsistency, and these may also impact cache performance.
>
> Most CPUs lacked a cache trace mechanism so you needed a CPU specific
> emulator which you run with the application of interest and this logs
> the cache behaviour for that application, so that you could look at
> cache occupancy and perform optimisations.
>
> Usually you will find that increasing cache size beyond typical loop
> span in an applications yields no further gains, and much the same
> happens with increasing set associativity beyond 4X.

This is basically it.

If you take an HPC (loopy) application, then you iterate over a block of
data, using each item once. If the block fits in cache you get a nice
performance benefit, if it doesn't then you stream from memory.

If you have a random access application, then you still have a block of
data, you get the best performance if the cache is large enough to fit
the entire block of data. However, since it's random access, you get a
curve as the cache size increases to the size of the working set.

You get a performance benefit if you are cache resident. What's more
"concerning" is that, at some point, for some (typically hpc)
applications, a slight increase in the size of the workload can cause it
to no longer be cache resident, and for performance to fall off a cliff.

A few years back I did some work looking at the working set size of the
cpu2006 benchmarks. This is not about cache sizing, more about figuring
out how much memory each code touched.

http://www.spec.org/cpu2006/publications/SIGARCH-2007-03/05_cpu2006_wss.pdf

Regards,

Darryl.

> ------------------------------------------------------------------------


> Dr Carlo Kopp, Associate Fellow AIAA, Senior Member IEEE, PEng
> Computer Scientist

> Email: Carlo...@monash.edu <mailto:Carlo...@monash.edu>
> W3: http://www.csse.monash.edu.au/~carlo
> <http://www.csse.monash.edu.au/%7Ecarlo>
> Ph: +61-3-9905-5229
> Cell: +61-437-478-224

> Clayton School of Information Technology, Faculty of Information Technology
> Monash University, Clayton, 3800, AUSTRALIA
> Monash Provider No. 00008C
> PRIVACY, CONFIDENTIALITY, DISCLAIMER AND COPYRIGHT NOTICE � 2007 - 2010
> This communication is copyright and intended for the named recipient/s
> only. Being an addressee on this Email does not imply and should not be
> inferred, in any way, as meaning the addressee endorses or agrees with
> its contents. The contents of this document, including any attachments,
> should not be copied, distributed, or disclosed to any third party
> person or organisation without written permission. If received in error
> or incorrectly onforwarded to you, kindly notify the sender by reply
> E-mail and permanently delete this message and its attachments.
>

> --
> You received this message because you are subscribed to the Google
> Groups "Guerrilla Capacity Planning" group.
> To post to this group, send email to
> guerrilla-cap...@googlegroups.com.
> To unsubscribe from this group, send email to
> guerrilla-capacity-...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/guerrilla-capacity-planning?hl=en.


--
http://www.darrylgove.com/

steve jenkin

unread,
Apr 21, 2012, 7:57:24 PM4/21/12
to guerrilla-cap...@googlegroups.com
Darryl Gove wrote on 22/04/12 4:38 AM:

> A few years back I did some work looking at the working set size of
> the cpu2006 benchmarks. This is not about cache sizing, more about
> figuring out how much memory each code touched.

My interest in the scaling of cache has to do with the upper bound of
current Silicon CMOS technology.
It could be as low as 4-10 times current size.

The maximum L3 cache size I've seen is ~1B transistors, around 24Mb.

At 64-200Mb, is that large enough to be treated as a Virtual Memory
system and managed as such with working sets?

So there'd be two levels of working memory (fast on-chip, slow RAM) and
"backing-store" on Flash/SCM/Disk.
SCM = Storage Class Memory. Things like spin torque memory.

Reply all
Reply to author
Forward
0 new messages