Which language for commercial project?
Joachim Durchholz
I'm going to choose the language for a commercial project. (In other
words: here's your chance to get commercial attention for your favourite
language, guys!)
Here's the list of criteria, in roughly decreasing order of relevance:
1. Maturity
The software will be mission-critical. If the software fails, the
business will fail. (There's also some of my professional standing at
stake, so please be careful in what you recommend and why!)
To the very least, the compiler and any other generating tools must be
rock-solid, able to deliver software that's ready for 24/7 performance.
If the compiler isn't quite there yet, this level of quality should be
within, say, a year's reach.
(Quality of other tools is part of requirement 2.)
2. Fun
It should be "fun" to work with the language.
This means: nonobtrusive syntax, up-to-date documentation on tools and
libraries, useful debuggers and profilers, garbage collection, security
from buffer overruns, maybe some other things.
This is difficult to assess, and most people adapt to strange things
anyway (witness the popularity of emacs *g*), so any language that has
advocates qualifies on this account :-)
3. Network transparency
Code shouldn't care whether an object that it's accessing is local or
remote.
If a design decision is changed so that formerly-local data becomes
remote, this must not affect more than a small, fixed number of places,
such as: a) when creating the root object; b) to handle network-related
failure modes (failure response will usually be "restart the entire
computation on a different machine, or locally").
4. Object (un-)marshalling
Moving an object with all referred-to subobjects should not require
writing an extra wrapper for every potentially referred-to object type.
Likewise, receiving an object through the network should not require
extra wrappers for unmarshalling.
5. Object persistency
It should be possible to write an object with all referred-to subobjects
to a sequential file, and to retrieve them back. This facility is not
needed during normal operation (the application will run from RAM in its
entirety), it's just for resuming operation after an expected or
unexpected interruption.
IOW it need not be fast, but it shouldn't require extra programmer
attention.
RDBMS support is irrelevant for this project.
6. Heavy-duty data structure libraries
Servers will keep gigabytes of data in RAM, and this data will live in
some dictionary objects. This means that the data structures that
implement these dictionaries will have to handle millions of keys with a
potential total size of dozens of megabytes, without suffering a nervous
breakdown.
6. FFI
A good and easy-to-use foreign function interface is almost a must.
Sometimes it's far easier to use an existing library or OS facility than
to reinvent the wheel.
7. Platforms
- Windows is a must.
- Linux would be a very strong plus.
- MacOS would be an added bonus.
Multi-platform support will not be helpful unless:
* relevant libraries are available for all supported platforms, and
* processes can exchange data even if running on different platforms.
8. Graphics
The project needs good 2D and/or 3D graphics support. A wrapper for
OpenGL or DirectX would fit the bill (OpenGL preferred due to
portability concerns).
9. GUI
A good, platform-independent GUI library would be a bonus.
(A GTK wrapper should work OK.)
I'm aware that this is a long list, and that it's unlikely that there is
a language and implementation that will satisfy all criteria. Life isn't
perfect, and I'm striving to do something interesting and worthwhile
even in an imperfect world :-)
Opinions, ideas, comments on criteria that I might have overlooked?
Please post, these all are highly appreciated!
Oh, and if the project makes it, it will have at least one open position
for a programmer in the chosen language and environment. It it all works
out in a timely manner, this will even happen within the year (fingers
crossed) :-)
Regards,
Jo
--
Didn't yet find a job, now preparing for a start-up :-)
Yoann Padioleau
I dont know erlang, but it seems close to what you want.
perhaps jcaml too.
and obviously Perl :)
--
Yoann Padioleau, INSA de Rennes, France,
Opinions expressed here are only mine. Je n'écris qu'à titre personnel.
**____ Get Free. Be Smart. Simply use Linux and Free Software. ____**
Basile STARYNKEVITCH
[I replied privately, suggesting Ocaml; here are the details]
See http://caml.org/ or http://caml.inria.fr/ about Ocaml
Joachim> Hi all, I'm going to choose the language for a commercial
Joachim> project. (In other words: here's your chance to get
Joachim> commercial attention for your favourite language, guys!)
Joachim> Here's the list of criteria, in roughly decreasing order
Joachim> of relevance:
Joachim> 1. Maturity The software will be mission-critical. If the
Joachim> software fails, the business will fail. [...]
There are some [few] companies developing business-critical software
in Ocaml. In particular, Lexifi and the other members of Ocaml
consortium.
Ocaml is currently developped by a team at INRIA (a french government
institution) which is currently one of the very few French research
agencies with a growing budget (the overall public R&D budget is
currently decreasing in France).
Ocaml is increasingly used, even in the US.
Joachim> 2. Fun It should be "fun" to work with the
Joachim> language. [...]
Definitely OCaml is fun. It is a statically typed eager functional
language of the ML family. It has interesting modules and functors,
and objects.
Joachim> 3. Network transparency Code shouldn't care whether an
Joachim> object that it's accessing is local or remote. If a
Joachim> design decision is changed so that formerly-local data
Joachim> becomes remote, this must not affect more than a small,
Joachim> fixed number of places, such as: a) when creating the
Joachim> root object; b) to handle network-related failure modes
Joachim> (failure response will usually be "restart the entire
Joachim> computation on a different machine, or locally").
Ocaml don't provide all of this out of the box, but you can carefully
code to have it. Some Ocaml software (Ensemble, Jocaml) might help.
Joachim> 4. Object (un-)marshalling Moving an object with all
Joachim> referred-to subobjects should not require writing an
Joachim> extra wrapper for every potentially referred-to object
Joachim> type. Likewise, receiving an object through the network
Joachim> should not require extra wrappers for unmarshalling.
There is a marshalling facility (dealing with arbitrary, including
circular, reference graphs) in the Marshal module.
Joachim> 5. Object persistency It should be possible to write an
Joachim> object with all referred-to subobjects to a sequential
Joachim> file, and to retrieve them back. This facility is not
Joachim> needed during normal operation (the application will run
Joachim> from RAM in its entirety), it's just for resuming
Joachim> operation after an expected or unexpected interruption.
Ocaml marshalling facility can provide the crude persistency you want.
Joachim> IOW it need not be fast, but it shouldn't require extra
Joachim> programmer attention. RDBMS support is irrelevant for
Joachim> this project.
Joachim> 6. Heavy-duty data structure libraries Servers will keep
Joachim> gigabytes of data in RAM, and this data will live in some
Joachim> dictionary objects. This means that the data structures
Joachim> that implement these dictionaries will have to handle
Joachim> millions of keys with a potential total size of dozens of
Joachim> megabytes, without suffering a nervous breakdown.
I don't know about this item, but Ocaml memory management (in
particular GC) is reputed for its performance. I never had access to a
multi-gigabyte RAM machine so can't tell.
Joachim> 6. FFI A good and easy-to-use foreign function interface
Joachim> is almost a must. Sometimes it's far easier to use an
Joachim> existing library or OS facility than to reinvent the
Joachim> wheel.
Ocaml C interface is a good and easy to use FFI machinery.
Joachim> 7. Platforms - Windows is a must. - Linux would be a
Joachim> very strong plus. - MacOS would be an added bonus.
Ocaml runs on all these plateform. Development is easier under Linux
that under Windows (because the debugger, able to travel back&forth in
time, heavily uses Unix primitives [fork, socket]).
Joachim> Multi-platform support will not be helpful unless: *
Joachim> relevant libraries are available for all supported
Joachim> platforms, and * processes can exchange data even if
Joachim> running on different platforms.
Processes can exchange data on different platform because the
marshalling format is neutral.
Joachim> 8. Graphics The project needs good 2D and/or 3D graphics
Joachim> support. A wrapper for OpenGL or DirectX would fit the
Joachim> bill (OpenGL preferred due to portability concerns).
Probably available.
Joachim> 9. GUI A good, platform-independent GUI library would be
Joachim> a bonus. (A GTK wrapper should work OK.)
Available (both TK & GTK interfaces).
Joachim> I'm aware that this is a long list, and that it's
Joachim> unlikely that there is a language and implementation that
Joachim> will satisfy all criteria. Life isn't perfect, and I'm
Joachim> striving to do something interesting and worthwhile even
Joachim> in an imperfect world :-)
--
Basile STARYNKEVITCH http://starynkevitch.net/Basile/
email: basile<at>starynkevitch<dot>net
aliases: basile<at>tunes<dot>org = bstarynk<at>nerim<dot>net
8, rue de la Faïencerie, 92340 Bourg La Reine, France
Garry Hodgson
> I'm going to choose the language for a commercial project.
> Here's the list of criteria, in roughly decreasing order of relevance:
>
> 1. Maturity
> 2. Fun
> 3. Network transparency
> 4. Object (un-)marshalling
> 5. Object persistency
> 6. Heavy-duty data structure libraries
> 6. FFI
> 7. Platforms
> 8. Graphics
> 9. GUI
> A good, platform-independent GUI library would be a bonus.
> Opinions, ideas, comments on criteria that I might have overlooked?
> Please post, these all are highly appreciated!
no doubt others can fill you in on the details with specific answers.
but hands down, no contest, the answer is erlang.
this is primarily due to your mission critical requirement. while many other
languages would be just fine for many of your requirements, no one
has thought through the issues and provided supporting libraries
and infrastructure like the OTP part of erlang.
----
Garry Hodgson, Senior Hacker, AT&T Labs
No act is more patriotic than speaking out when your government
is doing the wrong thing in your name. This is not your right
but your sacred duty. And none are more treasonous than those
who would silence these voices.
Christian Szegedy
for OCaml, but all other points fit very well to OCaml.
So, I would say, it is the best choice.
Another serious candidate is clean, but I don't know
it too well, so can't really say, whether it fits all
your points. The only think I know that its GUI
interface seems to work only under Windows (at least
I could not get it work under three different Linux distros,
all on different boxes).
Simon Helsen
>I don't know whether there is an OpenGL support
>for OCaml, but all other points fit very well to OCaml.
Simon Helsen
oh, and there is
http://sourceforge.net/projects/camlgl/
and also relevant might be
http://ocamlsdl.sourceforge.net/home.html
Anyways, if you want to know what Ocaml offers, then look in the hump:
http://pauillac.inria.fr/caml/humps/index_framed_caml.html
Regards,
Simon
Neelakantan Krishnaswami
> Joachim Durchholz <joac...@gmx.de> wrote:
>
> > I'm going to choose the language for a commercial project.
>
> > Here's the list of criteria, in roughly decreasing order of relevance:
> >
> > 1. Maturity
> > 2. Fun
> > 3. Network transparency
> > 4. Object (un-)marshalling
> > 5. Object persistency
> > 6. Heavy-duty data structure libraries
> > 6. FFI
> > 7. Platforms
> > 8. Graphics
> > 9. GUI
> > A good, platform-independent GUI library would be a bonus.
>
> > Opinions, ideas, comments on criteria that I might have overlooked?
> > Please post, these all are highly appreciated!
>
> no doubt others can fill you in on the details with specific answers.
> but hands down, no contest, the answer is erlang.
I would agree with Erlang, but I think it only just edges out Ocaml.
It's #3 that pushes it to Erlang. In most other respects I think
Ocaml is a little superior, but I don't think there is any other
language with concurrency support remotely as good as Erlang's.
--
Neel Krishnaswami
ne...@alum.mit.edu
Garry Hodgson
> Garry Hodgson <ga...@sage.att.com> wrote:
> > Joachim Durchholz <joac...@gmx.de> wrote:
> >
> > > I'm going to choose the language for a commercial project.
> > > Here's the list of criteria, in roughly decreasing order of relevance:
> > >
> > > 1. Maturity
> > > 2. Fun
> > > 3. Network transparency
> > > 4. Object (un-)marshalling
> > > 5. Object persistency
> > > 6. Heavy-duty data structure libraries
> > > 6. FFI
> > > 7. Platforms
> > > 8. Graphics
> > > 9. GUI
> > > A good, platform-independent GUI library would be a bonus.
>>
> > no doubt others can fill you in on the details with specific answers.
> > but hands down, no contest, the answer is erlang.
>
> I would agree with Erlang, but I think it only just edges out Ocaml.
> It's #3 that pushes it to Erlang. In most other respects I think
> Ocaml is a little superior, but I don't think there is any other
> language with concurrency support remotely as good as Erlang's.
i don't dispute this. i'm also rather fond of ocaml. i just think that
for this kind of application, the value of all that OAM infrastructure
vastly outweighs the language issues.
M H
> Hi all,
>
> I'm going to choose the language for a commercial project. (In other
> words: here's your chance to get commercial attention for your favourite
> language, guys!)
>
> Here's the list of criteria, in roughly decreasing order of relevance:
Which languages do you already know? This is an important piece of
information you forgot to mention in your posting...
Matthias
Ulf Wiger
> Here's the list of criteria, in roughly decreasing order of relevance:
>
> 1. Maturity
> The software will be mission-critical. If the software fails, the
> business will fail. (There's also some of my professional standing at
> stake, so please be careful in what you recommend and why!)
>
> To the very least, the compiler and any other generating tools must be
> rock-solid, able to deliver software that's ready for 24/7
> performance. If the compiler isn't quite there yet, this level of
> quality should be within, say, a year's reach.
> (Quality of other tools is part of requirement 2.)
Erlang has been running in mission-critical applications for almost
10 years, with e.g. the AXD 301 hitting the market in 1998. Two of
Ericsson's most important products, the AXD 301 and the GPRS node,
run Erlang. The transit switches in British Telecom's telephony
network are erlang-based, and show availability figures that are
better than "six nines" (99,9999%); this is the largest Voice over
ATM network in the world.
I have also heard that the U.S. DoD and the IRS rely on Erlang-based
systems from Nortel for handling their SSL traffic.
I'd say that Erlang covers this point pretty well.
The Erlang/OTP team has an excellent track record supporting
Erlang for mission critical applications since 1996. They
have never missed a deadline, and their software runs 24/7
in literally thousands of locations.
> 2. Fun
> It should be "fun" to work with the language.
> This means: nonobtrusive syntax, up-to-date documentation on tools and
> libraries, useful debuggers and profilers, garbage collection,
> security from buffer overruns, maybe some other things.
> This is difficult to assess, and most people adapt to strange things
> anyway (witness the popularity of emacs *g*), so any language that has
> advocates qualifies on this account :-)
Searching for the word "fun" on the Erlang mailing list yields
8530 messages -- 'nough said... except perhaps that "fun" is a
keyword in the Erlang language, which may account for some of
the hits. ;-)
Of course, Erlang is fun. Erlang also has outstanding tracing and
profiling capabilities, and has paid much attention to effective
fault localization (which is vital in mission-critical systems.)
Tools have been made using Erlang's tracing support that export
trace data to graphing tools or even model checkers.
(http://www.erlang.se/workshop/2002/Fredlund.pdf)
You can turn on tracing selectively per function (even adding
conditional guards on the function parameters). Trace data can
be handled within the node, be output via a TCP port, or written
to a log file.
Whether this qualifies as "fun" depends on your frame of
mind, but especially our testers (some of whom have worked
on Ericsson's venerable AXE switch - famous for its
powerful testing support) swear by Erlang
(see e.g. "Use of Erlang in System Test of AXD 301,
http://www.erlang.se/workshop/2002/Wiger.ppt)
The Erlang mailing list often receives high praise for its
positive tone and helpful experts, many of whom actually
build mission-critical systems for a living.
> 3. Network transparency
> Code shouldn't care whether an object that it's accessing is local or
> remote.
> If a design decision is changed so that formerly-local data becomes
> remote, this must not affect more than a small, fixed number of
> places, such as: a) when creating the root object; b) to handle
> network-related failure modes (failure response will usually be
> "restart the entire computation on a different machine, or
> locally").
This is where Erlang _really_ shines.
> 4. Object (un-)marshalling
> Moving an object with all referred-to subobjects should not require
> writing an extra wrapper for every potentially referred-to object type.
> Likewise, receiving an object through the network should not require
> extra wrappers for unmarshalling.
Message passing in Erlang is distribution transparent. There is no
need for any marshalling or unmarshalling.
> 5. Object persistency
> It should be possible to write an object with all referred-to
> subobjects to a sequential file, and to retrieve them back.
Yes, of course you can do this, e.g. with
file:write_file(File, term_to_binary(Data)).
and then read the same data back with
{ok, Binary} = file:read_file(File),
binary_to_term(Binary).
There are also other ways, disk-based hash tables, etc.
It's e.g. quite common to simply pretty-print data structures
to a text file, which can then be read and parsed using the
function file:consult(File).
> This
> facility is not needed during normal operation (the application will
> run from RAM in its entirety), it's just for resuming operation after
> an expected or unexpected interruption.
> IOW it need not be fast, but it shouldn't require extra programmer
> attention.
> RDBMS support is irrelevant for this project.
Understood, but you would probably still want to take a look at the
Mnesia DBMS, since it is exceptionally well integrated with the
Erlang runtime system and language. There is no semantic gap, and
you get persistence, distribution transparency, etc. Since Mnesia
is at heart a RAM-based DBMS, you may want to use it for your RAM-
based data as well, and simply flag some tables as persistent.
As a curiosity perhaps, Mnesia also supports table fragmentation,
so if your dictionary grows beyond your RAM budged, you can break
it into multiple fragments and create a multi-node database cluster.
Your application does not have to be modified, as Mnesia
provides location transparency.
The Mnesia fragmentation support has demonstrated linear
scalability up to at least 50 nodes (that was about as many
workstations that could be hijacked for the scalability test),
so you shouldn't have to worry about your application outgrowing
Erlang's capabilities.
> 6. Heavy-duty data structure libraries
> Servers will keep gigabytes of data in RAM, and this data will live in
> some dictionary objects. This means that the data structures that
> implement these dictionaries will have to handle millions of keys with
> a potential total size of dozens of megabytes, without suffering a
> nervous breakdown.
Erlang Term Storage, or ETS, is what's at the heart of Mnesia, but
you can use ETS tables directly too. I have a server of my own that
keeps millions of objects in hash tables and Btrees. It works very
well. Currently, Erlang is limited to 4 GB of addressable memory,
but there is a 64-bit version on its way, if this is not enough
for you.
> 6. FFI
> A good and easy-to-use foreign function interface is almost a must.
> Sometimes it's far easier to use an existing library or OS facility
> than to reinvent the wheel.
Erlang interfaces to other languages via sockets, pipes, or linked-in
drivers -- all of which appear to other erlang processes as "ports",
with which you communicate using messages, just like with normal
erlang processes. This makes it very easy to write erlang stubs for
foreign interfaces, for one thing.
> 7. Platforms
> - Windows is a must.
Erlang runs on Windows.
> - Linux would be a very strong plus.
Erlang runs on Linux. Linux and Windows are the two most widely
used Erlang platforms.
> - MacOS would be an added bonus.
Erlang runs on MacOS. Other platforms are Solaris, FreeBSD,
VxWorks, and (not yet released) OSE Delta.
> Multi-platform support will not be helpful unless:
> * relevant libraries are available for all supported platforms, and
> * processes can exchange data even if running on different
> platforms.
You can run a distributed network of Erlang nodes using a mix of
OSes. This is frequently done, too. Compiled Erlang code is
identical on all platforms.
> 8. Graphics
> The project needs good 2D and/or 3D graphics support. A wrapper for
> OpenGL or DirectX would fit the bill (OpenGL preferred due to
> portability concerns).
The Erlang-based Wings3D application (http://www.wings3d.com/)
uses OpenGL.
> 9. GUI
> A good, platform-independent GUI library would be a bonus.
> (A GTK wrapper should work OK.)
Ok, Erlang is usually not considered strong on GUIs, since it's
been very much focused on embedded systems, but there is a
tcl/tk library (actually a couple), and there is also a GTK wrapper.
(http://erlgtk.sourceforge.net/)
/Uffe
--
Ulf Wiger, Senior Specialist,
/ / / Architecture & Design of Carrier-Class Software
/ / / Strategic Product & System Management
/ / / Ericsson AB, Connectivity and Control Nodes
Jerzy Karczmarczuk
that proposals are perfectly respectable.
But I would like - for the sake of my fight against "eine Sprache
eine Kirche" ideology propose something else.
PYTHON
======
> 1. Maturity
Ho, ho. Rock solid, together with a plethora of development tools, libraries,
web programming experience, and thousands of people on the scene. Now, please.
I love functional languages, and I recommend them everywhere. But if people
say "we must use NOW in a commercial, collective project a serious language",
then the number of potential READY programmers may be an important factor. Then
Python is more than serious alternative.
> 2. Fun
> It should be "fun" to work with the language.
> This means: nonobtrusive syntax, up-to-date documentation on tools and
> libraries, useful debuggers and profilers, garbage collection, security
> from buffer overruns, maybe some other things.
Consider all that acquired. The debugging is much easier than in Caml (my
experience with Erlang is very superficial). Fun?
Subjective view. From my *substantial* pedagogical experience I can say
YES. Even for biology students, who are hard at accepting *any* programming
language... Personally, although I feel better within the Functionalist Church,
it is a pleasure to write programs in Python. For fast development of some
graphic stuff this is one of my favourite tools.
>
> 3. Network transparency
> Code shouldn't care whether an object that it's accessing is local or
> remote.
Yessir. Done. With Everything You Never Wanted To Jump Into. PySOAP. Java
support, and even Python/Java hybrides. With Mark Lutz's nice book to start
with. With the iceberg called Zope.
> 4. Object (un-)marshalling
Standard since the very early days. Marshalling for the code and also data
pickling, with all support you need. Of course GC is there.
> 5. Object persistency
How the Terminator II used to say: "No problemo, baby".
> 6. Heavy-duty data structure libraries
...
> potential total size of dozens of megabytes, without suffering a nervous
> breakdown.
SERIOUS (don't have bigger font...) dictionaries is one of most important
features of Python, language and implementation
> 6. FFI
> A good and easy-to-use foreign function interface is almost a must.
Major topic in Guido Van Rossum's check list. C etc. extensions are
considered so standard, that they are taught.
> 7. Platforms
> - Windows is a must.
> - Linux would be a very strong plus.
> - MacOS would be an added bonus.
Pffff... Existing for years. Alive, sane and popular.
> 8. Graphics
> The project needs good 2D and/or 3D graphics support. A wrapper for
> OpenGL or DirectX would fit the bill (OpenGL preferred due to
> portability concerns).
Everything is there, ready to eat and enjoy. OpenGL, some special OpenGl-
based goodies (Zoe, Visual Python, VTK modules, Alice, whatever you need.
Image processing library. Pygame and SDL bindings. More than you can eat).
Hm. Alice is rather DirectX based than OpenGL, but, oh well, you have
enough.
> 9. GUI
> A good, platform-independent GUI library would be a bonus.
> (A GTK wrapper should work OK.)
Yessir. From Tkinter to wxPython. Working, debugged, serious, efficient.
Oh you wanted GTK? No problem, It was just in my other pocket. PyGTK.
> I'm aware that this is a long list, and that it's unlikely that there is
> a language and implementation that will satisfy all criteria.
Well, I didn't find many handicaps in Python either. Speed is one, but it
seems you are not concerned with. For all my work, the speed was enough. For
the image processing, 3D modelling, etc. - well, you use extensions.
==
A bit of more ad, with your permission.
Python is a deeply OO language, but it has a decent functional layer, and
even there were some experiments on its "stackless" implementations using
continuations. Although higher-order programming is a bit handicapped,
it is perfectly usable. The language permits to write functionals, but
making dynamic closures is delicate, although Python has closures now.
But the optimisation issues were pushed in a direction I don't understand,
and, e.g., a sorting procedure using a standard, internal ordering function
is *MUCH* faster than a generic sort with user-supplied procedure.
Anyway all that works quite well. The scientific community appreciates Python
enormously, and with such libraries as Numpy which overloads math on composite
structures, Python is visibly attracting some clients from the Matlab etc.
zone.
Thank you.
Jerzy Karczmarczuk
Caen, France
[no personal attachment to Pythonist Sect; just liking them]
Ronald Legere
It seems to fit the bill very well. Python is one of
my standard 'utility' language, for getting work done :) Not as
elegant as many, but not as ugly as otheres either.
Cheers!
"Jerzy Karczmarczuk" <kar...@info.unicaen.fr> wrote in message
news:3E9FA2B4...@info.unicaen.fr...
Oleg Trott
> [Python] debugging is much easier than in Caml
In what way? What does Python debugging have that Caml (for UNIX) doesn't?
Oleg
Miles Egan
> I agree with Jerzy. I was just about to suggest python myself.
> It seems to fit the bill very well. Python is one of
> my standard 'utility' language, for getting work done :) Not as
> elegant as many, but not as ugly as otheres either.
If the Python road is appealing you might also consider Ruby. Most of
what's been said above about Python holds true for Ruby and I think
it's a better langauge. More logical, more flexible, and somehow
achieving that language design gestalt that Python, for all its
strengths, seems to lack.
--
miles egan
mi...@rddac.com
reverse the domain name to find me
OpenPGP/GPG Key 0x01F53D51 @ wwwkeys.us.pgp.net
Markus Mottl
> 6. Heavy-duty data structure libraries
> Servers will keep gigabytes of data in RAM, and this data will live in
> some dictionary objects. This means that the data structures that
> implement these dictionaries will have to handle millions of keys with a
> potential total size of dozens of megabytes, without suffering a nervous
> breakdown.
You will probably also have to add then that you need a language that
runs on 64-bit platforms (e.g. Alpha, Itanium), because you'd otherwise
not be able to address many gigabytes of data...
OCaml produces native code for both architectures.
Regards,
Markus Mottl
--
Markus Mottl http://www.oefai.at/~markus mar...@oefai.at
Joachim Durchholz
>
> Which languages do you already know? This is an important piece of
> information you forgot to mention in your posting...
I didn't forget it, I omitted it on purpose :-)
My CV lists 20 languages, of which I have actively used 13 in my life.
Besides, it's not too important. I'm not going to be the only programmer
on the job. (I'm not sure how much programming I'll be doing anyway, I
may be too occupied with other tasks.)
FWIW, here's the full list, for the curious:
I'm fully proficient in:
Eiffel (would do this again, though with reservations)
Turbo Pascal (obsolete)
C (if I must, I'll use it...)
I have programmed in
C++ (will use this at gun point)
Interlisp (enjoyable concepts, irritating philosophy)
Delphi (enjoyable but limited)
Clarion (fill-in-the-gaps programming with macros - useless)
ELAN (Pascal-like educational language - nice but too limited)
Java (wealth of libraries, good and bad... and limited language)
Clipper (urk)
8080/80x86 and 68000 assembly
MS QuickBasic (good thing it's dead)
RPG/400 (will not use this, even at gun point)
I know (read: have read the reference manual, end to end; in some cases,
even written some toy programs at <1000 LOC):
Mozart/Oz (wonderful concepts, strange syntax)
Haskell (nice syntax)
Common Lisp/Scheme (nice ideas, but unsafe)
Prolog (nice basic idea, but horrible for coding imperative stuff)
Forth (cute... but not my world)
Fortran (the mummy)
Smalltalk (interesting, but I never found my way in the libs)
I also know about ML/SML/OCaml, I even read parts of the OCaml manual,
but I never used any of these languages :-)
Hope this helps anybody :-)
Regards,
Jo
Joachim Durchholz
> Joachim Durchholz <joac...@gmx.de> wrote:
>
>>6. Heavy-duty data structure libraries
>>Servers will keep gigabytes of data in RAM, and this data will live in
>>some dictionary objects. This means that the data structures that
>>implement these dictionaries will have to handle millions of keys with a
>>potential total size of dozens of megabytes, without suffering a nervous
>>breakdown.
>
> You will probably also have to add then that you need a language that
> runs on 64-bit platforms (e.g. Alpha, Itanium), because you'd otherwise
> not be able to address many gigabytes of data...
Right - I overlooked that consequence.
I've getting SO used to processors with address spaces beyond any
imaginable harddisk size that I simply didn't notice that 4GB would be
the next barrier...
Well, good to know that 64 bits are enough to address every atom in the
universe and then some *g* (last time I asked, the estimate was 10e40 =
atoms, that's about 2^50 items). 64 bits might be the last generation
after 8-bit, 16-bit, and 32-bit processors... there's hope that this is
the last time that this problem rose from its grave ;-)
Regards,
Joachim
Joachim Durchholz
> http://www.erlang.se/workshop/2002/wiger.ppt
The link is broken.
>>4. Object (un-)marshalling
>>Moving an object with all referred-to subobjects should not require
>>writing an extra wrapper for every potentially referred-to object type.
>>Likewise, receiving an object through the network should not require
>>extra wrappers for unmarshalling.
>
> Message passing in Erlang is distribution transparent. There is no
> need for any marshalling or unmarshalling.
Can messages contain arbitrary data? I.e. entire networks of objects if
needed?
I'm also a bit concerned about a potential semantic gap.
One one hand, I have a language core that I can program functionally in.
On the other hand, I have message passing, a concept that's quite
imperative (and notorious for creating race conditions).
I'd expect that to be a deep semantic gap. How does Erlang close the gap?
> Understood, but you would probably still want to take a look at the
> Mnesia DBMS, since it is exceptionally well integrated with the
> Erlang runtime system and language. There is no semantic gap,
Really? I knew about Mnesia, but I didn't expect it to integrate seamlessly.
> As a curiosity perhaps, Mnesia also supports table fragmentation,
> so if your dictionary grows beyond your RAM budged, you can break
> it into multiple fragments and create a multi-node database cluster.
> Your application does not have to be modified, as Mnesia
> provides location transparency.
That's actually useful. I'm not sure that the system will ever break
that barrier, but SMP support scales only to a very limited number of
processors (4 or 8, I think). Having a database that allows a table to
be fragmented over multiple machines would offer a choice between
upgrading the server to have more CPUs or installing another server.
It might also offer additional choices for setting up tests.
> The Mnesia fragmentation support has demonstrated linear
> scalability up to at least 50 nodes (that was about as many
> workstations that could be hijacked for the scalability test),
Impressive.
Is fragmentation compatible with replication? (Particularly if some of
the replicated data is mutable - after all, even if I'm programming
functionally, there *is* a mutable world state somewhere *g*)
> so you shouldn't have to worry about your application outgrowing
> Erlang's capabilities.
Actually that's one of my least worries (at least for Erlang *g*).
I have read up on Erlang earlier, it wouldn't be used for large projects
like the AXP if it didn't scale.
> Ok, Erlang is usually not considered strong on GUIs, since it's
> been very much focused on embedded systems, but there is a
> tcl/tk library (actually a couple), and there is also a GTK wrapper.
> (http://erlgtk.sourceforge.net/)
Hmm... Alpha status for erlgtk. This doesn't sound promising.
But nothing is perfect :-)
I'll have to look into the tk wrappers.
GUI lib support wasn't too high on the priority list anyway :-)
Regards,
Jo
Benjamin Ylvisaker
Joachim Durchholz <joac...@gmx.de> wrote:
> (last time I asked, the estimate was 10e40 = atoms, that's about
> 2^50 items).
Umm... The rule of thumb is 2^(10x) ~= 10^(3x). So we would need
((40 / 3) * 10) ~= 130 bits in order to assign a unique memory
location to each atom (assuming your 10*10^40 figure is accurate).
Did I go wrong in my calculation somewhere?
Benjamin
har...@panix.com
Benjamin Ylvisaker <sletv...@yahoo.com> writes:
> So we would need ((40 / 3) * 10) ~= 130 bits in order to
> assign a unique memory location to each atom (assuming
> your 10*10^40 figure is accurate).
Some time ago, I think the estimate was 10^80
protons. [1] The recent results from WMAP [2] say regular
matter is only 4% of the mass. If you want to address them
as well, you are going to need more bits.
[2] http://map.gsfc.nasa.gov/m_mm/mr_content.html
The matter of which we are made is only a small portion of
the Universe, 4%. 23% is an exotic type of material known as
"cold dark matter". And 73% is an even more exotic "dark
energy". One possibility for the dark energy was introduced
by Albert Einstein, a so called "cosmological constant."
harley.
Joachim Durchholz
> Joachim Durchholz <joac...@gmx.de> wrote:
>>(last time I asked, the estimate was 10e40 = atoms, that's about
>>2^50 items).
>
> Umm... The rule of thumb is 2^(10x) ~= 10^(3x).
Ah, you're right. Grr.
OTOH: a few orders of magnitude more or less don't really matter at that
scale *gg*
Regards,
Jo
--
Currently looking for a new job.
Joachim Durchholz
> OK, my turn. OF COURSE other people proposed here Erlang or Caml, and
> that proposals are perfectly respectable.
> But I would like - for the sake of my fight against "eine Sprache
> eine Kirche" ideology propose something else.
>
> PYTHON
> ======
Thanks - such a separate vote, even if it's not heeded, is going to
sharpen the criteria for the rest of the language.
Besides, I like your style :-))
>> 6. Heavy-duty data structure libraries
>
> ....
>
>> potential total size of dozens of megabytes, without suffering a
>> nervous breakdown.
>
> SERIOUS (don't have bigger font...) dictionaries is one of most important
> features of Python, language and implementation
Since I didn't expect to get an imperative language recommendation,
here's a more precise requirement:
Are there efficienty *referentially transparent* dictionary data structures?
This is important. Data structures will be complex, referring to each
other in unforeseen ways. I can't accept a language/library where I have
to do in-place modifications that may break in the presence of
unexpected aliases.
The alternative would be to routinely clone every data structure before
modifying it. Essentially, this is what happens in a functional language
anyway - but the difference is that functional libraries are written to
be reasonably efficient under these conditions; how do the data
structures of Python fare?
(Same goes for Ruby. Actually I share Miles Egan's sentiments on Python
and Ruby. Not that my sentiment is very relevant: it's entirely based on
hearsay.)
Joachim Durchholz
This is an intermediate summary of languages proposed and the position
that they have at the moment.
Some criteria that I forgot should be mentioned here:
1. 64-bit support. Data above 4GB can't be handled in RAM unless you
have 64 bits in the pointer. (Well, 48 bits would probably be enough,
but the 48-bit addressing schemes that I have seen are too ugly for
consideration.)
2. Static typing
If in combination with type inference, that's a big plus.
Without type inference, I don't care too much; I'd like to see good
assertion support to do the type checking.
3. Graceful error handling
Exceptions or something equivalent. If a computation fails, it must be
possible to roll that computation back. (Haskell has this by default:
just don't run the IO monad if the function that produces it fails.) If
a computation acquires a new failure mode, it must be possible to
intercept failures of that mode at a higher level of the software
without having to modify intermediate software levels.
This is a must.
I know that exceptions work (except in Java, where you must annotate
every intermediate software level with the exceptions that can be thrown).
I don't know whether a Maybe type would work. To make intermediate
levels of software resistant to change, it would be necessary that the
types of intermediate functions be automatically inferred as a Maybe
type when a subcomputation acquires its first failure mode. I'm
uncertain whether type inference is good enough for this.
4. Planning for the future
The tools and run-time system must be either open source, or come from a
company that's "too big to fail".
Companies that are very large aren't usually the nicest to deal with,
which leaves the project with open source software.
I need additional information on the following languages (see below for
details):
SML
Haskell
Mozart/Oz
Also, the list of languages under consideration isn't closed yet; feel
free to enter additional language if you know of one!
Erlang: Looks extremely good.
Even when considering the additional requirements.
Also, I'll give a whole lot of bonus points for Erlang's ability to
replace the code within a running system, in a controlled manner. Since
the application is a WWW service, keeping the downtimes at a minimum is
a high priority, and Erlang excels in this area. (If I had made this a
key requirement, Erlang would indeed have won hands down...)
Lack of static typing is a small minus, but if no fundamentally new
facts are uncovered, it looks as if we have a strong candidate here.
OCaml: Looks good.
Having to homegrow network transparency is a strong minus though.
SML: No votes yet. Anybody out there?
I don't know anything about SML; for some reason, OCaml is more present
as the representative of strict-by-default FPLs.
I do know there's serious work in progress for SML though; there are
regular updates, there's a "compilation manager", there's an optimizing
compiler (MLton IIRC), and a lively community.
Haskell: No votes yet.
What are the problems here?
I can imagine maturity (the language seems to change), but it would be
good if somebody who actually used the language could confirm that.
I also have to admit that I have my reservations about by-default
laziness. It seems to be capable of creating exponential efficiency
holes, and that's something that I want to reliably excluded. OTOH I do
see that laziness gives an enormous boost to reusability... I think the
jury is still out on that issue. Which, in other words, means that
laziness isn't for commercial projects yet - is this common consensus,
or are there reasonable arguments against this stance?
Mozart/Oz: No votes yet.
[For the curious: Mozart is the run-time system, Oz the language.]
It's a nice language with a very strange syntax.
Networking is one of its strong points. It even seems to duplicate
Erlang's failover and code replacement mechanisms, but implemented by
library code (IIRC it's part of the language in Erlang).
Its multiparadigm mindset is extremely appealing as well, even in the
presence of minor problems.
Stability and maturity seem to be less than optimal at this time - but
between the lines I read a lot of determination to get there. I don't
have the slightest idea whether Mozart will be ready for show within a
year or two, though.
Also, I don't know how well this all works in practice. All that I have
is papers, a personal toy project (which is simply too small to show how
well Oz works, it just shows that run-time error messages are relatively
inscrutable), and a mailing list.
Python/Ruby: I appreciate Jerzy's separate vote. Jerzy's posts are
always fun to read, and even though I won't take his advice, it has
helped me sharpen my ideas about my selection criteria. This already has
proved invaluable - thanks!
I'd still like to stick within the functional realm, simply for
expressivity reasons. The project will need a competitive advantage, and
using a functional language is exactly that.
(As Jerzy noted, efficiency is not a main criterion. When software grows
large, system structure and data organization is much more important
than constant-factor differences. A faster processor resp. another
server are much cheaper than a single round of serious profiling and
post-hoc optimization *g*.
Smalltalk: Not on my list.
I see two show-stoppers here.
One is that every Smalltalk is incompatible with every other Smalltalk.
You live and die with the Smalltalk implementation that you chose. (Not
a problem with Squeak AFAIK, but I know of no other implementation that
doesn't have this Achilles heel.)
The other is that there is no efficient referentially transparent
container data structure (at least not in the standard classes).
Lisp: Not on my list, too.
I hold the same reservations as for Smalltalk, though on a somewhat
smaller scale.
However, I can't take a language seriously that combines run-time typing
and a requirement to use a different equality operator depending on the
run-time type. Worse: there are no equality operators that do the Right
Thing when comparing structures (for example, you can't reliably compare
lists that contain a mixture of large and small integers: the large
integers will be compared by reference, the small integers by value).
Then, add the total lack of restrictions. There are lots of OO
frameworks, each subtly different from the other (some of them crap,
most with some serious problems). There are other frameworks. When I use
a library, I have to understand the semantic framework that it uses;
when I use multiple libraries, I have to live with multiple frameworks
and multiple interpretations on how that framework should be applied.
That's exactly the stuff that nightmares are made of.
Actually, I'm surprised that Lisp is being used for serious work at all.
I suspect that every application implemented in Lisp uses a particular
niche, defined by one of the various semantic models, and guarded by
lots of Do's and Dont's. Sorry - the project doesn't have the resources
to acquire the knowledge how to use Lisp safely, it doesn't have the
resources to find a seasoned Lisp programmer who can teach to use the
language, or to teach new programmers to get up to speed with Lisp.
Sorry for ranting. Lisp has some important niches (such as experimenting
with new semantic concepts), but commercial development in small teams
with limited resources is definitely not one of them.
(The above critique also holds for Smalltalk, but on a MUCH smaller
scale. The list of "strange and wonderful things" that are done
routinely in Smalltalk is much smaller, and could be taught to a
programmer.)
Regards,
Jo
Neelakantan Krishnaswami
>
> Are there efficienty *referentially transparent* dictionary data
> structures? This is important. Data structures will be complex,
> referring to each other in unforeseen ways. I can't accept a
> language/library where I have to do in-place modifications that may
> break in the presence of unexpected aliases.
Yes -- they are usually versions of balanced binary trees, like
red-black trees, randomized treaps, Patricia trees, and so on. These
give you O(log N) access, addition, and removal of key/element pairs.
Most require a comparison function to implement, except for Patricia
trees, which want a hash function.
> The alternative would be to routinely clone every data structure
> before modifying it. Essentially, this is what happens in a
> functional language anyway - but the difference is that functional
> libraries are written to be reasonably efficient under these
> conditions; how do the data structures of Python fare?
Python's standard data structures are mutable -- growable arrays and
hash tables.
--
Neel Krishnaswami
ne...@alum.mit.edu
Francois-Rene Rideau
> typing and a requirement to use a different equality operator
> depending on the run-time type.
> Worse: there are no equality operators
> that do the Right Thing when comparing structures (for example, you
> can't reliably compare lists that contain a mixture of large and small
> integers: the large integers will be compared by reference, the small
> integers by value).
That's bullshit. EQUAL will work perfectly for this purpose.
For comparing numbers, you have a choice of = or EQL (or EQUAL).
If you hate EQ, you can forget about it and replace it with EQL everywhere.
EQ is just proposed as an optimization.
(In Scheme, make that EQ => eq?, EQL => eqv?, = => =, EQUAL => equal?)
Now, for arbitrary structures, you'll have to roll your own
comparison operator. Of course, if you have many structures
and a sensible, common principle upon which to build
your comparison operator, then you can wrap your defstruct/defclass
in a macro or metaclass that will automatically produce
the comparison operator *you* want, in a few tens of lines of code.
There may even already exist metaclasses that do that in a public library.
All in all, Lisp can do *better* than *any* other language in this respect.
> There are lots of OO frameworks, each subtly different
> from the other (some of them crap, most with some serious problems).
Apparently, you've been using Scheme, and not Common Lisp.
CL has CLOS, which is well-documented and standard,
and CLOS has a MOP which is also well-documented.
That said, if you use Scheme, no one forces you to use a bad object system.
There are plenty of satisfying object systems integrated to Scheme.
> Actually, I'm surprised that Lisp is being used for serious work at all.
If you use a lowest-common-feature approach instead of a best-choice approach,
then the trouble is in your own deficient approach,
and not in any particular system.
When you use C, you don't try hard to have a system that works on all
C compilers, even old or buggy releases, even versions for antiquated
exotic hardware, even with no C library at all, or buggy C libraries, etc.
You pick the most recent/decent C compiler and libraries
for an actual system that you target.
Why should Lisp programmers do things any differently?
Lispers pick a Lisp that suit their purpose,
that runs on their target machine of choice,
and provides all the features they need.
If there's a bug in the implementation or library, they have it fixed.
I'm not arguing here about the adequation of Lisp to any poster's own use.
But the arguments you use to disparage and discard it
are simply wrong and ignorant.
See http://alu.cliki.net and http://www.schemers.org/
for sites that promote Common Lisp and Scheme respectively.
As for distributed systems, a lot of them have been built based on Lisp:
In Scheme, see Dreme or the Tube. In CL, see ABCL/R or NetCLOS.
Not to talk about CORBA bindings, Java integration, etc.
[ François-René ÐVB Rideau | Reflection&Cybernethics | http://fare.tunes.org ]
[ TUNES project for a Free Reflective Computing System | http://tunes.org ]
Those who do not understand LISP are condemned to not even being able
to reinvent it poorly. -- Faré, without apologies to Henry Spencer.
gr...@cs.uwa.edu.au
: Jerzy Karczmarczuk wrote:
:> PYTHON
: Are there efficienty *referentially transparent* dictionary data structures?
: The alternative would be to routinely clone every data structure before
: modifying it. Essentially, this is what happens in a functional language
: anyway - but the difference is that functional libraries are written to
: be reasonably efficient under these conditions;
Just as importantly: functional languages create these clones lazily
(copy-on-write) _by_default_. The semantics of referentially
intransparent languages can sometimes make emulating this _very_ messy,
and it often needs doing on a per-data-structure basis. As good as
your libraries may be, you're going to have your own task-specific
data structures as well, and you don't want to have to fight the
language to keep them RT.
-Greg
gr...@cs.uwa.edu.au
: [What has prevented anyone recommending Haskell so far?]
: 1. Maturity
: The software will be mission-critical. If the software fails, the
: business will fail.
These are out there, but the language is evolving. Will the rock-solid
toochain you choose still be rock-solid when it adds the latest
feature you can't live without? The answer is "probably, but who is
going to stick their neck out and say definitely"? Perhaps you'll
find someone braver or better-informed than me.
: 3. Network transparency
: Code shouldn't care whether an object that it's accessing is local or
: remote.
The problem here is the word "object". If there is a formal object
framework for Haskell, I guess it is O'Haskell, which is relatively
experimental, (version 0.5) only implemented as an interpreter (AFAIK)
and probably not widely-used enough to give much assurance on item 1.
The alternative is roll-your own. Depending on your problem domain
(which only you know very well), the accuracy of your anticipation
of future needs, and your own abilities, it might be a short and
simple process or a never-quite-finished nightmare. It might even
turn out that you don't need a formal object system, but noone can
really tell you that yet. (OK, so they might tell you that anything
is possible w/out such a system, but if that is the way you perceive
your problem domain, then it'll take a very convincing argument to
change the way you look at it)
Other languages have been doing network-transparent object models
for donkey's years. That doesn't mean that object modelling is
right for your task, or even a good idea in general, but it does
seems to be the way you are looking at the problem.
: 4. Object (un-)marshalling
: Moving an object with all referred-to subobjects should not require
: writing an extra wrapper for every potentially referred-to object type.
: Likewise, receiving an object through the network should not require
: extra wrappers for unmarshalling.
Doesn't seem like it should be problem, given a good implementation
of this object system, but you can't be certain until it's clear
exactly where that object system is coming from.
: 5. Object persistency
: It should be possible to write an object with all referred-to subobjects
: to a sequential file, and to retrieve them back.
In principle, no problem. In practise, how do you deal with laziness?
(What if the structure you want to store us infinite?) It wouldn't
surprise me if this problem is already solved, but throw in the O-word,
and you are down to at most a slim proportion of advocates who can
assuredly tell you "yes, you can do this".
: 6. Heavy-duty data structure libraries
: Servers will keep gigabytes of data in RAM, and this data will live in
: some dictionary objects. This means that the data structures that
: implement these dictionaries will have to handle millions of keys with a
: potential total size of dozens of megabytes, without suffering a nervous
: breakdown.
Some very good data structure libraries exist. However, a degree of
fuss is made over them, because the efficient way of doing some things
in a referentially-transparent manner is not obvious. Do the existing
libraries cover everything you will need? Are you going to grok things
well enough to apply (perhaps even _find_) the right libraries, or fill
what gaps there might be? Hard for us to say.
: 6. FFI
: A good and easy-to-use foreign function interface is almost a must.
For sure. Is it standardized enough yet? Does it handle concurrency
well enough for your ambitions? Those are harder questions.
: 8. Graphics
: 9. GUI
Definitely out there, but fragmented.
So there, you are: some reasons that might help explain why noone
has advocated Haskell yet. I don't know how many, if any, of them are
valid, but I do know that there will be very few people who can give
you a clear and accurate picture of all of them, not least because
Haskellers will tend to look at problems in native ways and not
introduce object theories without serious duress. The other factor
is that Haskell is still (perhaps indefinitely) evolving. So far,
no flaw has been found that couldn't be (eventually) handled in an
elegant, orthogonal and usable way. But the price of that is
learning to do things in this (perhaps new) way, and knowing that
the language is never quite going to stop moving under your feet;
However well it manages to maintain compatibility, it still means
new toolchain versions if you want to use the new features, (and
how can you say _now_ that you won't need those features in the
future) and perhaps learning the Haskell way of seeing something,
even if it is not the way you looked at that thing originally.
-Greg
David Wagner
>I'm going to choose the language for a commercial project.
Java?
Less mainstream: Have you look at the E language?
>3. Network transparency
>Code shouldn't care whether an object that it's accessing is local or
>remote.
Are you sure you want this? I've seen people think they want this, but
my suspicion is that it's often a mistake. The semantics of touching
remote data are different from the semantics of touching local data:
there are different failure modes, different time scales, different
blocking/non-blocking behavior, different deadlock possibilities.
My suspicion is that it's often better to expose this to the programmer,
rather than hiding it from them. But then, you'd know your requirements
far better than I, so you've probably considered these issues.
Joachim Durchholz
> I'm not arguing here about the adequation of Lisp to any poster's
> own use.
> But the arguments you use to disparage and discard it
> are simply wrong and ignorant.
I think we'll have to agree to disagree, since slugging this out would
really go too far off-topic.
I know that not all of the points against Lisp are equally relevant, but
I find the multitude of problems and potential pitfalls deterring.
Inevitably, the project will attract average and sub-average
programmers; using all the facilities of Lisp correctly and avoiding all
the traps goes far beyond what I expect them to handle.
Joachim Durchholz
> Joachim Durchholz <joac...@gmx.de> wrote: <elsewhere>
> : [What has prevented anyone recommending Haskell so far?]
>
> : Code shouldn't care whether an object that it's accessing is local or
> : remote.
>
> The problem here is the word "object".
Oops... no. I didn't mean OO. I simply meant "data object", in the
traditional, pre-OO sense. I.e. data structures (including entire lists,
trees etc.).
Would you mind updating the rest of your post to that in
> : 5. Object persistency
> : It should be possible to write an object with all referred-to subobjects
> : to a sequential file, and to retrieve them back.
>
> In principle, no problem. In practise, how do you deal with laziness?
> (What if the structure you want to store us infinite?)
In that case, the system should simply store the closure.
Which might exactly be the problem (at least I have seen several people
mention that as a problem, it may be solved now - I think the Mozart/Oz
people solved the problem by simply storing the bytecode of the closure).
> : 6. Heavy-duty data structure libraries
> : Servers will keep gigabytes of data in RAM, and this data will live in
> : some dictionary objects. This means that the data structures that
> : implement these dictionaries will have to handle millions of keys with a
> : potential total size of dozens of megabytes, without suffering a nervous
> : breakdown.
>
> Some very good data structure libraries exist. However, a degree of
> fuss is made over them, because the efficient way of doing some things
> in a referentially-transparent manner is not obvious. Do the existing
> libraries cover everything you will need? Are you going to grok things
> well enough to apply (perhaps even _find_) the right libraries, or fill
> what gaps there might be? Hard for us to say.
Well, I think the Purely Functional Data Structures book by Chris
Okasaki does a pretty good job.
The main data structure that's needed is the Dictionary (aka search
tree, hash table, etc.). I don't care too much about lazily rebuilding
priority queues and whatnot - I can emulate all of these using a
Dictionary, which gives me an acceptable upper bound on performance, and
if anybody manages to write an even better implementation, I'll happily
use it without making a fuss over potential problems.
> : 6. FFI
> : A good and easy-to-use foreign function interface is almost a must.
>
> For sure. Is it standardized enough yet? Does it handle concurrency
> well enough for your ambitions? Those are harder questions.
The combination of concurrency and FFI is a generally unsolvable
problem, so I don't think that Haskell would have to go to extra lengths
here.
If all fails, I'll simply put all the FFI stuff into a separate thread
and enforce synchronization that way.
> The other factor
> is that Haskell is still (perhaps indefinitely) evolving.
This is probably the most serious problem.
The other potential problem is that I'm not sure how well Joe Avg.
Programmer will fare with laziness. It seems to be all too easy to write
code with horrible performance.
> So far,
> no flaw has been found that couldn't be (eventually) handled in an
> elegant, orthogonal and usable way. But the price of that is
> learning to do things in this (perhaps new) way, and knowing that
> the language is never quite going to stop moving under your feet;
> However well it manages to maintain compatibility, it still means
> new toolchain versions if you want to use the new features, (and
> how can you say _now_ that you won't need those features in the
> future)
Well - that will always be the problem with whatever language I choose.
Haskell may move quicker than other languages. I don't know. Actually, I
don't care too much - the software will have to be written with the
features that are stable right now. Added future features may or may not
be helpful, this will be decided on a case-by-case basis.
Version incompatibilities would be a serious problem.
Incompatibilities with persistent data might also be a problem.
Sometimes I want to inspect data that's two years old; if the current
version of whatever programming language is in use doesn't accept the
old data, this is trouble. (But then I'm pretty sure that there will
always be some way to convert the data.)
> and perhaps learning the Haskell way of seeing something,
> even if it is not the way you looked at that thing originally.
I don't think this is a real problem.
The real problem is not getting it, it's the time that it takes to get
it. That's time in which programmers are largely unproductive, because
everything that they write in that time will eventually have to be
rewritten.
(That's one of the things that make Erlang attractive. Some previous
posts by Erlang advocates have made quite strong statements that the
language is easy to "get".)
Regards,
Jo
Marcin 'Qrczak' Kowalczyk
> I know that exceptions work (except in Java, where you must annotate
> every intermediate software level with the exceptions that can be thrown).
> I don't know whether a Maybe type would work. To make intermediate
> levels of software resistant to change, it would be necessary that the
> types of intermediate functions be automatically inferred as a Maybe
> type when a subcomputation acquires its first failure mode. I'm
> uncertain whether type inference is good enough for this.
The preferred way is to include exceptions in a domain-specific monad
the library uses.
As long as a part of code has some patterns which can be done by introducing
a custom monad (uses a common contextual object everywhere, or threads some
state, or needs to capture continuations, or computation can fail), it's
very convenient to indeed wrap this in a monad. Changing the capabilities
of the monad then doesn't need to change the code using it.
--
__("< Marcin Kowalczyk
\__/ qrc...@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Dirk Thierbach
> Haskell: No votes yet.
> I also have to admit that I have my reservations about by-default
> laziness. It seems to be capable of creating exponential efficiency
> holes, and that's something that I want to reliably excluded.
The problem is not so much efficiency per se, but it is that it's
sometimes very hard to control space usage (or *space* efficiency, if
you want). In theory, you can fix that by using strict data
constructors, the 'seq' construct etc, but in practice you can still
run into unexpected memory overflow problems.
While I like Haskell a lot (IMHO it's the most elegant FP around),
this would prevent me from using it in a large mission critical
application.
Also, I'd say that network transparency is probably not developed
enough (if there is any good support at all besides the very basic
marshalling libraries -- no doubt someone will mention them now if
there are indeed some). Since that is high on your priority list, in
your situation Erlang seems to be one of the top candidates (even if
the lack of static typing is a minus -- but you can write more tests
to compensate).
- Dirk
Donn Cave
...
| The real problem is not getting it, it's the time that it takes to get
| it. That's time in which programmers are largely unproductive, because
| everything that they write in that time will eventually have to be
| rewritten.
| (That's one of the things that make Erlang attractive. Some previous
| posts by Erlang advocates have made quite strong statements that the
| language is easy to "get".)
All language advocates say that, don't they? Maybe it depends on what
you get. They say ``you can write FORTRAN in any language'' - so when
you've mastered a language well enough to do that, do you get it? It
sounds like you hold a higher standard. Is it possible that a system
that is initially more challenging might turn out easier at the end of
the term, as the more radical structural notions start to pay off?
Anyway, you may find the following to be an interesting read along those
lines about Erlang, though note that it is self promotional.
http://www.erlang-consulting.com/euc2001/index.htm
Donn Cave, do...@u.washington.edu
Joe Armstrong
> Thanks to everybody. The responses have gone beyond my wildest dreams.
>
First a comment - you haven't *ranked* your requirements - it would be
helpful to know your rankings.
Since you said earlier "mission critical" (bet you business stuff) I'd
have ranked points concerning fault-tolerance way higher than GUIs etc.
IMHO as projects get bigger and scale into industrial size the
stuff on fault-tolerance becomes more and more important.
*anybody* can get a 50 line program working in any language - with or
without a type system, debugger, ... you name it.
If you are talking mega lines of code how you handle errors and how you
organize large software projects becomes the critical factor.
How things behave as you scale them up in size is critical - I've
seen (too) many projects fail as an attempt was made to scale them
from a working small prototype to a full-scale project. Thinks like
memory fragmentation have killed C++ projects - because the phenomena
was not observed in small prototypes but was manifest in the full
product.
So whatever language you choose you need evidence that the
technology has been used in similar projects to whatever your project
is.
> This is an intermediate summary of languages proposed and the position
> that they have at the moment.
>
> Some criteria that I forgot should be mentioned here:
>
... cut ...
> 3. Graceful error handling
> Exceptions or something equivalent. If a computation fails, it must be
> possible to roll that computation back. (Haskell has this by default:
> just don't run the IO monad if the function that produces it fails.)
> If a computation acquires a new failure mode, it must be possible to
> intercept failures of that mode at a higher level of the software
> without having to modify intermediate software levels.
> This is a must.
> I know that exceptions work (except in Java, where you must annotate
> every intermediate software level with the exceptions that can be
> thrown).
> I don't know whether a Maybe type would work. To make intermediate
> levels of software resistant to change, it would be necessary that the
> types of intermediate functions be automatically inferred as a Maybe
> type when a subcomputation acquires its first failure mode. I'm
> uncertain whether type inference is good enough for this.
IMHO you need much more than this - being able to write code that
performs roll-backs traps exceptions etc is just the *beginning* not
the end -
To make code that degrades gracefully needs a carefully thought out
*architecture* as well as language features that support the
architecture.
The Erlang OTP architecture - indeed most of the design decisions
made when developing Erlang were made for reasons of fault tolerance
rather than efficiency or any other consideration.
The *essential* design requirement behind Erlang is that it was
designed for building fault tolerant systems - that it is functional
is more of an accident than by design :-)
IMHO true independence of concurrent activities is critical for
design and programming fault-tolerant software - I know no other way of
making fault-tolerant software. Fault tolerance *implies* TWO of more
machines (sorry for shouting) - no matter how good your software is
the hardware might fail - thus you need (at least) two machines.
This (the fact that you need two or more machines) gets you into
distributed processing (whether you like it or not) - You need
therefore a language that can detect "remote" errors (ie exceptions in
a remote machine) - This is what Erlang and the OTP libraries were
*designed* to do.
Does this work? - the Ericsson experience with the AXD301 with 1.7
million lines of Erlang and extremely high reliability figures shows
that the technology works (within Ericsson).
Is it transplantable? - Yes (ish) - the Nortel SSL accelerator etc.
show that the experience is reproducible outside Ericsson (but beware,
the Nortel work was done by half of the original Erlang developers).
The technology seems to work far from the home base (Vale systems -
from memory) use Erlang/OTP in a demanding switching application which
was developed far from Sweden - using only the open source code.
---
To make a fault-tolerant system you could design a fault-tolerant
architecture and then program the abstractions you need in
*any* language - this would be more or less difficult depending upon
the language which you chose.
The OTP system *is* just this - it's in fact about the third total
re-write of such a system - using this (The OTP behaviors) will save
you a *lot* of work. IMHO the difficult bit is not the language but
coming up with the right abstractions to use when programming a large
fault-tolerant system - this takes many years and is very difficult to
get right first time.
/Joe
Galen Menzel
>>
>> >
>> > 1. Maturity
>> > 3. Network transparency
>> > 4. Object (un-)marshalling
>> > 5. Object persistency
>> > 6. Heavy-duty data structure libraries
>> > 7. Platforms
>> > 8. Graphics
>> > 9. GUI
>> > A good, platform-independent GUI library would be a bonus.
>>
>> > Please post, these all are highly appreciated!
>>
>> no doubt others can fill you in on the details with specific answers.
>> but hands down, no contest, the answer is erlang.
>
> I would agree with Erlang, but I think it only just edges out Ocaml.
> It's #3 that pushes it to Erlang. In most other respects I think
> Ocaml is a little superior, but I don't think there is any other
> language with concurrency support remotely as good as Erlang's.
I realize efficiency wasn't explicitly mentioned as a language
requirement, but according to the The Great Computer Language Shootout
Ocaml's native compiler beats the pants off of Erlang in terms of both
CPU and memory usage, often rivaling gcc:
http://www.bagley.org/~doug/shootout/
On the whole, Erlang's preformance is pretty shabby. Granted, the
Shootout is far from a scientific undertaking, but it's something you
might want to think about.
galen
Joe Armstrong
> > It's #3 that pushes it to Erlang. In most other respects I think
> > Ocaml is a little superior, but I don't think there is any other
> > language with concurrency support remotely as good as Erlang's.
>
> I realize efficiency wasn't explicitly mentioned as a language
> requirement, but according to the The Great Computer Language Shootout
> Ocaml's native compiler beats the pants off of Erlang in terms of both
> CPU and memory usage, often rivaling gcc:
>
> http://www.bagley.org/~doug/shootout/
>
> On the whole, Erlang's preformance is pretty shabby. Granted, the
> Shootout is far from a scientific undertaking, but it's something you
> might want to think about.
These benchmarks do not compare like with like. Erlang was not designed
for fast sequential processing - it was designed for efficent handling
of concurrent processes - fast context switching and fast message passing
were the principle design criteria. Typical process spawning times in
Erlang are 2-3 orders of mangnitude faster than in languages like
Java or C# or systems using pthreads concurrency.
/Joe
>
> galen
Jerzy Karczmarczuk
> Jerzy Karczmarczuk wrote:
>>[Python] debugging is much easier than in Caml
>
> In what way? What does Python debugging have that Caml (for UNIX) doesn't?
In Python you have a dynamic acess to all dictionaries which represent local
environments. You can make a desintegrating debugger which analyses almost
everything. Now, it is rare that you will need all this, and I am not claiming
that this will save your soul. CAML compiler optimizes more things. Python is
slower, and dynamically typed. My 'ad' was quite restricted, I answered just
the question posed.
Miles Egan wrote:
> If the Python road is appealing you might also consider Ruby. Most of
> what's been said above about Python holds true for Ruby and I think
> it's a better langauge. More logical, more flexible, and somehow
> achieving that language design gestalt that Python, for all its
> strengths, seems to lack.
Be careful. One of requirements was: maturity. Besides, graphics and number
of tools/librairies on the actual market is much more evolved for Python. As
far as elegance of the language is concerned - hmmmm... for me Ruby is a little
too bit Frankensteinic, glued together from many sources for the sake of
"pragmatism". But I won't fight against anybody here, newer language *must*
appear, their criticism by old dinosaurs like myself *should* be counterbalanced
by the fascination of younger people. But they won't convince me that $var, @var
etc. Perlish conventions are elegant... Anyway...
http://www.rubycentral.com/faq/rubyfaq-2.html#ss2.1
======
Joachim Durchholz:
> Are there efficienty *referentially transparent* dictionary data structures?
> This is important. Data structures will be complex, referring to each other
> in unforeseen ways. I can't accept a language/library where I have to do
> in-place modifications that may break in the presence of unexpected aliases.
> The alternative would be to routinely clone every data structure before
> modifying it. Essentially, this is what happens in a functional language
> anyway - but the difference is that functional libraries are written
> to be reasonably efficient under these conditions; how do the data
> structures of Python fare?
Python is an OO imperative language, and all 'alias' problems you would have to
live with, are the same as elsewhere. If you write
a = {'x':1, 'y':2}
b = a
b['x'] = 8
then of course a['x'] becomes 8. This is not like referential transparency in
a language where the modifications of a data structures are forbidden. Now,
did you say 'unforeseen ways'? I wonder what do you mean. In an industrial
strength applications there are no unforeseen ways, by postulate...
Now, if you are afraid that this kind of aliasing may destroy your life, family
etc., you define a class where you desactivate/control the assignment, and you
are done. No more problems. I repeat, one of Python strengths is that you
control almost everything. With all the dangers and inefficiency which comes
with this godly gift... [To be a president of a totalitarian contry is an
appealing perspective, but not all of them had the same luck. For example, you
can put into prison for 25 years some intellectuals only if you don't have some
important oil deposits, otherwise you might get into trouble for moral reasons].
=====
Anyway, in a further posting you demanded static typing, so I take my toys and
proudly ride away westwards, in the direction of falling Sun. And the darkness
comes from the East...
One more word: be a little more just wrt. Smalltalk.
> I see two show-stoppers here. One is that every Smalltalk is incompatible
> with every other Smalltalk. You live and die with the Smalltalk implementation
> that you chose. (Not a problem with Squeak AFAIK, but I know of no other
> implementation that doesn't have this Achilles heel.)
> The other is that there is no efficient referentially transparent
> container data structure (at least not in the standard classes).
Smalltalk *is* pretty well standardized, main differences between Squeak, Visual
Works, etc. concern the interfacing, the structure of your categories, system
browsing utilities, etc. But you *have* this interfacing business at your
disposal. When you choose a neutral language, build your wonderful interfacing
stuff upon it, then you will have to stick to it for some time. (Although not
until your death, presumably. The Smalltalkers don't, either...) The referential
transparence is another story. With Smalltalk objects you can do everything you
wish, you may protect them as well.
I begin to suspect that you will have to have a second look on Java, despite
your justified doubts concerning exceptions...
Jerzy Karczmarczuk
Ulf Wiger
> Ulf Wiger wrote:
> > http://www.erlang.se/workshop/2002/wiger.ppt
>
> The link is broken.
So it is. Try http://www.erlang.se/workshop/2002/wiger.pdf
instead.
> >>4. Object (un-)marshalling
> >>Moving an object with all referred-to subobjects should not require
> >>writing an extra wrapper for every potentially referred-to object type.
> >>Likewise, receiving an object through the network should not require
> >>extra wrappers for unmarshalling.
> > Message passing in Erlang is distribution transparent. There is no
> > need for any marshalling or unmarshalling.
>
> Can messages contain arbitrary data? I.e. entire networks of objects
> if needed?
Anything that you can bind to a variable in Erlang can also be
sent in a message. Some data objects uniquely identify local
resources. Most of these can be passed to another processor
and still be useable. One example where it is not true is
local built-in dictionaries ("ETS tables", it used to be true, but
that feature was removed), and another is references to locally registered
processes. But otherwise, you can send just about anything - most
notably process identifiers (which always point to the same process),
but also e.g. anonymous functions(*). The following short example
illustrates the passing of a file descriptor to a remote node:
[1] erl -sname f1
Erlang (BEAM) emulator version 5.2.3.3 [source] [threads:0] [kernel-poll]
Eshell V5.2.3.3 (abort with ^G)
[4] (f1@cbe1066)1> {ok,Fd} = file:open("testf",[write]).
{ok,<0.44.0>}
[5] (f1@cbe1066)2> {foo,f2@cbe1066} ! {f,Fd}.
{f,<0.44.0>}
[8] (f1@cbe1066)3> file:close(Fd).
ok
[9] (f1@cbe1066)4> os:cmd("cat testf").
"hello from (<3706.50.0>)\n"
[2] erl -sname f2
Erlang (BEAM) emulator version 5.2.3.1 [hipe] [threads:0]
[3] (f2@cbe1066)1> register(foo,self()).
true
[6] (f2@cbe1066)2> {f,Fd} = receive {f,_} = Msg -> Msg end.
{f,<4215.44.0>}
[7] (f2@cbe1066)6> io:fwrite(Fd,"hello from (~p)~n", [self()]).
ok
The order in which the above commands were executed has been
noted within brackets (I ran the "erlang nodes" in two xterm
windows on my workstation.) A curiosity regarding the sequence
is perhaps that the receiving process did not actually go into
receive mode before the message was sent. It could have (then,
[5] and [6] would have been reversed), but it doesn't matter.
> I'm also a bit concerned about a potential semantic gap.
> One one hand, I have a language core that I can program functionally in.
> On the other hand, I have message passing, a concept that's quite
> imperative (and notorious for creating race conditions).
> I'd expect that to be a deep semantic gap. How does Erlang close the
> gap?
There are different concurrency models, and some of them are indeed
notoriously difficult to manage. Erlang handles concurrency in a
manner similar to CSP. Due to active message reception and
selective receive with implicit buffering, Erlang programs can
avoid many of the problems with race conditions with relative ease.
The short explanation is that you are not forced to handle
messages in the exact order that they arrive, but can prioritize
between different senders (and even between messages from a single
sender). This leads to much more stable and scalable programs.
In many cases, a program can be re-structured through the addition
of a process (e.g. changing from a local data structure to a
client-server model) without changing much of the code, if any.
If you had defined proper access functions for your local data
structure, you can often switch to a communication-based solution
without changing the access interface. I recently did so in an
example, where I took the standard 'dict' (hash dictionary) module
and made an API-compatible fully replicated dictionary.
The basic idea is this:
lookup(Key, Dictionary) ->
dict:lookup(Key, Dictionary)
is changed to
lookup(Key, Dictionary) ->
Ref = make_ref(),
Dictionary ! {self(), Ref, {lookup, Key}},
receive
{Ref, Result} ->
Result
end.
(Certain details omitted.) To the surrounding code, this still
looks like a local function call. The solution is safe from race
conditions, since the message reception clause will ignore (buffer)
any messages that do not fit the patern {Ref, Result} (where Ref
is bound, and Result is not).
Oh, and Erlang processes do not (logically) share memory.
Exceptions in one process are contained there, and do not
affect other processes, unless you've arranged for them to
do so through process links.
> > The Mnesia fragmentation support has demonstrated linear
> > scalability up to at least 50 nodes (that was about as many
> > workstations that could be hijacked for the scalability test),
>
> Impressive.
>
> Is fragmentation compatible with replication? (Particularly if some of
> the replicated data is mutable - after all, even if I'm programming
> functionally, there *is* a mutable world state somewhere *g*)
Yes, each fragment has the same capabilities as a normal mnesia
table. A normal pattern would be to replicate each fragment and
spread fragments and replicas evenly across a cluster of computers.
This way, if one computer fails, that computer's load can be
reallocated evenly across the remaining ones.
A bit of clarification is perhaps in place. All objects stored
in Mnesia are mutable by default. It is possible to flag a table
as read-only, if you want to.
/Uffe
(*) OK, a slight warning here. The function objects can be
passed to another node and used there as long as the module,
from which it originated, is also present there, in roughly
the same version (preferrably exactly the same) as on the
originating node. The local scope of the function instance
is of course preserved even when it is passed to another
node.
--
Ulf Wiger, Senior Specialist,
/ / / Architecture & Design of Carrier-Class Software
/ / / Strategic Product & System Management
/ / / Ericsson AB, Connectivity and Control Nodes
Ulf Wiger
> Neelakantan Krishnaswami wrote:
>
> I realize efficiency wasn't explicitly mentioned as a language
> requirement, but according to the The Great Computer Language Shootout
> Ocaml's native compiler beats the pants off of Erlang in terms of both
> CPU and memory usage, often rivaling gcc:
>
> http://www.bagley.org/~doug/shootout/
>
> On the whole, Erlang's preformance is pretty shabby. Granted,
> the Shootout is far from a scientific undertaking, but it's
> something you might want to think about.
Oh yes, the Doug Bagley Shootout...
According to it, C and C++ also beat the pants off of Erlang.
Joachim wants to build some kind of WWW service. Thus, this
comparison between an Erlang-based web server and Apache
http://www.sics.se/~joe/apachevsyaws.html
might be of interest. Unless Apache is written in awk, scheme,
bash, or one of the languages trailing Erlang in the shootout,
one may have reason to suspect that perhaps the Shootout
(interesting as it is) may not be the Final Word on performance.
Also, Nortel's Erlang-based SSL offload appliance seems to be
the fastest of its kind, according to Network Computing
http://www.networkcomputing.com/1212/1212f4.html
"Alteon WebSystems' iSD 2.0 SSL Accelerator is our pick in the
external device category. It offers phenomenal performance,
management and scalability, and only F5's offering came close
to the top spot."
The performance of the major Erlang-based products is everything
but shabby. In many cases, they outperform competing products
designed in mainstream languages.
Micro benchmarks is one thing - building powerful non-stop
systems is another.
/Uffe
Christian Szegedy
>
>
> Be careful. One of requirements was: maturity. Besides, graphics and number
> of tools/librairies on the actual market is much more evolved for
> Python.
Ruby is ten years old. It is quite mature and stable right now.
> As far as elegance of the language is concerned - hmmmm... for me Ruby is a
> little
> too bit Frankensteinic, glued together from many sources for the sake of
> "pragmatism". But I won't fight against anybody here, newer language *must*
> appear, their criticism by old dinosaurs like myself *should* be
> counterbalanced
> by the fascination of younger people. But they won't convince me that
> $var, @var
> etc. Perlish conventions are elegant... Anyway...
I don't like Perls way, but Ruby has a much cleaner syntax, much
less conventions than Perl, and it is much more comparable to Python
than to Perl. It is a matter of taste, but for me Ruby is extremely
consistent. Particularly its functional features are more consistent
and natural than it Python.
Anyway, I would not suggest Ruby (nor Python or Perl) for large scale
mission critical code. I would say that the static typing is
very important for larger projects, since it adds an additional
layer of security to a project.
Therefore, I think, that OCaml (or Haskell) is superior to dynamic
languages, since the type inference allows you to omit most type
declarations, bit still having type-safeness in your programs.
Best Regards, Christian
Garry Hodgson
> But they won't convince me that $var, @var
> etc. Perlish conventions are elegant... Anyway...
indeed.
from http://www.rubycentral.com/faq/rubyfaq-2.html#ss2.1:
>
> The Ruby collection classes and iterators are outstanding, much
> more powerful and elegant than the ad hoc solutions that Python is
> sprouting (lambdas and list comprehensions).
yeah, we all know ad hoc those lambda things and list comprehensions are.
(though admittedly python's lambdas are a bit crippled).
----
Garry Hodgson, Senior Hacker, AT&T Labs
No act is more patriotic than speaking out when your government
is doing the wrong thing in your name. This is not your right
but your sacred duty. And none are more treasonous than those
who would silence these voices.
Michael Schuerig
> M H wrote:
>>
>> Which languages do you already know? This is an important piece of
>> information you forgot to mention in your posting...
>
> I didn't forget it, I omitted it on purpose :-)
> My CV lists 20 languages, of which I have actively used 13 in my life.
> Besides, it's not too important. I'm not going to be the only
> programmer on the job. (I'm not sure how much programming I'll be
> doing anyway, I may be too occupied with other tasks.)
>
> FWIW, here's the full list, for the curious:
> I'm fully proficient in:
> Eiffel (would do this again, though with reservations)
> Turbo Pascal (obsolete)
> C (if I must, I'll use it...)
> I have programmed in
[snip]
> I know (read: have read the reference manual, end to end; in some
> cases, even written some toy programs at <1000 LOC):
> Mozart/Oz (wonderful concepts, strange syntax)
> Haskell (nice syntax)
> Common Lisp/Scheme (nice ideas, but unsafe)
> Prolog (nice basic idea, but horrible for coding imperative stuff)
> Forth (cute... but not my world)
> Fortran (the mummy)
> Smalltalk (interesting, but I never found my way in the libs)
> I also know about ML/SML/OCaml, I even read parts of the OCaml manual,
> but I never used any of these languages :-)
[snip]
I'm surprised. To me it seems very adventurous to choose a kind of
language you're not proficient in for a seemingly very important
project. I hope you enjoy it and succeed, but I wouldn't bet money on
the later unless you work with others who know the chosen the language
intimately.
Michael
--
Michael Schuerig Failures to use one's frontal lobes
mailto:schu...@acm.org can result in the loss of them.
http://www.schuerig.de/michael/ --William H. Calvin
Galen Menzel
> Galen Menzel <galen...@yahoo.com> writes:
>
>> Neelakantan Krishnaswami wrote:
>>
>> I realize efficiency wasn't explicitly mentioned as a language
>> requirement, but according to the The Great Computer Language
>> Shootout Ocaml's native compiler beats the pants off of Erlang in
>> terms of both CPU and memory usage, often rivaling gcc:
>>
>> http://www.bagley.org/~doug/shootout/
>>
>> On the whole, Erlang's preformance is pretty shabby. Granted, the
>> Shootout is far from a scientific undertaking, but it's something
>> you might want to think about.
>
> Oh yes, the Doug Bagley Shootout...
>
> According to it, C and C++ also beat the pants off of Erlang.
> Joachim wants to build some kind of WWW service. Thus, this
> comparison between an Erlang-based web server and Apache
>
> http://www.sics.se/~joe/apachevsyaws.html
>
> might be of interest.
The idea of the Shootout is comparing identical algorithms in
different languages. Yaws outperforms Apache because it uses Erlang's
super-duper built-in user-space threading, instead of OS threading
like Apache. A web server written in C with a good user-space
threading library would most likely beat the pants off of Yaws. Not
that I'm defending C or anything :-)
> Unless Apache is written in awk, scheme, bash, or one of the
> languages trailing Erlang in the shootout, one may have reason to
> suspect that perhaps the Shootout (interesting as it is) may not be
> the Final Word on performance.
I never said that it was -- quite the opposite, in fact, if you'd care
to read my message.
> Also, Nortel's Erlang-based SSL offload appliance seems to be
> the fastest of its kind, according to Network Computing
> http://www.networkcomputing.com/1212/1212f4.html
>
> "Alteon WebSystems' iSD 2.0 SSL Accelerator is our pick in the
> external device category. It offers phenomenal performance,
> management and scalability, and only F5's offering came close
> to the top spot."
This is impressive, but one wonders what sort of speed is bought by
specialized hardware.
> The performance of the major Erlang-based products is everything
> but shabby. In many cases, they outperform competing products
> designed in mainstream languages.
>
> Micro benchmarks is one thing - building powerful non-stop
> systems is another.
My intention was not to say Erlang is worthless or anything of the
sort. It has obviously proven its worth, both practically by running
some enormous systems (as well as SSL cards) and evangelically by
drawing some well-deserved attention to the FP community. But as
Joe's response to my first posting said, "Erlang was not designed for
fast sequential processing - it was designed for efficent handling of
concurrent processes". The results of the Shootout are a consequence
of this. If I were choosing between two languages for a major
project, I would be interested in any comparisons of their
performance, which is why I pointed this out; I give Joachim the
intellectual credit to be able to discern which tests are pertinent to
his project and which are not.
galen
Donn Cave
...
| Anyway, I would not suggest Ruby (nor Python or Perl) for large scale
| mission critical code. I would say that the static typing is
| very important for larger projects, since it adds an additional
| layer of security to a project.
|
| Therefore, I think, that OCaml (or Haskell) is superior to dynamic
| languages, since the type inference allows you to omit most type
| declarations, bit still having type-safeness in your programs.
I haven't ever done large scale mission critical code, but in
the less ambitious work I do, I have noticed that static typing
catches a class of frustrating, annoying errors that plague me
when programming in Python, specifically.
But the Python enthusiasts claim that this is irrelevant in just
this case of mission critical code, because the development methodology
must include rigorous testing that so catches everything the type
system would catch and then some.
Donn Cave, do...@drizzle.com
Neelakantan Krishnaswami
> Galen Menzel <galen...@yahoo.com> writes:
>
> > Neelakantan Krishnaswami wrote:
> >
> > I realize efficiency wasn't explicitly mentioned as a language
> > requirement, but according to the The Great Computer Language Shootout
> > Ocaml's native compiler beats the pants off of Erlang in terms of both
> > CPU and memory usage, often rivaling gcc:
> >
> > http://www.bagley.org/~doug/shootout/
> >
> > On the whole, Erlang's preformance is pretty shabby. Granted,
> > the Shootout is far from a scientific undertaking, but it's
> > something you might want to think about.
Just a note that the quoting is off: I didn't write this!
--
Neel Krishnaswami
ne...@alum.mit.edu
Galen Menzel
> I realize efficiency wasn't explicitly mentioned as a language
> requirement, but according to the The Great Computer Language Shootout
> Ocaml's native compiler beats the pants off of Erlang in terms of both
> CPU and memory usage, often rivaling gcc:
>
> http://www.bagley.org/~doug/shootout/
>
> On the whole, Erlang's preformance is pretty shabby.
I was just reading through this and realized that my statement isn't
so clear. I meant on the whole, Erlang's performance *in the
Shootout* is pretty shabby, not necessarily in general. Sorry for any
confusion.
galen
Miles Egan
> Be careful. One of requirements was: maturity. Besides, graphics and number
> of tools/librairies on the actual market is much more evolved for Python.
This is true but I rarely find in practice that something I need isn't
available for Ruby.
> far as elegance of the language is concerned - hmmmm... for me Ruby is a little
> too bit Frankensteinic, glued together from many sources for the sake of
> "pragmatism". But I won't fight against anybody here, newer language *must*
> appear, their criticism by old dinosaurs like myself *should* be counterbalanced
> by the fascination of younger people. But they won't convince me that $var, @var
> etc. Perlish conventions are elegant... Anyway...
A common misconception. $ and @ in perl are type prefixes associated
with all kinds of crazy semantics. In Ruby they're scope prefixes and
serve the same purpose that "self." and the convention of m_ and g_ do
in Python.
Python feels far more organic and disorganized to me than Ruby does.
In Ruby *everything* is an object and the standard library goes to
great lengths to make sure that similar operations on similar types
are accomplished in similar ways. I spend half of my Python time
fumbling around the module reference looking for the functions I need.
Some functions in Python are destructive while others return a copy,
with no logic or convention to distinguish them. Why is sort() only
available as an in-place sort? Why is len() a freestanding function
and why is there no list.length function? Ruby rigorously
distinguishes in-place from non-destructive operations (sort vs
sort!). In Ruby I can just assume that any container with a countable
number of elements will have a size method and I don't have to look in
the docs.
Python continues to sprout extensions. Generators, properties, object
slots, list comprehensions, type unification for basic types. It's
all catchup to Ruby and the resulting mishmash is a mess.
And don't get me started on the syntax. Syntactically significant
whitespace is a *bad* idea. I liked it when I started with Python,
but I've seen too many production hotfixes blown because someone used
the local vi and inserted a tab when the file was all spaces. I've
spent too much time tediously reformatting code by hand from a
newsgroup cut & paste that emacs should be able to beautify for me.
The whole model falls apart when you start embedding code in files
with other syntax like html. You have to invent a whole new
expression language just so you can mix your code in the template.
Look at Zope's DTML for an example of the jumble this becomes. I'll
give Guido credit for trying this idea out, but it just doesn't work.
Ruby has its warts but Python's not even in the same league.
--
miles egan
mi...@rddac.com
reverse the domain name to find me
OpenPGP/GPG Key 0x01F53D51 @ wwwkeys.us.pgp.net
Luke Gorrie
> (*) OK, a slight warning here. The function objects can be
> passed to another node and used there as long as the module,
> from which it originated, is also present there, in roughly
> the same version (preferrably exactly the same) as on the
> originating node.
But then, one of the Erlang modules available on every node is an
Erlang interpreter. You can use this to evaluate a genuinely arbitrary
function on any node, if you want to.
(The interpreter is included so that you bring up a shell (repl) on
your Erlang nodes.)
(I know that you know this already, Ulf :-))
Cheers,
Luke
Garry Hodgson
> Joachim Durchholz <joac...@gmx.de> writes:
> > The link is broken.
>
> So it is. Try http://www.erlang.se/workshop/2002/wiger.pdf
> instead.
that should be http://www.erlang.se/workshop/2002/Wiger.pdf
Garry Hodgson
> And don't get me started on the syntax. Syntactically significant
> whitespace is a *bad* idea.
ah, the dreaded Significant Indentation debate, always popping up
where you least expect it. the lack of visual noise is a truly wonderful thing.
does it bother you as much in haskell as it does in python?
> I liked it when I started with Python,
i hated it at first, but quickly became quite enamored. still am.
> but I've seen too many production hotfixes blown because someone used
> the local vi and inserted a tab when the file was all spaces.
you just can't get good help these days.
> I've spent too much time tediously reformatting code by hand from a
> newsgroup cut & paste that emacs should be able to beautify for me.
i find two quick global replaces does the trick nicely. i'd think that
emacs could handle that.
> The whole model falls apart when you start embedding code in files
> with other syntax like html.
now, *there* is a Bad Idea.
Joachim Durchholz
> Joachim wants to build some kind of WWW service.
It's not going to be an HTTP server, but the general structure is indeed
the same: a central server (possibly distributed over several physical
machines), with many clients exchanging data with it.
> Thus, this
> comparison between an Erlang-based web server and Apache
>
> http://www.sics.se/~joe/apachevsyaws.html
>
> might be of interest.
That's in fact one of the advocacies I'm aware of :-)
I do agree with Galen that Erlang is probably not the fastest language
around, at least not for heavy data processing.
On the other hand, I don't think it's *that* important. The project will
do without a database; this alone should improve performance by a hefty
factor.
Of course, a faster language would still allow improvements - but if the
language gives an improved programmer productivity of 10%, the salary of
a single underpaid under-productive sub-average programmer will make up
for an additional server.
It's essentially a question what the money should be spent for: for a
faster machine, or for a faster programmer.
Since Moore's Law doesn't hold for programmers, I know what side I'll
place my bets on :-)
All of this assuming that the slowdown for an interpreted language is
just a constant factor.
Nonlinear inefficiencies would be an entirely different game. For
example, if the system's garbage collection algorithm had some hidden
quadratic behaviour in it, this project would almost certainly detect
it. I have seen at least one report of such an event, so that's
something that I'd most definitely want to test for before deciding for
a language.
Regards,
Jo
Joachim Durchholz
>
> I'm surprised. To me it seems very adventurous to choose a kind of
> language you're not proficient in for a seemingly very important
> project.
On the other hand, I know that this project cannot be done in a
traditional, imperative programming language with the required
programmer productivity.
Besides, I'm pretty quick at picking up new languages and paradigms.
Actually I've been in love with functional concepts since my first
encounters with Lisp (gosh - that's over two decades in the past).
> I hope you enjoy it and succeed, but I wouldn't bet money on
> the later unless you work with others who know the chosen the language
> intimately.
Hmm... guess why I'm going to post a "job offered" message here once the
project is underway :-)
(IF it gets underway. It's the people with the money who decide...)
Regards,
Jo
Joachim Durchholz
> Anything that you can bind to a variable in Erlang can also be
> sent in a message. Some data objects uniquely identify local
> resources. Most of these can be passed to another processor
> and still be useable.
Heck, that's beyond what the project will need. I was just talking about
RAM-based data. (Well, in the case of Erlang, also Mnesia-based data -
but if it indeed replicates as described, it's locally available anyway.)
> There are different concurrency models, and some of them are indeed
> notoriously difficult to manage. Erlang handles concurrency in a
> manner similar to CSP. Due to active message reception and
> selective receive with implicit buffering, Erlang programs can
> avoid many of the problems with race conditions with relative ease.
> The short explanation is that you are not forced to handle
> messages in the exact order that they arrive, but can prioritize
> between different senders (and even between messages from a single
> sender). This leads to much more stable and scalable programs.
Hmm... right... that takes out the problems with race conditions that
I'd call "low-level". No need to worry about misprogrammed ring buffers
if the library offers a powerful message queuing simulation.
"High-level" race conditions could still happen if a complex stateful
protocol of messages with lots of "impossible" states would be used.
That's a (slight) design constraint. But then the main code should be
stateless, so I don't expect this to be a problem :-)
> In many cases, a program can be re-structured through the addition
> of a process (e.g. changing from a local data structure to a
> client-server model) without changing much of the code, if any.
>
> If you had defined proper access functions for your local data
> structure, you can often switch to a communication-based solution
> without changing the access interface. I recently did so in an
> example, where I took the standard 'dict' (hash dictionary) module
> and made an API-compatible fully replicated dictionary.
>
> The basic idea is this:
>
> lookup(Key, Dictionary) ->
> dict:lookup(Key, Dictionary)
>
> is changed to
>
> lookup(Key, Dictionary) ->
> Ref = make_ref(),
> Dictionary ! {self(), Ref, {lookup, Key}},
> receive
> {Ref, Result} ->
> Result
> end.
>
> (Certain details omitted.) To the surrounding code, this still
> looks like a local function call. The solution is safe from race
> conditions, since the message reception clause will ignore (buffer)
> any messages that do not fit the patern {Ref, Result} (where Ref
> is bound, and Result is not).
Good thing.
> A bit of clarification is perhaps in place. All objects stored
> in Mnesia are mutable by default. It is possible to flag a table
> as read-only, if you want to.
Ah, right, now I recall.
I'm not sure whether that's too relevant. If the language makes
programming in a stateless manner easy and fun, the number of
state-related bugs should be low.
> (*) OK, a slight warning here. The function objects can be
> passed to another node and used there as long as the module,
> from which it originated, is also present there, in roughly
> the same version (preferrably exactly the same) as on the
> originating node. The local scope of the function instance
> is of course preserved even when it is passed to another
> node.
I think this will be handled by an update service anyway :-)
Regards,
Jo
Joachim Durchholz
>
> First a comment - you haven't *ranked* your requirements - it would be
> helpful to know your rankings.
Let me quote from my original post:
"Here's the list of criteria, in roughly decreasing order of relevance:"
I admit it's easy to overlook details in an overly long post like mine.
Also, I have been intentionally vague on some points - I didn't want to
see answers that are slanted towards my expectations of relative
relevance. (For the latter, I have gone to more lengths than is obvious:
there are some criteria that I haven't written, simply because they just
reflect personal prejudices of mine. For example, if I were to select
the language strictly according to personal preferences, static typing
would have been quite high on the list, but I realized that this is not
one of the things that make or break a project, so I dropped it from the
list.)
> Since you said earlier "mission critical" (bet you business stuff) I'd
> have ranked points concerning fault-tolerance way higher than GUIs etc.
>
> IMHO as projects get bigger and scale into industrial size the
> stuff on fault-tolerance becomes more and more important.
>
> *anybody* can get a 50 line program working in any language - with or
> without a type system, debugger, ... you name it.
>
> If you are talking mega lines of code how you handle errors and how you
> organize large software projects becomes the critical factor.
Hmm... that's something that I forgot: project size.
I'd say that the project is in the 3-10 person-year league. Not really
large, but large enough that information hiding and the whole stuff
begin to really matter.
> So whatever language you choose you need evidence that the
> technology has been used in similar projects to whatever your project
> is.
The one unusual aspect is sheer in-RAM data volume. Fortunately, this is
something that one can write tests for with reasonable effort :-)
(E.g. generate a few large trees, then forget the root. Explicitly start
GC a few times. Do some networds of randomly connected nodes, then start
to forget the various nodes that point into it. Allocate memory blocks
with an inversely exponential size distribution and forget them; repeat
the latter and measure times, let the machine run for a night and look
at the results in the morning. Once the test suite runs, ask for
volunteers with really large machines who'll re-run the tests. Etc.)
> IMHO true independence of concurrent activities is critical for
> design and programming fault-tolerant software - I know no other way of
> making fault-tolerant software. Fault tolerance *implies* TWO of more
> machines (sorry for shouting) - no matter how good your software is
> the hardware might fail - thus you need (at least) two machines.
Agreed.
> This (the fact that you need two or more machines) gets you into
> distributed processing (whether you like it or not) - You need
> therefore a language that can detect "remote" errors (ie exceptions in
> a remote machine) - This is what Erlang and the OTP libraries were
> *designed* to do.
Good point.
> To make a fault-tolerant system you could design a fault-tolerant
> architecture and then program the abstractions you need in
> *any* language - this would be more or less difficult depending upon
> the language which you chose.
I'm not sure that this is that easy.
Assume you have several servers, each processing different slices of
data. I may find that the load is badly balanced, and want to move some
data over to the other machine.
This means that I have to "cut" the computation on one machine "in two"
so that I can move it over to the other machine. With, say, Java Beans,
that's incredibly difficult: you have to write marshalling and
unmarshalling code for all the routines that get exchanged over the net,
you have to trap additional exceptions, and probably do a lot of other
things that I haven't even begun thinking about. (Of course, you could
pack everything into a Java Bean, but that would be prohibitively slow.
I'm not an efficiency fetishist, but I do have my limits *g*)
> The OTP system *is* just this - it's in fact about the third total
> re-write of such a system - using this (The OTP behaviors) will save
> you a *lot* of work. IMHO the difficult bit is not the language but
> coming up with the right abstractions to use when programming a large
> fault-tolerant system - this takes many years and is very difficult to
> get right first time.
Hmm... I can see your argument.
I can't tell where the limits of its validity lie - I don't have any
experience with hardware fault tolerance. But yes, fault tolerance is
definitely a major factor; the project would do well with, say, four
nines (instead of the six-nines reliability required for Ericson's telco
switches), but even 99.99% is still more than you get without planning for.
Regards,
Jo
Joachim Durchholz
> Quoth Joachim Durchholz <joac...@gmx.de>:
> ....
> | The real problem is not getting it, it's the time that it takes to get
> | it. That's time in which programmers are largely unproductive, because
> | everything that they write in that time will eventually have to be
> | rewritten.
> | (That's one of the things that make Erlang attractive. Some previous
> | posts by Erlang advocates have made quite strong statements that the
> | language is easy to "get".)
>
> All language advocates say that, don't they?
Well, not really.
Haskellers tend to moan about the complexities of monads (don't ask me
*which* complexities). Lispers often take pride in being a special tribe
of especially gifted and talented programmers (which they often are).
The attractive part of the Erlang advocacy was that "we have normal
application programmers working in Erlang, and even these became more
productive in Erlang, with minimal training".
Of course, all of these statements are to be taken with a grain of salt.
I'll have to take a look into the Erlang mailing lists and see what the
recurring themes are for Erlang :-)
> Maybe it depends on what
> you get. They say ``you can write FORTRAN in any language'' - so when
> you've mastered a language well enough to do that, do you get it? It
> sounds like you hold a higher standard.
Most definitely... *grin*
Regards,
Jo
Joachim Durchholz
> Ulf Wiger <ulf....@CUT-ericsson.com> wrote:
>
>>Joachim Durchholz <joac...@gmx.de> writes:
>>
>>>The link is broken.
>>
>>So it is. Try http://www.erlang.se/workshop/2002/wiger.pdf
>>instead.
>
> that should be http://www.erlang.se/workshop/2002/Wiger.pdf
Ah... finally... many thanks.
(The thing is printing right now.)
Regards,
Jo
Joachim Durchholz
> Joachim Durchholz wrote:
>
>>I'm going to choose the language for a commercial project.
>
Seen it, programmed in it, got the t-shirt, rejected it.
Main reason: static typing without type inference. It's just too much
noise that needs to be written (and, far too often, corrected: Having to
re-run the compiler just because a type declaration was run has stolen
me too much productive time).
Other main reason: it's "too imperative for my taste". Whatever that
means :-)
Well, more specifically: I'm subscribing to the Church Of Aliasing
Considered Evil When Objects Are Mutable, as I have seen too many
aliasing problems in my life to have any confidence in large software
that persistently uses mutable objects. In other words, if there are
mutable objects, I insist that a single person must be able to
understand all interactions. (The bad thing about aliases is that it
isn't modular: module interfaces won't tell you whether aliases will
upset them or not, you'll have to inspect the entire code to find that out.)
> Less mainstream: Have you look at the E language?
I don't know about E.
Any URLs? Specific strengths? Specific weaknesses (I know I'm asking
much *g*)
>>3. Network transparency
>>Code shouldn't care whether an object that it's accessing is local or
>>remote.
>
> Are you sure you want this?
Actually, no.
I just want to be able to move some data and associated computations
from one machine to the other, without having to rewrite the entire system.
> I've seen people think they want this, but
> my suspicion is that it's often a mistake. The semantics of touching
> remote data are different from the semantics of touching local data:
> there are different failure modes, different time scales, different
> blocking/non-blocking behavior, different deadlock possibilities.
> My suspicion is that it's often better to expose this to the programmer,
> rather than hiding it from them. But then, you'd know your requirements
> far better than I, so you've probably considered these issues.
Partly - one learns new things with every project, and this is one of
them for me.
My (tentative) ideas about the issues that your raise go like this:
* different failure modes: should be handled at the higher levels of the
software, and result in an abort of the computation (and, depending on
context, either a reissue of the entire job on a different server, or a
panic message and/or a machine reboot or whatever is necessary to handle
the failure).
* different time scales: yes, but that's more a problem of identifying
the right set of data to migrate. I know of no better method like
analyzing access patterns, and lots of trial and error. Does anybody
have a better approach?
* different blocking/non-blocking behaviour: this is exactly the type of
problems that I want *not* to have. One of the good things about
functional programming is that this is not an issue as long as the
software is just computing functions. Of course, the differences
reappear as soon as the software effects changes, but the functional
approach reduces the stateful part of the computation, so it should be
less of a problem than usual.
* different deadlock possibilities: not really an issue, deadlocks are
relevant if you have limited resources. Again, in a largly-functional
setting, most of the time you're computing function results, which is an
unlimited resource.
Regards,
Jo
Miles Egan
> Miles Egan <mi...@rddac.com> wrote:
>
>> And don't get me started on the syntax. Syntactically significant
>> whitespace is a *bad* idea.
>
> ah, the dreaded Significant Indentation debate, always popping up
> where you least expect it. the lack of visual noise is a truly wonderful thing.
> does it bother you as much in haskell as it does in python?
Python strikes a particularly bad balance between lisp-like syntactic
regularity and perl-like terseness. The syntax is so homogeneous that
it's hard to parse, yet not regular enough that you can do anything
like lisp macros with it.
I haven't done any Haskell programming yet. Lots of Ocaml and a
little Clean.
>> but I've seen too many production hotfixes blown because someone used
>> the local vi and inserted a tab when the file was all spaces.
>
> you just can't get good help these days.
This will happen to you one day. You heard it here first.
>> I've spent too much time tediously reformatting code by hand from a
>> newsgroup cut & paste that emacs should be able to beautify for me.
>
> i find two quick global replaces does the trick nicely. i'd think that
> emacs could handle that.
Emacs is supposed to reconstruct lost semantic information? Sorry, no
software is that smart. How does it know I meant this:
class A:
"""i'm a properly commented class"""
def name(): return a
vs this:
class A():
"""i'm a properly commented class"""
def name(): return a
If you have emacs code that fixes this you should sell it for billions
and retire. Cut & paste becomes a lossy operation with this syntax.
>> The whole model falls apart when you start embedding code in files
>> with other syntax like html.
>
> now, *there* is a Bad Idea.
Embedding code in html is a bad idea. Here's a better one:
Tapesty, a java web framework, lets you script events in any
BSF-compatible language, including jython. You can do stuff like
this:
<component id="test">
<listener>
def python_func(bean):
bean.set_foo("jython")
</listener>
</component>
Suddenly making sure all the xml & python combined whitespace lines up
just right doesn't sound like so much fun, does it?
Garry Hodgson
> In article <HDrIB...@research.att.com>, Garry Hodgson wrote:
> > Miles Egan <mi...@rddac.com> wrote:
> >> but I've seen too many production hotfixes blown because someone used
> >> the local vi and inserted a tab when the file was all spaces.
> >
> > you just can't get good help these days.
>
> This will happen to you one day. You heard it here first.
i doubt it. but of course, one cannot reasonably refute such a statement.
by the way, you'll get hit by a bus someday. sorry. you heard it here first.
> Emacs is supposed to reconstruct lost semantic information? Sorry, no
> software is that smart. How does it know I meant this:
>
> class A:
> """i'm a properly commented class"""
> def name(): return a
>
> vs this:
>
> class A():
> """i'm a properly commented class"""
> def name(): return a
but you've deleted code here. delete the curly brackets from a
C program and you have the same problem.
> Embedding code in html is a bad idea. Here's a better one:
>
> Tapesty, a java web framework, lets you script events in any
> BSF-compatible language, including jython. You can do stuff like
> this:
>
> <component id="test">
> <listener>
> def python_func(bean):
> bean.set_foo("jython")
> </listener>
> </component>
same difference. why anyone would want to do this is beyond me.
if i never had to look at another <tag> again, it'd be too soon,
let alone mixing real code into it. ugh.
> Suddenly making sure all the xml & python combined whitespace lines up
> just right doesn't sound like so much fun, does it?
mixing xml and any language qualifies as No Fun in my book.
in any event, i doubt this group is real interested in The Significant Indenting debate.
nor am i, for that matter. if it causes you problems, by all means, don't use it. in
my experience, these drawbacks are just not a problem in practice. your mileage may vary.
Mark Alexander Wotton
> Jerzy Karczmarczuk wrote:
>
>> [Python] debugging is much easier than in Caml
>
> In what way? What does Python debugging have that Caml (for UNIX) doesn't?
If I might hijack the discussion for a second: this is one of the killer
features of python for me. It is *wonderful* to be able to inspect modules
from the REPL. From the python interpreter, I can see inside modules, and I
can generally find what I'm looking for very quickly. I really miss having
this in ghci: it's a wonderful feature for library exploration.
mrak
--
realise your life was only bait for a bigger fish
-- aesop rock
gr...@cs.uwa.edu.au
:> | it.
Assuming you're employing people with some functional background, the
only things they'll need to get used to are laziness and possibly the
static type system and interpreting type error messages, depending on
where they came from. The latter is a minor speedbump, the former will
only occassionally rear its head, and when it does, you will probably
make the leap from bizarrely poor performance to where's the profiler
yourself, but remembering laziness while you're trying to work out
what's happening might take some getting used to.
:> | (That's one of the things that make Erlang attractive. Some previous
:> | posts by Erlang advocates have made quite strong statements that the
:> | language is easy to "get".)
I think its easier to "get" Erlang to the point where you can write
your FORTRAN/Java/whatever programs in it, but I actually think it's
much easier to "get" functional programming in Haskell.
:> All language advocates say that, don't they?
: Well, not really. Haskellers tend to moan about the complexities
: of monads (don't ask me *which* complexities).
I used to moan about this, but now I think that the complexity is
almost entirely in trying to explain how they maintain referential
transparency, how they fit in with the type system, and why these
things are so important. If you just plonk someone down and tell
them "some functions return (IO Sometype) instead of Sometype. If
you want to get at their return value, you'll have to do that in a
function that also returns (IO SomeThingOrOther). Write those with
the do notation, but try to keep them to a minimum and at the top
level." they will get the hang of it in no time. And it will still
be a bugger to teach them the theory. :)
-Greg
Ketil Malde
>> Well, not really. Haskellers tend to moan about the complexities
>> of monads (don't ask me *which* complexities).
> I used to moan about this, but now I think that the complexity is
> almost entirely in trying to explain how they maintain referential
> transparency, how they fit in with the type system, and why these
> things are so important.
And not the least the fact that if you look for documentation, you'll
find heaps of papers describing how to accomplish all sorts of
mysterous and wonderful things with them. Not the kind of things you
want to tangle with if you're just trying to read a file for the first
time...
-kzm
--
If I haven't seen further, it is by standing in the footprints of giants
Ketil Malde
> If I might hijack the discussion for a second: this is one of the killer
> features of python for me. It is *wonderful* to be able to inspect modules
> from the REPL. From the python interpreter, I can see inside modules, and I
> can generally find what I'm looking for very quickly. I really miss having
> this in ghci: it's a wonderful feature for library exploration.
Didn't somebody (Jan Skibinski?) suggest a :browse command a while
ago? Along with a graphical shell integrated with Smalltalk, IIRC?
I would imagine ghci already keeps information about what functions
are available from where (cf. the :info command), so listing them
(preferably along with comments) shouldn't be too difficult?
Mark Alexander Wotton
Oh, cool. :b gives at least the type signatures and names of functions,
which is certainly a start. No documentation strings a la Python, but
just being able to browse is a big help.
Ulf Wiger
A thousand apologies. I must learn to copy-and-paste...
/Uffe
--
Ulf Wiger, Senior Specialist,
/ / / Architecture & Design of Carrier-Class Software
/ / / Strategic Product & System Management
/ / / Ericsson AB, Connectivity and Control Nodes
Ken Rose
> Since I didn't expect to get an imperative language recommendation,
OK, since you've thrown down the gauntlet...
If I were trying something like what you describe, and being forced (or
even just strongly inclined) to use an imperative language, I'd do it in
Modula-3. I think it meets all of your criteria. Most of them, anyway.
OCaml or Erlang are probably better choices, though.
- ken
Stefan Axelsson
> I haven't ever done large scale mission critical code, but in
> the less ambitious work I do, I have noticed that static typing
> catches a class of frustrating, annoying errors that plague me
> when programming in Python, specifically.
Well I have (and do), and even though I'm on the static typing side of
the debate, it is my experience that the kind of errors you refer to
(in our case it's Erlang) are never detected by our customers. Even
though we *make* those errors all the time.
> But the Python enthusiasts claim that this is irrelevant in just
> this case of mission critical code, because the development methodology
> must include rigorous testing that so catches everything the type
> system would catch and then some.
Certainly. My main gripe is then with the time in the project those
errors are detected. I'd like to detect such "simple" errors earlier,
when they cost less. The AXD301 people then like to rub our noses in
their much shorter test turnaround, which makes this particular point
moot, but that's life... ;-)
That's not to say that they're worth bothering about from a project
perspective. The real cost is in the "real" errors, the ones that (at
least a naive application of a) static typesystem wouldn't
catch. I.e. misunderstood/miscommunicated/ misstandardised
specifications.
Stefan,
--
Stefan Axelsson (email at http://www.ce.chalmers.se/staff/sax)
Ulf Wiger
> In article <1051024787.646071@yasure>, Donn Cave wrote:
> > I haven't ever done large scale mission critical code, but in
> > the less ambitious work I do, I have noticed that static typing
> > catches a class of frustrating, annoying errors that plague me
> > when programming in Python, specifically.
>
> Well I have (and do), and even though I'm on the static typing side of
> the debate, it is my experience that the kind of errors you refer to
> (in our case it's Erlang) are never detected by our customers. Even
> though we *make* those errors all the time.
Personally, I try to always be on the winning side of the debate(*),
whichever side that is. ;)
Lately, I've spent some time working with Joe Armstrong's UBF
(http://www.sics.se/~joe/ubf/site/home.html). In Joe's own words:
"UBF is a language for transporting and describing complex
data structures across a network. It has three components:
* UBF(A) is a data transport format, roughly equivalent
to well-formed XML.
* UBF(B) is a programming langauge for describing types
in UBF(A) and protocols between clients and servers.
UBF(B) is roughly equivalent to to Verified XML,
XML-schemas, SOAP and WDSL.
* UBF(C) is a meta-level protocol between used between
UBF servers."
The whole idea is "programming by contract" in a language-
neutral fashion (by sheer accident, the syntax is somewhat
reminiscent of Erlang). I'm currently using it for designing
the server part of a networked application with Java clients
and an Erlang server.
The UBF contract checker makes sure that all messages coming
in from the outside are readable (UBF(A)), type-correct
(UBF(B)), and that they are relevant to the current state of
the server (UBF(C)). Replies are also checked, of course.
I find it extremely helpful to have a contract checker that
tells me whenever I make type errors in my code. Whether
these errors are detected dynamically or statically is
secondary, I think (messages from an external client,
written in some other language, usually have to be
manually verified -- UBF generates that code.)
I've written a small program that generates Erlang record
definitions from an UBF contract. This allows me to get
some compile-time checks even in Erlang, knowing that
if the data structure arrived via a UBF-enabled port, all
parts of the structure have been type-checked.
I also think, although I haven't verified it yet, that
one could define a contract with UBF, and then generate
most of what's needed for XML-RPC or SOAP clients. I see
no reason why these services could not be added quite simply
so that they also pass through the contract checker.
This would be a lot more productive than working with XML
directly.
It would be interesting to see how UBF would work with
other functional languages, esp. those with greater emphasis
on static typing than Erlang
/Uffe
(*) Winning side = whichever allows me to do my job
in the most productive and elegant fashion, according
to my own subjective standards.
Peter G. Hancock
Thanks! That looked pretty interesting.
I looked mostly at the paper, which talks only about ubf(a) and ubf(b).
Dif ubf(b) split into 2? Is it possible to give a hint about why/how
it split?
A metaphor of contracts is used also in the refinement calculus of
Back and von Wright (at Turku). The RC notion of contract is vastly
more powerful, allowing for exact expression of specifications, and
(therefore) no basis for such a nice simple checking tool. But maybe
it might be worth having a look at it as background if you ever want
to extend the notion of type in ubf, or extend its scope.
Has anyone tried to use it in situations where the server can callback the
client, or there are cycles in the arrangement of components?
Peter Hancock
Stefan Axelsson
> Personally, I try to always be on the winning side of the debate(*),
> whichever side that is. ;)
Always a good side(**) to be on... ;-)
> Lately, I've spent some time working with Joe Armstrong's UBF
> (http://www.sics.se/~joe/ubf/site/home.html). In Joe's own words:
This looks interesting, and lighter weight than other proposals
(.NOT). I might have a closer look in my copious free time... ;-)
> (*) Winning side = whichever allows me to do my job
> in the most productive and elegant fashion, according
> to my own subjective standards.
(**) Good side = which ever allows me to get my papers published with
as little effort as possible according to the standards set by
the relevant program committee. ;-)
Bijan Parsia
[snip]
> Smalltalk: Not on my list.
> I see two show-stoppers here.
> One is that every Smalltalk is incompatible with every other Smalltalk.
There is an ANSI Standard for the core of Smalltalk. You might look at the
Camp Smalltalk efforts for cross (all) dialect projects and the
difficulties (or easinesses) thereof.
> You live and die with the Smalltalk implementation that you chose. (Not
> a problem with Squeak AFAIK, but I know of no other implementation that
> doesn't have this Achilles heel.)
Er...do you mean that since Squeak is open source...uhm. you still live or
die with the implementation. Or did you mean *vendor*. There are other
open source Smalltalks (see GNU Smalltalk).
> The other is that there is no efficient referentially transparent
> container data structure (at least not in the standard classes).
[snip]
Can't help you there. Given the sorta function container manipulators
(#collect:, #select:, etc) fixing this is a Small Matter of Programming. I
believe the Gemstone dialect might have a bit of this.
Cheers,
Bijan Parsia.
axxackall
> I would agree with Erlang, but I think it only just edges out Ocaml.
> It's #3 that pushes it to Erlang. In most other respects I think
> Ocaml is a little superior, but I don't think there is any other
> language with concurrency support remotely as good as Erlang's.
How about Mozart? Doesn't it have the same level of distributed concurrency?
IMHO Mozart combines good sides of both Erlang, Ocaml and Haskell.
Neelakantan Krishnaswami
I thought about listing it, but I decided against it. I've never
programmed in Mozart, so take what I say with a grain of salt: it
seemed to me to be like SML/NJ's Concurrent ML. Like CML, it seems
like it has a really good basic model, but it doesn't really seem to
have been through the baptism-of-fire of actually being used out in
the field on big projects.
That kind of general bullet-proofing is very valuable, and I think
Erlang is the only concurrent language with a serious claim to having
it.
--
Neel Krishnaswami
ne...@alum.mit.edu
Joachim Durchholz
figures have been too impressive to be ignored, despite I'm more a
static typing guy :-)
Regards,
Jo
Hartmann Schaffer
it would be interesting to see occasional experience reports (e.g. how
it matches up to your expectations)
hs
--
ceterum censeo SCO esse delendam
Ulf Wiger
Good for you. (:
I strongly suggest that you sign up to the
erlang-q...@erlang.org mailing list. Many helpful
erlangers do not read this newsgroup.
/Uffe
Joachim Durchholz
> I strongly suggest that you sign up to the
> erlang-q...@erlang.org mailing list. Many helpful
> erlangers do not read this newsgroup.
I'm already in it, though I have been just lurking :-)
I'm still in the phase of absorbing as many characters of documentation
per second as I can (besides market analysis, writing and rewriting
business plans, redefining the scope of the business, negotiating for an
Internet server, finding a good name for the business and for the
Internet presence, and a thousand other entirely marketroid-style
business suit things that aren't really related to programming...).
But things are finally getting ahead, I'll be starting with Erlang Real
Soon Now ;-)
Regards,
Jo
Joachim Durchholz
>>To those who may interest it: I have chosen Erlang.
>
> it matches up to your expectations)
OK, I'm staring this under the topic "A Newbie's Erlang Diary".
Regards,
Jo