Having struggle in understanding FP (and Clojure question)
cwyang
First of all, thanks for the joyful language, Clojure.
I am having struggle in understanding functional programming,
which Clojure dictates.
Though I'm not a big fan of object oriented programming,
I totally agree with the model of objects - objects and methods,
entities and functions in other words, or
states and means to change the states.
So, I understand as follows:
- OOP: keeping objects, which has states and methods.
methods are encapuslated to the corresponding object.
- FP: keeping objects(or structs or variables, whatever it is called)
and functions on the objects.
functions does not encapsulated to specific objects, for some
reasons.
Other characteristics such as function as first-order data structure
and
enforcing functional way (discourated side effects and immutable
variables)
can be important, but not in modeling the world.
So, main question is these.
(1) I might know that there are some benefits not to bind functions to
specific
objects (class). But I think that functions can be tied to
specific objects (class)
and it's more convienient in many situations. By tieing, I do mean
logical concept,
not physical syntax.
So, I guess that any small piece of programs in FP consist of
entity definitons and
corresponding functions which operates on the entity.
Am I right? Is it correct or idiomatic ways in FP?
(2) Second question is specific to Clojure.
Let's think of an example. We need to make a Web server.
Session is typical modeling entites in this problem.
Session begins(instantiates) upon the arrival of a client request,
receives HTTP request, parses the request, search the storage for
given
requests, generates the response, and sends the response.
Since Session has to keep states such as client information,
received requests
and corresponding data, I'll create Session as an object in
conventional OOP and
call the method like this on a dedicated thread.
do_session() {
receive_and_parse();
lookup();
generate();
send();
}
I know it's easy way to do like this, rather than do
asynchronosly.
In Clojure, what's the best way?
My first guess is:
(def session ..)
(defn do_session [..] (receive_and_parse) (lookup) (generate)
(send))
Since the default stack size of a Java thread (over 512KB) is much
bigger than
that of native threads (8KB NPTL), above approch may suffer from
low number of
maximum concurrent threads. So my second guess is using _agent_ in
Clojure.
As far as I understands, agent in Clojure is a state which can
send and
receive messages. Then, by using agent in Clojure, does one must
do
a kind of state-machine programming?
With above example,
session agent can
have :initial, :after_receive, :after_lookup, :after_generate,
and :end.
I may use an asynchornous IO facility (I never used Java, but I
guess there is one)
to dispatch IO completion notification and send messages to the
agents.
But this is too complex! Everybody favors thread programming
because it's easy.
In each state of session agent, there can be sub-state like:
:in_receving, :in_sending, :in_looking, ... for nature of IO
operations.
So complex. Asynchornous IO with state management is inherent
comple, in my opinion.
So my third guess is "There must be a right way to do this."
Maybe this question is related to the first one.
But for the lack of experience in FP and clojure, I don't know how
to.
Thank you.
- cw
Stuart Halloway
I have been converting the examples from "Practical Common Lisp" into
Clojure. A big part of this is switching from imperative to functional
mindset. You might want to look at Chapters 3, 16, and 17.
http://blog.thinkrelevance.com/2008/9/16/pcl-clojure
Cheers,
Stuart
Konrad Hinsen
> So, I understand as follows:
> - OOP: keeping objects, which has states and methods.
> methods are encapuslated to the corresponding object.
> - FP: keeping objects(or structs or variables, whatever it is called)
> and functions on the objects.
> functions does not encapsulated to specific objects, for some
> reasons.
>
> Other characteristics such as function as first-order data structure
> and
> enforcing functional way (discourated side effects and immutable
> variables)
> can be important, but not in modeling the world.
I think the difference is more fundamental than what you describe. On
the other hand, I don't think that OOP and FP are contradictory.
Here is my view of things:
OOP = data abstraction. The program structure is defined by the data
that the program works on. Important data structures are identified
and implemented together with their associated operations as classes.
FP = algorithmic abstraction. The program structure is defined by the
algorithms that are used. Important algorithmic patterns are
identified and implemented as functions that take functional
arguments, or as macros. Functional arguments make it possible to
insert specific actions into a skeleton.
There are additional properties typically associated with OOP
(polymorphism, inheritance, ...) and FP (referential transparency,
immutable data structures, ...), but I think the approch to program
structure is the aspect where the two methods differ most. On the
other hand, there is no reason not to use both data abstraction and
algorithmic abstraction in a program. In that sense OOP and FP are
orthogonal and combinable.
As an illustration of the two approaches, consider a program to sort
data. In OOP, one would define an abstract class "comparable" with a
method "sort" that works by calling methods such as "greater" and
"equal" implemented in concrete subclasses. In FP, one would write a
function "sort" that takes as arguments a list of things to sort plus
a function to do the comparisons. At the top level of the program,
you'd see "interface comparable" in the OOP version and "function
sort" in the FP version. A mixed OOP-FP program might call the FP
function "sort" and pass the method "compare" of a subclass of
"comparable" as the comparison function.
Konrad.
mb
On 4 Nov., 15:03, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> As an illustration of the two approaches, consider a program to sort
> data. In OOP, one would define an abstract class "comparable" with a
> method "sort" that works by calling methods such as "greater" and
> "equal" implemented in concrete subclasses. In FP, one would write a
> function "sort" that takes as arguments a list of things to sort plus
> a function to do the comparisons. At the top level of the program,
> you'd see "interface comparable" in the OOP version and "function
> sort" in the FP version. A mixed OOP-FP program might call the FP
> function "sort" and pass the method "compare" of a subclass of
> "comparable" as the comparison function.
one can easily sort the same data in different ways in the FP style.
While implementing an interface Comparable defines *one* way
of sorting, the FP style separates the functions from the data. So
it is eg. easily possible to sort a list of email objects by subject,
author or date, or any combination thereof.
Sincerely
Meikel
Randall R Schulz
> ...
>
> As an illustration of the two approaches, consider a program to sort
> data. In OOP, one would define an abstract class "comparable" with a
> method "sort" that works by calling methods such as "greater" and
> "equal" implemented in concrete subclasses. In FP, one would write a
> function "sort" that takes as arguments a list of things to sort plus
> a function to do the comparisons. At the top level of the program,
> you'd see "interface comparable" in the OOP version and "function
> sort" in the FP version. A mixed OOP-FP program might call the FP
> function "sort" and pass the method "compare" of a subclass of
> "comparable" as the comparison function.
Even the current Java libraries and object model belie this comparison.
An array of intrisically comparable instances (those that "implement
Comparable") can be sorted without supplying a Comparator. But an array
of arbitrary Objects can be sorted in arbitrary and flexible ways by
supplying a Comparable that accepts the types submitted to it.
> Konrad.
Randall Schulz
Konrad Hinsen
> And as an effect of not forcing the sorting logic into some class,
> one can easily sort the same data in different ways in the FP style.
> While implementing an interface Comparable defines *one* way
> of sorting, the FP style separates the functions from the data. So
> it is eg. easily possible to sort a list of email objects by subject,
> author or date, or any combination thereof.
Sorting is indeed an example where the FP approach is superior. But
then, it is not representative in size and complexity of a typical
program or library. One could easily cite other examples where OOP
would be the better choice. My ideal language would support both as
much as possible.
An example of where Clojure lacks OO features, in my opinion, is your
lazy-map library. It provides an additional implementation for an
existing interface, which is a typical OO approach, and a very useful
one. But although all your code is written Clojure, it looks like a
kludge in having to use a feature (genclass) intended for Java
compatibility and forcing an unnatural file structure (one file for
the methods, one for the code generation, and one for the end-user
wrapper) on the implementation.
Konrad.
Mark H.
> So, I understand as follows:
> - OOP: keeping objects, which has states and methods.
> methods are encapuslated to the corresponding object.
> - FP: keeping objects(or structs or variables, whatever it is called)
> and functions on the objects.
> functions does not encapsulated to specific objects, for some
> reasons.
side effects at least. A program's output is a _function_ of its
input, which means that the same input always results in the same
output. This is very helpful for reasoning about the correctness of
your program, because it means that a particular function's output
does not depend on the state of your program. It's especially helpful
for multithreaded programs where the threads share state, because one
thread's state won't ever be changed without its knowledge.
Of course sometimes you need side effects in order to accomplish
useful work, and a good FP language will provide abstractions that
help you limit side effects and reason about them. Clojure does this
by means of refs, agents, and transactional updates. refs are
different than regular values so that you know they are mutable.
An object-oriented program may also use FP. OOP and FP are not
opposites. However, a lot of OO programming these days involves
changing the state of opaque objects (therefore, not purely
functional) via member functions. I've heard this characterized as
"the new spaghetti code" because it's hard to reason about thread-
safety when objects are being modified all the time, and when these
modifications aren't syntactically obvious. For example, the member
function call
y = x.f()
might modify x, so you have no idea whether f() is thread-safe (unless
you look at the source code and try to reason it out yourself). f()
might modify x in nonobvious ways which the authors of f() might not
have documented. For example, f() might write to a temporary buffer
inside the x object. Calling f() on the same object in multiple
threads without the appropriate locking may corrupt that buffer and
cause f() to return an incorrect value (or worse; f() might overrun
the buffer). OOP doesn't have to do this, but a lot of OOP code does
this.
mfh
Raoul Duke
i think that is a good point. there is not a single succinct
definition of OO when one looks at the various ways it has been
actually implemented in languages: Smalltalk is not the same thing as
Java. I think the ways in which they differ can have a serious impact
on what it means in the end to "be" OO, especially when contrasting
with a different paradigm (be it FP or Declarative or Procedural or
whathaveyou).
sincerely.
Konrad Hinsen
> Of course sometimes you need side effects in order to accomplish
> useful work, and a good FP language will provide abstractions that
> help you limit side effects and reason about them. Clojure does this
> by means of refs, agents, and transactional updates. refs are
> different than regular values so that you know they are mutable.
Vars are also mutable, even if that mutability is local to a thread.
I guess one could write state-modifying code in Clojure just like in
most imperative languages, it is merely discouraged and made hard to
hide. That looks like a very reasonable compromise to me.
> An object-oriented program may also use FP. OOP and FP are not
> opposites. However, a lot of OO programming these days involves
> changing the state of opaque objects (therefore, not purely
> functional) via member functions. I've heard this characterized as
> "the new spaghetti code" because it's hard to reason about thread-
> safety when objects are being modified all the time, and when these
> modifications aren't syntactically obvious.
Even in the absence of threads, changing state can be a cause of
trouble, in particular if it is well hidden under a few layers of
nice-looking APIs.
On the other hand, I am not sure that all important algorithms
already have purely functional equivalents that are sufficiently
efficient for real life. Is there an efficient purely functional
algorithm for matrix inversion, for example?
Konrad.
Meikel Brandmeyer
Am 04.11.2008 um 16:06 schrieb Konrad Hinsen:
> An example of where Clojure lacks OO features, in my opinion, is your
> lazy-map library. It provides an additional implementation for an
> existing interface, which is a typical OO approach, and a very useful
> one. But although all your code is written Clojure, it looks like a
> kludge in having to use a feature (genclass) intended for Java
> compatibility and forcing an unnatural file structure (one file for
> the methods, one for the code generation, and one for the end-user
> wrapper) on the implementation.
I disagree. Clojure brings everything needed: multimethods. One defines
a set of multimethods (the interface) and different arguments may have
different implementations.
Inheritance can also be replaced with delegation without problems. And
in fact lazy-map does not inherit from any maptype. Instead it delegates
to another instance without caring for whether it's a struct map, hash
map or sorted map. With inheritance each of this types would have needed
an own lazy-x-map class. This is again some style of functional OO
programming.
Suppose get, contains?, etc. were multimethods. I would not have needed
gen-class. I just would register my own implementations for the lazy-
map.
But the problem is: Clojure's map interface is not implemented with
multimethods. They have a Java side. If I want to provide a drop-in
replacement I have to also serve that side. Hence I need gen-class.
Or Java itself for that matter in case gen-class is too kludgy.
I don't think that Clojure is that bad for object-oriented programming.
It just looks a bit different than usual.
Sincerely
Meikel
Mark H.
> On the other hand, I am not sure that all important algorithms
> already have purely functional equivalents that are sufficiently
> efficient for real life. Is there an efficient purely functional
> algorithm for matrix inversion, for example?
Long answer: SISAL is an example of a functional parallel language
designed for performance. Its compiler could optimize away
temporaries in purely functional code (the equivalent of loop/recur).
Optimizing away temporaries (esp. temp vectors and submatrices turns
out to be important for several reasons:
* Parallel GC is scary (not impossible, though)
* Memory usage is tight for big problems
* A lot of HPC code is limited by memory bandwidth so creating
temporary vectors (for example) rather than overwriting existing ones
is slow
One could express solving linear systems (which is what I presume you
mean by "matrix inversion," unless you really want the entries of the
inverse) in a purely functional way using a language like SISAL with a
compiler that optimizes away temporaries, and the resulting
implementation may very well be highly efficient. Krylov methods like
conjugate gradient for sparse linear systems would be even easier for
the compiler to optimize since they don't modify the matrix, only
vectors.
HPC coders tend to resist functional implementations because
* they like to control memory allocation themselves
* the existing code infrastructure is not functional
* for cultural reasons
That was basically why SISAL didn't take off at Livermore. HPC coders
do, however, write a lot of control code, scripts, etc. that can
benefit from FP. This is why I follow Clojure (for example).
mfh
Konrad Hinsen
> Suppose get, contains?, etc. were multimethods. I would not have
> needed
> gen-class. I just would register my own implementations for the
> lazy-map.
> But the problem is: Clojure's map interface is not implemented with
> multimethods. They have a Java side.
That's exactly my point. Multimethods may well be sufficient or even
superior for implementing OO concepts useful in Clojure. We will see
when someone actually uses them this way (or has it already
happened?). But as long as the most important interfaces (sequences,
maps, numbers), which happen to be the ones also used by the built-in
types, don't use the same mechanism, there will always be first-class
and second-class citizens in Clojure's datatype universe.
Konrad.
Konrad Hinsen
> Long answer: SISAL is an example of a functional parallel language
Ah, right, there was SISAL... unfortunately long forgotten.
> One could express solving linear systems (which is what I presume you
> mean by "matrix inversion," unless you really want the entries of the
> inverse) in a purely functional way using a language like SISAL with a
> compiler that optimizes away temporaries, and the resulting
Could one? That was actually the core of my question. Of course HPC
applications would require smart compilers that, but first of all
there must be a purely functional algorithm that can be transformed
automatically into an efficient one. Do you know any references to
such algorithms in linear algebra?
Of course, HPC applications dealing with large data sets would always
want algorithms that modify matrices in place instead of allocating
more memory. I don't expect a purely OO HPC world any time soon. But
it would still be interesting to know how many of the traditional CPU-
hungry algorithms already have known efficient functional equivalents.
> That was basically why SISAL didn't take off at Livermore. HPC coders
> do, however, write a lot of control code, scripts, etc. that can
> benefit from FP. This is why I follow Clojure (for example).
Me too. HPC is only one aspect of my work. And I think languages like
Clojure can be useful for generating specialized HPC code as well,
just like FFTW uses Caml code for generating C routines.
Konrad.
mb
On 5 Nov., 08:31, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> That's exactly my point. Multimethods may well be sufficient or even
> superior for implementing OO concepts useful in Clojure. We will see
> when someone actually uses them this way (or has it already
> happened?). But as long as the most important interfaces (sequences,
> maps, numbers), which happen to be the ones also used by the built-in
> types, don't use the same mechanism, there will always be first-class
> and second-class citizens in Clojure's datatype universe.
They
just have a different aspect. Having pure Clojure "classes" with
multimethods might well be integrated and nice, but you get problems
when you have to interface to Java. On the other hand using Java
classes gives you easy interface, but you have to live with gen-class.
So which one is first, which one second class? I think each solution
has just different Pros and Cons.
What would interesting, is the question, whether both approaches
could be combined. We make a Java class (via gen-class), which
gets a default implementation for each method, which references
a similar named multimethod instead of the Class-methodName.
Clojure "classes" deriving from that class do not provide an
a Java method themselves, but register with the multimethod.
The Clojure code could use the multimethods, while there are still
the methods available for a Java interface. So the downside would
be that one can only inherit from one such prototype class (since
it needs to be a class, not an interface, for the default
implementation
of the methods) or one has to respecify the method - multimethod
translation in all sub classes. Maybe a modified gen-class could
support?
(These are just random thoughts. Maybe they make sense, maybe
not.)
For the built-in datatypes: I implemented nth and get as multimethods
(as some kind of proof-of-concept), but Rich was not very interested
due to performance issues and the fact, that the datatypes are used
internally in very low-level areas. I don't really follow the argument
about monkey patching. I think it's not monkey patching at all, since
the multimethods only define the interface. So I don't modify a thing
which is returned by hash-map. That's still a hash map using the
hash-map implementation.
- http://clojure-log.n01se.net/date/2008-10-06.html#13:18
Sincerely
Meikel
Konrad Hinsen
> I don't think that there are first-class and second-class citizens.
> They just have a different aspect.
Right, one can't say one is superior to the other. There are just two
separate worlds, each having its own characteristics.
> Having pure Clojure "classes" with
> multimethods might well be integrated and nice, but you get problems
> when you have to interface to Java. On the other hand using Java
> classes gives you easy interface, but you have to live with gen-class.
But does gen-class have to look the way it does? Couldn't the same
functionality be provided in a way that looks more like a proper part
of the language?
> What would interesting, is the question, whether both approaches
> could be combined. We make a Java class (via gen-class), which
> gets a default implementation for each method, which references
> a similar named multimethod instead of the Class-methodName.
That sounds like an interesting idea.
> For the built-in datatypes: I implemented nth and get as multimethods
> (as some kind of proof-of-concept), but Rich was not very interested
> due to performance issues and the fact, that the datatypes are used
> internally in very low-level areas.
I can understand performance arguments, of course. And I don't really
want to modify the built-in types in any way, I just want to be able
to use the same interfaces in completely independent datatype
implementations.
The current implementation of nth provides a special implementation
for each kind of datatype that it knows about, and fails for unknown
types. Shouldn't it be possible to add a default case at the end that
calls a Clojure multimethod? That shouldn't have any performance
impact on the built-in datatypes.
> I don't really follow the argument about monkey patching. I think
> it's not monkey patching at all, since
> the multimethods only define the interface.
I agree. It's no more monkey-patching than any use of multimethods
would be monkey-patching.
Konrad.
mb
On 5 Nov., 15:40, Konrad Hinsen <konrad.hin...@laposte.net> wrote:
> But does gen-class have to look the way it does? Couldn't the same
> functionality be provided in a way that looks more like a proper part
> of the language?
and that defclass doesn't look to different in the form the function
is called, a mix of keywords and arguments.
My main concern with gen-class is that it doesn't use the namespace
itself. I posted a patch[1] yesterday which addresses the namespace
and -/_ translation. I rose this also some weeks ago on the list but
there was no response. So the general interest seems to be pretty
low. On the other hand Rich, seems to have something like this on
his TODO list[2]. But maybe everything is different again because
of AOT. So I'm not sure, what his plans are for gen-class right now.
[1]: http://groups.google.com/group/clojure/browse_thread/thread/386d90a8b757aee4
[2]: http://richhickey.backpackit.com/pub/1597914
> The current implementation of nth provides a special implementation
> for each kind of datatype that it knows about, and fails for unknown
> types. Shouldn't it be possible to add a default case at the end that
> calls a Clojure multimethod? That shouldn't have any performance
> impact on the built-in datatypes.
Sincerely
Meikel
Mark H.
> > Long answer: SISAL is an example of a functional parallel language
>
> Ah, right, there was SISAL... unfortunately long forgotten.
of the Livermore folks who promoted it. Pretty neat!
> > One could express solving linear systems (which is what I presume you
> > mean by "matrix inversion," unless you really want the entries of the
> > inverse) in a purely functional way using a language like SISAL with a
> > compiler that optimizes away temporaries, and the resulting
>
> Could one? That was actually the core of my question. Of course HPC
> applications would require smart compilers that, but first of all
> there must be a purely functional algorithm that can be transformed
> automatically into an efficient one. Do you know any references to
> such algorithms in linear algebra?
practice" in this case ;-) SISAL didn't have multidimensional arrays
so I doubt very much that it could have optimized away the creation of
matrix temporaries.
The usual trick to make things look functional is to index vector
variables with the current iteration, so that you aren't overwriting
anything. Then if the compiler can prove that there are no read-after-
write conflicts, it can collapse the vector variables into one vector
and replace copies with destructive writes. I haven't seen a purely
functional formulation of LU factorization but it could be done
without too much trouble. Of course there's no reason to go through
that effort because people spend so much time optimizing LU and its
constituent components that you would be better off reusing their
work.
> Of course, HPC applications dealing with large data sets would always
> want algorithms that modify matrices in place instead of allocating
> more memory. I don't expect a purely OO HPC world any time soon. But
> it would still be interesting to know how many of the traditional CPU-
> hungry algorithms already have known efficient functional equivalents.
splice existing HPC libraries into a functional framework, without
losing the ability to reason functionally about the components.
> Me too. HPC is only one aspect of my work. And I think languages like
> Clojure can be useful for generating specialized HPC code as well,
> just like FFTW uses Caml code for generating C routines.
to generate stencil codes. He's been thinking about switching to
Clojure ever since he and I worked on a thorny Lisp problem
together ;-)
mfh
Konrad Hinsen
> and replace copies with destructive writes. I haven't seen a purely
> functional formulation of LU factorization but it could be done
> without too much trouble. Of course there's no reason to go through
> that effort because people spend so much time optimizing LU and its
> constituent components that you would be better off reusing their
> work.
For the immediate future, yes. But with changing computer
architectures, the existing algorithms and routines may lose much of
their interest in the future.
> To me the more interesting and rewarding task is to figure out how to
> splice existing HPC libraries into a functional framework, without
> losing the ability to reason functionally about the components.
Unfortunately, it is already a bit of a pain to link existing HPC
libraries (written in Fortran, C, or C++) to functional code in any
decent language. Clojure won't help there, as there are still very
few HPC libraries for the JVM and JNI adds too much of an overhead.
> Definitely! We've got at least one fellow here who uses Common Lisp
> to generate stencil codes. He's been thinking about switching to
> Clojure ever since he and I worked on a thorny Lisp problem
> together ;-)
I am looking forward to a nice Clojure library then :-)
Konrad.
cwyang
I've read all the replies, and I might understand something.
(Though it seems that discussion goes to beyond my current
understanding in the middle of this discussion threads)
To explore further, I become another owner of 'Programming
Clojure' :-)
For my second question, I'll keep thinking and repost question.
I'm really concern about concurrency of Clojure,
since the concurrency in my world means
_the C10K problem_, that is supporting simultaneous
10000 connections or over for networking applications.
(http://www.kegel.com/c10k.html)
In YAWS, Erlang have shown one solution, so I'm curios that
Clojure could be another viable alternative,
and that how well-suited Clojure is for that kind of concurrency.
My expectation is these:
1) For C10K problem (or C100K), application must not use
native threads. Big stack size means low concurrency
2) For application developer, thread model is easy to develop
and follow.
3) Therefore, underlying system should support
concurrency facility like micro-thread, lightweight process (in
Erlang)
, agent (in Clojure), or whatever.
In this case, underlying facility should have high-performance.
4) At the same time, there must be ways for connecting conceptual
gap between 2) and 3). In other words, the way for suspending current
execution of function, saving current execution snapshot
(normally native thread stack, but may be different),
and switching to other functions are needed when the request for I/O
should be blocked. It's important that the size of current execution
snapshot should be small, since it determine the degree of
concurrency.
After I invest some time to learn Clojure, I'll repost.
Thank you.
- cw
Mark H.
> On 05.11.2008, at 17:16, Mark H. wrote:
> For the immediate future, yes. But with changing computer
> architectures, the existing algorithms and routines may lose much of
> their interest in the future.
at least).
> > To me the more interesting and rewarding task is to figure out how to
> > splice existing HPC libraries into a functional framework, without
> > losing the ability to reason functionally about the components.
>
> Unfortunately, it is already a bit of a pain to link existing HPC
> libraries (written in Fortran, C, or C++) to functional code in any
> decent language. Clojure won't help there, as there are still very
> few HPC libraries for the JVM and JNI adds too much of an overhead.
numbers? I know Java has messed-up floating-point (Sun stupidly took
away x87's 80-bit temps and other good things) but I'd at least like
to avoid copy overhead for matrix and vector ops. I don't mind the
extra function call out of JVM space as long as I can amortize it over
the cost of a big matrix operation. (No way I'm writing loops in
Clojure for that.)
There are a number of Python-based projects to do what you mentioned
(link HPC libraries in Fortran or C to a higher-level language), so
it's not impossible. It's just a lot of tedious work for some unlucky
coders.
> > Definitely! We've got at least one fellow here who uses Common Lisp
> > to generate stencil codes. He's been thinking about switching to
> > Clojure ever since he and I worked on a thorny Lisp problem
> > together ;-)
>
> I am looking forward to a nice Clojure library then :-)
he may not bother migrating it. Plus he likes the ECL / C linkup;
Java may not be so helpful for him.
mfh
Konrad Hinsen
> Does the JNI require copy-in / copy-out for arrays of floating-point
> numbers?
The JNI requires the C code to call JNI routines for converting the
Java array to a native array and back. Whether or not a copy is made
depends on the JNI implementation.
> There are a number of Python-based projects to do what you mentioned
> (link HPC libraries in Fortran or C to a higher-level language), so
> it's not impossible. It's just a lot of tedious work for some unlucky
> coders.
Python has a much nicer C interface than Java. The NumPy library
provides an array implementation whose underlying data structure can
be used directly as a C/Fortran array. No copying, no conversion, and
there are lots of tools that do much of the unpleasant work.
Konrad.
Graham Fawcett
> My expectation is these:
> 1) For C10K problem (or C100K), application must not use
> native threads. Big stack size means low concurrency
Hi,
I assume you mean 'new native thread per request' is bad for CnK.
Clojure's thread-pooling is not very different from the N:M threading
model that Erlang-OTP/Yaws uses, in a general sense.
> 4) At the same time, there must be ways for connecting conceptual
> gap between 2) and 3). In other words, the way for suspending current
> execution of function, saving current execution snapshot
> (normally native thread stack, but may be different),
> and switching to other functions are needed when the request for I/O
> should be blocked. It's important that the size of current execution
> snapshot should be small, since it determine the degree of
> concurrency.
Sounds like you're looking for first-class continuations here.You
won't find that on any JVM language, there's no JVM support for them.
(It is true that JVM-based Scheme languages have continuations, but
they are severely limited by the constraints of the JVM).
If you really believe you need this, you might be interested in
Termite, an Erlang-style actor system written for Gambit Scheme (which
has serializable first-class continuations and very lightweight
threads, but no SMP support if I remember correctly).
An alternative would be to use non-blocking I/O multiplexing -- an
event-driven model like you would write with select() or epoll(). I
don't know the Java universe very well, but I know JVM has
asynchronous I/O, so this must be an available option. Writing such
programs can sometimes lead to hard-to-understand, fragmented
application code, but Clojure's flexible syntax might be able to help
"unify" the fragments into a more readable form.
Best,
Graham