My first look at Clojure

[Kearsley Schieder-Wethy]

So, I spent my afternoon actually sitting down and doing something in
Clojure, to try to see how it fit together. Only, unlike my previous
forays into language-learning, I kept a journal.

Most of this was written as I wrote the code, with only occasional
backtracking, and that mostly editing for consistency.

It's not a tutorial or a guide so much as it is a very dog-eared list
of my own mistakes. But it is a nothing-to-working-software
examination of the learning curve.

It also started out as a short document.

## Preliminaries

So, Clojure is a pretty little language, a Lisp[1] for the Java
Virtual Machine. That, in and of itself, doesn't say much for or
against it.

I've been playing with it for a few days now and really finding that I
like more than I dislike and that the bits that I found to be
difficult were, well, being fixed or changed.

In the spirit of Dive Into Python, this is the place where I should
offer up a
snippet of finished code and then walk through it. Unfortunately, only
having
used Clojure a few days, I'm still in the first romance stage, rather
than the
stacks of working code stage.

So, rather than offering a block of code and then dissecting it, or
alternately
doing the thing that seems common in ruminations on functional
languages[2] and offering up an example like:

(defn fibs
([] (fibs 0 1))
([a] (fibs a 1))
([a b]
(lazy-cons a (fibs b (+ a b)))))

(defn nthfib [n]
(first (reverse (take n (fibs)))))

That, while pretty in demonstrating infinite sequences and a simple
form of multiple dispatch, does little to offer a reason to actually
use the language[3]. Instead, I'm going to try my hand at two things
that have frustrated me in the past with Lisp. Namely sockets and
threads[4].

## Goals

So, what I want is:

A client that:
* Connects to a server.
* Listens and prints out what it hears.

A server that:
* Listens on a port
* Creates a new thread on connection.
* Sends the current ctime to the client.
* Disconnects.

Or, essentially, the most primitive version of NTP imaginable.

As Clojure runs on the JVM, first step becomes to open up JavaDoc and
check what objects I actually want. I'll start by importing
ServerSocket and Socket.

user=> (import '(java.net ServerSocket Socket))
nil

## The most primitive client and server possible

At this point, all I want to do is make functions to open the
connections, so:

(defn listener-new [port]
(new ServerSocket port))

(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(new Socket address port)))

listener-new is a function that binds on a port and returns a Java
ServerSocket
bound to that port. connection-new is a function that takes either a
port (in
which case it connects to localhost) or an address and a port and
returns a
Socket connected to that.

I'll try to explain new terms and forms as I go. The ones introduced
here are defn, which creates a function, with the arguments in square
brackets, returning whatever the last value in its body is.

So, (defn [port] (new ServerSocket port)) makes a function that takes
one argument and then calls the new function, which instantiates a
Java object with whatever arguments are passed. It then returns
whatever new returns, which, unless there's an error, should be a
ServerSocket.

defn can also accept multiple sets of arguments, in order to either do
a certain amount of pattern matching on them or, in this case, provide
a default value to a function.

The next step, then, is to try each of these and see if they work.

user=> (def serversocket (listener-new 56894))
#<Var: user/serversocket>
user=> serversocket
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=56894]
user=> (. serversocket (close))
Reflection warning, line: 17 - call to close can't be resolved.
nil

So, the serversocket will open just fine, though I receive a
(relatively harmless) type warning when I close it. That can be
removed either by ensuring that the close method call ensures (via a
type annotation) that what is passed to it is indeed a serversocket
and that what is returned by listener-new is also a serversocket.

For the present, though, I don't actually care about this
warning. Let's try connection-new.

user=> (def socket (connection-new 56894))
java.lang.reflect.InvocationTargetException
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at clojure.lang.Reflector.invokeConstructor(Reflector.java:
125)
at user.connection_new.invoke(Unknown Source)
at user.connection_new.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:173)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler$DefExpr.eval(Compiler.java:253)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
... 13 more

Oh dear. Well, that didn't work. Though, I suppose having a port open
to connect to might just help.

user=> (def ssocket (listener-new 56894))
#<Var: user/ssocket>
user=> (def socket (connection-new 56894))
#<Var: user/socket>
user=> (. socket close)
Reflection warning, line: 45 - reference to field close can't be
resolved.
java.lang.IllegalArgumentException: No matching field found
at clojure.lang.Reflector.getInstanceField(Reflector.java:175)
at clojure.lang.Compiler$InstanceFieldExpr.eval(Compiler.java:
744)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
user=> (. socket (close))
Reflection warning, line: 53 - call to close can't be resolved.
nil
user=> (. ssocket (close))
Reflection warning, line: 54 - call to close can't be resolved.
nil

And that does it. The error in that block was because I referred to
close as a field: close, rather than a method: (close). On the other
hand, this doesn't do anything interesting. It just makes and closes
the listener and the connection.

Now, the . operator. I've used it a few times so far, so I should
clarify what it does. It looks up the second value within the first
one. Thus, (. socket (close)) looks for a close method inside the
instance socket, then calls it.

Next, I need to add behaviour to the server to detect connection and
send the ctime, as well as making the client recognise that data is
being passed to it.

## Adding data

(import '(java.net ServerSocket Socket)
'(java.util Date))

(defn current-time []
(. (new Date) (toString)))

(defn listener-new [port]
(new ServerSocket port))

(defn listener-wait [listener]
(. listener (accept)))

(defn listener-send [lsocket]
(.. lsocket (getOutputStream) (write (current-time)))
lsocket)

(defn listener-close [listener]
(. listener (close)))

(defn listener-run [port]
(let [listener (listener-new port)]
(listener-send (listener-wait listener))
(listener-close)))

So now I've added a function that dumps the current time as a string,
as well as functions to accept a connection, send that current time
and disconnect.

Trying it out, I get:

user=> (current-time)
"Sat Feb 02 12:16:19 EST 2008"
user=> (listener-run 51245)

And then it hangs. Unsurprisingly, really, as it's blocking at
(. listener (accept)), which waits for a connection before returning
the socket referring to that connection. While this would be a good
place to make the thread that I was talking about earlier, for the
moment, I just want to see if this actually works. So, what I'll do
is start the server in a separate REPL[5].

So, in the one REPL, I create the server again:

user=> (listener-run 51345)

And in the other, I connect to it:

user=> (connection-new 51345)
Socket[addr=/127.0.0.1,port=51345,localport=1668]

Which results in this in the first REPL.

java.lang.IllegalArgumentException: No matching method found: write
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:
59)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:
26)
at user.listener_send.invoke(sockets.clj:15)
at user.listener_run.invoke(sockets.clj:23)
at clojure.lang.AFn.applyToHelper(AFn.java:173)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)

## Debugging data transmission

Now, despite this being an error, it does show that the listener made
it to listener-send, as that's where write is called. So, I know that
it's accepting network connections. Unfortunately, it's crashing when
it receives them.

So, my next step is to look at what that command actually does:

user=> (macroexpand '(.. lsocket (getOutputStream) (write (current-
time))))
(. (. lsocket (getOutputStream)) (write (current-time)))

The macroexpand function takes a macro passed to it and evaluates it
(as well as macros inside that one, recursively). So I know that write
is being called on the outputstream of the socket. For debugging
purposes, I'll rewrite listener-send procedurally.

(defn listener-send [lsocket]
(println lsocket)
(let [outputstream (. lsocket (getOutputStream))]
(println outputstream (current-time))
(. outputstream (write (current-time)))
lsocket))

This is the same function, only it's spitting out its state at each
stage.

user=> (listener-run 51345)
java.lang.IllegalArgumentException: No matching method found: write
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:
59)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:
26)
at user.listener_send.invoke(sockets.clj:18)
at user.listener_run.invoke(sockets.clj:26)
at clojure.lang.AFn.applyToHelper(AFn.java:173)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
Socket[addr=/127.0.0.1,port=1789,localport=51345]
java.net.SocketOutputStream@26dbec Sat Feb 02 13:08:22 EST 2008

This time, with output, I can see that I indeed get an
OutputStream. Time to play with it and see what it's doing.

user=> (import '(java.io InputStream OutputStream)
nil
user=> (def li (listener-new 51345))
#<Var: user/li>
user=> (def sock (listener-wait li))
#<Var: user/sock>
user=> (def by (. (current-time) (getBytes)))
Reflection warning, line: 46 - call to getBytes can't be resolved.
#<Var: user/by>
user=> (map (comp char (appl aget by)) (range (alength by)))
(\S \a \t \space \F \e \b \space \0 \2 \space \1 \3 \: \4 \9 \: \4 \4
\space \E \S \T \space \2 \0 \0 \8)
user=> (def ostream (. sock (getOutputStream)))
Reflection warning, line: 48 - call to getOutputStream can't be
resolved.
#<Var: user/ostream>
user=> (. ostream (write by))
Reflection warning, line: 49 - call to write can't be resolved.
nil
user=> (listener-close li)
nil

So, there's the catch. When I pass current-time directly to the
OutputStream's write method, what's expected is a write( String s
). What actually exists is write( Byte[] barr ).

The map done on by is worth looking at, though. I used it to find out
what exactly by was set to, it being a Java array. So, first off, I
get a list of all numbers from 0 to the length of by: (range (alength
by)). That's the indices to the array. Then (comp char (appl aget
by))[6] is called on each of those.

## Converting to and from byte arrays.

This actually raises a valid assertion. When creating a byte array
from a string, the character values in the byte array should be the
same as those in the string.

So, I'll create a function (that I'll use in a minute anyway) that
makes a Java byte array out of a string. And, because I'm starting to
get more cautious after my repeated bugs, I'll make a simple test to
assert that its results are the same as the string from which it was
made.

(defn byte-arr-from-string [str]
(. str (getBytes)))

(defn test-byte-array-from-string
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))
(== 0
(count (filter false?
(map eql?
bseq
chseq))))))))

There. A String to Byte[] converter and a unit test. Now, to run the
test:

user=> (test-byte-array-from-string)
true
user=> (test-byte-array-from-string "qwertyuiop")
true

And the test indicates that there is no loss per se in converting the
string to a byte array. That done, I'll add the conversion to a byte
array into listener-send. If that works, I'll collapse the expanded
debuggy listener-send back down.

(defn listener-send [lsocket]
(println lsocket)
(let [outputstream (. lsocket (getOutputStream))]
(println outputstream
(current-time)
(instance? outputstream java.io.OutputStream))
(. outputstream (write (byte-arr-from-string (current-time))))
lsocket))

And now a further round of testing:

user=> (listener-run 51345)
Socket[addr=/127.0.0.1,port=2564,localport=51345]
java.net.SocketOutputStream@88e2dd Sat Feb 02 15:23:04 EST 2008 true
nil

So, the server says (more or less) that it did its steps and nothing
went wrong. Let's check this from the other side:

user=> (def conn (connection-new 51345))
#<Var: user/conn>
user=> (def ins (. conn (getInputStream)))
Reflection warning, line: 69 - call to getInputStream can't be
resolved.
#<Var: user/ins>
user=> ins
java.net.SocketInputStream@453807
user=> (. ins (read))
Reflection warning, line: 71 - call to read can't be resolved.
83
user=> (. ins (read))
Reflection warning, line: 72 - call to read can't be resolved.
97

And, at the other end, I get the bytes that (I think) I sent. Time to
wrap up my client connection and get it to read the entire input
stream.

## Reading the transmitted data

(defn string-from-byte-sequence [coll]
(reduce strcat coll))

(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(new Socket address port)))

(defn connection-read [conn]
(let [instream (. conn (getInputStream))]
(loop [bytes nil
current-byte (. instream (read))]
(if (== current-byte -1)
bytes
(recur (conj bytes current-byte)
(. instream (read)))))))

(defn connection-close [conn]
(. conn (close)))

(defn connection-run [port]
(let [conn (connection-new port)
str (string-from-byte-sequence (connection-read conn))]
(connection-close conn)
str))

So, what this is trying to do is open a connection, read from it until
the number -1 is found and then returning the resulting sequence of
bytes.

Why -1? Because the JavaDoc for InputStream::Read) says:

The value byte is returned as an int in the range 0 to 255. If no byte
is available because the end of the stream has been reached, the value
-1 is returned. This method blocks until input data is available, the
end of the stream is detected, or an exception is thrown.

So I'm trying -1 first, as by the time I'm reading, listener-close
should have been called, closing the socket.

## Bugs with reading the data

user=> (connection-run 51348)
"5648485032848369325048584949585449325048329810170321169783"

So, it reads fine, but it appears that something went wrong in
translating the byte sequence into a string again. Looking at
string-from-byte-sequence, it concatenates the string values of the
items in the collection.


(defn string-from-byte-sequence [coll]
(reduce strcat coll))

So, I'll try concatenating two numbers:

user=> (strcat 64 65)
"6465"

Right, not what I was intending at all. So, before I concatenate the
numbers, I'll need to convert them to characters.

user=> (strcat (char 64) (char 65))
"@A"

That worked a lot better. And tells me what I need to do to fix the
function:

(defn string-from-byte-sequence [coll]
(reduce strcat
(map char coll)))

Because map is lazy, it means that simply using (strcat (map char
coll)) gets me the string value of map's returned FnSeq instead of the
concatenated characters. The reduce acts to apply strcat to every
element of the entire sequence.

And, connecting to a new server, I get:

user=> (connection-run 51349)
"8002 TSE 15:13:61 20 beF taS"

Whoops, almost it! Only backwards. Checking the byte sequence as it
comes in, I find that it is indeed reversed when returned from
connection-read. So, I test what connection-read does to build the
byte sequence:

user=> (conj (conj nil 1) 2)
(2 1)

So I check the documentation for conj and indeed:

Conj[oin]. Returns a new collection with the item 'added'. (conj nil
item) returns (item). The 'addition' may happen at different 'places'
depending on the concrete type.

An amendment to connection-read later:

(defn connection-read [conn]
(let [instream (. conn (getInputStream))]
(loop [bytes nil
current-byte (. instream (read))]
(if (== current-byte -1)
bytes
(recur (concat bytes (list current-byte))
(. instream (read)))))))

And I finally get the desired result:

user=> (connection-run 51354)
"Sat Feb 02 16:44:09 EST 2008"

## Bugfixing the primitive NTP server

So, what I now have is a server that waits for a connection on a given
port, then sends the time and a client that makes a connection and
reads the time. What I don't have is a bug-free or elegant version of
this.

Time for me to run down the list of bugs and desires that I know of at
the moment:

* The server fails to clean up its port correctly, meaning that
throughout this testing, I've been incrementing ports any time I
choose to keep the server's REPL open.
* The client will hang, waiting for data in its stream, when fed a
port
that is open but not responding.
* The server is single-threaded still. This is the only one of my six
test-application goals left undone. But making it not block the
REPL
while waiting for input will make testing far easier.
* No Java exceptions are currently handled, despite their being
thrown.
* Ten reflection warnings when I load the file into my REPL.
* test-byte-array-from-string is the only test attached. The
behaviour
of all the functions should be testable.
* It would be nice (though not necessarily useful in this case) to
represent the data stream as a lazy sequence, rather than expecting
that it all be here.

So, I'll address these in order, starting with the server port.

### The server fails to clean up its port...

I'll start by looking at what listener-run actually does.

(defn listener-run [port]
(let [listener (listener-new port)]
(listener-close (listener-send (listener-wait listener)))
listener))

It binds listener to a new ServerSocket listening on port.

It passes that ServerSocket to listener-wait, which calls the accept
method, returning the opened socket.

That socket is passed to listener-send, which gets its OutputStream
and writes the current time to the stream, returning the socket.

That socket is passed on to listener-close, which closes it.

The listener is returned.

And thatfourth step would likely be the problem here. Rather than
closing the listener, it's closing the socket acquired from accept,
which means that rather than five, I only have four goals complete.

So, I'll try:

(defn listener-run [port]
(let [listener (listener-new port)]
(listener-send (listener-wait listener))
(listener-close listener)
listener))

But that has the net result that now, the server still holds onto that
address, but the client hangs, waiting for data. A little bit of
prodding later and I have this:


(defn listener-send [lsocket]
(.. lsocket
(getOutputStream)
(write (byte-arr-from-string (current-time))))
(.. lsocket (getOutputStream) (close))
lsocket)

(defn listener-run [port]
(let [listener (listener-new port)]
(. (listener-send (listener-wait listener)) (close))
(listener-close listener)
listener))

Which successfully sends the time across the (loopback) network, then
releases the port.

The client will hang...

Not exactly true. I wrote that forgetting that I had random
ServerSockets poking around. In fact, what happens when the client is
passed a closed port is:


user=> (connection-new 1500)
java.lang.reflect.InvocationTargetException
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at clojure.lang.Reflector.invokeConstructor(Reflector.java:
125)
at user.connection_new.invoke(Unknown Source)
at user.connection_new.invoke(Unknown Source)
at user.connection_run.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:173)
at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
... 13 more

Conclusion: I need to not only handle connection timeouts but also
refused connections. As this happens in connection-new, which only has
one instruction, I need to wrap the new socket in a try/catch block.


(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(try (new Socket address port)
(catch InvocationTargetException except
(println (strcat "Could not connect to "
address
" on port "
port))))))

(defn connection-run [port]
(let [conn (connection-new port)
str (when conn (string-from-byte-sequence (connection-read
conn)))]
(when conn
(connection-close conn))
str))

So now trying to connect to an invalid port spits an error message and
returns nil. On the other hand, it still hangs when pushed to connect
to a port that doesn't behave like it expects (send data, close
socket).

user=> (connection-new 1280)
Could not connect to 127.0.0.1 on port 1280
nil

The answer, then, is to give the client a timeout after which it
assumes that no data is coming.

(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(try (doto (new Socket address port)
(setSoTimeout 5000))
(catch InvocationTargetException except
(println (strcat "Could not connect to "
address
" on port "
port))))))

(defn connection-read [conn]
(let [instream (. conn (getInputStream))
reader (fn [] (try (. instream (read))
(catch InvocationTargetException except
-1)))]
(loop [bytes nil
current-byte (reader)]
(if (== current-byte -1)
bytes
(recur (concat bytes (list current-byte))
(reader))))))

So now, if nothing comes in in five seconds, connection-read signals
an error and flags the current byte as -1, signalling an end of the
transmission.

doto, which I used to set the socket timeout, takes an instance,
evaluates the following body on it, then returns the changed
instance. I used it in this case because setSoTimeout returns void
while changing the object's state.


user=> (connection-run 80)
""

### The server is single-threaded...

Oh bugger. I really should have put this lower on the list. I did not
want to deal with threading today.

So, that leaves me the question: What should the server do? For
starters, I'll make its instances persistant. That way, with a single
server open, I can connect repeatedly without restarting the server
each time.

(defn listener-run
([port]
(listener-run (listener-new port)
port))
([listener port]
(loop [socket nil]
(when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener)))
listener))

However, what I want is for that same function to be pushed into the
background. So, for starters, I'll give the loop an exit condition:

(defn listener-run [listener port]
(loop [socket nil]
(if (. listener (isClosed))
listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

Now, if I happen to have captured the listener, I can kill the server
process by closing the listener. This isn't useful yet, as the server
will still loop infinitely when it's launched.

To make sure that I'm not holding on to inaccessable listeners, I also
removed my earlier convenience form that created a listener for me.

So, time to push listener-run into its own execution thread.


(defn listener-run-in-background [port]
(let [listener (listener-new port)]
(listener-run listener port)
listener))

There's a starting form. Though all it does at the moment is run the
listener in the same way as the bits I removed above, I'll now poke at
adding Java's concurrency bits into it.


(defn listener-run-in-background [port]
(let [listener (listener-new port)
exec (. Executors (newSingleThreadExecutor))
run (appl listener-run listener port)]
(. exec (submit run))
listener))

So, that's a first attempt. It creates a pool of one thread, wraps
running the server in an anonymous function and punts the function off
to the thread pool.

And testing it:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (connection-run 51345)
"Sat Feb 02 20:49:15 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:16 EST 2008"
user=> (connection-run 51345)
"Sat Feb 02 20:49:17 EST 2008"
user=> (. li (close))
Reflection warning, line: 560 - call to close can't be resolved.
nil
user=> li
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345]
user=> (. li (isClosed))
Reflection warning, line: 562 - call to isClosed can't be resolved.
true
user=> (connection-run 51345)
Could not connect to 127.0.0.1 on port 51345
nil

You know, I was expecting that to not work. That really shouldn't have
been that easy. To be fair, this doesn't do anything with that thread
pool that I created. So I'll shut it down once I'm done running the
server.

(defn listener-run-in-background [port]
(let [listener (listener-new port)
exec (. Executors (newSingleThreadExecutor))
run (fn [] (do (listener-run listener port)
(. exec (shutdown))))]
(. exec (submit run))
(list exec listener)))

user=> (connection-run 51345)
""
user=> li
(java.util.concurrent.Executors
$FinalizableDelegatedExecutorService@19bb25a
ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=51345])
user=> (connection-run 51345)
""
user=> (. (first li) (isShutdown))
Reflection warning, line: 114 - call to isShutdown can't be resolved.
true

And that, on the other hand, fails miserably, as do my next six
attempts. Then it finally occurs that, since in my original version,
the executor is empty and unreferenced, all I'm really doing, as I try
to bind exec inside itself with increasingly complex combinations of
let, do, appl and fn, is creating circular references, and that I'd
really just be a lot better off letting that ExecutorService fall on
the floor and assuming that, because it can't be reached, it'll get
picked up by the garbage collection[7].

### No Java exceptions are currently handled...

Well, I dealt with two of the three exceptions that I'm seeing
routinely. Better catch the other one, the BindException when trying
to launch two servers at the same port.


(defn listener-new [port]
(try (new ServerSocket port)
(catch BindException except
(println "Address is already in use."))))

And checking that with an open server:

user=> (def li (listener-run-in-background 51345))
#<Var: user/li>
user=> (def li (listener-run-in-background 51345))
Address is already in use.
#<Var: user/li>
user=> li
nil

## Afterthoughts

I'll count that one as done and take a look back at what needs
doing. Type/reflection warnings, unit tests and cleaning my hacky
"just get it done" code. Really, not bad for an afternoon's learning.

Further things to play with, from this tangent at least, seem to be:
serialization: seeing just what I can push as an array of bytes,
poking at java.lang.Concurrent and seeing how it reacts.

But as someone who's written more Lisp than Java in his life, I have
to say, Clojure had me at the (.) function. Lisp plus library support
is making this a very neat language to learn, and unlike my previous
forays into Common Lisp, I haven't hit a wall where I found myself
needing to reimplement something common[8].

Also, despite fairly routinely using (basic) Java objects in this
exercise, I never felt like I was having to coerce things to fit a
foreign function interface. For that matter, this approach felt more
or less like the right way to use objects: as a very rich system of
types, rather than as a way to control program structure and flow.

## Notes

1. It's not Common Lisp and it's not Scheme. It's its own dialect.

2. Clojure counts, for all intents and purposes, as a functional
language. It's certainly less strict than Haskell, but on a par with
the ML family in terms of being functional.

3. That naive implementation ran out of heap space at 100000
numbers. Likely binding it to a var would help.

After testing, apparently not.

4. I'm aware that there's lisps with good socket implementations and
ones with good threading implementations. It's just that, for me at
least, I've never found the two things in the same place at the same
time when I needed them.

5. REPL: Read Evaluate Print Loop. A shell in which you interact with
your program and the language.

In theory, I could have written all of this within the Clojure
REPL. In practice, it's far simpler to write functions in a separate
file and then send them to the REPL.

This means that, when I do something silly like causing the REPL to
hang due to waiting for a network connection, I can just reload my
file of functions.

Also, it does mean that, at the end of the day, I have my functions
in useable and modifiable form, rather than as java bytecote that will
be lost when I close the REPL.

6. This is an admittedly Haskellish way to build a function. In this
case, its results are directly equivalent to the anonymous function
(fn [x] (char (aget by x))).

To dissect this a little more, appl takes a function and at least
one argument, then returns a new function that takes the rest of the
original function's arguments. In other words, it applies the
arguments given to it partially.

This means that my (appl aget by) creates a new function equivalent
to (fn [a] (aget by a)).

comp, on the other hand, is function composition. For f(a) and g(b)
f .comp. g is equivalent to f( g( b ) ) or (f (g a)). In other words,
it takes functions and makes a new function that applies them from
right to left to its arguments.

Were I expanding out this function completely, on expanding appl, it
becomes: (comp char (fn [a] (aget by a))) and then on composing char
to it, (fn [b] (char ((fn [a] (aget by a)) b))).

In this case, I used comp and appl because I found the resulting
form easier to write and apply. By contrast, there are many
circumstances where I'd prefer a simple anonymous function.

7. In my defense here, I should point out that I was reading the C++
Frequently Questioned Answers last week, and it may have been
channeling a few bad memories.

8. To be fair, one of those involved the aforementioned choice between
getting sockets or threads, before throwing up my hands in dismay.

[Kearsley Schieder-Wethy]

And, because it's fragmented throughout the post, here's my resulting
source.

;;; A simple application that acts as a very primitive time server and
;;; client.

(import '(java.net BindException ServerSocket Socket)
'(java.lang.reflect InvocationTargetException)
'(java.util Date)
'(java.io InputStream OutputStream)
'(java.util.concurrent Executors))


;;; Utility functions

(defn current-time []
(. (new Date) (toString)))

(defn byte-arr-from-string [str]
(. str (getBytes)))

(defn string-from-byte-sequence [coll]
(reduce strcat
(map char coll)))

;;; Listener functions
;;; These control the server

(defn listener-new [port]
(try (new ServerSocket port)
(catch BindException except
(println "Address is already in use."))))

(defn listener-wait [listener]
(. listener (accept)))

(defn listener-close [listener]
(. listener (close)))

(defn listener-send [lsocket]
(.. lsocket
(getOutputStream)
(write (byte-arr-from-string (current-time))))
(.. lsocket (getOutputStream) (close))
lsocket)

(defn listener-run [listener port]
(loop [socket nil]
(if (. listener (isClosed))
listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

(defn listener-run-in-background [port]
(let [listener (listener-new port)
exec (. Executors (newSingleThreadExecutor))
run (appl listener-run listener port)]

(when listener
(. exec (submit run)))
listener))

;;; Connection functions
;;; These control the client.

(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(try (doto (new Socket address port)
(setSoTimeout 5000))
(catch InvocationTargetException except
(println (strcat "Could not connect to "
address
" on port "
port))))))

(defn connection-read [conn]
(let [instream (. conn (getInputStream))
reader (fn [] (try (. instream (read))
(catch InvocationTargetException except
-1)))]
(loop [bytes nil
current-byte (reader)]
(if (== current-byte -1)
bytes
(recur (concat bytes (list current-byte))
(reader))))))

(defn connection-close [conn]
(. conn (close)))

(defn connection-run [port]
(let [conn (connection-new port)

[Rich Hickey]

On Feb 2, 11:16 pm, Kearsley Schieder-Wethy <turv...@gmail.com> wrote:
> So, I spent my afternoon actually sitting down and doing something in
> Clojure, to try to see how it fit together. Only, unlike my previous
> forays into language-learning, I kept a journal.

> ...

Stunning - what a fantastic contribution - thanks!

Rich

[arsala...@gmail.com]

You know, having some sort of wiki for clojure where people can add
stuff like this would be nice... :-)

--Arsalan

[Rich Hickey]

On Feb 4, 1:28 am, "arsalan.za...@gmail.com" <arsalan.za...@gmail.com>
wrote:

> You know, having some sort of wiki for clojure where people can add
> stuff like this would be nice... :-)
>

One was created over the weekend:

http://en.wikibooks.org/wiki/Clojure_Programming

Rich

[arsala...@gmail.com]

Yes, I saw that, but are you sure you want to dump everything into one
wiki book? Shouldn't that page be just for the canonical "Programming
in Clojure" stuff rather than snippets, tips and tutorials like this?

--Arsalan

[pawel.o...@gmail.com]

Hello,

On Feb 3, 5:22 am, Kearsley Schieder-Wethy <turv...@gmail.com> wrote:
> And, because it's fragmented throughout the post, here's my resulting
> source.

I have been playing with your code and I could not connect to listener
established with (listener-run-in-background 31337). After some
debugging I have found out, that listener quits just after receiving
client connection. I added debug println to listener-run:

(defn listener-run [listener port]
(loop [socket nil]
(if (. listener (isClosed))

(do
(println (strcat "listener quit: " (. listener (isClosed)))) ;
<<<------ my debug
listener)

(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

And here is my clojure session (warn on reflections skipped):

----------------------------------------------
[(130) pasza@paszaibm clojure]$ rlwrap java -cp clojure.jar
clojure.lang.Repl src/boot.clj
Clojure
user=> (load-file "/home/pasza/programming/clojure/client-server.clj")
#<Var: user/connection-run>
user=> (def l (listener-run-in-background 31337))
#<Var: user/l>
user=> (connection-run 31337)
listener quit: false
""
user=> (. l (isClosed))
false
user=> (when (. l (isClosed)) (println "bug?"))
bug?
nil
user=> (when (eql? (. l (isClosed)) true) (println "bug?"))
nil
----------------------------------------------

So the problem seems to be that even though (. l (isClosed)) returns
false, conditional expresions (if, when) treat it as if it were true.
Am I missing something or is it a clojure bug?

My setup:
[pasza@paszaibm ~]$ java -version
java version "1.6.0_02"
Java(TM) SE Runtime Environment (build 1.6.0_02-b05)
Java HotSpot(TM) Server VM (build 1.6.0_02-b05, mixed mode)
[pasza@paszaibm ~]$ uname -a
Linux paszaibm 2.6.20-16-generic #2 SMP Sun Sep 23 19:50:39 UTC 2007
i686 GNU/Linux
[pasza@paszaibm ~]$

I am using 20080106 clojure release.

[Kearsley Schieder-Wethy]

On Feb 4, 11:10 am, "pawel.ostrow...@gmail.com"

<pawel.ostrow...@gmail.com> wrote:

> I have been playing with your code and I could not connect to listener
> established with (listener-run-in-background 31337). After some
> debugging I have found out, that listener quits just after receiving
> client connection. I added debug println to listener-run:
>

... REPL transcript snipped ...

>
> So the problem seems to be that even though (. l (isClosed)) returns
> false, conditional expresions (if, when) treat it as if it were true.
> Am I missing something or is it a clojure bug?

I forgot, when all was said and done, to close my REPL, reload my
source, and test it. Mea culpa.

I was indeed able to reproduce exactly this.

It seems that when boolean ServerSocket::isClosed() gets called, the
value that it returns is interpreted as a non-nil, non-false value.
Explicitly casting it to Boolean did fix it for me, though.

My patch to fix that:

(defn #^Boolean listener-closed? [#^ServerSocket listener]
(. listener (isClosed)))

(defn listener-run [listener port]
(loop [socket nil]

(if (listener-closed? listener)
listener

(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

And, from a fresh Clojure REPL:

user=> (load-file "/clojure/src/sockets.clj")
#<Var: user/connection-run>
user=> (def li (listener-run-in-background 31337))
#<Var: user/li>
user=> (listener-closed? li)
false
user=> (connection-run 31337)
"Mon Feb 04 14:28:38 EST 2008"
user=> (. li (close))
Reflection warning, line: 5 - call to close can't be resolved.
nil
user=> (listener-closed? li)
true
user=> (connection-run 31337)
Could not connect to 127.0.0.1 on port 31337
nil

[John Cowan]

On Feb 4, 2008 2:36 PM, Kearsley Schieder-Wethy <tur...@gmail.com> wrote:

> It seems that when boolean ServerSocket::isClosed() gets called, the
> value that it returns is interpreted as a non-nil, non-false value.

Arguably that's a bug in Java. When you make a reflective call to a
Java method that returns a boolean, it is boxed into a freshly
allocated java.lang.Boolean rather than returning either Boolean.TRUE
or Boolean.FALSE, which are what Clojure recognizes as true and false
respectively. For true returns it doesn't matter, but for false
returns it's busted.

> Explicitly casting it to Boolean did fix it for me, though.

You should always do so when calling a Java method that returns a
boolean, unless Clojure is changed to make the fix for you.

--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures

[Rich Hickey]

On Feb 4, 3:00 pm, "John Cowan" <johnwco...@gmail.com> wrote:

> On Feb 4, 2008 2:36 PM, Kearsley Schieder-Wethy <turv...@gmail.com> wrote:
>
> > It seems that when boolean ServerSocket::isClosed() gets called, the
> > value that it returns is interpreted as a non-nil, non-false value.
>
> Arguably that's a bug in Java. When you make a reflective call to a
> Java method that returns a boolean, it is boxed into a freshly
> allocated java.lang.Boolean rather than returning either Boolean.TRUE
> or Boolean.FALSE, which are what Clojure recognizes as true and false
> respectively. For true returns it doesn't matter, but for false
> returns it's busted.
>
> > Explicitly casting it to Boolean did fix it for me, though.
>
> You should always do so when calling a Java method that returns a
> boolean, unless Clojure is changed to make the fix for you.
>

Clojure does take care of this now (since rev 616)

Rich

[John Cowan]

On Feb 4, 2008 3:26 PM, Rich Hickey <richh...@gmail.com> wrote:

> Clojure does take care of this now (since rev 616)

Thanks. I have also asked on jvm-la...@groups.google.com (are you
on that list?) for someone to file a bug against Java.

[Rich Hickey]

On Feb 4, 11:07 am, "arsalan.za...@gmail.com"

I'm sure we could split it up once it's got some content. I don't see
it replacing the Clojure site as a reference. But, being a wiki, I
hope it will become whatever the community needs it to become and
makes it.

Rich

[Kearsley Schieder-Wethy]

So, I decided to keep the conceit going as I fixed the bug with
detecting connection. And then on into setting up a test framework.
If you find watching someone fumble about and generally make a mess of
a lovely language by hitting it with a hammer and wrongheaded ideas to
be painful, be warned, at one point I make something that makes a
bubble sort look pretty, then run it overnight.

## Casting and complements

As demonstrated to me, (. listener (isClosed)) returns a fresh
boolean, rather than Boolean.TRUE or Boolean.FALSE. So, while the
symbol looks fine in the REPL, when passed to (if (. listener
(isClosed)) ...), it is always true.

So, I made the following pair of complementary[1] utility functions:

(defn #^Boolean listener-closed? [#^ServerSocket listener]
(. listener (isClosed)))

def listener-open? (complement listener-closed?))

complement takes a function as its argument and returns a function
that does exactly the same thing as that function, only with its
return value inverted. Thus, (complement listener-closed?) returns a
function that does exactly what listener-closed? does and then returns
the opposite value.

In this case, it saves me from having to repeat listener-closed?'s
details. That way, if I ever need to change listener-closed, in order,
say, to handle more than simple Java ServerSockets, I can do so
without having to make any changes to listener-open?.

Note, though, that listener-open? is a def and not a defn. That's
because defn is a macro that wraps an fn around its body. complement,
on the other hand, returns an fn, so I only need to bind it to a Var.

I then amended listener-run so that it uses this explicit cast rather
than just taking what ``ServerSocket gives it.

(defn listener-run [listener port]
(loop [socket nil]

(if (listener-closed? listener)

listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

So, while I'm assigning casting, I should likely take care of the
warnings on loading this file, if for no reason other than that
they're starting to get a little excessively verbose.

user=> (load-file "/clojure/src/sockets.clj")
Reflection warning, line: 17 - call to getBytes can't be resolved.
Reflection warning, line: 53 - call to accept can't be resolved.
Reflection warning, line: 56 - call to close can't be resolved.
Reflection warning, line: 397 - call to getOutputStream can't be
resolved.
Reflection warning, line: 396 - call to write can't be resolved.
Reflection warning, line: 397 - call to getOutputStream can't be
resolved.
Reflection warning, line: 396 - call to close can't be resolved.
Reflection warning, line: 70 - call to close can't be resolved.
Reflection warning, line: 78 - call to submit can't be resolved.
Reflection warning, line: 88 - call to java.net.Socket ctor can't be
resolved.
Reflection warning, line: 97 - call to getInputStream can't be
resolved.
Reflection warning, line: 98 - call to read can't be resolved.
Reflection warning, line: 109 - call to close can't be resolved.
#<Var: user/connection-run>

Simply, I went through and placed type hints on functions that call
out to Java in order to ensure that their methods resolve on load.

Since I also noticed a few new things while reformatting the Clojure
manual into a single texinfo document for my own use, I'll also change
my use of (. var (toString) to use Clojure's str function:

(str x) Returns x.toString(). (str nil) returns ""


## Building a testing framework

All of those casts are fairly minor refactorings that just make the
statement of what a listener and connection are to the compiler. On
the other hand, the bug that I started with this time would have been
caught by the other thing I've been putting off. Namely, actual an
testing framework, rather than just eyeballing results in the REPL.

Now, I could go find JUnit, install that, and then see how well
Clojure links into it. But, well, that sounds like a new project[1].

So, instead, what I'll look at is defining what it is that my
functions are supposed to be doing, write a few tests, then add a
framework to glue those tests together.


### Ugliness ensues...

Which ended up looking like:
(def DEFAULT-PORT 51345)

(defn test-byte-array-from-string
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))

(nil? (first (filter false?
(map eql?
bseq
chseq))))))))

(defn test-listener-new
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))

(defn test-listener-predicates [])
(defn test-all []
(loop [test-list '(("string->byte array conversion"
test-byte-array-from-string)
("creating a listener",
test-listener-new)
("listener open?/closed? predicates"
test-listener-predicates))
failure false]
(let [this-test (first test-list)
test-list (rest test-list)]
(if (not this-test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests succeeded."))
(let [test-result (time (eval (rest this-test)))]
(println "Testing" (first this-test) ":" test-result
(if (first test-list) "\n" ""))
(recur test-list
(when (or failure
(not test-result))

And, running it, I get:

user=> (test-all)
"Elapsed time: 8.437664 msecs"
Testing string->byte array conversion : true

"Elapsed time: 68.38411 msecs"
Testing creating a listener : true

"Elapsed time: 2.831645 msecs"
Testing listener open?/closed? predicates : nil
---
Some tests failed.
nil

So, it works. (Noting that I deliberately built one of those tests to
fail, to make sure that the testing function catches failure.) Except
that, well, how best to put it... Eww. I know that so far this is only
150 lines, but I'm not going to remember to update the list of tests
down in the testing function every time I make a new one. So, I'll run
test-all, realise that my new test is missing, then go back and add
the new test.


### Adding structure...

No, I think what I need here is for the test itself to hold its own
payload. Sounds like a reasonable time to add in a struct, really. A
struct is just a defined hashmap taking keywords as arguments. For
starters, I'll build one that just duplicates the list that I have
below:

(defstruct unit-test :string :function)

Well, that does let me wrap tests into structures, but doesn't exactly
let me manipulate them:

user=> (struct unit-test :string "creating a listener" :function test-
listener-new)
{:function user.test_listener_new@d5eb7, :string "creating a
listener"}

What I'm going to need is a way to build these tests and load them
when I load the file, otherwise I'm just making the process of
eyeballing my results in the REPL harder on myself.

After a bit of thought, I settle on this as my test list:

(defstruct unit-test :name :string :function)

(def ALL-TESTS {})

(defn unit-tests-add [test]
(def ALL-TESTS
(assoc ALL-TESTS (get test :name) test)))

(defn unit-test-new
([description function]
(unit-tests-add (struct unit-test
:name function
:string description
:function function))))

Nothing special at the moment. Just a simple structure defining a unit
test as a name, description and function. Now, let's use that one one
of the three rudimentary tests that I already have:

(defn test-listener-new
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))

(unit-test-new "creating a listener" test-listener-new)

All right. Improving a tiny bit. Except that, after evaluating the
file twice, I still have:

user=> ALL-TESTS
{user.test_listener_new@1a0d866 {:function
user.test_listener_new@1a0d866, :string "creating a listener", :name
user.test_listener_new@1a0d866}, user.test_listener_new@3f96ee
{:function user.test_listener_new@3f96ee, :string "creating a
listener", :name user.test_listener_new@3f96ee}}

Makes sense, really. My new instance of test-listener-new has a new
symbol. So, even though I know that they're the same function, there's
no good reason for the compiler to.

What I really want, though, is to make defining a new test as simple
as adding a new (albeit specialised) function, rather than playing
around with remembering to call unit-test-new after. So, rather than
trying to get unit-test-new to parse out the name of the function,
I'll rethink my approach.

What do I actually want? Well, the function, name and description
added to my set of tests seamlessiy. I think it's fair to assume that
before I'm done, I'll want a few more qualifiers on tests as well.
Since I don't know what those are, I won't add them to my structure
yet.


### Admitting that I need macros...

However, I'm in a conundrum here, because I don't want to add logic to
my test definition every time I think of something new. Really, this
is starting to sound like a good argument for a macro.

I'll start by just imitating what defn does. Because if I can't
imitate that in a macro, the rest of this is going to start smelling
like week-old moose.

(defmacro deftest [fname description extras & fdecl]
`(defn ~fname ~@fdecl))

So that takes a function name, a descriptive string and a bunch of
unspecified "extras" and then proceeds to ignore the second two before
shoving the function name and the declaration into a defn. The
backtick (`) marks the following list as a template, meaning that the
defn won't be resolved right away.

The ~ and ~@ macro characters inside that template mean, respectively,
to unquote fname (so that I'm not defining a function called user/
fname) and to resolve fdecl into the sequence of values which it
represents.

I have to do this because the & rest argument to an fn is considered
to be a sequence. Were I simply passing it as an unquoted symbol, I'd
end up with an extra set of parentheses around the function body when
it's evaluated, which I don't want.

So, having rationalised every jot and tittle of what I just did, I'm
going to dump test-listener-new - that is to say, a working function -
into deftest via macroexpand-1 and see if it generates a plausible
defn. (macroexpand-1 because I don't really want to see the full
expanion of defn's internals, just this first layer that I added.

user=> (macroexpand-1 '(deftest test-listener-new "creating a
listener" '()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(clojure/defn test-listener-new ([] (test-listener-new DEFAULT-PORT))
([port] (with-open listener (listener-new port) (when listener
true))))

And that resulting definition looks just like the definition of test-
listener-new, only on one line and with defn as a qualified namespace/
symbol pair.

Seeing as that worked, it means that all I should have to do is wrap
defining the function in a do and add, as the second half, a function
call adding the test to my list of tests.

(defmacro deftest [fname description extras & fdecl]
`(do (defn ~fname ~@fdecl)
(unit-tests-add (struct unit-test
:name (name '~fname)
:string ~description
:function ~fname
~@extras))))

Which, when expanded, gats me:

user=> (macroexpand-1 '(deftest test-listener-new "creating a
listener" '()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-
PORT)) ([port] (with-open listener (listener-new port) (when listener
true)))) (user/unit-tests-add (clojure/struct user/unit-test :name
(clojure/name (quote test-listener-new)) :string "creating a
listener" :function test-listener-new quote ())))

Which counts as an almost-but-not-quite. Because I'm quoting the empty
list in the arguments that I'm passing to deftest, it expands to
(quote ()), which is then unquoted to quote (). Which isn't exactly
what I want.

On the other hand, the working syntax:

user=> (macroexpand-1 '(deftest test-listener-new "creating a
listener" ()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true)))))
(do (clojure/defn test-listener-new ([] (test-listener-new DEFAULT-
PORT)) ([port] (with-open listener (listener-new port) (when listener
true)))) (user/unit-tests-add (clojure/struct user/unit-test :name
(clojure/name (quote test-listener-new)) :string "creating a
listener" :function test-listener-new)))

It works fine, and saves me a spurious quote. Furthermore, values in a
struct default to nil, so if I add more bits to the unit-test
structure later, I don't have to cope with accounting for them in any
way other than doing nothing for nil, and my previously-defined tests
will still work.
Applying these new tests.

So, I modify my test to use this macro and get:

(deftest test-listener-new "creating a listener" ()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))

And evaluating that adds it to ALL-TESTS.

user=> ALL-TESTS
{"test-listener-new" {:function
user.test_listener_new@1dcc2a3, :string "creating a
listener", :returns nil, :name "test-listener-new"}}

I could have put the extras (that part that I have a suspicion that
I'll need, but don't know why yet,) at the end of the deftest macro.
However, that means that I'd likely forget to add them. Also, it means
that I have to make more changes to an existing test function than
simply adding two pieces of information.

On the other hand, because it returns the result of unit-tests-add,
deftest isn't quite a drop-in replacement to defn yet. Time to get it
to return the function instead.

So, knowing the function name, and establishing the reasonable
qualification that the function has been defined (which it should have
been, as defining it is a part of the macro, what I want to return is
the Var named by fname. Which leads me to:

(defmacro deftest [fname description extras & fdecl]
`(do (defn ~fname ~@fdecl)
(unit-tests-add (struct unit-test
:name (name '~fname)
:string ~description
:function ~fname
~@extras))
#'~fname))

Which, when I evaluate my test, returns the test function, ensuring
that this is now a wrapper around defn that actually returns the right
values.

That done, I can finally get to converting my remaining two tests and
the test-all function. The tests are easy:

(deftest test-byte-array-from-string "string->byte array
conversion" ()

([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))

(nil? (first (filter false?
(map eql?
bseq
chseq))))))))

(deftest test-listener-predicates "listener open?/closed?
predicates" () [])

To make the testing function work, it needs to be able to get at my
vector of tests and to examine the tests.

(def unit-test-name (accessor unit-test :name))
(def unit-test-string (accessor unit-test :string))
(def unit-test-function (accessor unit-test :function))


### A side-trip into accessors...

Here, however, I'm noticing that I'm duplicating what's already
written once in my struct, just to have accessors. Also, :string is a
daft name.

So, I'll make :string into :description and generate my accessors
based directly off of the structure.

(map (fn [key] (def (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))
key))
(keys unit-test))

And testing that:

user=> clojure.lang.Compiler$CompilerException: REPL:1026: Second
argument to def must be a Symbol
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.access$400(Compiler.java:38)
at clojure.lang.Compiler$BodyExpr$Parser.parse(Compiler.java:
2740)
at clojure.lang.Compiler$FnMethod.parse(Compiler.java:2626)
at clojure.lang.Compiler$FnMethod.access$1300(Compiler.java:
2540)
at clojure.lang.Compiler$FnExpr.parse(Compiler.java:2327)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3056)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.access$400(Compiler.java:38)
at clojure.lang.Compiler$InvokeExpr.parse(Compiler.java:2255)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3060)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.Exception: Second argument to def must be a
Symbol
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:
296)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 17 more

Hmm, not so good. Ok, time to poke at what exactly it is that I'm
doing wrong here.

user=> (sym "user" "a")
user/a
user=> (def (sym "a") 1)
clojure.lang.Compiler$CompilerException: REPL:1030: Second argument to
def must be a Symbol
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.Exception: Second argument to def must be a
Symbol
at clojure.lang.Compiler$DefExpr$Parser.parse(Compiler.java:
296)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 4 more
user=> (instance? clojure.lang.Symbol (sym "user" "a"))
clojure.lang.Compiler$CompilerException: REPL:3: Unable to resolve
classname: clojure.lang.PersistentList@d165fe67
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3065)
at clojure.lang.Compiler.analyze(Compiler.java:3011)
at clojure.lang.Compiler.analyze(Compiler.java:2986)
at clojure.lang.Compiler.eval(Compiler.java:3085)
at clojure.lang.Repl.main(Repl.java:59)
Caused by: java.lang.IllegalArgumentException: Unable to resolve
classname: clojure.lang.PersistentList@d165fe67
at clojure.lang.Compiler$InstanceExpr
$Parser.parse(Compiler.java:1868)
at clojure.lang.Compiler.analyzeSeq(Compiler.java:3058)
... 4 more

So, it appears that both def and instance? are seeing the unexpanded
list before it gets turned into a symbol. Curiouser and curiouser.

user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*)
(strcat "unit-test-" (name
key)))]
my-sym))
(keys unit-test))
(user/unit-test-name user/unit-test-function user/unit-test-
description)
user=> (map (fn [key] (let [my-sym (sym (name *current-namespace*)
(strcat "unit-test-" (name
key)))]
(def my-sym 1)))
(keys unit-test))
(#<Var: user/my-sym> #<Var: user/my-sym> #<Var: user/my-sym>)

Ok, so I redefined my-sym three times. After I got as far as:
(map (fn [li] (def ~@li))
(map (fn [key] (list (sym (name *current-namespace*)
(strcat "unit-test-" (name key)))
(accessor unit-test key)))
(keys unit-test)))

I've decided that I'll no longer make a big deal out of building my
accessors from my structure, as I seem to have a merry clash between
what def wants and what I know how to offer to it.

I'll admit that this failure irks me, but I'm going to leave it as
something to poke at later, as I'm getting confused as to operator and
macro precedence.

Sobeit:

(def unit-test-name (accessor unit-test :name))
(def unit-test-description (accessor unit-test :description))
(def unit-test-function (accessor unit-test :function))


### Giving up and going back to the test function.

Now, on to making the test function actually work with this structure,
rather than playing with trying to be clever with generating
accessors.

(defn test-all []
(loop [test-list (vals ALL-TESTS)
failure false]
(let [this-test (first test-list)
test-list (rest test-list)]
(if (not this-test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests succeeded."))
(let [test-result (time (unit-test-function this-test))]
(println "Testing" (unit-test-description this-test) ":"
test-result
(if (first test-list) "\n" ""))
(recur test-list
(when (or failure
(not test-result))

However, testing that, it shows only that the unit test functions
exist. Nothing more.

user=> (test-all)
"Elapsed time: 0.181588 msecs"
Testing creating a listener : user.test_listener_new@1878144

"Elapsed time: 0.028216 msecs"
Testing string->byte array conversion :
user.test_byte_array_from_string@137d090

"Elapsed time: 0.019277 msecs"
Testing listener open?/closed? predicates :
user.test_listener_predicates@15db314
---
All tests succeeded.
nil

I need it to evaluate the returned function, preferably with the
capacity to insert arguments as needed. Hmm.

(defn test-all []
(loop [test-list (map (fn [x] (list (unit-test-description x)
(unit-test-function x)))
(vals ALL-TESTS))
failure false]
(let [test (first test-list)
test-list (rest test-list)]
(if (not test)
(if failure
(println "---\nSome tests failed.")
(println "---\nAll tests passed."))
(let [test-result (time (eval (list (second test))))]
(println "Testing" (first test) ":" test-result)
(recur test-list
(when (or (not test-result)
failure)
true)))))))

Will leave that as a working test framework for the moment and
actually move back to writing tests.

(deftest test-listener-predicates "listener open?/closed?
predicates" ()
([]
(test-listener-predicates DEFAULT-PORT))
([port]
(let [listener (listener-new port)
tests (and (not (listener-closed? listener))
(listener-open? listener))]
(. listener (close))
(and tests
(not (listener-open? listener))
(listener-closed? listener)))))

(deftest test-listener-close "closing a listener" ()
([]
(let [listener (listener-new DEFAULT-PORT)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)))))

(deftest test-connection-new "creating a connection" ()
([]
(let [listener (listener-run-in-background)
connection (connection-new)]
(and (connection-new)
(not (listener-close listener))
(not (connection-new))))))

(deftest test-connection-run "running a complete connection" ()
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))

So there's my complete set of tests to date. Which I can then without
changing the test-all that I had already defined.

user=> (test-all)
"Elapsed time: 11.890338 msecs"
Testing creating a listener : true

Could not connect to 127.0.0.1 on port 51345

"Elapsed time: 1194.393981 msecs"
Testing creating a connection : true
"Elapsed time: 4.478223 msecs"
Testing string->byte array conversion : true
"Elapsed time: 35.4397 msecs"
Testing running a complete connection : false
"Elapsed time: 11.453132 msecs"
Testing closing a listener : true
"Elapsed time: 13.804548 msecs"
Testing listener open?/closed? predicates : true
---
Some tests failed.
nil

Ok, so I notice two things here. One is that every test works except
what should be the last one. The other is that, well, it isn't the
last one. In fact, the tests are in no particular order when, in fact,
they should have a certain level of ordering in them.


### Setting up after statements...

Sounds like I finally have a use for that extras argument that, until
now, I haven't been using.

(defstruct unit-test :name :description :function :after)

(def unit-test-after (accessor unit-test :after))

(deftest test-listener-predicates "listener open?/closed? predicates"
(:after "test-listener-new")
([]
(test-listener-predicates DEFAULT-PORT))
([port]
(let [listener (listener-new port)
tests (and (not (listener-closed? listener))
(listener-open? listener))]
(. listener (close))
(and tests
(not (listener-open? listener))
(listener-closed? listener)))))

(deftest test-listener-close "closing a listener"
(:after "test-listener-predicates")
([]
(let [listener (listener-new DEFAULT-PORT)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)))))

(deftest test-connection-new "creating a connection"
(:after "test-listener-close")
([]
(let [listener (listener-run-in-background)
connection (connection-new)]
(and (connection-new)
(not (listener-close listener))
(not (connection-new))))))

(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))

There. Now tests that should have sequence can have them. But this
means that I need to actually extract my function for generating the
list of unit tests and make it respect the rules that I just added.

(defn unit-test-list
([]
(map (fn [x] (list (unit-test-description x)
(unit-test-function x)))
(vals ALL-TESTS))))

Which gives me the same result for test-all. Now, to sort this data
rather than just dropping it in a list.

(defn unit-test-list-helper [unsorted sorted]
(let [test (first unsorted)
unsorted (rest unsorted)]
(cond (or (not (unit-test-after test))
(and (eql? :all (unit-test-after test))
(not unsorted))
(first (filter (fn [x] (eql? (unit-test-name x)
(unit-test-after test)))
unsorted)))
(list unsorted
(concat sorted
(list test)))
:t
(list (concat unsorted
(list test))
sorted))))

So what that's supposed to do is run through the unsorted list, adding
things to the sorted list when they either have no after statement or
their after statement is met.

Now, I'll bind that helper into a loop:

(defn unit-test-list
([]
(cond (rest (filter (fn [test] (eql? :all
(unit-test-after test)))))
(do (println "Error: Only one test may be marked as"
"coming after all others.")
[])
:t
(unit-test-list (vals ALL-TESTS) (list))))
([unsorted sorted]
(if (not (first unsorted))
(map (fn [test] (list (unit-test-description test)
(unit-test-function test)))
sorted)
(let [helped-list (unit-test-list-helper unsorted sorted)
unsorted (first helped-list)
sorted (second helped-list)]
(recur unsorted sorted)))))

And running it:

user=> (unit-test-list (vals ALL-TESTS) nil)

It hangs.

Stepping through the helper function in a long series of examinations
tells me that it's hanging at the last step. Then, after changing it
to reflect my actual intent (I typoed sorted as unsorted. I start to
think about whether this is actually a good way to build a list. After
all, it will take up to the factorial of the length of the list of
tests to actually build it.

I poke with comparators for a bit and find that my somewhat fuzzy
requirements don't exactly meet comparator's exacting demands[2]. Then
I decide to revisit my iffy list function, but move blocks of tests at
a time.

(defn unit-test-list-helper [unsorted sorted]
(let [nils (filter (comp not unit-test-after)
unsorted)
alls (filter (comp (appl eql? :all) unit-test-after)
unsorted)
others (filter (fn [t] (not-any? (fn [x] (eql? (unit-test-
after t)
(unit-test-
after x)))
(concat nils alls)))
unsorted)
in-sorted (fn [t] (some (fn [x] (eql? (unit-test-name x)
(unit-test-after t)))
(concat nils sorted)))]
(cond (first nils)
(list (concat alls
(filter (complement in-sorted)
others))
(concat nils
(filter in-sorted others)))
(== (count alls)
(count unsorted))
(list nil
(concat sorted alls))
:t
(list (concat alls
(filter (complement in-sorted)
others))
(concat sorted
(filter in-sorted others))))))

This one takes the unsorted list, breaks it into three parts, and
moves what it can over to sorted at each step. It sorts my list of
functions in three steps, rather than twenty. However, those steps are
relatively expensive. I might as well compare the two functions'
performances while I've them both on my screen.

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println
"----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 3.240635 msecs"
"Elapsed time: 0.478832 msecs"
"Elapsed time: 0.459835 msecs"
"Elapsed time: 0.713778 msecs"
"Elapsed time: 0.462908 msecs"
"Elapsed time: 0.450616 msecs"
"Elapsed time: 0.433575 msecs"
"Elapsed time: 0.506489 msecs"
"Elapsed time: 0.453689 msecs"
"Elapsed time: 5.946846 msecs"
----------
"Elapsed time: 8.428725 msecs"
"Elapsed time: 1.182832 msecs"
"Elapsed time: 1.732902 msecs"
"Elapsed time: 1.130031 msecs"
"Elapsed time: 1.121931 msecs"
"Elapsed time: 1.130869 msecs"
"Elapsed time: 1.137016 msecs"
"Elapsed time: 1.390959 msecs"
"Elapsed time: 1.163555 msecs"
"Elapsed time: 1.132546 msecs"

Interesting. The old one actually did better than the new. Now, I
could count their operations, but it would be more fun to get the REPL
testing this for me.

user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake
test" (:after nil) [] true)))

There. Made a hundred test macros with no structure to them. gensym
just makes a symbol with a guaranteed-unique name. That ensures that,
in generating these tests, I don't have to first come up with my own
means to make unique names.

Now to try again:

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println
"----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 108.745944 msecs"
"Elapsed time: 44.778876 msecs"
"Elapsed time: 38.915002 msecs"
"Elapsed time: 47.836247 msecs"
"Elapsed time: 41.699434 msecs"
"Elapsed time: 40.098393 msecs"
"Elapsed time: 41.917059 msecs"
"Elapsed time: 40.536157 msecs"
"Elapsed time: 39.040995 msecs"
"Elapsed time: 39.437135 msecs"
----------
"Elapsed time: 21.603863 msecs"
"Elapsed time: 17.80338 msecs"
"Elapsed time: 17.744992 msecs"
"Elapsed time: 19.674568 msecs"
"Elapsed time: 13.584687 msecs"
"Elapsed time: 19.662834 msecs"
"Elapsed time: 17.736891 msecs"
"Elapsed time: 18.160967 msecs"
"Elapsed time: 13.494174 msecs"
"Elapsed time: 16.875608 msecs"

A gap is starting to form, I think. What about adding some test macros
that do rely on other tests?

user=> (import '(java.util Random))
nil
user=> (def RNG (new Random))
#<Var: user/RNG>
user=> (. RNG (nextInt))
Reflection warning, line: 948 - call to nextInt can't be resolved.
1410779829
user=> `(deftest ~(gensym) "z" (:after nil) [] true))
user=> `(deftest ~(gensym) "z" (:after nil) [] true)
(user/deftest G__2659 "z" (:after nil) [] true)
user=> `(deftest ~(gensym) "z" (:after ~(let [keys (keys ALL-TESTS)]
(first (drop (rem (. RNG (nextInt)) (count keys)) keys)))) [] true)
Reflection warning, line: 951 - call to nextInt can't be resolved.
(user/deftest G__2665 "z" (:after "G__2491") [] true)
user=> (dotimes x 1000 (eval `(deftest ~(gensym) "z" (:after ~(let
[keys (keys ALL-TESTS)] (first (drop (rem (. RNG (nextInt)) (count
keys)) keys)))) [] true)))

So, that adds a thousand tests in that do follow a structure, as they
all have an after statement this time.

And I run the test again ...

And I go get coffee ...

5 minutes...

It seems to be still running...

Maybe a thousand was a bad seed...

Oh, there it is:

user=> (do (dotimes x 10 (time (unit-test-list-old))) (println
"----------") (dotimes x 10 (time (unit-test-list))))
"Elapsed time: 21438.660779 msecs"
"Elapsed time: 24259.176236 msecs"
"Elapsed time: 20528.833693 msecs"
"Elapsed time: 20170.246777 msecs"
"Elapsed time: 23642.198659 msecs"
"Elapsed time: 20390.313015 msecs"
"Elapsed time: 19931.102975 msecs"
"Elapsed time: 22232.556093 msecs"
"Elapsed time: 21692.653954 msecs"
"Elapsed time: 20286.386119 msecs"
----------
"Elapsed time: 5885.515896 msecs"
"Elapsed time: 7089.577027 msecs"
"Elapsed time: 6119.980053 msecs"
"Elapsed time: 6135.949681 msecs"
"Elapsed time: 6105.716229 msecs"
"Elapsed time: 6035.137985 msecs"
"Elapsed time: 6120.137894 msecs"
"Elapsed time: 5858.970903 msecs"
"Elapsed time: 5882.34203 msecs"
"Elapsed time: 6120.462796 msecs"

So, definitely, as the structure gets more complicated, the slightly
more sensible approach is pulling ahead.

But just for fun, as I'm going to bed:

user=> #<Var: user/ALL-TESTS>
user=> (dotimes x 100 (eval `(deftest ~(gensym) "generated fake
test" (:after nil) [] true)))
nil
user=> (dotimes x 100000 (eval `(deftest ~(gensym) "generated fake
test" (:after ~(let [keys (keys ALL-TESTS)] (first (drop (rem (. RNG
(nextInt)) (count keys)) keys)))) [] true)))
Reflection warning, line: 957 - call to nextInt can't be resolved.
(do (dotimes x 10 (time (unit-test-list-old))) (println "----------")
(dotimes x 10 (time (unit-test-list))))

Ok, somewhere before generating a 100000-length tree of tests, it
crashed, due to a lack of symbols. Fair enough. That test was just
waving things around to see if I can.

clojure.lang.Compiler$CompilerException: REPL:749: PermGen space
user=> (count (keys ALL-TESTS))
42577


### Revisiting generating my accessors.

However, going back one step, all that silly mucking around with
gensyms and defines gives me an idea:

(map (fn [key] (eval `(def ~(sym (name *current-namespace*)
(strcat "unit-test-"
(name key)))
(accessor unit-test ~key))))
(keys unit-test))

There. Now I know that, should I later decide to add more keys to that
structure, the accessors will be there and waiting for me. Except
that, in a freshly-loaded REPL (with this file loaded via load-file
rather than dumping it in,) I find that my accessors are no longer
loaded.

All right, I'll take the hint, stop picking at this, and leave them as
normal functions rather than generated ones.


### Revisiting actually running the tests

So, I try running all my tests again. And the test function (which I
haven't changed since I started that little digression into sorting my
tests) still does what it's told, albeit with a tiny wart and a failed
test.

user=> (test-all)
"Elapsed time: 306.707138 msecs"
Testing creating a listener : true
"Elapsed time: 216.1839 msecs"
Testing string->byte array conversion : true
"Elapsed time: 11.579405 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 10.170846 msecs"
Testing closing a listener : true

Could not connect to 127.0.0.1 on port 51345

"Elapsed time: 1064.018472 msecs"
Testing creating a connection : true
"Elapsed time: 46.276552 msecs"
Testing running a complete connection : false
---
Some tests failed.

And here's the offending test. I'll step through it one bit at a time
and see what's going wrong in actuality.

(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(> 0 (. result (length)))
(== 0 (. (connection-run) (length)))))))

user=> (def listener (listener-run-in-background))
#<Var: user/listener>
user=> (def result (connection-run))
#<Var: user/result>
user=> (listener-open? listener)
true
user=> (listener-close listener)
nil
user=> (listener-closed? listener)
true
user=> (> 0 (. result (length)))
Reflection warning, line: 306 - call to length can't be resolved.
false
user=> (. result (length))
Reflection warning, line: 307 - call to length can't be resolved.
28
user=> (> (. result (length)) 0)
Reflection warning, line: 308 - call to length can't be resolved.
true

Ok, so I messed up the ordering of greater-than and less-than. I find
that I do this a lot in Lisp, because of how I read (> 0 ... mentally.
Namely, as a single function, testing if what follows is greater than
the integer that I've put there. Or, in other words, exactly
backwards.

So, change that to (> (. result (length)) 0) and re-run the test
function:

user=> (test-all)
java.lang.NullPointerException
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:
24)
at user.test_connection_run.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:171)

at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)

at clojure.eval.invoke(boot.clj:635)
at user.test_all.invoke(Unknown Source)
at clojure.lang.AFn.applyToHelper(AFn.java:171)

at clojure.lang.AFn.applyTo(AFn.java:164)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2213)
at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)

"Elapsed time: 9.756547 msecs"
Testing creating a listener : true
"Elapsed time: 4.988344 msecs"
Testing string->byte array conversion : true
"Elapsed time: 26.513984 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 10.239011 msecs"
Testing closing a listener : true

Could not connect to 127.0.0.1 on port 51345

"Elapsed time: 997.681828 msecs"
Testing creating a connection : true

Could not connect to 127.0.0.1 on port 51345

Erk. All right, that's getting a bit odd. Changing it back gets me the
previous, broken test, but that's not exactly helpful. Looking at that
dump, I can see that, first off, the error happened in test-connection-
run, but, secondly, it happened when trying to invoke an instance
method.

Further, on that line, the only one I've been changing, there is
indeed an instance method. However, on a successful connection, even
one with no data, it should return an empty string.

The answer here lies with the next line.

user=> (== 0 (. (connection-run) (length)))
Reflection warning, line: 894 - call to length can't be resolved.
java.lang.NullPointerException
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:
24)
at clojure.lang.Compiler$InstanceMethodExpr.eval(Compiler.java:
950)
at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:2212)

at clojure.lang.Compiler.eval(Compiler.java:3086)
at clojure.lang.Repl.main(Repl.java:59)

Could not connect to 127.0.0.1 on port 51345

Before, the failure at the successful length was causing and to stop
and return false, in a "short-circuiting" behaviour that's fairly
normal[3].

Now, what that reminds me is that (connection-run) is going to return
nil when it can't open a connection at all. So thus, I'm calling
(length) on nil: (. nil (length)) which, deservedly, throws an error.
So, in this case, the reflection warnings about length not being
resolved were helping me rather than nagging.

(defn #^Int length-of-string [#^String string]
(if string
(. string (length))
0))

user=> (length-of-string "asdf")
4
user=> (length-of-string nil)
0

(deftest test-connection-run "running a complete connection"
(:after :all)
([]
(let [listener (listener-run-in-background)
result (connection-run)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)
(<= 0 (length-of-string result))
(== 0 (length-of-string (connection-run)))))))

user=> (test-all)
"Elapsed time: 75.76773 msecs"
Testing creating a listener : true
"Elapsed time: 3.349029 msecs"
Testing string->byte array conversion : true
"Elapsed time: 9.635583 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 9.517131 msecs"
Testing closing a listener : true

Could not connect to 127.0.0.1 on port 51345

"Elapsed time: 944.630393 msecs"
Testing creating a connection : true

Could not connect to 127.0.0.1 on port 51345

"Elapsed time: 1001.659708 msecs"
Testing running a complete connection : true
---
All tests passed.

And, testing all of that, it does indeed work out the way it ought to,
except for the aforementioned wart. To point it out, "Could not
connect to 127.0.0.1 on port 51345" would appear to indicate an error,
whereas I know that it's actually a valid part of both the tests for
which it appears.

So, here's the problem: those errors are useful rather than throwing
an exception when connecting manually, but they give the wrong
indication (to me at least) when connecting automatically.

So, what I'd like is to shove any messages generated into something
other than standard output, where I can inspect them if I so choose.

(defn test-all []
(let [test-out (new StringWriter)]
(loop [test-list (unit-test-list)
failure false]
(let [test (first test-list)
test-list (rest test-list)]
(cond (not test)
(do (if failure
(println "---\nSome tests failed.")
(println "---\nAll tests passed."))
(list failure test-out))
:t
(let [test-result (time (binding [*out* test-out]
(eval (list (second test)))))]
(println "Testing" (first test) ":" test-result)
(recur test-list
(when (or (not test-result)
failure)
true))))))))

So, what that does is, before doing anything, creates a StringWriter
outside the loop. Then, when it comes time to perform the test,
temporarily re-binds out (the variable telling Clojure where to print
to) to that StringWriter.

This means, now, that I can return the success/failure of the set of
tests, as well as any error messages, as a list.

user=> (def test-results (test-all))
"Elapsed time: 8.808941 msecs"
Testing creating a listener : true
"Elapsed time: 3.364394 msecs"
Testing string->byte array conversion : true
"Elapsed time: 8.342401 msecs"
Testing listener open?/closed? predicates : true
"Elapsed time: 63.791348 msecs"
Testing closing a listener : true
"Elapsed time: 982.201928 msecs"
Testing creating a connection : true
"Elapsed time: 1002.468191 msecs"
Testing running a complete connection : true
---
All tests passed.
#<Var: user/test-results>
user=> test-results
(nil Could not connect to 127.0.0.1 on port 51345

Could not connect to 127.0.0.1 on port 51345

)


## Afterthoughts

So, now I have a very basic (albeit ostensibly extensible) testing
framework, as well as a setup that generates that framework
automatically.

So, now a few (code-dump-free) thoughts as I actually make sure that
my in-file commentary is up to date.


### Mistakes

What are the current downsides to this system and this approach? Well,
for starters, the test framework only recognises nil and true. This
means that, when an early test manages to hang and throw an exception,
it brings down the test framework as well.

In laziness, I chose to use return values in order to not have to play
too much with caught/thrown errors, and instead passed truth/falsity
around. Not necessarily the best approach, but I haven't decided
whether throwing/catching is better in this case[5].

Also, I'll be the first to admit that I'm handling errors in a bit of
a cheap way, and one that, were I actually turning this into a real
component, rather than something that I'm building as the "build one
to throw away" of my learning process, I'd have started to question at
about the point that I was wrapping a binding around println in order
to isolate my error handling away from my testing function.

Further, the 100 lines defining my simple testing framework should
really be extracted from this client-server application. They're only
loosely attached to each other, so I should be able to give them their
own namespaces with minimal hassle. (I'll wait for the next Clojure
release to do that, though, because I know I'll have to change a bunch
in order to handle upcoming changes, and would prefer to discuss
namespaces with myself once the new system is in.)


### Lessons

On contrast, what have I learned? Well, for one, do not try to
generate 17 000 lines of Lisp at once via an elisp command. Or, at
least, save your document first. (I lost all of my poking at
comparators via that gaffe.)

Also, that macros really aren't especially terrifying. Believe me or
don't, but deftest was the first Lisp macro that I've ever written.
I'm still tripping over order of operations and how forms get expanded
and names get resolved, especially when dealing with things like def
and load_file, but I do think that that falls simultaneously with me
trying to be too clever and not actually understanding what it is that
I'm doing.

As I write more, I keep finding new boot.clj functions that let me
write a verbose expression a lot more succinctly and clearly. My new
ones are with-open, some, not-any? and anything beginning with sort.

I'm finding that having the manual as a single, indexed document is
making me try harder to use Clojure's functions for doing something,
rather than rolling my own off of Java's library.

As I've found before, having a single unified environment in which to
write code, then evaluate it selectively is a great asset. It means,
for me at least, that I spend less time having to mentally page-swap
as I try to remember what I was going to do next.

There's some caveats to that, though: Number one is that my current
environment state doesn't necessarily match what I have written down.
I found myself, this session, killing by REPL on occasion just to make
sure that it was matching my work as exactly as possible.

Also, it makes me lazy about making sure that, e.g., functions are
declared before another function references them. Which works just
fine, right up until I try to load the file and realise that my
selective evaluation has (yet again) created a file that needs minor
reordering in order to load.

But that's a trivial irritation. And far outweighed by being able to
write this (as HTML-heavy Markdown), manipulate the actual source of
this program and run the REPL, all in the same environment, and all
sharing data6.


## Future / Goals

Well, beyond the nebulous goals I dropped at the end of my initial
look, I now have a few more.

* Cease using nil as an error value.
* Move to separate test / application namespaces.
* Look at existing functions that do the same thing (the closes and
length-of-string come to mind) and push them into multimethods
instead.
(Yes, the long-term geal here is to meander through all 21 chapters
of the manual as I find needs.)
* Explore wrapping an extant Java unit testing facility in Clojure
rather than rolling my own (a nice learning exercise but not exactly a
valid solution to anything beyond making myself try out the language.


## Notes

1. Ok, it sounds like a fine idea for a project, but I'm trying not to
let my focus wander all over the place, so I'll keep this experiment/
application self-contained.

2. To clarify my failure to use comparators a little bit, a comparator
is an object/function that grabs some items, compares them, and
replies which one is greater.

user=> (sort (comparator >) [5 4 8 4 3 8 4 6])
(8 8 6 5 4 4 4 3)
user=> (sort (comparator (fn [x y] (println x y (> x y)) >)) [5 4 8 4
3 8 4 6])
5 4 true
4 8 false
8 4 true
3 8 false
8 4 true
4 6 false
4 3 true
(5 4 8 4 3 8 4 6)
user=> (sort (comparator (fn [x y] (println x y (>= x y)) >)) [5 4 4
4])
5 4 true
4 4 true
4 4 true
(5 4 4 4)

3. Here's where that falls apart for my purposes, though: when sorting
numbers, letters, names, etc., I know the relative ordering of any
two. On the other hand, with these tests, unless one of four
conditions is met, the ordering of the two items is unknown.

1. :after is nil. The test comes before all others.

2. :after is :all. The test comes after all others.

3. Both tests have the same :after. They are equivalent.

4. One test's :name is the other test's :after. They have to go in
that order.

My problem is that most of my comparisons do not meet these
standards. Hence my sort functions both having a means to defer
looking at an item until a condition is met.

4. This meaning, in a logical condition, to stop evaluating the
condition as soon as its truth / falsity becomes known. Here, and in
other tests, I was using it as somewhat of a shorthand to say, "If the
test fails at any point, the entire test has failed."

On the other hand, using logical predicates (and, or, etc.) as
control structure can quickly get unreadable at a glance.

(or (or (and (test1) (test2))
(test3))
(or (and (test4) (test5) (test6))
(and (test6) (test7) (or (test8)
(test8a)
(test8b)))
(test9)))

A contrived example, sure, but I lost the flow of that logical tree
midway through writing the test. I also tried to devise it so that the
tree structure would bear at least a passing resemblance to something
that had been modified to deal with a later condition[4]

4. (For what it's worth, it says: Perform test1. On success, perform
test2. On success of test1 and test2, return success. Otherwise,
perform test3. On success of test3, return success. On failure of
test3, try test4, test5, test6, stopping if there's failure. And so
on, until, if all other tests have failed, return the success/failute
of test9.)

5. Yes, I'm aware that the consistent (and possibly even correct)
answer is throw/catch. However, for the purposes of a simple demo
application, building Throwables to toss around strikes me as an
unnecessarily obfuscating the mechanics of what I'm doing. Back

6. Emacs, 164 characters wide, divided into two columns, with a full
column devoted to the REPL, 20 lines to the source and 40 to this
text. Which raises a question: With the admonition to not write any
function longer than a single screen, with whose screen length?

For the record, the longest function in this example is 28 lines and
the shortest is 1 line long. That's likely a direct consequence to my
having kept my source scrunched up with only a little window into
it[7]. Back

7. Done via the script:
(save-excursion
(goto-char (point-min))
(re-search-forward "^ *$" (point-max) nil)
(message (mapconcat (lambda (lc) (format "%d" lc))
(let ((lcount nil))
(while (not (eobp))
(forward-sexp)
(setq lcount
(append lcount
(list (count-lines
(save-excursion
(backward-sexp) (point))
(point))))))
lcount)
" ")))


And that's why Emacs one-liners are scary. Because before you know
it, they're actually 15 lines long, all scrunched into an Eval: prompt.

[Kearsley Schieder-Wethy]

And again, here's the snapshot of the full (now rather grown) source
after these latest changes.

;;; A simple application that acts as a very primitive time server and
;;; client.

(import '(java.net BindException ServerSocket Socket)
'(java.lang.reflect InvocationTargetException)
'(java.util Date)

'(java.io InputStream OutputStream StringWriter)
'(java.util.concurrent Executors))



;;; Globals

(def DEFAULT-PORT 51345)

;; A structure defining the data in a unit test, as well as a global
;; list of all the tests.

(defstruct unit-test :name :description :function :after)

(def ALL-TESTS {})



;;; Testing functions

;; Unit test accessors

(def unit-test-name (accessor unit-test :name))
(def unit-test-description (accessor unit-test :description))
(def unit-test-function (accessor unit-test :function))

(def unit-test-after (accessor unit-test :after))

;; Add a new test structure to the list of unit tests, or replaces the
;; existing test of that name if it already exists.

(defn unit-tests-add [test]
(def ALL-TESTS

(assoc ALL-TESTS (unit-test-name test) test)))

;; Takes an unsorted and a sorted list of unit tests and moves / sorts
;; some of the tests in the unsorted list over to the sorted list

;; Returns a list of unit tests, sorted according to their :after
;; statement.
(defn unit-test-list
([]
(unit-test-list (vals ALL-TESTS) nil))

([unsorted sorted]
(if (not (first unsorted))
(map (fn [test] (list (unit-test-description test)
(unit-test-function test)))
sorted)
(let [helped-list (unit-test-list-helper unsorted sorted)
unsorted (first helped-list)
sorted (second helped-list)]
(recur unsorted sorted)))))

;; Run all the defined tests

(defn test-all []
(let [test-out (new StringWriter)]
(loop [test-list (unit-test-list)
failure false]
(let [test (first test-list)
test-list (rest test-list)]
(cond (not test)
(do (if failure
(println "---\nSome tests failed.")
(println "---\nAll tests passed."))
(list failure test-out))
:t
(let [test-result (time (binding [*out* test-out]
(eval (list (second test)))))]
(println "Testing" (first test) ":" test-result)
(recur test-list
(when (or (not test-result)
failure)
true))))))))

;; Define a testing function. Same syntax as defn, only with a
;; description string and a list of extras (may be empty)

(defmacro deftest [fname description extras & fdecl]
`(do (defn ~fname ~@fdecl)
(unit-tests-add (struct unit-test
:name (name '~fname)

:description ~description

:function ~fname
~@extras))
#'~fname))

;;; Utility functions

(defn current-time []
(str (new Date)))

;; Generate an array of bytes from a string, suitable for passing to a
;; write() method.
;;
;; Has a unit test.
(defn byte-arr-from-string [#^String str]
(. str (getBytes)))

(deftest test-byte-array-from-string "string->byte array
conversion" ()
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)
bseq (map (comp char (appl aget barr))
(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))

(some true?
(map eql?
bseq
chseq))))))

;; Takes a sequence of bytes (not a byte array, but a sequence) and
;; concatenates it into a string.

(defn string-from-byte-sequence [coll]
(reduce strcat
(map char coll)))

;; Gets the integer length of a string. Returns nil for "" and nil.

(defn #^Int length-of-string [#^String string]
(if string
(. string (length))
0))

;;; Listener functions
;;; These control the server

;; Make a new listener. Returns that listener or nil if no listener
;; can be created on that port.
;;
;; Has a unit test.
(defn #^ServerSocket listener-new
([]
(listener-new DEFAULT-PORT))
([port]

(try (new ServerSocket port)
(catch BindException except

(println "Address is already in use.")))))

(deftest test-listener-new "creating a listener" ()
([]
(test-listener-new DEFAULT-PORT))
([port]
(with-open listener (listener-new port)
(when listener
true))))

;; Logical predicates for whether a listener is open or closed.
;;
;; Have unit tests.

(defn #^Boolean listener-closed? [#^ServerSocket listener]
(. listener (isClosed)))

(def listener-open? (complement listener-closed?))

(deftest test-listener-predicates "listener open?/closed? predicates"
(:after "test-listener-new")
([]
(test-listener-predicates DEFAULT-PORT))
([port]
(let [listener (listener-new port)
tests (and (not (listener-closed? listener))
(listener-open? listener))]
(. listener (close))
(and tests
(not (listener-open? listener))
(listener-closed? listener)))))

;; Wait for a connection to a given listener the return the socket to
;; that connection.
;;
;; NOTE: This function is blocking.
(defn #^Socket listener-wait [#^ServerSocket listener]
(. listener (accept)))

;; Closes a given listener. Really only for use from the REPL or from
;; unit tests, as actual code should be using (with-open ...)
;;
;; Has a unit test.
(defn listener-close [#^ServerSocket listener]
(. listener (close)))

(deftest test-listener-close "closing a listener"
(:after "test-listener-predicates")
([]
(let [listener (listener-new DEFAULT-PORT)]
(and (listener-open? listener)
(not (listener-close listener))
(listener-closed? listener)))))

;; Takes a socket, sends the current time through it, closes the
;; socket's output and returns the socket.
(defn #^Socket listener-send [#^Socket lsocket]

(.. lsocket
(getOutputStream)
(write (byte-arr-from-string (current-time))))
(.. lsocket (getOutputStream) (close))
lsocket)

;; Launches the supplied listener and then loops, waiting for
;; connections.
;;
;; NOTE: Because it depends of (listener-wait ...), which is blocking,
;; this is qlso blocking.

(defn listener-run [listener port]
(loop [socket nil]
(if (listener-closed? listener)
listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

;; Launches a listener as a background thread. Returns that listener.
(defn listener-run-in-background
([]
(listener-run-in-background DEFAULT-PORT))

([port]
(let [listener (listener-new port)

exec (. Executors (newSingleThreadExecutor))
run (appl listener-run listener port)]

(when listener
(. exec (submit run)))

listener)))

;;; Connection functions
;;; These control the client.

;; Makes a new connection to a listener. Times out in 5 seconds if no
;; connection is made.
;;
;; Has a unit test.
(defn connection-new
([]
(connection-new DEFAULT-PORT))

([port]
(connection-new "127.0.0.1" port))
([address port]
(try (doto (new Socket address port)
(setSoTimeout 5000))
(catch InvocationTargetException except
(println (strcat "Could not connect to "
address
" on port "
port))))))

(deftest test-connection-new "creating a connection"
(:after "test-listener-close")
([]
(let [listener (listener-run-in-background)
connection (connection-new)]
(and (connection-new)
(not (listener-close listener))
(not (connection-new))))))

;; Reads data from a supplied socket as an array of bytes. Stops on
;; timeout and when receiving the -1 end of stream byte.
(defn connection-read [#^Socket conn]

(let [instream (. conn (getInputStream))
reader (fn [] (try (. instream (read))
(catch InvocationTargetException except
-1)))]
(loop [bytes nil
current-byte (reader)]
(if (== current-byte -1)
bytes
(recur (concat bytes (list current-byte))
(reader))))))

;; Closes an open socket.
(defn connection-close [#^Socket conn]
(. conn (close)))

;; Opens a connection to a listener, reads out a byte sequence, then
;; returns that sequence as a string.
;;
;; Has a unit test.
(defn connection-run
([]
(connection-run DEFAULT-PORT))
([port]

(let [conn (connection-new port)
str (when conn
(string-from-byte-sequence (connection-read conn)))]
(when conn
(connection-close conn))

str)))

[Kearsley Schieder-Wethy]

On Feb 4, 11:07 am, "arsalan.za...@gmail.com"
<arsalan.za...@gmail.com> wrote:

> Yes, I saw that, but are you sure you want to dump everything into one
> wiki book? Shouldn't that page be just for the canonical "Programming
> in Clojure" stuff rather than snippets, tips and tutorials like this?
>
> --Arsalan

At the moment, I'm keeping a reference copy (with formatting and
syntax hilighting) on a blogspot page that I use to think about my
works-in-progress. I don't know that this would make a good tutorial
or even a good set of tips, because it's deliberately meandering;
discussing more how I go about fixing my own mistakes than actually
commenting on why working code works.

I am, however, watching the wikibook and using this meandering to get
my thoughts in order before being so presumptuous as to tell anyone
that something is a good or bad way to build Lisp software.

[John]

Thanks for the awesome tutorial. That really helped me become a
little more familiar with Clojure. I'm a novice, self-taught
programmer with just enough knowledge about Java to get around. So
this tutorial was very helpful. I'll be picking through this for bits
of knowledge for a while, I'm sure. If you find more time to write
more tutorials in this way, you can be sure I'll read them carefully.

As I was going through it though, I had to change some of the words,
like appl (to partial) and strcat (to str). So I've ended up with a
working copy of the program, along with imports and whatever else was
necessary, and I thought I'd post it here so that others can benefit
from it too. Thanks again.


;; written by Kearsley Schieder-Wethy at
;; http://groups.google.com/group/clojure/msg/f038decc18c7da37?
;;
;; updated by johnmn3 at gmail
;;
;; runs on clojure_20080213


;;;;;;; To run it ;;;;;;
;
;;; Copy and paste this whole doc into the REPL.
;;; Then then just do like this:
; user=> (def li (listener-run-in-background 54323))
;; #<Var: user/li>
; user=> (connection-run 54323)
;; "Sun Mar 16 23:42:47 GMT 2008"
;
;;;;;;;;;;;;;;;;;;;;;;;;


;;;;;;;;;;;;;;;;;; Server Side ;;;;;;;;;;;;;;;;;


(import '(java.io IOException InputStream
OutputStream)
'(java.net ServerSocket Socket
BindException)
'(java.util Date)
'(java.util.concurrent Executors))

(defn current-time []
(. (new Date) (toString)))

(defn listener-new [port]

(try (new ServerSocket port)
(catch BindException except
(println "Address is already in use."))))

(defn listener-wait [listener]
(. listener (accept)))

(defn byte-arr-from-string [str]
(. str (getBytes)))

(defn listener-send [lsocket]
(.. lsocket
(getOutputStream)
(write (byte-arr-from-string (current-time))))
(.. lsocket (getOutputStream) (close))
lsocket)

(defn listener-close [listener]
(. listener (close)))

(defn listener-run [listener port]
(loop [socket nil]
(if (. listener (isClosed))
listener
(do (when socket
(. (listener-send socket) (close)))
(recur (listener-wait listener))))))

(defn listener-run-in-background [port]
(let [listener (listener-new port)
exec (. Executors
(newSingleThreadExecutor))

run (partial listener-run listener port)]

(. exec (submit run))
listener))

;;;;;;;;;;;; Client Side ;;;;;;;;;;;;;;;;;;;;;


(import '(java.io BufferedReader InputStreamReader
IOException OutputStreamWriter
PrintWriter)
'(java.net Socket)
'(java.lang.reflect InvocationTargetException))

(defn string-from-byte-sequence [coll]
(reduce str
(map char coll)))

(defn connection-new
([port]
(connection-new "127.0.0.1" port))
([address port]
(try (doto (new Socket address port)
(setSoTimeout 5000))
(catch InvocationTargetException except

(println (str "Could not connect to "

address
" on port "
port))))))

(defn connection-read [conn]
(let [instream (. conn (getInputStream))
reader (fn [] (try (. instream (read))
(catch InvocationTargetException
except -1)))]
(loop [bytes nil
current-byte (reader)]
(if (== current-byte -1)
bytes
(recur (concat bytes (list current-byte))
(reader))))))

(defn connection-close [conn]
(. conn (close)))

(defn connection-run [port]
(let [conn (connection-new port)

str (when conn (string-from-byte-sequence
(connection-read conn)))]
(when conn
(connection-close conn))
str))

;;;;;;;;;;;;;; testing conversion from string to
array ;;;;;;;;;;;;;;;;

(defn test-byte-array-from-string
([]
(test-byte-array-from-string (current-time)))
([str]
(let [barr (byte-arr-from-string str)

bseq (map (comp char (partial aget barr))

(range (alength barr)))
chseq (map char str)]
(and (== (alength barr)
(count bseq)
(count chseq))
(== 0
(count (filter false?

(map =
bseq
chseq))))))))

;;;;;;;;;;;;;;

[John]

Yea, scratch that. Curse Google. It could be that I'm browsing over
a shoddy satellite internet connection. As it turned out, when I
found your tutorial, your first post was the only thing that showed
up. All your subsequent posts (and the posts of others) weren't
there. So I did all that and I didn't even need to :/ Oh well, I
learned a good bit and now I can read the rest to learn some more. At
least my update has the name changes.

> ...
>
> read more »

[Joubert Nel]

Hi,

I was looking at this tutorial and am wondering about the 2 code
extracts below.

In my tests the string-from-byte-sequence does not work for all
strings (e.g. string with characters further down the Unicode table),
since it does a per-byte to character processing (whereas more than
one byte may be required to represent a "character").

To rectify this, I was trying to do (new String coll), or even (new
String coll "UTF-8"). However, coll seems to be a lazy sequence (if
obtained from connection-read (which I've included below as well), and
so the String constructor fails (it expects a byte array).

Now I am wondering what is the *recommended* way to take a lazy seq
and create an array from it?

Joubert


CODE:

Snippet A
========
On Mar 16, 1:25 am, John <john...@gmail.com> wrote:
> > (defnstring-from-byte-sequence [coll]

> >  (reduce str
> >         (map char coll)))

Snippet B
=========

[Rich Hickey]

On May 14, 12:57 am, Joubert Nel <joubert....@gmail.com> wrote:
> Hi,
>
> I was looking at this tutorial and am wondering about the 2 code
> extracts below.
>

Those were less tutorials than recordings of impressions of initial
explorations of Clojure, so caveat emptor. If you want to deal with
characters correctly, you need to use some variant of Reader, not raw
bytes.

Rich

[Joubert Nel]

I have figured out a solution - for those who might be curious (or
getting to know the Java class library as you go along like I am), the
key is to use the DataInputStream and DataOutputStream - these 2
classes have built-in support for reading/writing UTF data.

Joubert

[Michael Beauregard]

Agreed. I would not have gone through all that string/byte conversion
manually. Just use a BufferedReader and BufferedWriter to get the job
done.

Something like:

(import '(java.io BufferedReader InputStreamReader))
(def reader (new BufferedReader (new InputStreamReader (. socket
(getInputStream)))))

for creating a buffered reader around the client socket would work
fine. Then you can read strings from the stream like:

(. reader (readLine))

Michael