concat component?
Serge Wroclawski
looking at the existing one and the concat one seems like the
canonical answer. Here it is as of last checkout:
(def-component {:name "concatenate"
:category "Transform"
:description "Concats record sequences"
:output-type "Record Sequence"
:args {"inputs" {:doc "The record sequences to concatnate"
:type "Record Sequence"
:min-required 1}}}
(fn [pipe pipe-args args]
((fn cat-inputs [inputs]
(concat
(lazy-seq
(if (seq inputs)
(concat (first inputs) (cat-inputs (rest inputs)))
'())))
(args "inputs"))))
All the components should assume that they're taking sequences as
inputs. If someone throws a non-sequence at a component, they're
taking their life into their own hands (and an exception should be
thrown).
And concat takes as many arguments as you can throw at it and returns
a lazy-seq. Such as in:
(take 2 (concat '({:a 1 :b 2}) '({:a 2 :b 3}) '({:a 3 :b 4})))
So why does this function construct a lazy-seq recursively?
Why doesn't this do the same job?
(def-component {:name "concatenate"
:category "Transform"
:description "Concats record sequences"
:output-type "Record Sequence"
:args {"inputs" {:doc "The record sequences to concatnate"
:type "Record Sequence"
:min-required 1}}}
(fn [pipe pipe-args args]
((fn cat-inputs [inputs]
(apply concat args)))))
My question is do I:
a) Misunderstand concat?
b) Misunderstand the def-component macro?
c) Misunderstand something else?
- Serge
Luke VanderHart
identical and obviously a lot cleaner.
Somehow I was having a brain fart and didn't realize concat was
already lazy. Doh. I think I'd been working with so many non-lazy
sequence functions right before that I wrapped it in a lazy-seq by
default, which is very unnecessary.
But if it WASN'T lazy, that's how you'd do it ;)
Sorry about that.
-Luke
Serge Wroclawski
<luke.va...@gmail.com> wrote:
>
> Um... you're right. The version you cite is actually semantically
> identical and obviously a lot cleaner.
>
> Somehow I was having a brain fart and didn't realize concat was
> already lazy.
I was thinking that maybe your def-component macro did something and
un-lazyied the structures.
I know we're thinking of rewriting everything from scratch, but I'd
still like to have you explain the macro. It hurt my head.
- Serge
Luke VanderHart
symbols and macro-language.
I'll refer to thinks by line number on the following link, by the way.
The macro definition starts on line 73.
http://github.com/levand/flowjure/blob/b10b4efd59d08731e056c045e7765f61cc01d149/src/flowjure/engine.clj
The macro takes two parameters. The first is simply a map which will
be attached as metadata. The second is a form which returns a function
(most commonly, a function definition).
Line 81 starts the body of the macro with a backtick, and opens with a
defn. The macro as a whole, then, expands to a "defn" expression. The
fn it defines is actually what is called during pipe creation.
Note that on line 81, the actual name of the function is created by
taking the :name field of component-cfg, creating a symbol from it,
and appending metadata to it. So the ~ expression in line 80 simply
resolves to a symbol with attached metadata.
Line 82 adds the "description" as the "documentation" line in a normal
defn statement.
Line 83 declares that the function will take one argument. We add a #
to invoke the auto-gensym and prevent our parameter name from
capturing any external variable.
The "let" statement on line 84 binds "action-fun#" to the function
definition that was passed in to the macro. Note that we expand it -
this means that we can then _call_ "action-fun#" within our "let".
85-86 just define certain local bindings from the vars *pipe* and
*pipe-args* for convenience. *pipe* and *pipe-args* are clojure Vars,
which obey a stack discipline - the engine ensures that the function
we are defining ought only to be called in the context of loading a
pipe, and these two vars are guaranteed to be set appropriately to the
pipe we're loading.
87 creates a do-sync, because *pipe* and *pipe-args* (now pipe# and
pipe-args#) are refs and we will be modifying them.
88 lets pipe-key# to the ID of the pipe, unless it is an "output"
component, in which case it gives it the special name of "pipe-
output". All pipes have a component named pipe-output, so the system
knows which one to pull from first.
Line 91 just adds a key/val to *pipe*. The key is the pipe-key# from
line 88. The value is a simple function that basically wraps the
action-fun# - the function we originally passed in - in some error-
handling code and passes it the *pipe-args* ref.
So - it looks complicated, but all that is happening is that we are
defining a function that we will call when _defining_ a pipe. The
function we define, when called, takes a ref called *pipe*, and adds a
key and a value, where the key is the ID of a component and the value
is the function we want to call when _running_ the component.
Sorry it's hard to explain more clearly in text. I can certainly go
over it next time we get together.
Thanks,
-Luke
On May 5, 7:15 am, Serge Wroclawski <emac...@gmail.com> wrote:
> On Mon, May 4, 2009 at 4:54 PM, Luke VanderHart
>