macro woes (deftype generator)

[Karsten Schmidt]

Hi, I'm (once again) despairing over some issues trying to integrate
one macro with another. Both are supposed to be used together to
generate a bunch of deftypes. Firstly, here's the one working
correctly (stripped out error checking for simplicity):

(defmacro swizzle*
"Takes a class name, source instance, keyword and metadata map,
returns new instance of type with fields populated based on
decomposed single letter key lookups. E.g.

(macroexpand-1 '(swizzle* Vec3 v :zyx))
#_=> (new Vec3 (. v z) (. v y) (. v x) {})

(macroexpand-1 '(swizzle* Vec3 v :xxz))
#_=> (new Vec3 (. v x) (. v x) (. v z) {})
"
[clz src k meta]
`(new ~clz
~@(->> k
(name)
(map (comp symbol str))
(map (fn [f] `(. ~src ~f))))
~meta))

Now, where things go wrong is when I try to use this swizzle macro
from within the 2nd macro which generates the deftype skeleton (also
stripped down):

(defmacro defvec
[n]
(let [vname (symbol (str "Vec" n))
syms (map (fn [s] (-> s str symbol)) (take n "xyzw"))]
`(deftype ~vname [~@syms __meta#]

clojure.lang.IObj
(meta [this#] __meta#)
(withMeta [this# mm#] (new ~vname ~@syms mm#))

clojure.lang.ILookup
(valAt [this# k#] (swizzle* ~vname this# k# __meta#)) ;; FIXME
)))

(defvec 3)
CompilerException java.lang.IllegalArgumentException: No matching ctor
found for class compile__stub.user.Vec3,
compiling:(NO_SOURCE_PATH:1:1)

=:_(

I've been trying a lot of other versions (e.g. inline swizzle*,
defining it as local fn etc.), but honestly I don't understand why
this fails and would really appreciate some further insights into
these aspects of the dark art of macro voodoo.

Furthermore (for the future), in that second macro how can I add type
hints to the generated field names (they should all be doubles)...

Thanks a lot!

[Mikera]

Hi Karsten,

It looks to me like you're trying to define some small fixed-size numeric double vectors and related operations? For graphics, games or physical modelling perhaps?

Perhaps you've already evaluated all the options and decided that a bunch of custom deftypes is the best way go, but in case you haven't seen it already, can I recommend considering my little library vectorz-clj? 

I designed it specifically to handle this kind of use case in Clojure, and it includes:

- Fixed size 1-4D Vectors, with many specialised methods for high performance

- Specialised 2D and 3D transformation matrices (rotations, scaling etc.)

- Affine transformations

- core.matrix  fully supported (which gives you a lot of more general matrix/vector features with a nice Clojure API)

See: https://github.com/mikera/vectorz-clj
And core.matrix: https://github.com/mikera/matrix-api

If vectorz-clj and/or core.matrix doesn't quite fit your needs, could you let me know why? I'm keen to address all the common vector use cases that people have in Clojure, so that people can avoid reinventing the wheel.

[Konrad Hinsen]

Karsten Schmidt writes:

> Now, where things go wrong is when I try to use this swizzle macro
> from within the 2nd macro which generates the deftype skeleton (also
> stripped down):

...

> (defvec 3)
> CompilerException java.lang.IllegalArgumentException: No matching ctor
> found for class compile__stub.user.Vec3,
> compiling:(NO_SOURCE_PATH:1:1)
>
> =:_(
>
> I've been trying a lot of other versions (e.g. inline swizzle*,
> defining it as local fn etc.), but honestly I don't understand why
> this fails and would really appreciate some further insights into
> these aspects of the dark art of macro voodoo.

Recursive macro expansion is an important debugging aid for macro writers:

(use 'clojure.tools.macro)
(mexpand-all '(defvec 3))

returns

(let* []
(deftype* Vec3 user.Vec3 [x y z __meta__1154__auto__]
:implements [clojure.lang.IObj clojure.lang.ILookup clojure.lang.IType]
(user/valAt [this__1155__auto__ k__1157__auto__]
(new Vec3
(. this__1155__auto__ k)
(. this__1155__auto__ _)
(. this__1155__auto__ _)
(. this__1155__auto__ 1)
(. this__1155__auto__ 1)
(. this__1155__auto__ 5)
(. this__1155__auto__ 7)
(. this__1155__auto__ _)
(. this__1155__auto__ _)
(. this__1155__auto__ a)
(. this__1155__auto__ u)
(. this__1155__auto__ t)
(. this__1155__auto__ o)
(. this__1155__auto__ _)
(. this__1155__auto__ _)
__meta__1154__auto__))
(clojure.core/meta [this__1155__auto__] __meta__1154__auto__)
(user/withMeta [this__1155__auto__ mm__1156__auto__] (new Vec3 x y z mm__1156__auto__)))
(do (clojure.core/import* "user.Vec3"))
(def ->Vec3 (fn* ([x y z __meta__1154__auto__] (new user.Vec3 x y z __meta__1154__auto__))))
user.Vec3)

So the problem is that your swizzle* macro takes apart the symbol
generated from k#.

I suspect you want to pass a keyword such as :x to valAt at
runtime. In that case, you can't decompose it using a macro at compile
time.

Konrad.

[Karsten Schmidt]

Hi Mike, thank you, I've checked out your vectorz lib in the past and
am also aware of Zach's clj-tuple (which this concrete part is much
closer too), but I'm looking for a pure Clojure & CLJS compatible
solution (using CLJX). This alI is part of a longwinded (on/off 2+ yrs
now) refactoring/total re-imagining effort of my http://toxiclibs.org/
project, which too has a comprehensive set of vector ops (and the
Clojure version many more), e.g:
http://hg.postspectacular.com/toxiclibs/src/tip/src.core/toxi/geom/Vec3D.java

K.

> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--
Karsten Schmidt
http://postspectacular.com | http://toxiclibs.org | http://toxi.co.uk

[Karsten Schmidt]

Hi Konrad, yeah, I already saw that the k# is being unravelled in this
version, but I don't understand why. Calling swizzle* with an existing
deftype instance directly is working absolutely fine, just not when
called from within the other macro. So my confusion remains about
maybe needing somewhere another layer of syntax quoting? But where?
But why?

Also, the keywords passed in to valAt/swizzle* would be in this case 3
chars long, e.g. :zyx, :xxy, : yyy - their role is basically to
redefine the order of fields in the new instance...

On 12 September 2013 07:58, Konrad Hinsen

[Mike Anderson]

Understood. vectorz-clj is going to stay JVM-only for the foreseeable future, so it won't fit if you need ClojureScript.

On the other hand, core.matrix is pure Clojure and totally protocol-based, so it should (I think!) be pretty easy to port to ClojureScript, and it's easy to extend the protocols to new implementations. Also it would be nice to have a pure Clojure small vector implementation (which we don't currently have). Can I therefore suggest some collaboration on this front, in the interests of saving effort and avoiding further fragmentation in the Clojure library space?

It would mean a few things:

1) Adding core.matrix API functions / protocols that you think are important for your use cases

2) Having your vector library implement the basic core.matrix protocols (only a few protocols are needed, the rest are optional)

3) Extending core.matrix to support ClojureScript. Several people have expressed a general interest in this.

4) Making sure formats are compatible. It should be possible, for example, for someone to use vectorz-clj on the server and your library for the ClojureScript frontend.

 

If we collectively do all this then your library will get full core.matrix compatibility, core.matrix will work on ClojureScript, and people will get a fast pure Clojure/ClojureScript small vector implementation. And we all avoid the "Lisp Curse" for a little while longer. Does that sound like a win/win/win goal to work towards?

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/09LJvmFxVNY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[Meikel Brandmeyer (kotarak)]

Hi,

maybe this gives you a hint, one why it happens:

user=> (let [k :xyz] (macroexpand-1 '(swizzle* Vec3 v k {})))
(new Vec3 (. v k) {})

The problem is that swizzle* is a macro. It has no access to runtime information! It sees only the symbol not its value. Calling it with a literal keyword works, thusly. You could hardwire some functions in case the number of coordinates is always three:

(defmacro defswizzlers
  []
  (let [v (gensym "v")]
    `(do ~@(for [x "xyz"
                 y "xyz"
                 z "xyz"]
             `(defn ~(symbol (str x y z))
                [~(with-meta v {:tag `Vec3})]
                (Vec3. ~@(map (fn [s] `(. ~v ~(symbol (str s)))) [x y z])
                       (meta ~v)))))))

(defswizzlers)

Then you can simply call like (zyx v) instead of (:zyx v).

But all that is very, very static.

Kind regards
Meikel