AOT'd namespaces lose their metadata

[Phil Hagelberg]

I noticed an odd bug when working on the help command for leiningen. It
uses docstrings on metadata for help output, but when AOTing the
project, the docstrings (as well as all other metadata) would be
lost. Note that this doesn't happen when metadata is added to the
namespace after the fact as is the case with clojure.core and
alter-meta!.

Any idea what might be causing this? I'd like to dig deeper but am
unsure where to look.

thanks,
Phil

[Lauri Pesonen]

2009/11/23 Phil Hagelberg <ph...@hagelb.org>:

There's a ticket for it

https://www.assembla.com/spaces/clojure/tickets/130-Namespace-metadata-lost-in-AOT-compile

but I don't think anyone's spent any time in trying to fix it.

> thanks,
> Phil

--
! Lauri

[Tom Faulhaber]

I did take a little bit of a look at it and it is both simple and
complex.

When you're not AOT compiling code, the symbol that's creating the
namespace gets wrapped with metadata when the ns macro is evaluated.
This metadata is then explicitly moved on to the namespace.

When the compiler runs, it doesn't seem that it is so explicit about
namespace creation. It essentially does (in-ns 'foo) which will create
the ns foo on the fly if appropriate. There is currently no code
generated to do anything *explicit* with the namespace and, therefore,
no place to decide that the namespace has metadata and tack it on.

I got about as far as I could eyeballing the code and wanted to get it
up in the debugger, but I don't have a Java debug environment set up
and my Clojure time has been super limited lately owing to all sorts
of other things happening in my life (I am *so* jealous of you guys
who are doing makor Clojure projects). If I get some time, I'll dig in
enough to make a patch, but it might be a few more months. I feel like
the fix will be pretty easy, once the real flow is well-understood.
This is my first foray into the compiler, so there's a lot to
understand (though it's all pretty straightforward).

So, Phil, if you want to take it from there, it would be great. If you
don't, I'll keep it on my list.

Tom


On Nov 23, 2:38 am, Lauri Pesonen <lauri.peso...@iki.fi> wrote:
> 2009/11/23 Phil Hagelberg <p...@hagelb.org>:

>
>
>
> > I noticed an odd bug when working on the help command for leiningen. It
> > uses docstrings on metadata for help output, but when AOTing the
> > project, the docstrings (as well as all other metadata) would be
> > lost. Note that this doesn't happen when metadata is added to the
> > namespace after the fact as is the case with clojure.core and
> > alter-meta!.
>
> > Any idea what might be causing this? I'd like to dig deeper but am
> > unsure where to look.
>
> There's a ticket for it
>

> https://www.assembla.com/spaces/clojure/tickets/130-Namespace-metadat...

[Phil Hagelberg]

Tom Faulhaber <tomfau...@gmail.com> writes:

> So, Phil, if you want to take it from there, it would be great. If you
> don't, I'll keep it on my list.

Thanks for the notes; nice to see someone has done some detective work
already. I'll probably try to take a look at this once I get Leiningen
1.0 out. It looks like ticket #130 in Assembla is about this problem, so
I'll be sure to add a note to there once I start working on it.

-Phil

[Tom Faulhaber]

Yup, that's right - I filed ticket 130 to reflect this. If I get there
first, I'll mark the ticket as well and we can compare notes.

I'm super-excited about Leiningen, btw. I've been thinking about how
to turn my autodoc stuff into a Leiningen plugin so that we can get
easy doc for any project.

On Nov 24, 8:48 pm, Phil Hagelberg <p...@hagelb.org> wrote:

[Phil Hagelberg]

Tom Faulhaber <tomfau...@gmail.com> writes:

> I'm super-excited about Leiningen, btw. I've been thinking about how
> to turn my autodoc stuff into a Leiningen plugin so that we can get
> easy doc for any project.

Awesome. We've already got someone started working on a doc plugin;
please join the leiningen mailing list if you'd like to coordinate work
on that. The contrib HTML documentation is really great, and I'd love it
if it were easier for more projects to produce output like that.

http://groups.google.com/group/clojure

-Phil

[James Reeves]

On Nov 25, 12:53 am, Tom Faulhaber <tomfaulha...@gmail.com> wrote:
> When the compiler runs, it doesn't seem that it is so explicit about
> namespace creation. It essentially does (in-ns 'foo) which will create
> the ns foo on the fly if appropriate. There is currently no code
> generated to do anything *explicit* with the namespace and, therefore,
> no place to decide that the namespace has metadata and tack it on.

I'm not sure it does. I haven't gotten very far in my investigations
of this issue, but the problem only seems to occur when the namespace
has at least one 'def' in it. A namespace with no interned vars seems
to keep the metadata okay; so it is possible to have a compiled
namespace with metadata, indicating the functionality for compiled
namespace metadata exists.

- James