[ANN] Grimoire: up to date Clojure web docs

[Reid McKenzie]

Hey guys,

If you're like me while using clojure.repl/doc works for the most part
there are just times that you need to send someone a link to the
official docs and navigating the docstrings in the core Clojure
repository is a pain.

After several months of being frustrated that clojuredocs is out of
date, I finally sat down and built an alternative which I'm pleased to
present here: http://www.arrdem.com/grimoire/

Grimoire is still alpha and subject to change, especially with regards
to the example contribution process which needs streamlining beyond the
current "click a link, edit on github and submit a PR".

I'd like to give a shout out to César for porting clojure.repl/source
into the doc generation script and to Nicola for his random usability
criticisms.

Reid McKenzie

[Francesco Bellomi]

Very nice work.

I've recently done something similar, by adding to http://crossclj.info the ability to navigate auto-generated docs from docstrings and metadata.

The user interface balance is a bit different, the navigation is by namespace or by project, for example:

http://crossclj.info/doc/org.clojure/clojure/1.6.0/clojure.string.html

you see all the docs for the whole namespace, and you can jump to the source of a function (within the whole namespace source).

you can navigate the namespace hierarchy and switch from source to docs view using the control on the top left 

There is no way to add examples, but you can search for usages from all the indexed open source projects, which is usually pretty effective (you can try by clicking on one of the Usages link in the page above).

The auto generated doc is available and cross-referenced not only for the clojure standard lib, but also for all the indexed open-source projects (you can search them from the home page, or by navigating the references in the sources), although with different quality and coverage levels, due to some practical issues in working with arbitrary sources.

Other examples:

http://crossclj.info/doc/instaparse/1.3.2/instaparse.core.html

http://crossclj.info/doc/clj-time/0.7.0/clj-time.core.html

Overall, I think it's useful and enriching to see different UI approaches at work.

Francesco

[Daniel Compton]

What is the status of Clojuredocs? It has lots of good examples and Google juice but is out of date. It would be great if we could update it to 1.6. I'd be happy to help in whatever way is needed. 

Is this possible/feasible?

Daniel.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andy Fingerhut]

Zachary Kim has mentioned in an issue comment that he might be able to find time to update the source code and doc strings to Clojure 1.6.0, here:

    https://github.com/zk/clojuredocs/issues/66

I don't know when/if that will happen.

Andy

[cldwalker]

Hey Reid,

Nice work! I think this is a much more maintainable approach for a community-based examples repository.  Have you thought about seeding examples from clojuredocs using their api e.g. http://api.clojuredocs.org/examples/1.3.0/clojure.core/map? I think this would help if the goal is to become the canonical site for clojure fn examples.

It is worth noting that clojuredocs is undergoing a rewrite, https://github.com/zk/clojuredocs/tree/clj-rewrite. If this rewrite lands soon than it may be worth adding a Motivation section to the home page to distinguish it from clojuredocs.org.

On Tuesday, July 1, 2014 7:34:38 PM UTC-4, Reid McKenzie wrote:

Hey guys,

If you're like me while using clojure.repl/doc works for the most part
there are just times that you need to send someone a link to the
official docs and navigating the docstrings in the core Clojure
repository is a pain.

After several months of being frustrated that clojuredocs is out of
date, I finally sat down and built an alternative which I'm pleased to
present here: http://www.arrdem.com/grimoire/

Grimoire is still alpha and subject to change, especially with regards
to the example contribution process which needs streamlining beyond the
current "click a link, edit on github and submit a PR".

I think this is a reasonable approach. It's better than having a login just for clojuredocs.

 

I'd like to give a shout out to César for porting clojure.repl/source
into the doc generation script and to Nicola for his random usability
criticisms.

Cheers,

Gabriel

 

Reid McKenzie

[Reid McKenzie]

I have, and I spent a little bit playing with extending my build script to scrape their API, however it wasn't dead easy so I haven't thrown a whole lot more time at it. An issue I'm running into is that the build clojure program can't tell if there are examples for a given symbol and clobbers the existing example and doc files on every pass. I expect to remove that script soon and freeze the doc files at which point I'll probably kick out a minimal bash script to pirate what examples I can from clojuredocs.

I'd also welcome a PR if you care to hack such a thing yourself :P

As to having a better frontpage and generally better navigation, I will be attempting to address those issues in the next full build/release of Grimoire because at present the homepage is close to useless and navigation in general could be better. Motivation and contribution pages will likely be part of this rework. I don't honestly think that Grimoire is likely to supplant clojuredocs. My issue with clojuredocs is that I'm looking at years-old docs and examples (which may still work), and Grimoire solves that issue by making me the bottleneck in updating the docs I use rather than zk's free time. Besides, clojuredocs already has a _lot_ of google-foo and I don't expect *.arrdem.com/* to ever catch up, but that's fine. It surves my purposes and if it can serve yours as well so much the better.

Reid

--

[Reid McKenzie]

I'm delighted to announce that thanks to the official Clojuredocs client (https://github.com/dakrone/clojuredocs-client) Grimoire now features every example posted on Clojuredocs.

Ex. http://www.arrdem.com/grimoire/1.6.0/clojure.core/DASH__GT__GT/#example-0

Currently at 0.0.13, which represents a slew of changes since the original announcement including the dropping of 1.5.1 as a documented version (it's a patch release, nothing changed from 1.5.0) and a major rework of the way I munge symbols and include examples between versions.

I'd like to give Alex (puredanger) a huge shout out for handing me a SQL dump of the clojuredocs examples table right after I got a dakrone's client working.

More tooling thanks to Phil (technomancy), lein's profiles enabled me to ditch a make target which downloaded old Clojure versions in favor of simply having one lein profile for each documented Clojure version.

Reid

On 07/07/2014 04:54 PM, cldwalker wrote:

--

[Mark Mandel]

Nice work! I've been using Grimoire all day :)

Two small suggestions, which you have probably thought of: (1) search  (2) a copy of the clojure cheat sheet that points to grimoire instead. (Maybe Grimoire needs it's own domain?)

CrossClj is *awesome* as well. I've been wanting something similar for a looong time. Thanks!

Mark

--
E: mark....@gmail.com
T: http://www.twitter.com/neurotic
W: www.compoundtheory.com

2 Devs from Down Under Podcast

http://www.2ddu.com/

[Andy Fingerhut]

Mark, creating separate versions of the Clojure cheat sheet that link to Grimoire instead of ClojureDocs.org should be fairly straightforward, but due to other work I won't get to it for at least a few days.  If someone else is interested, and not put off by my code, they are welcome to go for it and submit a pull request:

    https://github.com/jafingerhut/clojure-cheatsheets

The time-consuming part isn't getting the first version up and running, but checking that all the symbols with odd characters in them actually have working links in both the HTML and PDF versions, although arguably the links in the PDF versions aren't as important.

I can publish the results here, which is one click away from http://clojure.org/cheatsheet

    http://jafingerhut.github.io

Once done, Clojure folks other than myself can consider whether the version with links to Grimoire belongs at http://clojure.org/cheatsheet, too.

Andy

[Colin Fleming]

If we think that Grimoire should be the official ClojureDocs replacement, why don't we do that? Could we just host Grimoire under the clojuredocs.org domain, perhaps structuring the URLs to match?

One thing that I couldn't see - does Grimoire offer an API to get access to the examples?

[Andy Fingerhut]

On Tue, Jul 8, 2014 at 3:28 AM, Colin Fleming <colin.ma...@gmail.com> wrote:

If we think that Grimoire should be the official ClojureDocs replacement, why don't we do that? Could we just host Grimoire under the clojuredocs.org domain, perhaps structuring the URLs to match?

Depends upon who you mean by "we".  The clojuredocs.org domain is owned by Zachary Kim, according to a search at whois.net.  He gets to decide what is published there.  Lately he has been actively working on a rewrite of the ClojureDocs.org server code in Clojure, intended to replace the original version written in Ruby:

    https://github.com/zk/clojuredocs

Andy

[Colin Fleming]

That was sort of a royal "we, the community". I think everyone probably agrees that it's not in anyone's interest to have two separate clojure doc sites serving essentially exactly the same purpose (docs + examples). It looks like Zachary is working fairly actively on his new ClojureDocs version, is he on the mailing list here? Does anyone know what the ETA for a new version is?

[Reid McKenzie]

1) doesn't make a whole lot of sense. I'm an adherent to the "simplest thing that could possibly work" philosophy, and as a result Grimoire is entirely static HTML. Search and symbol quick access could be implemented by adding some javascript, but that's a low priority item at the moment as there are still basic usability and layout concerns being addressed. Feel free to open an improvement issue on Github (http://github.com/arrdem/grimoire/). Down the road it may be worth questioning this static HTML approach as there are a number of tricks one could play such as ranking symbols by the frequency with which their documentation is visited which are impossible in pure static HTML.

2) this probably does make sense. I think it's worth doing some more general study of how people discover Clojure functions, but the categorized quick reference provided by the various cheat sheets is an index I'd love to have in Grimoire, perhaps even as part of the homepage?

3) http://grimoire.arrdem.com/ exists, however it's pretty broken and there's some configuration changes that need to happen (and which will break http://arrdem.com/grimoire) in order for it to work. I'm open for suggestions WRT having a dedicated domain, however remember I'm a GSoC student on a shoestring budget building this in my free time :P. I'm open to comment as to which URL you find more generally useful.

Reid

[Reid McKenzie]

Not as such. Strictly speaking one can navigate to say https://github.com/arrdem/grimoire/edit/master/_includes/1.4.0/clojure.core/DASH/examples.md and find all the examples for what is in this case clojure.core/-. However there is no examples API at present and due to the flat file nature of Grimoire I consider one more or less out of scope until such time as Grimoire gets reworked into a "real" webapp if ever.

As Grimoire contains only examples from clojuredocs at present, the clojuredocs REPL client (https://github.com/dakrone/clojuredocs-client) (which is what Grimoire uses to access clojuredocs' examples) should give you identical results for the time being.

I'm open to comment and issues as to whether or not having REPL access to examples is potentially useful and a desired feature.

Reid

[Alex P]

There's also a version of what you wrote (at least something very similar), where one can specify libraries he wants docs for and have it all running either locally or on some webserver: https://github.com/ifesdjeen/gizmo-cloc

Adds code snippets with highlights and lucene-backed search.

On Wednesday, July 2, 2014 1:34:38 AM UTC+2, Reid McKenzie wrote:

[cldwalker]

On Tuesday, July 8, 2014 3:07:03 PM UTC-4, Reid McKenzie wrote:

1) doesn't make a whole lot of sense. I'm an adherent to the "simplest thing that could possibly work" philosophy, and as a result Grimoire is entirely static HTML. Search and symbol quick access could be implemented by adding some javascript, but that's a low priority item at the moment as there are still basic usability and layout concerns being addressed. Feel free to open an improvement issue on Github (http://github.com/arrdem/grimoire/). Down the road it may be worth questioning this static HTML approach as there are a number of tricks one could play such as ranking symbols by the frequency with which their documentation is visited which are impossible in pure static HTML.

2) this probably does make sense. I think it's worth doing some more general study of how people discover Clojure functions, but the categorized quick reference provided by the various cheat sheets is an index I'd love to have in Grimoire, perhaps even as part of the homepage?

3) http://grimoire.arrdem.com/ exists, however it's pretty broken and there's some configuration changes that need to happen (and which will break http://arrdem.com/grimoire) in order for it to work. I'm open for suggestions WRT having a dedicated domain, however remember I'm a GSoC student on a shoestring budget building this in my free time :P. I'm open to comment as to which URL you find more generally useful.

I'd also love to see a custom domain for this as well. I'm happy to pay for it if no one else will. How about clj-docs.org or clj-doc.org (which are available)? I do see clojuredoc.org is available but could easily be confused with clojuredocs.org.

[George Oliver]

I'd also love to see a custom domain for this as well. I'm happy to pay for it if no one else will. How about clj-docs.org or clj-doc.org (which are available)? I do see clojuredoc.org is available but could easily be confused with clojuredocs.org

There's also clojure-doc.org.  

[cldwalker]

Once Grimoire starts getting custom examples, we could provide a corresponding client like clojuredocs-client that slurps _includes/**/examples and strips text only between raw and endraw tags. That would allow you to keep Grimoire as simple static HTML while still allowing for painless example contributions.

[Reid McKenzie]

Or something. I think what's gonna wind up happening with examples is that each "example" will become a Jekyll "post" and that as a result I'll be able to break examples out into single files, strip the user=> prefix, provide output validation and do some fun symbol analysis stuff to try and link to the documentation of other used functions.

Jekyll has proved rather finicky and ill behaved for large numbers of includes. The day may be approaching rapidly that I (or someone braver) yak shaves a replacement static site engine or reworks Grimoire to generate markup for one of the existing Clojure static site generators.

idk. I think that a REPL documentation/example client is still quite a ways in the future in terms of how much Grimoire needs to mature internally first. It may well be that I rework the directory structure tomorrow so that I can accommodate user submitted docs of other libraries. Or of the Clojure contrib libraries. Only time will tell.

Reid

[Reid McKenzie]

While I appreciate the interest, I think that attempting to "officiate"
Grimoire is a bad move.

We already have "clojure.org". "clojure.org" is the only official site.
Grimoire is not condoned by Rich. Using #"clo?j.*\.org would detract
from any future documentation effort Rich and co. may make and implies a
blessing which I do not have, do not seek and do not expect to get. See
http://arrdem.com/2014/07/12/of_mages_and_grimoires/ for more on the
subject.

Furthermore the #"clo?j.*" is also confusing due to existing namespace
contention from clojuredocs.org and clojure-doc.org let alone
clojure.org should it overnight grow better docs. Case in point: I lost
the link to http://getclojure.com/ for several months until it was
mentioned in IRC in the last week because it fell down the back of my
mental couch with the rest of the "*jure" namespace.

Reid

[Tim Gilbert]

Well, clojure-grimoire.com is available, FWIW.

Tim