[ANN] Hoplon: web applications in Clojure and ClojureScript

[Micha Niskin]

Documentation is here: http://hoplon.io 

We continue to add documentation all the time. Serverside stuff not yet documented yet. Feedback welcomed!

-- Micha Niskin and Alan Dipert

[Clinton Dreisbach]

So excited to see this officially released! Hoplon is some cool shit.

-- Clinton

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Ray Willig]

I'm an early adopter who's been using Hoplon for a while. Here's a demo of software for solving the traveling salesman problem that uses the full hoplon suite. 

[James Reeves]

It looks interesting, but it really needs an overview of the syntax. The purpose of the examples is clear, but the mechanism is a mystery.

- James

--

[Micha Niskin]

There is a section in the "Getting Started" page, titled "Sexp Markup Syntax" that attempts to explain the syntax. What needs to be improved there?

[Ray Willig]

Hi, I've notice a bunch of activity on my demo and I appreciate the interest. I apologize for the lack of documentation on specifically how to use it; it's in the works. I've noticed that several users are attempting to use the multiple vehicle feature which solves the capacitated vehicle routing problem. The assumption is that each stop represents a load of 1. So to see how multiple trucks would pan out, you'll want to a) select all stops so you have a good working number b) slide the capacity down to some number that will guarantee that all but the last truck will be filled to capacity. 

A good example might be to select 4 trucks, select all 50 stops and slide capacity to 13. This will fill the first three trucks, leaving 11 items in the last one (3 *13 + 11 = 50)

[James Reeves]

I wasn't expecting an overview of the syntax to be under "Getting Started". The documentation there makes things clearer, but I still have a few questions:

1. What differentiates the top-level page definition from DOM objects returned from functions? Is the compiler just looking for the last expression in the source file?

2. Do the HTML forms (div, span, etc.) act like functions, or more like macros?

3. Are the HTML forms generated on the fly, or is there a fixed list of tags available?

4. Is defc equivalent to def and cell? And defc= equivalent to def and cell=?

5. What's the difference between cell and cell=? Is one like an atom, and the other like a reactive function?

6. What's the difference between a cell an an atom?

- James

[Micha Niskin]

Ah, yes. There are some gaps in the docs that need to be fixed. To answer your questions:

1. The main thing that Hoplon does is unify the DOM and the dynamic environment (the JS environment). This makes it possible to refer to things from the dynamic environment in your markup. Normally you can reference things in the DOM from JavaScript, but not the other way around. So in a unified environment that includes both JavaScript and the DOM there needs to be a way to develop both of them in the same page. A Hoplon page has 3 parts: the page declaration, a number of optional top-level forms, and finally the DOM markup. So yes, the Hoplon compiler selects the last expression in the file as the DOM markup.
2. The primitive HTML5 elements (div, span, etc.) are, in fact ClojureScript functions in the tailrecursion.hoplon namespace. These functions return new DOM elements of the appropriate type. However, the Element type is extended to implement the IFn protocol such that they maintain the HTML semantic. That is, function application means append child / set attribute. So yes, div and span etc. are functions, but they return DOM elements that implement IFn, so they return functions, in a way, too.
3. Everything is generated on the fly, pretty much. The entire document is replaced with the result of evaluating the DOM markup when the page loads. In other words, when the page loads the last expression in the source file is evaluated in the browser, and document is replaced with the result. Since the DOM markup is just a ClojureScript expression that is evaluated as ClojureScript, you can create your own custom tags. All you have to do is define functions that abide by the HTML semantic. There are examples of this in the "Getting Started" page, especially the "Creating a tabs abstraction" section.
4. Sorry, I should have put a link in the docs there to Javelin, the dataflow library we use for the cells. That page explains in detail, but the basic premise is that there are two kinds of cells in spreadsheets, input cells that you update manually, and formula cells that update themselves reactively according to a formula that might refer to other cells. So defc is equivalent to def and cell, and that defines an input cell. Likewise, defc= is equivalent to def and cell=, and that defines a formula cell.
5. All cells, both input cells and formula cells, contain values. The difference is in how those values are updated. In order to obtain the value contained in any cell you may dereference the cell. So both types of cell can be dereferenced like atoms. However, only input cells can be updated using swap! or reset!. Formula cells will update themselves automatically, and it's an error to try to do it directly using those functions.
6. The difference between a cell and an atom with watchers attached is that cells guarantee consistency. That is to say that the evaluation mechanism ensures that a formula cell is never updated until all of the cells it depends on have been updated, that the formula is evaluated at most one time, and that the cell's formula is evaluated only when the value of a cell it depends on has changed. A cool property here is that the entire graph of cells updates atomically and consistently, even though the individual cells are updating themselves one at a time. The consistency guarantee ensures that each cell sees the world as if it updates atomically; no cell can ever see other cells in a half-evaluated state; each cell acts as if it were the last cell to update. You can think of the entire graph as a single value.

[Cedric Greevey]

On Wed, Dec 18, 2013 at 10:45 PM, Micha Niskin <micha....@gmail.com> wrote:

The difference between a cell and an atom with watchers attached is that cells guarantee consistency. That is to say that the evaluation mechanism ensures that a formula cell is never updated until all of the cells it depends on have been updated, that the formula is evaluated at most one time, and that the cell's formula is evaluated only when the value of a cell it depends on has changed. A cool property here is that the entire graph of cells updates atomically and consistently, even though the individual cells are updating themselves one at a time. The consistency guarantee ensures that each cell sees the world as if it updates atomically; no cell can ever see other cells in a half-evaluated state; each cell acts as if it were the last cell to update. You can think of the entire graph as a single value.

What happens if the formula cells have circular dependencies?

[Micha Niskin]

If formula cells have circular dependencies then you have an infinite loop at runtime. It's not really a problem when making real applications, though, at least for me. The spreadsheet model simplifies things enough that there really aren't any surprises; it's very clear what's happening in the state machine, so it's not like there is a problem with inadvertently introducing circular deps.

--

Micha Niskin

--

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.

To post to this group, send email to clo...@googlegroups.com

Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/gRFyzvRfPa8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

[James Reeves]

Thank you for taking the time to write such a detailed explanation. That clears up my understanding of the library considerably.

- James

[Murtaza Husain]

Hi,

Thanks for the great framework. I am loving it and will give it a try in a pet project. A few nitpicks - 

1. Why not use hiccup data structure for representing DOM, instead of the custom fns and macros. This provides several advantages where your DOM is just data and you can manipulate it like data. 

2. The destructing and looping used within the DOM forms is custom. Why cant we have regular clojure all the way ? Introduction of a another syntax for templating is not very appealing. So a regular 'map' or 'for' could be used instead of 'loop-tpl'

3. String interpolation for the bindings - '~{x}' . Cant we use something more idiomatic, either the var itself or atleast a symbol :x ?

4. It will also be helpful to have a leiningen example for the project. Nothing against boot, however it will be easy to get started, and also IDE's are already configured to work with project.clj

Thanks,

Murtaza

[Alan Dipert]

Hi Murtaza, thanks for your interest!  Responses below.

Alan 

1. Why not use hiccup data structure for representing DOM, instead of the custom fns and macros. This provides several advantages where your DOM is just data and you can manipulate it like data.

 

The hlisp part of Hoplon - the compiler bit that converts HTML to ClojureScript - was designed and implemented specifically to avoid having to do this.  It's our opinion (and experience) that admission of the DOM as a data structure instead of a piece of the program introduces tremendous complexity.  This data structure must be explicitly interpreted, evaluated, and updated in complicated ways that tend to couple tightly to both overall state logic and DOM quirks.  That said, Hoplon markup can be manipulated as data the way any other Lisp program sanely can - via hygienic macros.  Also, the semantics we apply to HTML to compile it to ClojureScript extend to other representations.  For instance, I'm currently finishing up boot tasks for compiling HAML and Slim markup to ClojureScript programs.  A hiccup task would be just as straightforward, but IMO pointless.

2. The destructing and looping used within the DOM forms is custom. Why cant we have regular clojure all the way ? Introduction of a another syntax for templating is not very appealing. So a regular 'map' or 'for' could be used instead of 'loop-tpl'

I'm sympathetic to your concern here, but loop-tpl is an entirely different animal than map or for and unfortunately they don't line up.  The main difference between loop-tpl and map/for is that loop-tpl is a macro that statically allocates DOM elements to deposit the elements it loops over into - it is not a runtime construct.  For more on loop-tpl please see the end of "Task 6" on http://hoplon.io/#/getting-started/.  Plus, hey, you're already ClojureScripting - how bad can just a little more weirdness be? ;-)

3. String interpolation for the bindings - '~{x}' . Cant we use something more idiomatic, either the var itself or atleast a symbol :x ?

If interpolation just used var symbols, we'd need to analyze the global environment and look for a possible var match for every word we found in the string - a complicated process.  If we used :keywords, we'd have to escape them in order to write the string \:keyword - highly unintuitive IMO.  ~{} is odd enough to probably not occur in a string, and follows how interpolation works in e.g. Ruby (which uses #{}).  Also, ClojureScript's js* compiler macro uses ~{} to achieve kinda the same thing.

 

4. It will also be helpful to have a leiningen example for the project. Nothing against boot, however it will be easy to get started, and also IDE's are already configured to work with project.clj

A Leiningen plugin is unlikely, but we're open to improving the documentation, friendliness of, and tool support around boot.  For more on Why Boot, please see http://www.reddit.com/r/Clojure/comments/1t6wgs/hoplon_a_simpler_way_to_program_the_web/ce5zntf

[Ryan Spangler]

The hlisp part of Hoplon - the compiler bit that converts HTML to ClojureScript - was designed and implemented specifically to avoid having to do this.  It's our opinion (and experience) that admission of the DOM as a data structure instead of a piece of the program introduces tremendous complexity.  This data structure must be explicitly interpreted, evaluated, and updated in complicated ways that tend to couple tightly to both overall state logic and DOM quirks.  That said, Hoplon markup can be manipulated as data the way any other Lisp program sanely can - via hygienic macros. 

I am curious about this.  I have had the opposite experience.  Whenever I use a library based on macros I find myself wanting to do something with them the macro writer did not expect, and being forced to wrap the macros in functions so I can compose things programmatically (which seems to me like unnecessary complexity).  I can't pass macros around as arguments, store them in a list, or apply them to arguments in a general way.  

On the other hand, if I am just working with a data structure, I can write functions to manipulate the data in novel ways and even use many functions that already exist to manipulate data without having to recreate that functionality for a specific domain (like looping, merging, search and replace etc).  

So I am curious specifically what flexibility you are gaining by using macros (things you can't do with functions and data) that would offset the flexibility you sacrifice with macros?  I am open to being convinced on this point!

[Alan Dipert]

Hi Ryan, thanks for your interest - and for remaining open, because things will get weird here for a moment :-)

I didn't mean to say that macros are a substitute for data manipulation; they are just a related thing, of arguable utility depending on context, that come along with the idea of Lisp.

In general, I too prefer function composition.  The problem hlisp addresses is that out of the box, the DOM doesn't come with anything like function composition at all.  Many people resort to something worse than even 'data' as we know it to deal with this issue - they write big DOM programs with C-preprocessors they call template engines!  Something better than this is to represent DOM in the hiccup style and perform higher level data manipulation using a programming language.  I see the relationship between string-munging template languages and Clojure-style data manipulation of hiccup as roughly the relationship between C-preprocessor macros and Lisp macros.  Data driven is clearly better, and most of us in Clojure-land are already here when it comes to representing and emitting DOM or DOM fragments.

Conspicuously missing from this continuum is the thing we really want: functional composition.  This is the flavor of composition we use most when we're programming.  How do we get it?  Well, in order for data to compose with other like data in the way we're used to functions doing, we must apply 'denotational semantics'.  Lisp's semantics are things like: a list means a function call.  The first thing in the list is the function.  The rest of the things are arguments.  If we provide a set of "primitive" operations - "the 7 things" [1] - and then apply the combination of our primitives and our semantics to Lisp data - we have a programming language, Lisp.  Once we have Lisp, we have a way to perform functional composition and do all the other great stuff one can do in Lisp.  As a programming language now, our data has sprouted the ability to compose with other like data in possibly novel ways without us necessarily adding more base semantics to our interpreter.  In a sense, our data has become infinitely meaningful, because it is now Turing complete.

This is what hlisp is: it's a means to a real Lisp, ClojureScript, atop not traditional lists, but DOM elements.  Where Lisp is that thing that happens when you apply the 7 things and semantics to lists, hlisp is the thing that happens when you apply the set of primitives and semantics Micha Niskin designed [2] to the DOM.

The result, in my opinion, is mindblowingly beautiful and extremely powerful.  By making the DOM a Lisp - and not just any Lisp, ClojureScript - we have a means to *real* DOM component modularity.  The exact same modularity that you get with things like namespaces and closures.  Where various other HTML component systems drag in their own reference types and scoping rules and evaluation semantics,  we have just ClojureScript's usual rules.  That's because, via the IFn protocol, it's possible for us to make DOM elements invokable.  As implementers of IFn, hlisp-generated DOM nodes can participate fully in the ClojureScript world without introducing new scoping constructs or DOM-specific composition rules.  As a result of all this, we have no need to represent HTML anywhere as data.  Everything is a program, and expressions in this program may evaluate to DOM.  There are no templates, just expressions.  I encourage you to check out our "Getting Started" page to see just how powerful functional composition can be when applied over Lispified DOM nodes.

It's worth noting that hlisp is not an interpreter - it actually compiles HTML to ClojureScript.  As a result, our entire program including 'markup' can now benefit from Google Closure stuff like optimization and dead-code analysis.

My reference to macros in my previous post was just this point: you can write macros in Clojure to write hlisp the same way you can write macros in Clojure to write ClojureScript.  Again, this is something that just falls out of Lisp and doesn't necessarily have much to do with hiccup or the idea of the DOM as data.

I hope this cleared things up for you.  Don't hesitate to join us on our mailing list or in IRC #hoplon if you have further questions!

Happy Lisping,

Alan

1. http://www.paulgraham.com/rootsoflisp.html

2. https://github.com/tailrecursion/hlisp-starter/blob/master/PROJECT.md#html-is-semantically-compatible-with-lisp

[Catonano]

2013-12-20 21:42 GMT+01:00 Alan Dipert <al...@dipert.org>:

The result, in my opinion, is mindblowingly beautiful and extremely powerful.  By making the DOM a Lisp - and not just any Lisp, ClojureScript - we have a means to *real* DOM component modularity.  The exact same modularity that you get with things like namespaces and closures.  Where various other HTML component systems drag in their own reference types and scoping rules and evaluation semantics,  we have just ClojureScript's usual rules.  That's because, via the IFn protocol, it's possible for us to make DOM elements invokable.  As implementers of IFn, hlisp-generated DOM nodes can participate fully in the ClojureScript world without introducing new scoping constructs or DOM-specific composition rules.  As a result of all this, we have no need to represent HTML anywhere as data.  Everything is a program, and expressions in this program may evaluate to DOM.  There are no templates, just expressions.  I encourage you to check out our "Getting Started" page to see just how powerful functional composition can be when applied over Lispified DOM nodes.

Maybe I'm off track here, but this resounds to me as similar to the arguments that I listened to in lecture 2a of the SICP course.

In that lecture, they introduce the representation and manipulation of pictures.

They introduce a single primitive and then build on top of that.

They create a dsl specific to pictures processing that is  "embedded" into lisp.

So it inherits from lisp the common means of expression (abstraction, combination, recursion and whatnot)

I see a sort of kinship, here, between DOM elements and SICP pictures.

In fact, they stress that "data" representing pictures are the same kind of lisp citizens we already knew. Data are blurred with the language, in lisp.

In the DOM case macros are necessary because there's the html to deal with.

So I guess that the solution is to use the macros to end up with lisp stuff and then work with that, rather than the other way around.

I hope I didn't make myself a fool, here