Being productive with clojurescript

[Arnaud Bailly]

Hello,

I have been developing with Om and clojurescript for a few months now, and I do feel I am missing something to get as productive as I am when working with Haskell (or java for that matter). 

I would appreciate some advices or pointers resources from more experienced people on how to get to the point where I can develop confidently, easily and with decent speed in clojurescript. 

Here are my main concerns and some experiences I can share to give more context:

 - compiler is slow: I have setup cljsbuild profile to use advanced optimisation which heavily slows down compiler. Even simple optimisation is rather slow. No optimisation is rather fast, but not that fast (several seconds...). Note that I "often" have to remove generated compiled file and target/ dir to get usable result when doing advanced optimisation, otherwise I got seemingly unrelated runtime errors in the UI

 - compiler is not very helpful in catching errors: Error messages are ok, at least better than default clojure and its sometimes huge stack trace (or is it something from cljsbuild?), but of course not having types means fewer errors got caught... This is not an issue when one has the fast feedback of the REPL but large compilation time makes it really unwieldy.

  This is particularly annoying in Om because it requires state to be stored in an atom which in some contexts can be manipulated directly and in some others need to be dereferenced... And it is very easy, at least for me, to forget/add an @ 

 - unit tests are hard: code is mostly Om/react DOM building and ajax requests, so pure unit testing is not really useful or would need too much mocking. There seems to be a testing framework for React but I could not really figure how to use it given react is actually encapsulated under Om... I tried to write unit tests manipulating DOM elements but got stuck on how to simulate a change event. My JS-fu is probably lacking here but although I was able to synthesize a click, it was not possible for a change. 

 - I resorted to selenium tests, but of course those are fragile and slow and I would like to avoid having to fire a server for testing part of client application

 - I tried to LightTable, but not being able to use repl/instarepl means I am left with a text editor less powerful than emacs. I user LightTable with clojure code and found it really nice so I would like to be able to use for clojurescript too.

As I said, I have been coding for a living mostly in statically (or not so statically) typed  languages (Java, Haskell), but I have been using clojure for side projects in the last few years and I really enjoy coding in clojure. So when I discovered Om I was really enthusiastic about its capabilities and the way it expanded my horizon to make programming in the browser a more pleasing experience. 

But my coding flow is not satisfying, it takes a long time to iterate so I tend to do larger steps which creates bigger mistakes which take a long time to track... And I positively hate using a debugged, especially with optimised clojurescript compiled code :-)

So this post is really not a rant at all, I just got the feeling I am not doing things the right way and would like to be enlightened by people more knowledgeable than me.

Thanks in advance,

-- 

Arnaud Bailly

[David Nolen]

I'll let others speak to your other points. But if :none is taking more than hundreds of milliseconds something is wrong with your build settings.

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

[Arnaud Bailly]

Thanks for your answer.

Actually, I am not using :none. As advocated on cljsbuild page and Om tutorial, I have tried to use two configurations, one for dev with no optimisation, one for prod with advanced optimisations. Problem is that structure of JS within HTML pages is not the same so one has to change HTML between the two settings, which is not very practicle. It seems that boot-cljs handles that for you but did not tried it.

Moreover, I feel uncomfortable using different settings for prod and dev in something as important as the produced javascript code. I would like to be sure I find compiler issues (mine or others') in dev, not in production. Of course, I have ETE tests to catch that kind of stuff but again, I tend to think it is best to have dev and prod settings as close as possible. Or maybe I am wrong?

How do people code in practice with clojurescript? Bear with me, I am pretty sure the way I do things is wrong but I think it has more to do with the dev process itself than the actual compiler settings I use.

Regards,
Arnaud

[David Nolen]

On Fri, Dec 19, 2014 at 11:29 AM, Arnaud Bailly <arnaud...@gmail.com> wrote:
> Thanks for your answer.
>
> Actually, I am not using :none. As advocated on cljsbuild page and Om tutorial, I have tried to use two configurations, one for dev with no optimisation, one for prod with advanced optimisations. Problem is that structure of JS within HTML pages is not the same so one has to change HTML between the two settings, which is not very practicle. It seems that boot-cljs handles that for you but did not tried it.

There's an open ticket to address this which needs much review & consideration.

In the meantime this is entirely solvable in the same way it has been
solved in many web frameworks (Rails, Django, etc.) - use environment
variables to configure the generated markup. environ,
https://github.com/weavejester/environ is good for this.

> Moreover, I feel uncomfortable using different settings for prod and dev in something as important as the produced javascript code. I would like to be sure I find compiler issues (mine or others') in dev, not in production. Of course, I have ETE tests to catch that kind of stuff but again, I tend to think it is best to have dev and prod settings as close as possible. Or maybe I am wrong?

Your discomfort seems very misplaced to me. Deployed JS/CSS artifacts
in most production websites are rarely ever the same as what was
dev'ed with for good reason - you absolutely want concatenation &
minification in production.

David

[Arnaud Bailly]

On Friday, December 19, 2014 5:47:53 PM UTC+1, David Nolen wrote:

> On Fri, Dec 19, 2014 at 11:29 AM, Arnaud Bailly wrote:
> There's an open ticket to address this which needs much review & consideration.

OK, I think I will wait a little bit before trying to offer a PR :-)

>
> In the meantime this is entirely solvable in the same way it has been
> solved in many web frameworks (Rails, Django, etc.) - use environment
> variables to configure the generated markup. environ,
> https://github.com/weavejester/environ is good for this.
>

Sure, I can do this, thanks for the pointer.

>
> Your discomfort seems very misplaced to me. Deployed JS/CSS artifacts
> in most production websites are rarely ever the same as what was
> dev'ed with for good reason - you absolutely want concatenation &
> minification in production.
>

Well, I would like to have it in dev too... What if there is a bug or misused feature in the minifier/compiler? I want to catch it as quickly as possible. And any test I can write in the browser or using some other tool like selenium does not care about what the code looks like on the wire.
As a matter of fact, when I do Haskell or Java development I use the same compiler flags in dev and production, because these beasts are so complex I do not want to get caught into believing I made a mistake where actually this is something related to the compiler flags.

Thanks for your advices,

Arnaud

[David Nolen]

On Fri, Dec 19, 2014 at 12:10 PM, Arnaud Bailly <arnaud...@gmail.com> wrote:
> Well, I would like to have it in dev too... What if there is a bug or misused feature in the minifier/compiler? I want to catch it as quickly as possible. And any test I can write in the browser or using some other tool like selenium does not care about what the code looks like on the wire.
> As a matter of fact, when I do Haskell or Java development I use the same compiler flags in dev and production, because these beasts are so complex I do not want to get caught into believing I made a mistake where actually this is something related to the compiler flags.

This is the reason why in my experience at least three environment
settings are good idea: dev, staging, production. staging looks like
production but you test here before going to production to catch bugs
that might crop up under production flags.

David

[Arnaud Bailly]

Sure, that's what I do too. In the present stage of my application there is really no production (app is not live) so it boils down to 2, but you are definitely right.

Thinking about it, this probably means I could:
- use profiles/env for different compilation targets
- have some templating in HTML (currently static but could be easily generated) to swithc between no and advanced optim, or use cljs-boot, while waiting for cljsbuild to handle it
- run system-level tests against staging
- run unit tests against dev (and probably staging too if possible...)

This leaves me with the question: How can I unit test my stuff? Have you had experience with integrating React.js testing fwk?

Thanks again for your help,
Arnaud

[David Nolen]

On Fri, Dec 19, 2014 at 1:24 PM, Arnaud Bailly <arnaud...@gmail.com> wrote:
> This leaves me with the question: How can I unit test my stuff? Have you had experience with integrating React.js testing fwk?
>
> Thanks again for your help,
> Arnaud

I haven't looked at existing React.js testing solutions too closely
but I'd surprised if they can't be made to work w/ Om/React/etc.. If
they can't the discussion should be started how to improve this.

David

[Sven Richter]

Hi Arnaud,

I had a good time with the chestnut template: https://github.com/plexus/chestnut
It is no magic bullet, but generates a leiningen project with front and backend with some sane defaults for development and production.
I don't know about tests, but thats a different topic I guess.


Another player in the boat might be the build tool boot. I have not tried it yet, but it seems like it does provide good defaults too for dev / production environments.

Best Regards,
Sven

[Arnaud Bailly]

I had a look at http://facebook.github.io/react/docs/test-utils.html which seems interesting but could not figure out how to extract actual DOM component from Om... Was a couple of months ago, and I am not very good at searching the web so I might have overlooked something.

-- 

Arnaud Bailly

twitter: abailly

skype: arnaud-bailly

linkedin: http://fr.linkedin.com/in/arnaudbailly/

--
Note that posts from new members are moderated - please be patient with your first post.
---

You received this message because you are subscribed to a topic in the Google Groups "ClojureScript" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojurescript/7o7tX7ktOvM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojurescrip...@googlegroups.com.

[Arnaud Bailly]

On Sat, Dec 20, 2014 at 12:42 AM, Sven Richter <sve...@googlemail.com> wrote:

Hi Arnaud, 

I had a good time with the chestnut template: https://github.com/plexus/chestnut

thanks a lot for the pointer sven, will definitely have a look at it and get some inspiration from it. Cannot use it asis because I do not use clojure as backend.

 

It is no magic bullet, but generates a leiningen project with front and backend with some sane defaults for development and production.
I don't know about tests, but thats a different topic I guess.

You don't write tests for your clojurescript? You use the REPL to interact with the application until satisfied? Or work in the debugger? 

I have been test-infected for several years now so developing a piece of code without first writing a test is really something painful to me. But I can understand people are productive and efficient doing otherwise, and would like to know better. Might be related to my lack of (recent) experience and interest with front-end dev...

 

 

Another  player in the boat might be the build tool boot. I have not tried it yet, but it seems like it does provide good defaults too for dev / production environments.

Yes. A friend of mine also pointed me at this on the french ML.

Regard,

Arnaud 

[Sven Richter]

Am Samstag, 20. Dezember 2014 12:03:51 UTC+1 schrieb Arnaud Bailly:
> On Sat, Dec 20, 2014 at 12:42 AM, Sven Richter <sve...@googlemail.com> wrote:Hi Arnaud, 
>
> I had a good time with the chestnut template: https://github.com/plexus/chestnut
>
>
> thanks a lot for the pointer sven, will definitely have a look at it and get some inspiration from it. Cannot use it asis because I do not use clojure as backend.

There is no need, just take the cljs related things from it. It basically does what was suggested here already.
Using optimizations :none for development and injecting some extra html code for this mode.

> I don't know about tests, but thats a different topic I guess.
>
>
>
>
>
> You don't write tests for your clojurescript? You use the REPL to interact with the application until satisfied? Or work in the debugger? 

Disclaimer: I use clj / cljs only for my private projects and learning.
And yes, I did not write cljs tests yet. I don't say one does not need cljs tests, however, setting this up and using phantomjs seemed to much of a burden for me for fun projects.
Of course, if something goes into production I strongly advise to have tests for it, which you obviously seem to follow anyway.

However I recently started using cljx in one of my projects and was pleased to find that I can unit test them like clojure code, which I did in some cases.



Best Regards,
Sven

[pand...@gmail.com]

Hi Arnaud,

I feel your pain. I've been developing with cljs nearly from the beginning (though as a newbie to Clojure) and the road has been bumpy but plenty rewarding. My advice is to stay at it; your flow will improve and the language and tooling have been improving a lot, too.

I'll speak to a couple points about testing. I'm far from a veteran but with a couple apps undergoing heavy refactoring I've felt the pain of components breaking b/c of subtle changes to data structures, so I've been exploring this a bit.

In terms of testing, I am using React.addons.TestUtils.Simulate (I use react-with-addons.js in development, which includes it), and it is vital for being able to test components, because JavaScript event handlers like 'onChange' aren't triggered programmatically. Also I should note I'm using reagent, allowing me to be explicit about passing references to state in as a parameter(s), which makes mocking simple. Finally I've made a component mounting macro that just creates and tears down a DOM node target just for the test. The result of this is that I can pass a certain state in to a component, simulate some clicks, input, or external state change, and then inspect the resulting state in the test. I am using simple (assert ...)'s and run my tests in a separate tests.html (which refreshes on every compilation via ring-refresh). Failing asserts point right to the failure in devtools with sourcemaps enabled and change the favicon for a quick visual indicator w/o having to switch tabs.

So like I said, keep at it and I'm sure you'll find your flow. A lot is already possible.
M

[Nikita Prokopov]

Arnaud,

what’s wrong with LightTable? I use it for CLJS development, it works fine, live evaluation is here, watches are working, results are displayed inline. It needs open browser and a connection string added to the html page, but otherwise it works, at least for me.

[Arnaud Bailly]

Thanks a lot for your comforting words! 

Here is the test code I started to write couple of months ago which did not work: I could not manage to trigger onChange event. Is it really not triggerable programatically? 

(ns capital-match.users-test

  (:require-macros [cemerick.cljs.test

                    :refer (is deftest with-test run-tests testing test-var done)]

                   [cljs.core.async.macros :refer [go]]

                   [dommy.macros :refer [node sel sel1]])

  (:require [cemerick.cljs.test :as t]

            [capital-match.user.registration :as u]

            [om.core :as om :include-macros true]

            [om.dom :as dom :include-macros true]

            [cljs.core.async :refer [chan <! >! pub sub]]

            [dommy.utils :as utils]

            [dommy.core :as dommy]))

(defn simulate-click-event

  "From https://github.com/levand/domina/blob/master/test/cljs/domina/test.cljs"

  [el]

  (let [document (.-document js/window)]

    (cond

     (.-click el) (.click el)

     (.-createEvent document) (let [e (.createEvent document "MouseEvents")]

                                (.initMouseEvent e "click" true true

                                                 js/window 0 0 0 0 0

                                                 false false false false 0 nil)

                                (.dispatchEvent el e))

     :default (throw "Unable to simulate click event"))))

(defn simulate-change-event

  "From https://github.com/levand/domina/blob/master/test/cljs/domina/test.cljs"

  [el]

  (let [document (.-document js/window)]

    (cond

     (.-onChange el) (do (print "firing on change on "  el) (.onChange el))

     (.-createEvent document) (let [e (.createEvent document "HTMLEvents")]

                                (print "firing  " e " on change on "  (.-id el))

                                (.initEvent e "change" true true)

                                (.dispatchEvent el e))

     :default (throw "Unable to simulate change event"))))

(def sink

  "contains a channel that receives messages along with notification type"

  (chan))

;; see http://yobriefca.se/blog/2014/06/04/publish-and-subscribe-with-core-dot-asyncs-pub-and-sub/

(def source

  (pub sink #(:topic %)))

(defn change-field!

  [id value]

  (let [el (sel1 (keyword (str "#" id)))]

     (dommy/set-value! el  value)

     (simulate-change-event el)

     ))

(deftest ^:async password-confirmation

  (testing "do not submit if passwords are not equal"

    (let [subscription (chan)]

      (sub source :user-registration subscription)

      (om/root

       (partial u/registration-view source sink)

       nil

       {:target (sel1 :#view)})

      (go

       (let [m (<! subscription)]

         (is (= :error (:state m)))

         (done)

         ))

      (change-field! "userRequestedEmail"    "f...@bar.com")

      (change-field! "userRequestedPassword" "secret")

      (change-field! "confirmPassword"       "nosecret")

      (simulate-click-event (sel1 :#submitRegistration))

      )))

The component is mounted on a dedicated DOM node which is part of the test HTML. 

I would like to be able to test this without a browser, e.g. using something like phantomjs, in order to run all tests as part of build of the system the UI is part of. We currently have setup ETE testing using selenium w/in docker components so it would not be a big deal to use that to run "unit" tests too but these kind of tests are slow and selenium is somewhat brittle (although it too has improved and seems a lot more stable now than it was couple of years ago...). 

Do you have an example test available that I could get inspiration from? I am not using reagent but I guess it should be possible to use React testutils with Om, I simply don't seem to understand how to do this...

Thanks a lot 

-- 

Arnaud Bailly

twitter: abailly

skype: arnaud-bailly

linkedin: http://fr.linkedin.com/in/arnaudbailly/

[Arnaud Bailly]

Hi Nikita,

Surely there is nothing wrong with LightTable: I have been using it a lot at the beginning. However, I found that: 

 1. adding connection string to HTML introduces another point where dev is different from staging/prod, which is annoying and requires cooperation from the build system

 2. live reloading/repl does not seem to work very well with advanced optimisations on... 

 3. the backend is not clojure but haskell and I code in emacs. Given emacs handles edition pretty well and I was not able to use the REPL features of LightTable, I found it painful to have to switch between LT and emacs, so I stick to the latter

I suspect there are ways to use live reloading and browser REPL in emacs too but did not spent too much time on it, I confess.

Anyway, if I follow David's advices and get back to a setting with different dev/prod HTML, I could get back to use LT fruitfully.

Best regards,

-- 

Arnaud Bailly

twitter: abailly

skype: arnaud-bailly

linkedin: http://fr.linkedin.com/in/arnaudbailly/

On Fri, Dec 26, 2014 at 6:09 PM, Nikita Prokopov <prok...@gmail.com> wrote:

Arnaud,

what’s wrong with LightTable? I use it for CLJS development, it works fine, live evaluation is here, watches are working, results are displayed inline. It needs open browser and a connection string added to the html page, but otherwise it works, at least for me.

[Colin Yates]

Hi Arnaud,

Are you using lein cljsbuild auto? If so then :none should be sub-second? Things that significantly improved my cljs workflow (with om):

- figwheel (chestnut template is a great starting point) pushes javascript changes to the browser (and css as well)
- defonce the atom so state persists figwheel pushes
- pristmatic's schema (or core.typed)
- pristmatic's schema (or core.typed)
- pristmatic's schema (or core.typed)
- pristmatic's schema (or core.typed)
- (yeah, realy, schema is that useful)
- pristmatic's schema (or core.typed)
- (I mean it :))
- https://github.com/cemerick/clojurescript.test (haven't tried David's shiny new clojure.test port).
- ruthlessly separating out logic from components. If I need to unit test a component I first see if I can split the interesting logic into a separate method which I can test without a browser. If I can't then I go get some coffee and try again.
- in a previous life (CoffeeScript and ExtJS) I had great times with Jasmine.
- Cursive Clojure (https://cursiveclojure.com/). To be honest you might want to stick with LightTable if it isn't clj backend.
- https://github.com/noprompt/lein-garden (but I suspect you might be using a non-clojure CSS generator?)
- https://github.com/gdeer81/lein-marginalia for documentation (https://github.com/zcaudate/lein-midje-doc also looks awesome but I haven't used it).

And did I mention schema? :).

For general programming I highly recommend https://github.com/jonase/kibit and https://github.com/jonase/eastwood.

Hope this helps.

[Arnaud Bailly]

Thanks a lot colin for your insights. I will definitely take the time to investigate all that, and also stick to a more standard dual compilation approach as suggested by David. 

Some comments:

 - ok, schema looks cool! But then it makes me think why would I want to use a dynamically typed language like clojure if I can have a real type system with ghcjs... 

 - yes, I also do enforce separation of biz logic from presentation stuff, unit testing the logic part. However the real point of UI component is that the DOM they produce depends on some state/interaction and this is that part I need to test, not biz logic

 - I don't use clojure for backend

Cheers (and Happy new year)

-- 

Arnaud Bailly

twitter: abailly

skype: arnaud-bailly

linkedin: http://fr.linkedin.com/in/arnaudbailly/

[Colin Yates]

It must be obvious by now the value I put on schema, but for om, where
you necessarily end up with large hierarchies it has saved my bacon
multiple times.

I think of it less to do with type enforcement (although obviously the
s/validate or (s/defn ^:always-validate) can do that) and more to do
with a very lightweight and composable form of documenting my
expectations.

For example, I have things like (from memory not code):

(def TreeFilter {:expanded-ids [s/Num]})
(def SelectableTreeFilter (assoc TreeFilter :selected-ids [s/Num]))
;; unfortunately Clojurescript schema can't self refer - boo :).
(def TreeNode {:id s/Str :text s/Str (s/maybe :children) [s/Any]))

(def world (atom {}))

(def TreeComponent {:filter (merge TreeNode SelectableTreeFilter)
:node TreeFilter})
(s/defn ^:always-validate build-tree [data :- TreeComponent ...] (om/.....))

The sooner you use it the less painful it will be and almost every
single time I have gotten a stack trace it is because the data wasn't
as I expected. Coupled with Clojure's forgiveness of nils and empty
sequences it has been indespensible (e.g. a very high signal->noise
ratio).

[Ivan L]

+1 for Schema... you tend to not need a full static type system when using clojure. Schema is useful as its designers intended - living documentation and much better error messages.

[Arnaud Bailly]

I could imagine generating schemas from server-side (Haskell) code... Givenpat data structures used in the client actually come from the server. 

Wouldn't EDN help? Don't know how this works actually but given I have full control of server I could generate EDN instead of Json, or any format for that matter.

Cheers

Arnaud 

Le lundi 5 janvier 2015, Ivan L <ivan.l...@gmail.com> a écrit :

+1 for Schema... you tend to not need a full static type system when using clojure.  Schema is useful as its designers intended - living documentation and much better error messages.

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to a topic in the Google Groups "ClojureScript" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojurescript/7o7tX7ktOvM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

--

[Colin Yates]

On Monday, 5 January 2015 18:26:19 UTC, Arnaud Bailly wrote:
> I could imagine generating schemas from server-side (Haskell) code... Givenpat data structures used in the client actually come from the server. 

Schema's declaration is done through pure data (yet another win for data everywhere) so sure.

>
> Wouldn't EDN help? Don't know how this works actually but given I have full control of server I could generate EDN instead of Json, or any format for that matter.

EDN is a nicer format for Clojure to consume but it does have some costs. Have you looked at transmit?

[Arnaud Bailly]

-- 

Arnaud Bailly

twitter: abailly

skype: arnaud-bailly

linkedin: http://fr.linkedin.com/in/arnaudbailly/

On Tue, Jan 6, 2015 at 2:29 PM, Colin Yates <colin...@gmail.com> wrote:

> Wouldn't EDN help? Don't know how this works actually but given I have full control of server I could generate EDN instead of Json, or any format for that matter.
EDN is a nicer format for Clojure to consume but it does have some costs. Have you looked at transmit?

Not yet :-) Could not remember the name but have seen the news months ago...