[ANN] Dieter Asset Pipeline

[John Andrews]

I'd like to announce a little project I've been working on for pre-
compiling static assets. If you're familiar with Ruby's Sprockets
library (or Rails' asset pipeline) the concept will be familiar to
you.

Dieter is ring middleware which concatenates javascript and css files
into one file with optional minification. It also has a couple of
precompilers built in for turning less.js into css, and handlebars.js
files into javascript. More can be added but I haven't had the need
for others yet...

Maybe someone else will find this useful. If so, the package is on
http://clojars.org/dieter and the code is at https://github.com/edgecase/dieter.

-John

[Chris Granger]

Looks like really cool stuff and a very welcomed addition to the ecosystem :) I'll try to take a deeper look in the next couple of days.

Cheers,
Chris.

[James Reeves]

On 3 February 2012 21:53, John Andrews <john.m....@gmail.com> wrote:
> Maybe someone else will find this useful. If so, the package is on
> http://clojars.org/dieter and the code is at https://github.com/edgecase/dieter.

Feel free to add a link to this on the Ring wiki:

https://github.com/mmcgrana/ring/wiki/Third-Party-Libraries

- James

[John Szakmeister]

On Friday, February 3, 2012 4:53:11 PM UTC-5, John Andrews wrote:

[snip] 

Dieter is ring middleware which concatenates javascript and css files
into one file with optional minification. It also has a couple of
precompilers built in for turning less.js into css, and handlebars.js
files into javascript. More can be added but I haven't had the need
for others yet...

Maybe someone else will find this useful. If so, the package is on
http://clojars.org/dieter and the code is at https://github.com/edgecase/dieter.

Thanks for providing this John!  I took it for a whirl on a little project I'm working on, mainly for turning the less into css.  For the most part, it works as advertised (yay!).  The only issue I ran into was dealing with the assets in an uberjar.  It doesn't try to access the assets in the jar, and then the asset-cache has to be on disk somewhere too.  I'm curious if you have some model of how to package things up?

Thanks again!

-John

 

[John Andrews]

On Sat, Mar 31, 2012 at 5:32 PM, John Szakmeister
<jszakm...@gmail.com> wrote:
> It doesn't try to access the assets in the jar, and then the asset-cache has to be on disk somewhere too.

Hi John. Thanks for giving Dieter a try. I don't come from a Java
background so I'm definitely open to suggestions on how to handle this
case. My initial thought is that we can add support for loading assets
from the system resource path. Then we create a lein task to
pre-compile your assets when building the uberjar. Thoughts?

-John

[James Reeves]

On 2 April 2012 15:04, John Andrews <john.m....@gmail.com> wrote:
> Hi John. Thanks for giving Dieter a try. I don't come from a Java
> background so I'm definitely open to suggestions on how to handle this
> case. My initial thought is that we can add support for loading assets
> from the system resource path. Then we create a lein task to
> pre-compile your assets when building the uberjar. Thoughts?

You could make the assets just normal Java resources, and the
preprocessors as standard Ring middleware. That would be a more
idiomatic approach, I think.

- James

[John Andrews]

Loading assets from Java resources makes sense. In order to avoid
recompilation, Dieter writes the compiled assets to the
filesystem so they can be served directly with wrap-file. Then the
Dieter middleware is only called when there is a cache miss.

I'm guessing that in the deployment strategy of deploying an uberjar
(presumably to tomcat or similar application server) would prevent us
from writing files to the filesystem. Is this a correct assumption?
If we can't write files we're left with 2 options

1. Cache the final asset in memory
2. Precompile the assets into Java resources as part of uberjar
creation. Serve those resources directly via file middleware.

James, If I understand your suggestion correctly, you would advocate for the
memory cache. It would probably be the easier option for me to
implement as well, but I want to understand the resource picture a
little better. Is it discouraged to build resources at deploy-time?

-John

[James Reeves]

On 2 April 2012 21:09, John Andrews <john.m....@gmail.com> wrote:
> James, If I understand your suggestion correctly, you would advocate for the
> memory cache. It would probably be the easier option for me to
> implement as well, but I want to understand the resource picture a
> little better. Is it discouraged to build resources at deploy-time?

That's not quite what I'm saying.

Caching and compiling are two separate problems. A cache does not need
to know what it is caching, and a compiler does not need to know it is
being cached.

Idiomatic Clojure is about reducing a complex problem to simple
components. In this case, I think you need to separate out compiling
and caching.

For example:

(-> handler
wrap-coffeescript
(wrap-cache (file-cache "/var/cache/myapp))

The cache could be in memory, or a location on disk, or some external
system like S3. Caching to a file on disk isn't forbidden, so long as
the web app has read/write access to the directory you supply.

- James

[James Reeves]

Re-reading that, I think in my attempts to be concise, I made it sound
a little more curt and unfriendly than I intended it to. Please excuse
my tone.

- James

[John Szakmeister]

I'd love to tell you my background is different, but... :-)

Being able to load assets from Java resources would be nice, and it
probably the bit I'm most concerned about. I can give it a place on
disk to store data for the actual cache it a production system. I
also like James's suggestion of making the preprocessors standard
middleware too (I only care about the less preprocessor for now).

-John

[John Andrews]

Thanks for all of the input. I've created an issue on github where you
can track progress if you're interested.
https://github.com/edgecase/dieter/issues/8

-John

[Hoàng Minh Thắng]

Hi all,
I couldn't figure it out how to do unit testing in noir.validation.
This is how I did:

(ns lorem.ipsum.test
(:use clojure.test)
(:use midje.sweet)
(:require [noir.validation :as vali]))

;then I added this
(declare ^:dynamic *errors*)

(def good-item {:title "pass!" :body "this should pass"})
(def bad-item {:title "this should fail" :body ""})

;stolen from noir-blog
(defn lorem-valid? [{:keys [title body]}]
(vali/rule (vali/has-value? title)
[:title "There must be a title"])
(vali/rule (vali/has-value? body)
[:body "There's no post content."])
(not (vali/errors? :title :body)))

(facts
;neither this
(lorem-valid? good-item)) => nil
;nor this worked
(binding [^:dynamic *errors* (atom {})]
(lorem-valid? good-item)) => nil
)

They printed out these:

java.lang.ClassCastException: clojure.lang.Var$Unbound cannot be cast
to clojure.lang.IDeref
noir.validation$get_errors.invoke(validation.clj:48)

and of course a lot of error lines following.

[Hoàng Minh Thắng]

Solved!

I changed the tests to this

(facts

  (binding [^:dynamic vali/*errors* (atom {})]

    (lorem-valid? good-item)) => true

  (binding [^:dynamic vali/*errors* (atom {})]

    (lorem-valid? bad-item)) => false

)

and it worked just fine

[Chris Granger]

Take a look at noir.util.test/with-noir :)

Cheers,
Chris.

2012/4/4 Hoàng Minh Thắng <phuthuycuo...@gmail.com>

[Nick Bauman]

with-noir works great!

I find myself using send-request more often (also in noir.util.test).

I have also been having a lot of fun testing my validation through the
routes with my test2 harness. This makes testing a bit more
declarative (avoiding things like managing bindings and introducing
mini state management like you're having to do with your tests there,
Hoàng Minh Thắng)

It's here:

https://github.com/nickbauman/noir.util.test2

By the way, there is nothing wrong with testing on a lower level, like
you are, Hoàng Minh Thắng. Chris and I are just pointing out other
ways of doing it.