Clojure/Pedestal vs Go

[Alan Thompson]

Hi,

I'm about to start a new web project and they are thinking about using Go (golang) instead of a JVM (preferably Clojure) based approach.  The idea is "BARE METAL SPEED!!!", but I really think the network and DB will be the bottlenecks, not Clojure vs Go.

Is anybody out there aware of any speed comparisons using Clojure/Pedestal and/or Go?  I'm thinking basic measurements like connections/sec, latency, simultaneous users, etc.

Thanks,

Alan

[Alan Thompson]

P.S.  I have seen the results at https://github.com/ptaoussanis/clojure-web-server-benchmarks  although I'm not sure exactly how to interpret them w.r.t. "keepalive" and the "errors" graph.  Also, the plotted results don't seem to include latency.

Alan

--
You received this message because you are subscribed to the Google Groups "pedestal-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pedestal-user...@googlegroups.com.
Visit this group at http://groups.google.com/group/pedestal-users.

[Max Countryman]

Hi,

I'd love to see some discussion about this as well: I've struggled to justify the JVM in a production environment that's dominated by Go. My experience with my team has been that they are very unwilling to use the JVM and will go to great lengths to avoid it. The argument seems to be that Go produces light, extremely fast binaries, and has phenomenal networking libraries built into the standard lib, making it ideal for writing server-side components that may handle large amounts of data and need to scale horizontally. Despite my attempts to have members of my team checkout Clojure the final decision has always been to use Go instead. I'm personally a bit concerned that Go has started to become the replacement for Java and that the relevance of the JVM may be declining. It would be great to see some counterpoints to Go vis-à-vis Clojure and the JVM. 

Max

Sent from my iPhone

--

You received this message because you are subscribed to the Google

Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[James Reeves]

There's the TechEmpower benchmarks at: https://www.techempower.com/benchmarks

Interestingly, Clojure is currently ahead of Go in the single and multiple database query benchmarks.

- James

--

[Rui Paulo]

On Sun, 2015-09-13 at 12:44 -0700, Alan Thompson wrote:
> Hi,
>
> I'm about to start a new web project and they are thinking about
> using Go (golang) instead of a JVM (preferably Clojure) based
> approach. The idea is "BARE METAL SPEED!!!", but I really think the
> network and DB will be the bottlenecks, not Clojure vs Go.

Depends a lot on the application code, doesn't it?

This might be of some interest to you, but it doesn't really compare
any JVM language:

https://blog.repustate.com/migrating-entire-api-go-python/

--
Rui Paulo

[Christian Weilbach]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

> I'm about to start a new web project and they are thinking about
> using Go (golang) instead of a JVM (preferably Clojure) based
> approach. The idea is "BARE METAL SPEED!!!", but I really think
> the network and DB will be the bottlenecks, not Clojure vs Go.

You might also be interested in
http://docs.paralleluniverse.co/pulsar/ The go take on CSP is limited
when you really want to scale, while having an actor Erlang-like model
can also give you benefits, so in Clojure (or on the JVM), you can
evolve your application differently, while go basically has one
(well-done, but fairly primitive, e.g. error handling) concurrency
concept baked into the language semantics. I haven't used go myself
though and I guess best would be a small prototype application both in
go and Clojure + some experiments. It would be nice if you publish the
results somewhere and ping back.

>
> Is anybody out there aware of any speed comparisons using
> Clojure/Pedestal and/or Go? I'm thinking basic measurements like
> connections/sec, latency, simultaneous users, etc.
>
> Thanks, Alan
>

Christian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBAgAGBQJV9fW6AAoJEKel+aujRZMkxxoH/3bWMiFW/brS2t+o9qcq7gH6
NsIx+WL9YuSe0Fy2/lp/WKD7owLLM6gIoSfQIbYYurRqSlb4+hjKoei5Wqnxg6TF
Meq1I3D0uSJmMcQ7ux3d6Nn/XloXbUR9QDL8IVUZKNiCM+jz9s4XNBhTVRyhp2UI
0ubV7uXJGZV6XVun2UV2zoLDZV4PwXS0O45GB+ta+q+RhJ1bKZtdKtEBq6p449Hs
f6mdHWo+0MtWnHzOpOo7r2sLwkdwkRSi9ItWLB0UUbp4/5UQqmE+zUjYQR3wVIKe
307R8NFAb0sZp9fk/rtklIemIeUXslOTid//sdDHmKBWRCa8vKuEKiYDwQvR0t4=
=itJZ
-----END PGP SIGNATURE-----

[Thomas Heller]

Hey,

this might not be too relevant to your project but since you said "web project" it might be.

I cannot speak for Go but Clojure is "fast enough". I went from Ruby to Clojure in one app and the difference was huge (150ms-ish -> 10ms-ish). I'm actually somewhat of a performance junkie and even got most Clojure numbers down to 2ms but NONE OF IT MATTERED! At the time of transition most of the frontend was still a large ball of jQuery-sphagetti and absolutely no proper HTTP caching config in the backend. While the initial request got faster we were still looking at average page load times of 3+ seconds. When I started rewriting all of the speed related stuff in the frontend the performance gains were huge, it more than halved the average page load time which had a far greater impact overall than the choice of programming language in the backend.

None of these techniques are specific to any programming language. What had the biggest impact for me was that in Clojure you are just writing Clojure(Script). You don't have to switch between 2 languages for the backend/frontend and get some of the most amazing tools (Closure Compiler) for "free". When it comes to raw speed Clojure and Go will probably produce rather similar numbers, but for me Clojure definitely has the better development story. (Disclaimer: I actually don't know anything about Go frontend story.) 

I don't care much for micro benchmarks in any language and you should not make decisions based on them. Look at the whole stack. The initial response time for a "hello world" request will never properly reflect your production app. In the end all that matters is your code, it won't be fast just because it is Go and not a JVM.

Just my 2 cents,

/thomas

[Paul deGrandis]

Hi Alan,

Thanks for reaching out on the Pedestal mailing list!
Most web benchmarks care more about throughput and concurrent connection than latency, but sadly, nearly all benchmarks show these measurements as scalars instead of distributions.  The "errors" number is the number of requests that resulted in a server error (because of overload/mishandling).  A robust server that correctly understands back-pressure and can efficiently process requests on a queue should result in no errors, assuming that it can process the queued requests at the rate in which they're being received.

The "keepalive" flag refers to TCP keepalive, which will affect speed and connection timeout.

While I'm not aware of any benchmark between Pedestal and Go, I would be highly skeptical of such a benchmark.  Web benchmarks are often very misleading - they represent an application you'd never write, integrated into a server in a way you'd never do, just to get the highest overall number possible.  I would instead encourage you to look for measurements related directly to the quality attributes you care about, for the specific pieces of your overall application.  I'd also think about what you're really trying to optimize.

With Clojure you get the power of Lisp - you get to use go-channels, property-based generative testing, pattern matching, logic programming - all as libraries.  You can extend the language however you need, to build the most efficient system.  Hell, you can even write directly to the LLVM compiler to get bare-metal speed (again, just a library).  You get the power and speed of interactive development, with the focus of small, composable, functional units (first class functions, immutable data structures, protocols).
I recently needed to do very fast binary processing for a wire protocol, and I was able to utilize Netty within Clojure to do the entire thing off-heap (no GC impact), and with native byte handling - Clojure gives you immediate access to the powerful library ecosystem on the JVM.  You can even connect your editor to a running system to inspect, debug, evolve, and patch it.

While Golang's community is growing and great libraries are appearing in the ecosystem (Pedestal's router was heavily influenced by httprouter), the language and runtime aren't nearly as powerful as Clojure, for me.  I have far greater flexibility and power with Clojure, and the opportunity to optimize my application and process where it matters most.

We benchmark Pedestal against other web libraries with real production systems.  In every benchmark we've done so far, Pedestal has come out on top.  We also look at more than just runtime performance characteristics - we consider code size, system complexity, etc.
Additionally Pedestal is secure by default - all of the best security practices are automatically enabled in Pedestal (you have to explicitly asked for them to be turned off).  We're also only one of two Clojure web systems that is truly asynchronous.  You can also take a look at Pedestal's other notable features.

Hope this helps!  Reach out with any other questions you may have.

Regards,
Paul

[Alan Moore]

I'll second Paul's comments and raise you two:

1) Depending on your app's use cases, speed going forward will be gained primarily from parallelism. I think Clojure has a better story there than Go but that is just my opinion.

2) It is very hard to fight against cultural bias against the JVM. I work in embedded systems where anything but C/C++ (or Lua, Python) is taboo. Your best bet is to "Go" with their momentum and when they run into a roadblock in Go (probably something related to mutability/locks in the face of heavy load), give a shot at the same problem with Clojure.

Obviously this has to be done in a non-intrusive way (on the other end of a socket) but will give you a chance to prove Clojure and the JVM can handle the job. Unfortunately your company won't gain the other benefits of Clojure beyond just performance (e.g. clarity, simplicity, etc.) because the rest of the code base will be in Go but... clearly that ship has already sailed.

Good luck! Let us know how things turn out.

Alan

[Alan Moore]

One more thing: if you are truly stuck with Go you can still adopt functional patterns, style and data structures to good effect. Just because the opening paren is on the wrong side of the function name doesn't mean you can't find (or write?) an immutable data structure library for Go and promote functional programming ideas within your team.

Here is are some simple examples from C++ (from one of the authors of the D language):

http://bartoszmilewski.com/2014/06/09/the-functional-revolution-in-c

http://bartoszmilewski.com/2013/11/25/functional-data-structures-in-c-trees

https://github.com/BartoszMilewski/Okasaki

I'm pretty sure you could port these or some other implementation to Go fairly easily - YMMV.

Alan

--
You received this message because you are subscribed to the Google

Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/p_cZoGCpvbE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

-- 
"Whatever you can do, or dream you can do, begin it. Boldness has genius, power, and magic in it. Begin it now." - Goethe

[Max Countryman]

Hi Alan,

From my experience this is not true: Go does not provide generics and actively resists what most of us would consider good functional programming--Go is very opinionated and doesn't allow much deviation from these opinions, by design.  So implementing practical immutable data structures would be difficult and likely of limited utility in Go. That said, you should still try yourself if given the opportunity. 

Max

You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Alan Moore]

Max,

You obviously know way more than I do about Go... I stand corrected, thanks. I did know that it doesn't support TCO so it doesn't surprise me that other language features went the wrong way too.

I did not mean to misinform anyone, my apologies for speaking beyond my core competency (Go is not one of them.) I was trying to give our friend some hope... Those of us with day jobs in C/C++ (and Go) need some of that to keep us sane. :-(

Alan

[Max Countryman]

Alan,

Absolutely no need to apologize! :) I would hardly consider myself an expert either, I’m basing my understanding on the experiences I’ve had at my day job and the large body of developing articles and blogposts related to the language that seem to pop up at a near-constant rate on your favorite programming news aggregator.

If people are doing FP with Go somewhere, that’s awesome and I would love to see it. My suspicion is it’s difficult and generally not what you “want” to be doing with the language. Go’s popularity is a big disappoint for me personally, but again your experience may differ.


Max

[kovas boguta]

At least in one area -- data infrastructure -- the JVM has no
competitors for off-the-shelf solutions.

Hadoop, Spark, Storm, Kafka, Cassandra, HBase, etc etc are all JVM-based.

In the alpha-nerd set, one can easily argue that the relevance of Go
is fading and its being replaced by Rust.

I'm not too worried.

[Christopher Small]

I also am no Go expert. All I'll add is that I overheard that the lead author rejected a proposal to add `map` to the language, stating that it is "too niche". Doesn't bode well for FP in Go...

[Mikera]

Go is fine for system-level programming, but I personally wouldn't use it as a web application programming language. Ten reasons to consider.:

1. There isn't really much speed difference on micro-benchmarks between JVM code and Go. See e.g.: http://benchmarksgame.alioth.debian.org/u64q/go.html

2. In many real-world cases JVM code will outperform Go, see e.g. http://zhen.org/blog/go-vs-java-decoding-billions-of-integers-per-second/

3. The JVM GC is superior to the GC in Go. This makes a big difference for complex applications with a lot of live objects on the heap, especially when dealing with stuff like immutable persistent data structures (and you want to use those, right?). Overall, I'd expect Clojure to easily outperform Go for these types of applications. 

4. The JVM startup time / memory overhead is irrelevant for long running server applications. If you are restarting the JVM regularly, you are definitely doing something wrong. The JVM is well tuned for long-running server side web applications in general, rather than the lightweight processes / scripts that are more suited to Go.

5. Functional programming is natural in Clojure and is IMHO the future for building modern applications

6. If you like the CSP stuff (goroutines, channels etc.), Clojure has the excellent core.async, which basically lets you do all the nice CSP type things in Clojure you can do in Go. However that isn't forced on you: Clojure supports a lot of other paradigms for concurrent programming. Choose what works best for you.

7. The open source library ecosystem on the JVM is awesome. There's nothing like it for any other language. Even if nobody has written a Clojure library that wraps up the functionality you need yet, using Java libraries from Clojure is very painless (often easier than using Java libraries from Java!)

8. Virtually all the key big data stuff depends on the JVM. Spark, Hadoop ecosystem etc. Being on the JVM is a valuable strategic choice if you are into data processing. / analytics.

9. Clojure has an compelling full stack development story with ClojureScript on the browser and Clojure on the server. It's pretty useful to be able to share code between the server and the client.

10. Once you've experienced interactive REPL driven web application development with stuff like Figwheel, you probably won't want to go back to anything else

[Michael Gardner]

On Sep 15, 2015, at 20:45, Mikera <mike.r.an...@gmail.com> wrote:
>
> 7. The open source library ecosystem on the JVM is awesome. There's nothing like it for any other language.

I like your other points, but in my experience this one is (arguably) no longer true. I've often found the JVM library ecosystem to be lacking in comparison to Python's, especially for newer problem domains. I'd attribute this to Java no longer being a "hip" language, which counts for a lot in OSS development!

[Josh Kamau]

11.  Leiningen and painless dependency management.  

Josh

[Colin Yates]

12. Community (although I have no idea what Go's community is like to be fair)

And an upgrade to the "full stack" comment is full stack including over the wire translation with edn and transmit.

Sent from my iPhone

[Brian Hurt]

I have no flipping clue what "system-level programming" means these days, and I'm hoping someone could spell it out.  I used to think (back when I was doing that sort of stuff) that it mean operating systems level stuff- device drivers, embedded, real time, stuff like that.  But no language with a GC or non-trivial run time is a good fit for that.  From usage, I now think it means "we refuse to use Java".

--

[Ernestas Lisauskas]

There was a question at LangNext 2014 conference: what is a systems programming language in 2014? https://youtu.be/BBbv1ej0fFo?t=3m43s