Hello everyone,
In this document, I'm going to take a look at Elixir as a product (with its goals and target audience) and try to start a discussion to try and answer the following questions:
* which goals is it pursuing?
* does it do a good job at it?
* what can be improved?
All the views and opinions here are my own and are not necessarily shared by Elixir authors or other members of the community, although José did take a look at this document before and suggested a few corrections regarding Elixir's goals, and I'd like to thank him for that.
## TL;DR ##
This is a long message. You may prefer to read it on the wiki here --
https://github.com/alco/elixir/wiki/Elixir-Identity
Here are the main points I'm going to discuss in the following sections:
* the need to identify the steps needed to lower the entry barrier to Elixir
* insufficient examples for Elixir's meta-programming features that make it stand out
* lack of code in the wild (libraries and examples)
* hide Erlang from the docs for as long as possible at first so that newcomers can begin building OTP applications with zero knowledge of Erlang
If you're curious about the reasoning behind those points, read on.
I would like to note here the I'm going to be saying things like "we need more docs" or "we need more examples". That's certainly easier said that done, but the purpose of the present document is to make sure we're all on the same page and that we acknowledge that issues discussed here are indeed real issues. The next step will then be to define a set of tasks for Elixir developers and other members of the community to help improve the situation. Those are going to be discussed in a future document.
## Intro
I got to know Elixir about a year ago. Read through the getting started guide, wrote some toy programs. Then I learned the basics of Erlang, contributed a little to Elixir's libraries, docs, and tutorials. This all happened over the course of two month, then I dropped out of Elixir community completely for almost year.
Recently, I've got an opportunity to start playing with the language again. I was glad to discover that it has matured, became rather stable (no significant breaking changes for the last months, maybe even half a year?). The docs got improved too: it's now easy to start a new project or get and build someone else's project using mix. The mailing list and IRC channel got a bit livelier. All in all, Elixir has been growing at a steady pace during the last year.
In this document, I'm going to argue that it could go faster than that. And I don't mean merely growing raw number of users that start learning Elixir (that's the domain of SEO). I'm talking about natural growth of the community that will help the language itself and the ecosystem around it to grow faster on all fronts: core language, documentation, available libraries, projects in production, etc.
After all, that's the ultimate goal of the project -- spread the awareness of its existence and encourage more people to learn and use it for great good. And not only encourage but provide pragmatic reasons and benefits of doing so.
## What is the purpose of Elixir?
So, let us think about just how clearly Elixir states its purpose to new users. Judging from the site, it's meant to provide a different approach to building reliable, concurrent, and distributed systems using Erlang VM as its foundation and, consequently, making OTP more accessible to general audience.
Erlang itself is a niche language. It has very concrete and narrow goals of building highly reliable distributed systems and it implements those goals rather well, judging from general satisfaction of its user base. But the language itself hasn't got much traction in the modern development community, web development in particular because that's the domain that tends to welcome the ability to build highly concurrent and scalable distributed systems, and Erlang has been designed to do just that.
There's a number of reasons for why Erlang ended up where it is, at the moment, let us not worry about that. For the purposes of this document, it's just the way things happened.
What we should be worried about, however, is that Elixir wouldn't end up in a situation where it's used by a handful of people that mostly come from a relatively small Erlang community. Because the intent behind Elixir is to provide the goodness of the Erlang VM and its OTP ecosystem to the mainstream development community. In other words, Elixir aims to open up Erlang's magic box of tricks to a wider audience, in the fashion of what Ruby on Rails did for Ruby.
## Are we moving in the right direction?
Let's talk about Erlang a bit more. It is undeniably what made Elixir's existence possible in the first place. But at the same time it can harm Elixir's adoption among the broader developer audience that Elixir is targeting.
There's really not that much one can do with Elixir alone when it comes to building OTP applications. And not only web applications, there is enough space in software development landscape where Elixir could be a great fit. However, most of the docs are targeting Erlang the language. Many frameworks and libraries are written in Erlang, which by itself is actually one of the things Elixir is striving to achieve -- seamless compatibility with Erlang ecosystem. But it's a fact that there is no comprehensive documentation written specifically for Elixir that would describe how to use cowboy, for instance, or other often recommended libraries. And this is actually hurting Elixir's adoption.
Elixir doesn't fully deliver the promise of a more productive and accessible Erlang/OTP. Accessible for the mainstream community, that is.
## Improving the "getting started" experience
Let's see how we can improve the experience of starting out with Elixir.
Due to its nature, there are two kinds of users coming to Elixir: 1) those who have been using Erlang professionally and know what OTP is and how it helps make their job easier; 2) those who have background in other programming languages, having only heard of or even with no knowledge of Erlang at all.
We'll focus on the second group for the time being. Those who have background in other languages may simply not know much about the properties and advantages of the Erlang VM and niceties Elixir adds on top of that. It is in the best interest of Elixir's authors and community to educate those people and guide them through their first steps with this marvelous language.
### Educating people coming from other languages
The biggest entry barrier for people coming from other languages than Erlang is, paradoxically, the Erlang itself. Although it forms a solid foundation for Elixir, it should not be pitched as Elixir's primary selling point to this group of users. After all, Erlang is a niche language, and Elixir tries to open the Erlang VM/OTP combo to a larger audience. So while Erlang's strong points are recognized by those who are already using it (and they need other reasons for switching to Elixir, see above), to a wider audience outside of Erlang's ecosystem it is not as clear what's this all about.
So what I'm proposing instead is that Erlang should not be exposed to newcomers early in their learning process. In other words, Elixir has to show that it's self-sustainable enough to harvest the power of OTP and build real working applications in it without the need to dive into Erlang. Of course, this doesn't mean dumping cowboy, for instance. But writing a good doc that covers most of cowboy's use cases in Elixir's context and at the same time targets the audience coming to Elixir with zero knowledge of Erlang should definitely be considered.
Here are some bullet points listing the things that would demonstrate Elixir's maturity to be used for development without learning too much of Erlang along the way:
* 1st-class native support for all of the OTP (that's commonly used) in Elixir with comprehensive documentation. Yes, there is GenX which provides a great high-level abstraction on top of OTP behaviors. It's lacking the docs though. At present, our best advice for a novice starting with OTP in Elixir is to go and read the relevant chapters from Learn You Some Erlang. In order to support my arguments above, we need to provide a guide to OTP written specifically for Elixir, using its best practices and providing higher abstractions where it helps understanding and reducing the code size.
* IEx shell being able to do everything erl does and more. It may already do so, sorry if I've missed this fact. But the documentation is mostly that of erl, if I understand correctly. We don't need to replace `man erl`, but we need a guide that would show all the different ways of using IEx with a working app, written with Elixir users in mind.
* Make it easy to add Erlang libs to Elixir project. Just add a line to `mix.exs` and you're good to go. This too might already be implemented. Still, we're lacking in examples that would show how to use commonly recommended Erlang libs in Elixir.
* A comprehensive guide of practical applications of Elixir's features. Getting started guide shows the basics only. We need a guide that would assume the basics have already been learned and would delve into more advanced topics. This is in line with the point I made earlier about demonstrating the goodies of Elixir's macros and protocols to Erlang programmers.
* Examples of doing common tasks in Elixir (like building a web service) and using concurrency with OTP to do them (in the form of talks, articles, code, screencasts). Rob Pike, for example, has done a stellar job presenting many talks that demonstrate Go's concurrency primitives and how they help build reliable concurrent programs with very little effort.
I'd like to mention Dynamo here which is an excellent project to demonstrate exactly how the approach to writing a web service in Elixir is going to be different from that of Erlang's. But it's not finished and not documented enough. To move this project forward, a number of tasks can be defined that will allow newcomers that have found their own motivations to learn Elixir to come and help with Dynamo's progress.
* Lack of libraries written in Elixir that could boast less code that does more or simply better code that is easier to write, read, test, and maintain.
* Lack of real applications out there that could demonstrate Elixir's best practices and serve as role models for others that are only starting with Elixir. I'm not saying there are none (see here[1] for some), but still not many of those.
I'm mostly talking about documentation, because it's critical for newcomers with no background in Erlang or concurrent languages in general. There is still other kind of things Elixir is lacking like pointers to how one can contribute to Elixir as a whole and help grow its ecosystem.
We need an outline of all the different things newcomers can do to help move the project forward. Most of the points above are aimed at attracting users and educating them, but there are also programmers that like to delve straight into the code and figure it out themselves. Those people can help by contributing to the language or its libraries directly, but there's a lack of general guidance for them.
---
Footnotes with links:
[1]:
https://github.com/elixir-lang/elixir/wiki/Projects-in-the-wild## Conclusion
So let us reiterate on the issues present in this document and what we can take out from it.
What's hindering Elixir's adoption?
* Erlang burden. We can't require knowledge of Erlang if we want for Elixir to become popular. In short, learning Elixir shouldn't mean learning Erlang as well. At least, not at first.
* Insufficient examples for Elixir's meta-programming features that make it stand out.
* Little code in the wild (libraries and examples).
* Not clear what to do if I want to contribute.
Elixir has much potential and we still have much room to try harder making more accessible and growing the community. Afterwards, the community will be growing the language (provided that the language becomes stable in the long term and has an enhancement process in place).
Thanks for reading.
--
Best regards
Alexei Sholik