Deployment involves many issues of concern to every CB adopter.
I've started this thread with hope that we can explore the issues, expand the wiki on relevant topics and, perhaps, develop code to make deployment easier and safer for all.
Here are some of the issues. No doubt there are more:
- Remote vs. local deployment
- Dedicated server vs. VPS vs. cloud
- Security
- Static vs. dynamic DNS provisioning
- Reverse proxy configuration
- Multi-server installation and configuration
- Backup
- Deployment automation
- Server monitoring and maintenance
So four questions:
1) What are the most crucial issues involved in deployment?
2) What would a checklist for secure, efficient deployment look like?
3) What can we contribute to the wiki to make all issues of deployment easier for the noobie?
4) Can we develop useful code utilities to make deployment of CB and CB apps easier and safer for all?
Best wishes,
LRP
*********************************************
My books:
THE GOSPEL OF ASHES
Strength is not enough. Do they have the courage and the
cunning? Can they survive long enough to save the lives
of millions - March 2012 release
FREEIN' PANCHO
It takes a community to help a troubled boy find his way
AYA TAKEO
Star-crossed love, war and power in an alternative universe
Available through Amazon or by request
from your favorite bookstore
Websites:
**********************************************
Wow!! Congrats!
--
Evan Miller
http://www.evanmiller.org/
Thanks. I'll look into capistrano. Looks interesting.
But I, for one, need to know much more about all the issues surrounding deployment. How can we factor the topics to stimulate constructive discussion?
Thanks,
LRP
-----Original Message-----
From: "Jose Luis Gordo Romero" <jgo...@gmail.com>
Sent: Friday, February 10, 2012 1:59pm
To: chica...@googlegroups.com
Subject: Re: Deployment
I think you are trying to attack too many things at once.
One thing is the stack (ex. so, erl, balancers/proxy, ...) and their
automation including scaling (ex. chef), other the deployment strategy
(git/capistrano, migration task, ...) and other monitoring (like jmx in
java).
For now I'm happy with a simple capistrano recipe (in a separate project)
that fetchs-git/handles config/compiles/checks/reloads on every node of the
cluster.
I known that is not erlang-friendly and also highly inefficient, OTP
packaging can be our friend here, but requires another step in the
rebarization process.
Interesting topic!
--
Jose Luis Gordo Romero
@jgordor <http://twitter.com/jgordor> - http://www.freemindsystems.com
Many thanks. This thoughtful reply is very helpful.
Maybe I'm over complicating things.
Beyond a few simple sites (ayatakeo.com (VPS) and fromtheglen.com:8000 (dedicated server - still working on it. Hope to bring it over to CB)), I don't have experience with heavy-duty deployment. But I have much that I want to do in the near future so want to be efficient and reliable. And I'm conscious of the many security risks lurking in the shadows of Cyberspace.
> There is already a local server in every boss project, just run ./
> start_dev.sh
Yes, I'm using that locally. Exploring Nginx as a reverse proxy when I start deploying current work under CB.
Why? I'm thinking tighter security, faster response to static resource requests and, for awhile, I may need to run another web server in parallel to CB. Am I wrong in my thinking?
I guess what I'm kind of looking for are best practices.
> There is already a wiki page with recommended install steps here:
> https://github.com/evanmiller/ChicagoBoss/wiki/Deploy
> Anyone can add their own experiences
I do have an item or two to add to the wiki. Just need to find time to learn the mark-up and method. Will get to that as soon as I get my new book out.
> I think CB is already poised to implement some update and rollback
> features. And as you would expect from Evan
Yeaaa CB! An answer to a spare-time techie's prayers.
I have been studying and working with Rebar. It makes OTP sooo much easier.
> destroying an app will be project specific, no way around that, that I
> can tell
That I can manage. Have done it several times unintentionally.
> I would try to setup a rebar script that pulls from github(or my own
> private git repo).
This is very helpful. As soon as I can make time I'll try something like it and post whatever useful comes out on the wiki.
> Backups are a database concept and should be handled at that level. We
> use SCM for apps
I'm using rsync at present. Wiped out some very precious files last year with a dunderheaded rsync command. Works OK now, but scares me silly every time I think about it. This is one reason why I, as an amateur, cherish good user documentation. The quality of the user docs is number one on my why-I-like-CB list.
Monitoring deserves it's own topic
Perhaps down the road we can pick this one up again.
Ben, related to your interest in multi-server deployment---
A wispy thought has been circulating in my head involving embedded Erlang, Chicago Boss (maybe), and the new low-power-consumption, low-cost server technologies in the wind (See http://www.calxeda.com/ for one of the more out-edge ideas; maybe Rasbperry Pi or similar for really cheap).
So question: Do you have a sense of how minimalistic an Erlang/CB provisioned server can be yet still provide decent, relatively low-load performance?
I'm thinking that if the hardware is cheap enough we can start paralleling servers ala Erlang concurrency and do some interesting things.
Thanks again and all the best,
LRP
I think this better summarizes my philosophy
http://www.youtube.com/watch?v=7llLFbV97bs
Happy friday everyone
--
Evan Miller
http://www.evanmiller.org/
I'm very interested in these developments, and Erlang is certainly
poised to take advantage of these new ARM-based servers. (HiPE
supports ARM, and embedded Erlang has had tons of investment in it
over the years.) If you use Mnesia and SSDs then your RAM requirements
will be quite small, and for low loads CPU usage will be negligible.
If you don't need extensive logging then you might get away with say a
500 MHz chip, 64 MB of RAM (less?), and 256 MB of flash memory. I
wonder what the virtualization situation is on these Moonshot servers.
I think you are correct that Erlang's parallelism will be especially
appealing as ARM clusters become economical.
--
Evan Miller
http://www.evanmiller.org/
> I'm very interested in these developments, and Erlang is certainly
> poised to take advantage of these new ARM-based servers.
Have you spotted any low-cost boards/boxes that are particularly well-suited for experimentation?
Best,
L.
-----Original Message-----
From: "Evan Miller" <emmi...@gmail.com>
Sent: Friday, February 10, 2012 7:53pm
To: chica...@googlegroups.com
Subject: Re: Deployment
>> I may need to run another web server in parallel to CB. Am I wrong in my thinking?
> sounds like you are prematurely optimizing. How many page requests a
months do you get now? A single server will take you quite a long
ways.
I didn't express this well. I meant to say that I need to run Zotonic in parallel with CB and static sites for awhile until I can move content over from Zotonic to CB.
Incidentally, this is not to say anything negative about Zotonic. There are just too many moving parts in Zotonic for my feeble brain. I can't grasp how they fit together sufficiently to feel like I can fix things if they break.
So my set-up would be:
Nginx -->
-- Static 1
-- Static 2...N
-- CB (with multiple sites).
-- Zotonic
This all on one hardware server.
I'd like to be able to call each of them from unique domain names; but cross-link internally from one to another. I've got the Nginx virtual host config worked out on localhost. My one attempt at cross-linking failed. I just need to spend a bit more time on it.
Thanks much,
LRP
-----Original Message-----
From: "Ben G." <bng...@gmail.com>
Sent: Friday, February 10, 2012 10:19pm
To: "ChicagoBoss" <chica...@googlegroups.com>
Subject: Re: Deployment
Reliability and scalability are Erlang strengths.
Erlang IS easy to learn--- Excellent documentation, texts, on-line tutorials, supportive community, training consultancies
Rebar makes it easy to document, test, and deploy Erlang.
Erlang can fit on a USB drive. For extreme case see: http://www.erlang-embedded.com/
Erlang plays well with other languages.
Erlang does require a different mindset than you may be used to. Once you grasp it though, it's and easy and productive way to program.
Chicago Boss is Erlang.
Chicago Boss has superior user documentation.
Chicago Boss is fast restructuring under Rebar.
I'd suggest: Learn You Some Erlang for the Great Good ---
http://learnyousomeerlang.com/
All the best,
LRP
I have not. From my understanding the server-class boards won't be
available at retail for a while.
Evan
I picked up four Pogoplug V2 for experimentation -- $25/ea.
https://wiki.xkyle.com/PogoplugV2
May be a a bit memory shy.
I won't have time to play a few months, but welcome input and ideas.
Best,
LRP
Thanks for the info.
I prefer Linux; am competent with most tools.
I haven't yet used VMs, however. Plan to. Just need to find time to read up. Would certainly welcome any tips on that.
Aptana Studio looks interesting. I'll definitely explore that.
Thanks again,
LRP
I can well understand the frustration that you and Hwa feel.
I spent more than a year struggling to grasp Erlang and Erlang web infrastructure well enough to bring up sites I desperately need to support my books and small publishing business. Still struggling with it.
But "dumbing down," I think is an unfortunate way to think about the problem. What you're asking for, if I understand it, is a shrink-wrapped solution that allows you to load and go. Wouldn't that be nice. But there are any number of issues.
First, Erlang and Chicago Boss are open source. This means that smart and generous people have provided us with incredible software that we can study, deploy, and re-sell as we see fit.
The scores of talented developers who've created Erlang and Chicago Boss have no need to "market" their work. They already understand and use Erlang and CB. They have projects of their own to market.
I've been involved with computers and software development for more than 30 years. And I can vouch that both Erlang and Chicago Boss are truly outstanding. And, relative to many open source projects, are exceptionally well documented. Are the user docs perfect? No. Are they as easy to adopt and deploy as we might wish? No. Are they finished and polished software packages? No--- they're works in progress.
So this is where we come in. The one thing open source developers do hope for -- their most meaningful reward -- is users who will help carry the ball forward. As a user, my tech skills are considerably less than the exceptional talents behind Erlang and Chicago Boss. But I do hope, and try, that I can do my part to carry the ball forward by asking sensible questions when I don't understand something, sharing what I do know when I think I do, contributing to user documentation when I'm able, contributing useful code should I develop it, and supporting the generous developers who've provided this incredible platform in any way I can think of.
All well and good, but what about me? I'm as selfish and self-centered as the next guy. I've got books to write and publish. Time I spend on software is that much less time toward bringing success to my books and publishing business. I don't have many productive years left. But I do need the web to successfully market my books. Erlang and Chicago Boss give me exceptional access to the web. It's in my selfish, self-centered interest to learn everything I can about Erlang and Chicago Boss so I can fix stuff when it goes wrong.
And it will. Software breaks. Software goes out of date. Software systems are fragile. Software/hardware systems even more so. I'm a "army of one." I don't have an IT department. I can't afford expensive consultants. I thank my lucky stars that I've got the Erlang/Chicago Boss open source communities at my back.
So guys, you've already made a contribution toward greater success for Erlang and Chicago Boss by asking an important question. I do hope you stay with it, continue to ask about what confounds you, study more, and contribute back what you can. The success of the Erlang/Chicago Boss community depends upon it.
All the best,
LRP
-----Original Message-----
From: "Trevor Bortins" <trevor....@gmail.com>
Sent: Sunday, February 12, 2012 1:39pm
To: chica...@googlegroups.com
Subject: Re: Deployment
I'm especially about the possibilities with open-source servers built
with Erlang/CB. You'll be able to distribute your projects as turn-key
packages with ZERO compiling or prerequisites -- just unpack and run
on any machine. (Of course, you'll need to provide separate packages
for each platform/architecture.)
The functional-programming bar is probably too high for Erlang to be
the next Ruby (and CB the next Rails), but my hope is that once we
harness the power of OTP, Erlang/CB will be used to produce a
generation of high-quality websites and a plethora of open-source
projects that literally anyone will be able to deploy.
Evan
--
Evan Miller
http://www.evanmiller.org/
I do indeed agree. What can we do to make it easier?
Best wishes,
I volunteer.
Will be a week or two until I get my book out. But then I'll be in a panic to bring up my book sites. So I plan to spend serious focused time with CB. I'll probably be more hod carrier than brick layer, but will do whatever I can.
I have a working, still a bit superstitious, understanding of OTP and managing Erlang apps under Rebar. I say superstitious since I can sort of get things to work without fully understanding the underlying why. I was just beginning to look into releases when other priorities pulled me away.
If you can give me your thoughts on where you see the biggest obstacles for CB I'll put particular attention there. I totally agree that deploying CB as an Erlang release will be a big step forward, not that it's all that difficult now.
Re: functional programming, or at least, Erlang--- the light went on for me when I grasped the significance of three concepts: single assignment, pattern matching, and recursion. They all seemed so simple-minded when I read Armstrong that I glazed over them. Then the light finally dawned--- these are the beating heart of the language. I haven't fully unleashed the power of functional style yet, but I know enough now to use list comprehensions, map, and filter. Suddenly it feels like I've gone from 0 to 60 overnight.
It was when I started understanding edoc and eunit, which wasn't difficult, and some of the various debugging and support tools, that I realized that Erlang is an industrial-strength language intended for getting real work done. Well worth studying in depth.
Having cut my teeth with Forth it does pain me to type io:format("My string") when I could express the same thing as ."My string" in Forth. But there are those who mistakenly see Forth as a write-only language.
Your studies will be well rewarded. You'll be better able to understand Erlang code, expand your programming capabilities through the libraries, and be able to develop large-scale fault-tolerant systems.
Become skillful enough and you'll likely find many rewarding jobs out there.
And, it's not that hard.
Best,
Or don't do OTP releases and use OS packging. We have build our own infrastructure to debianize CB deployments (and Erlang Apps in general). It took a while to get the basics right, but now we can checkout a new version of out project from git, package it and publish to an repository in under 10min. The customer (or we) can then upgrade the test, stage and production system with just an apt-get update; apt-get upgrade. It can be that simple ;-)
The packing tool is already public (https://github.com/fjl/tetrapak), the application manager (a kind of init.d system for erlang) should follow.
Andreas
[...]