Pallet 0.8 Tutorial

[Jon Crosby]

This is just a quick check-in before duplicating efforts that might already be underway by others.

I am interested in producing an end-to-end tutorial for the 0.8 Pallet release. My goals are to show the series of exact steps required to go from nothing (i.e. not even leiningen) to a running local web app using vmfest (perhaps just a ring-based "hello world" using some kind of datastore), to finally running the same app on a cloud provider such as AWS or Rackspace, with a load balancer, two web nodes, and a separate database node or service.

Is something like this underway? If not, would a public gist be a good place to start sketching it out?

-Jon

[Hugo Duncan]

Jon Crosby <j...@joncrosby.me> writes:

> I am interested in producing an end-to-end tutorial for the 0.8 Pallet
> release. My goals are to show the series of exact steps required to go from
> nothing (i.e. not even leiningen) to a running local web app using vmfest
> (perhaps just a ring-based "hello world" using some kind of datastore), to
> finally running the same app on a cloud provider such as AWS or Rackspace,
> with a load balancer, two web nodes, and a separate database node or
> service.
>
> Is something like this underway? If not, would a public gist be a good
> place to start sketching it out?

Something like this has been on the todo list for way too long, and
would be most appreciated.

Even a simple "this is how you get your leiningen/ring based project
deployed", without the proxy and datastore, would be of use to many
people I think. Maybe that could be an early part of this?

A public gist sounds like a reasonable starting point (though it might
need a github project fairly quickly).

Look forward to helping see this come to fruition, and thanks for taking
the initiative.

Hugo


Some previous examples, now long in the tooth:

https://github.com/pallet/pallet-examples/tree/develop/webapp-haproxy-nodes
http://cemerick.com/2010/05/12/provisioning-administration-and-deployment-of-couchdb-java-tomcat/

[hunter hutchinson]

Don't forget about Otfrom's great quickstart for 0.8.x ... 

https://github.com/otfrom/qspallet

It's not up to date with 0.8.0-beta.1

--
You received this message because you are subscribed to the Google Groups "pallet" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pallet-clj+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Hunter Hutchinson

http://about.me/huntar

[Bruce Durling]

I hope it was helpful. I should update it, but if someone else updates
it before I get a chance, I'd happily accept a pull request.

cheers,
Bruce

@otfrom | CTO & co-founder @MastodonC | mastodonc.com
See recent coverage of us in the Economist http://econ.st/WeTd2i and
the Financial Times http://on.ft.com/T154BA

[Jon Crosby]

Thank you for the references.

For the first part of the tutorial, I would like to address the process of VM creation. Unless I am missing something, the methods to automate this are outside of the current pallet and vmfest scopes. Tools I am familiar with are veewee [1] and vboxn [2]. Both of them introduce new dependencies (ruby and python, respectively). I attempted to embed JRuby and use veewee transparently but veewee seems to have issues running on JRuby specifically.

I am now considering the following options for the tutorial, open for feedback on these or corrections related to the above:

1. Document as-is with veewee.

2. Document as-is with vboxn.

3. Use the bash script in vboxn [3] for reference and contribute to vmfest, wrapping with lein tasks.

What do you think? Go with 2, then work on 3 once the docs are out? Or is 3 a large enough pain point that it should be addressed first?

-Jon

[1] https://github.com/jedi4ever/veewee

[2] https://github.com/podados/vboxn

[3] https://github.com/podados/vboxn/blob/master/scripts/vboxn-init

[Antoni Batchelli]

Hi Jon,

Visaje is our solution, but it needs some love. For now it works for Debian/Ubuntu images: https://github.com/palletops/visaje/blob/feature/hook-web-server/src/visaje/core.clj#L183

I have no opinion on the other solutions, but know that the latest version of pallet-vmfest can use any vagrant box as image. This feature (vagrant boxes) is still in alpha, but so far so good. 

Toni.

[Jon Crosby]

On Tuesday, February 19, 2013 7:49:04 AM UTC-8, tbatchelli wrote:

Hi Jon,

Visaje is our solution, but it needs some love. For now it works for Debian/Ubuntu images: https://github.com/palletops/visaje/blob/feature/hook-web-server/src/visaje/core.clj#L183

Excellent! I will direct my efforts there. Thank you!

[Jon Crosby]

On Tuesday, February 19, 2013 8:05:00 AM UTC-8, Jon Crosby wrote:

On Tuesday, February 19, 2013 7:49:04 AM UTC-8, tbatchelli wrote:

Hi Jon,

Visaje is our solution, but it needs some love. For now it works for Debian/Ubuntu images: https://github.com/palletops/visaje/blob/feature/hook-web-server/src/visaje/core.clj#L183

Excellent! I will direct my efforts there. Thank you!

I have run into several issues getting off the ground with visaje and have begun a fork and series of pull requests. Part of the complexity seems related to artifacts/namespaces changing and versions across the chain of visaje, vmfest, pallet-vmfest, pallet-lein, and pallet undergoing some iteration. I plan to have a solid project.clj example in the tutorial to address this.

So as not to block progress on this end-to-end tutorial, I went ahead and used veewee locally to automate the creation of VM images. How does this look for a first cut outline of the tutorial?

1. Requirements
  Mac OS X (so that xpcom works)
2. Prepare your toolbox
  homebrew
  leiningen
  virtualbox
  ruby (temporary, for veewee)
3. Build an app
  compojure hello-world
4. Deploy to a local, virtual cloud
  set ruby version for project
  add Gemfile with veewee
  choose OS template
  build base OS image
  pallet project-init
  add vmfest provider
  run/test
5. Re-target the deployment for AWS
  add AWS credentials/provider
  deploy/test
6. Clean up
  shutdown AWS instance

[tbatchelli]

Hi Jon,

Sorry for the trouble with visaje, I'll look into it and update the documentation and dependencies. Glad to hear to hear veewee is working fine though!

I am concerned that with this plan might lead people to think that they need to build an image before they can use pallet, which is not the case, and we should be extra careful since this is a tutorial and tutorials are usually the first point of contact of users. The same way you don't need to build an image in AWS before you can use EC2, you don't need to build an vbox image before you can use pallet-vmfest. This is the reason why we provide our own vmfest images and we recently added support for Vagrant boxes.

I also want to clarify that pallet-vmfest will work on OSs other than OSX. As you said, xpcom is not working on some linux distros, but xpcom is only a convenient option and not a requirement, as vmfest 0.3.0 also supports the web services interface to vbox, currently working for all linux distros. The only drawback with the web services option is that it requires a one-time configuration of the the authentication method with the server.

So here is what I would suggest:

1- Requirements

     OSX

 2- Toolbox

     homebrew

     lein

     virtualbox

3 - Build app

     compojure hello world

4- Deploy to local virtual cloud

     pallet project init

     add vmfest provider

     download and install image

     run test

5 - Retarget deployment for AWS

     add AWS credentials/ provider

     deploy/test

6- Clean up

    shutdown AWS instance

8- Running local cloud on linux

     setup the VBox Server

     starting the VBox Server

7- Building your own local images

     set ruby version for project

     add Gemfile with veewee

     choose OS template

     build base OS image

Let me know what you think of this approach. If you need some real-time chat, you can find us on the #pallet IRC channel at freenode.net too.

Toni.

[Jon Crosby]

Thank you for the solid feedback. I like your ordering and the reasoning behind it. Having less work up front is better for a tutorial. At this point, I can reorder some of my notes and start sharing a document for more detailed feedback as it progresses.

-Jon