Merb Scoop

[Bill]

What's the skinny on Merb? Is it all about minimalism and
composability (e.g. choice of ORM module)?

Is there an interesting political backstory?

[Sam Livingston-Gray]

There's a backstory, but it's not especially political -- merb started
out as a supplemental thing that would let multiple users upload files
without tying up an entire Rails process for each one. Then it got
bigger. Basically, it's Less-Opinionated Software.

I'm definitely watching for the 1.0 release.

-Sam

[Bill Burcham]

I see Sam. That helped - I googled for "merb less opinionated" and found the first four hits illuminating.

Aside: when new bamboo says that "Merb…is thread safe" is that pertinent only when running on JRuby?

[Igal Koshevoy]

If someone wants to give a talk on Merb at the next meeting, that'd be
great.

As I understand it, it's advantages over Rails are that it's
multi-threaded (MRI works, JRuby work is in progress), lighter weight
(uses half the memory for a single instance, and additional threads
weigh nothing rather than requiring you to start another process), is
faster and more responsive, and lets you easily choose different ORM and
view technologies.

The disadvantages of Merb seem to be are that there is no usable
official documentation, it lacks many convenience features, and it
requires developers using it to understand it at a much deeper
implementation level to be able to do useful work.

I've poked at it a few times in the past, but the documentation was
little more than a few terse blog entries, there were no complete
examples, and the official site still has absolutely no actual
documentation -- their rdoc is useless unless you already know where to
use, say, a deep_array_node call. However, the architecture has mostly
stabilized and there's clearly effort being made to explain how Merb works.

Here are some promising Merb tutorials on my to-read list for those
interested:

"Life On The Edge With Merb, DataMapper & RSpec"
http://merb.4ninjas.org/
"Meet Merb" [It's $9, but I've always been happy with my PeepCode purchases]
http://peepcode.com/products/meet-merb-pdf-draft
"Slapp: A simple chat wall Merb tutorial."
http://www.socialface.com/slapp/

-igal

[Bill Burcham]

All the stuff you point out Igal, plus the fact that from a feature standpoint I see Merb retreading so much of Rails e.g. when you look at The Book and see this: 

has_many :through is in the pipes, but…

is why I wonder about the backstory. Somebody is putting many hours into this and at the same time giving up (for the time being) lots of depth, ostensibly in favor of efficiency.

[Sam Livingston-Gray]

Let's not confuse DataMapper (which is what the above quote applies
to) and Merb. Merb can also use ActiveRecord.

[Bill Burcham]

Rightiyo Sam. But I imagine all the candidate modules are in similar shape yes?

Maybe there's something else. That Merb Book mentions that:

Merb consists of a number of gems: merb-core, merb-more and merb-plugins. This means that it is possible to pick and choose the functionality you need, instead of cluttering up the framework with non-essential features

But rails consists of a set of gems right? And I can use ActionView without ActiveRecord right? Or is it the case that for Rails, while it's theoretically possible to do that it's not practical? DataMapper started life as an ActiveRecord replacement as I recall which implies that Rails is capable (or at least Sam thought it was capable) of this sort of pluggability.

So wonder if what's attracting folks to Merb is that it's a better ActionController+ActionView (the "core" of the appserver?) than Rails is?

[Igal Koshevoy]

Bill Burcham wrote:
> So wonder if what's attracting folks to Merb is that it's a better
> ActionController+ActionView (the "core" of the appserver?) than Rails is?

I don't think so, especially since Merb lacks many features of Rails and
likely always will.

People pick Merb because a single Merb process consumes half the memory
of a Rails process and is able to handle many simultaneous clients with
that one Merb process, whereas you must have one 60MB+ Rails process for
every single concurrent client.

Consider uploads (which Sam mentioned earlier). Let's say you have 4
Rails processes. If four people decide to upload a file at the same
time, then your website is effectively offline until one of those
uploads finishes -- each upload takes the total attention of a single
Rails process. In contrast, Merb can handle many uploads at once, and
your site also stays online for other users.

Many large Rails sites offload their uploads to Merb, while using Rails
and its many features for serving the main site. Merb is also able to
serve more requests per second, so some large Rails sites use Merb to
serve their most-heavily hit pages.

A note about threading: MRI supports green threads, whereas JRuby makes
use of Java's processor threads. Merb is still useful on MRI, but it's
benefit is more about making use of otherwise idle processes on a single
CPU core rather than distributing processes efficiently across many
cores. So if you're using MRI on a multicore system, you'll want to
start one Merb process per core.

-igal

[Bill Burcham]

So it sounds like it's all about that considerable efficiency gain. It's:

• Merb: many clients per 30MB process
  
• Rails: one client per 60MB process
  

For large values of "many" that's pretty important for lots of apps huh.

[rco...@gmail.com]

Running a Merb process per core will only help if you're actually consuming 100% of the CPU on one core. In cases where network I/O is actually the bottleneck, you may never peg even one core.

Because Ruby IO#read/write automagically turned into select() calls under the hood, you can get fairly good multiplexing of network I/O even within a single native thread.

Personally, though, I find Merb appealing simply because the core is compact enough to be easy to audit and understand as compared to Rails. Web development is only about 25% of what I do, which means that Rails is just too big and too chaotic for me to stay up to date.

That being said, Merb isn't quit there yet, as far as I know. After the 1.0, though, I expect I'll be spending a bit of quality time with at least the core modules.

-Lennon

Sent via BlackBerry from T-Mobile

-----Original Message-----
From: Igal Koshevoy <ig...@pragmaticraft.com>

Date: Wed, 14 May 2008 17:36:06
To:pdx...@googlegroups.com
Subject: [pdxruby] Re: Merb Scoop

[Ammon Christiansen]

In the discussion of file uploads being Merb's strength, I'd like to point out a couple of things.

 

If you use a front end web server like Apache or Nginx that buffers the file upload to memory or disk, depending on the size, until the upload is complete, rails or merb wouldn't get involved at all until the upload was complete. 

 

If you don't have a front end web server, yes, the rails processes will block on the upload.

 

I think that Merb doesn't block on the file upload because it can spawn a thread to handle the upload (and another one to handle the upload progress). 

 

The real difference is if you want to use AJAX to update the page state with the upload progress.  You would need an entirely different rails process to answer the upload status.  And if someone else decided to upload a file at the same time, then you need yet 2 more rails instances.
 

Does rails even give incremental status on file uploads or does actioncontroller not even get called until the file upload was complete?
 

[Dan Herrera]

One of mere's strengths is that a controller action can call render,
but continue to process after the response has been sent to the
client.

In the case of file uploads, you could render a response to the user
that their file has been uploaded and continue processing the
controller action to thumbnail the file.

If I recall correctly, in a rails controller action that thumbnails an
uploaded image, that process will be blocked while until the action
completes.

Dan

[Igal Koshevoy]

Ammon Christiansen wrote:
> If you use a front end web server like Apache or Nginx that buffers
> the file upload to memory or disk, depending on the size, until the
> upload is complete, rails or merb wouldn't get involved at all until
> the upload was complete.

Thanks for the clarification. I'd assumed nginx was behaving as a dumb
proxy, like pen or pound, which I'd spent more time fussing with before
switching over to nginx.

Here's Zed Shaw's 2 cents on the matter:
> nginx does buffer the whole request body before sending it to the
> backends. This is why pound, pen, or balance works but nginx doesn't
> when you do upload progress.

Also, here's a link with more discussion on how to monitor an upload's
progress using nginx and a Merb controller and jQuery AJAX client for
monitoring progress:

http://blog.new-bamboo.co.uk/2007/11/23/upload-progress-with-nginx

Although their sample code seems fairly straightforward, is anyone using
a good plugin or such that wraps this all up to provide an upload
progress monitor?

-igal

[Igal Koshevoy]

Dan Herrera wrote:
> One of mere's strengths is that a controller action can call render,
> but continue to process after the response has been sent to the
> client.
>
> In the case of file uploads, you could render a response to the user
> that their file has been uploaded and continue processing the
> controller action to thumbnail the file.
>
> If I recall correctly, in a rails controller action that thumbnails an
> uploaded image, that process will be blocked while until the action
> completes.
>

To the best of my knowledge, Rails blocks until it finishes rendering an
action. You can probably hack something by monkey patching
ActiveController, but yuck.

I've used the following strategies to execute long-running asynchronous
tasks:
(1) Invoke a method that forks, reconnects to the database, runs the
task, and exits to kill the forked child process. Pros: Easy to write
and manage because there's no need for to manually launch a separate
process to run tasks, since the processes are forked from your existing
Rails processes. Cons: Ugly, and you can end up with many forked
children running at the same time.
(2) Offload tasks to a queue or DRb server. Pros: Gives you total
control over how of how you run tasks and many resources you use, e.g.
number of workers, whether to use the Rails environment, etc. Cons: More
complicated and you have to launch/manage a separate process that'll run
tasks in the queue or answer DRb requests.

-igal

[jason_watkins]

I haven't used it, but I've heard that http://wiki.codemongers.com/NginxHttpUploadProgressModule
is generally better if upload progress is your only concern.

[jason_watkins]

> (1) Invoke a method that forks, reconnects to the database, runs the

One thing to be aware of is that on most OS's if the GC fires it'll
trigger a copying of all the memory pages. If you're ram limited and
get several long running forks at once that can push you into swapping
or other bad things.

For simple image resizing I've found that shelling out a background
convert/mogrify command works best.

[Igal Koshevoy]

jason_watkins wrote:

(1) Invoke a method that forks, reconnects to the database, runs the
    

One thing to be aware of is that on most OS's if the GC fires it'll
trigger a copying of all the memory pages. If you're ram limited and
get several long running forks at once that can push you into swapping
or other bad things.
  

Agreed, that approach is awful for the reasons you mentioned. However, the approach has some appeal because it's stupid simple, you get to keep your entire Rails environment with you, and there's no need to manage additional non-Rails servers to handle the queue.

For simple image resizing I've found that shelling out a background
convert/mogrify command works best.

This works well for a fair number of tasks, and we're using this approach on some sites to execute the UNIX copy of imagemagick because it's so much quicker and less fussy than any Ruby library.

However, this approach has its problems too. For instance, if a bunch of people upload images at the same time, the system might choke because it'd try to convert them all at once. Or if the jobs take a long time and the machine reboots, you'll probably lose all the jobs the machine was working on when it failed.

Queue-based solutions are great because they allow full control over how things are run and can easily track completion, but this all comes at the cost of additional complexity.

Anyway, the right solution varies quite a bit based on the task and available resources.

-igal