Streaming in Thin server

[Sagar Ranglani]

I tried the following rails code...

 self.response.headers["Cache-Control"] ||= "no-cache"
 self.response.headers["Transfer-Encoding"] = "chunked"
 self.response.headers['Last-Modified'] = Time.now.ctime.to_s
 self.response_body =  Rack::Chunked::Body.new(Enumerator.new do |y|
   10.times do

     sleep 1
     y << "Hello World\n"
   end
 end)

This works great in Unicron server but can't stream using Thin server. I tried 2.0.0.pre too, this is not working in thin.

I tried the following rack code,

class DeferredBody
  def each(block)
    @server_block = block
  end

  def send(data)
    @server_block.call data
  end
end

class RackStreamApp
  def self.call(env)
    Thread.new do
      sleep 2  # simulate waiting for some event
      body = DeferredBody.new
      response = [200, {'Content-Type' => 'text/plain'}, body]
      env['async.callback'].call response

      body.send 'Hello, '
      sleep 2
      body.send 'World'
    end

    [-1, {}, []]  # or throw :async
  end
end

The above code streams "Hello, World" if we use Unicorn Server, but the code doesn't stream using Thin server ( I tried 2.0.0-pre)

Is there anything I can do to stream data using the thin server?

[Marc-André Cournoyer]

 self.response.headers["Cache-Control"] ||= "no-cache"
 self.response.headers["Transfer-Encoding"] = "chunked"
 self.response.headers['Last-Modified'] = Time.now.ctime.to_s
 self.response_body =  Rack::Chunked::Body.new(Enumerator.new do |y|
   10.times do

     sleep 1
     y << "Hello World\n"
   end
 end)

This works great in Unicron server but can't stream using Thin server. I tried 2.0.0.pre too, this is not working in thin.

Do not use sleep w/ Thin unless you run it in threaded mode. Or else, this will block the entire server.

Once again, this one is probably not working because of sleep.

Here's an example w/ Thin async response here: https://github.com/macournoyer/thin/blob/v2/examples/async.ru

--

M-A

http://macournoyer.com

[Eamonn Webster]

I'm also having problems streaming data. In my case I have no sleep I'm just reading the rows from a database.
The streaming is required for one action in one controller. So I don't see how I can apply the example you gave
using Thin::AsyncResponse.

The problem is that all the data is received at once at the end, i.e. it isn't streamed.
The code works fine with mongrel, but not with thin.

Should it work? Bottom line is that if thin doesn't support streaming, I'll have to solve the problem some other way.
But if it should work (without major surgery on the app) then great.

The following is the action from the controller.

  def export

    self.response.headers['Cache-Control'] = 'no-cache'
    self.response.headers['Transfer-Encoding'] = 'chunked'

    self.response.headers['Last-Modified'] = Time.now.rfc2822
    respond_to do |format|
      format.csv   {

        self.response_body = Rack::Chunked::Body.new(Enumerator.new do |y|

          User.find_each(batch_size: 25) do |u|
            y << "#{u.id},#{u.login}\n"
          end
        end)
      }
    end
  end

[Nathan Woodhull]

I'm also having trouble using the streaming support in rails 3.2 in combination with Thin. 

The following code works flawlessly in unicorn, but in thin it waits until the loop completes before returning any result to the browser. 

class StreamingController < ApplicationController

  def index

    self.response.headers['Last-Modified'] = Time.now.httpdate

    self.response_body = Enumerator.new do |y|

      i = 0

      while 1 < 1000000

        i = i +1

        y << "This is line #{i}\n"

      end

    end

  end

end

Should this work, or do I need to rewrite the code to be a rackup app using Thin::AsyncResponse? 

I thought that this was the entire point of the streaming responses feature that was first introduced in rails 3.1. I'd prefer not to tie things to Thin explicitly as using Thin::AsyncResponse would seem to do -- our app is open source and people may need to deploy it in other sorts of environments.

Nathan 

[dafoo]

I also had the same problem. In development, I use thin, but in production & qa, I run unicorn for the streaming.

[Marc-André Cournoyer]

Rails streaming was not build for event driven severs like Thin. If you're looking to build a high throughput streaming app I would recommend using Thin async feature: https://github.com/macournoyer/thin_async. It's perfect for this. In fact I would say this is what Thin is best at! It's pretty easy to use this in a custom middleware and hook it up inside Rails.

The reason it doesn't work is because the control is not given back to EventMachine after each chunk is returned. So basically what happens is that each chunk of the body is buffered by EventMachine because the event loop doesn't have a chance to send the data between each Enumerator yield.

I don't know of any simple solution that would not compromise the non blocking nature of Thin atm. But I'll give it a bit more thinking.

Probably Thin will need to ship with a few patches inside Rails for streaming to work in the future. But I have no plans at the moment to do this.

On Tue, Jan 29, 2013 at 2:15 AM, dafoo <foo...@gmail.com> wrote:

I also had the same problem. In development, I use thin, but in production & qa, I run unicorn for the streaming.

--
You received this message because you are subscribed to the Google Groups "thin-ruby" group.

To unsubscribe from this group and stop receiving emails from it, send an email to thin-ruby+...@googlegroups.com.

To post to this group, send email to thin...@googlegroups.com.

Visit this group at http://groups.google.com/group/thin-ruby?hl=en.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

M-A

http://macournoyer.com

[macournoyer]

Scratch that. I might have found a solution! I'll try to commit a fix later this week.

[Nathan Woodhull]

I'm curious -- what is your proposed solution?

[Nathan Woodhull]

I don't entirely understand thin & Event Machine's architecture, but is there a way to pass control back to thin's EM event loop from inside my controller code? Inside the enumerator generating the response you could imagine triggering something that would pass control back to Thin so that the data could be flushed to the browser?  Then control would in theory pass back to my controller code.

As I said, no idea if this is even plausible given that I don't understand Thin or EM internals (beyond the broadest possible event loop concept). 

Nathan 

[Marc-André Cournoyer]

Here's my commit (in a branch) to make streaming work: https://github.com/macournoyer/thin/commit/06cdd8777d5a5dc46adf9a31f8152effea50be78

You can compile and install thin from the streaming branch to get this now.

I haven't tested it much yet. But once I do, I'll merge in master.

--

You received this message because you are subscribed to the Google Groups "thin-ruby" group.
To unsubscribe from this group and stop receiving emails from it, send an email to thin-ruby+...@googlegroups.com.
To post to this group, send email to thin...@googlegroups.com.
Visit this group at http://groups.google.com/group/thin-ruby?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

M-A

http://macournoyer.com

[Eamonn Webster]

Hi,

Thanks for your commit. 

I tried it out but it gave me a error (see below). I get the feeling that the recursive locking might be a result of the method not found exception. 

Or perhaps caused by a clash with some of the other middleware in use.

I will try an dig into it and see why it can't find tick_loop. 

thanks again

eweb

Completed 200 OK in 3147ms (Views: 2053.1ms | ActiveRecord: 179.8ms | Sphinx: 0.0ms)

!! Unexpected error while processing request: undefined method `tick_loop' for EventMachine:Module

ThreadError: deadlock; recursive locking

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/lock.rb:14:in `lock'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/lock.rb:14:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/actionpack-3.1.10/lib/action_dispatch/middleware/static.rb:61:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/airbrake-3.1.2/lib/airbrake/rack.rb:42:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/airbrake-3.1.2/lib/airbrake/user_informer.rb:12:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/engine.rb:456:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/application.rb:143:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/railtie/configurable.rb:30:in `method_missing'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/urlmap.rb:52:in `block in call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/urlmap.rb:46:in `each'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/urlmap.rb:46:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/static.rb:53:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/showexceptions.rb:24:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/content_length.rb:14:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/rack/log_tailer.rb:14:in `call'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/connection.rb:81:in `block in pre_process'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/connection.rb:79:in `catch'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/connection.rb:79:in `pre_process'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/connection.rb:54:in `process'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/connection.rb:39:in `receive_data'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/backends/base.rb:63:in `start'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/bundler/gems/thin-06cdd8777d5a/lib/thin/server.rb:159:in `start'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/handler/thin.rb:13:in `run'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/rack-1.3.8/lib/rack/server.rb:268:in `start'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/commands/server.rb:70:in `start'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/commands.rb:54:in `block in <top (required)>'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/commands.rb:49:in `tap'

/Users/eweb/.rvm/gems/ruby-1.9.2-p320@qstream-ruby19/gems/railties-3.1.10/lib/rails/commands.rb:49:in `<top (required)>'

script/rails:6:in `require'

script/rails:6:in `<main>'

!! Unexpected error while processing request: undefined method `tick_loop' for EventMachine:Module

[Marc-André Cournoyer]

Try updating EventMachine: $ gem update eventmachine

[James Tucker]

Marc made it concurrent, but you don't have a concurrency safe stack in your Rack build.

Rack::Lock is not designed to be concurrent.

In the context in which this is operating, you're basically multi-threaded.

[James Tucker]

I think also the Body#close requirements may have additional issues with the new code, but I'm too tired to do a full review.

On Feb 1, 2013, at 1:46 AM, Eamonn Webster <eamonn....@gmail.com> wrote:

[Marc-André Cournoyer]

Ah you're right on both points. Body#close was not called on exception, that's an easy fix, but...

This is not gonna work anyways w/ Rack::Lock... deadlock happen on concurrent requests.

Damn... will try to figure out another solution.

[Marc-André Cournoyer]

Ah well... you could just remove Rack::Lock from the stack which is useless w/ Thin anyways.

This won't get in v1 then as it can't be enable by default w/o breaking stuff.

--

M-A

http://macournoyer.com

[James Tucker]

On Feb 2, 2013, at 10:46 AM, Marc-André Cournoyer <macou...@gmail.com> wrote:

Ah you're right on both points. Body#close was not called on exception, that's an easy fix, but...

This is not gonna work anyways w/ Rack::Lock... deadlock happen on concurrent requests.

Damn... will try to figure out another solution.

Happy to talk over proposals some time, but single threaded apps need to be single threaded. You could simply enqueue future accepted requests, but this is going to cause load balancer instability (well, that problem already exists, but it's much smaller). You could additionally add code to stop accepting and so on, but that's going to up your complexity a lot, and I'm probably not covering all the angles either.

[James Tucker]

On Feb 2, 2013, at 11:31 AM, Marc-André Cournoyer <macou...@gmail.com> wrote:

Ah well... you could just remove Rack::Lock from the stack which is useless w/ Thin anyways.

mmm, I'd be careful with that recommendation. Rack::Lock prevents concurrent execution when thin is in threaded mode, which is a good thing if that is what your app requires. In my experience, many users seem to think that eventmachine has some kind of magic concurrency model that just ushers away all need for concurrency control.

This won't get in v1 then as it can't be enable by default w/o breaking stuff.

Using fibers by default is fraught with problems generally. They interact badly with thread locals, and have limited size stacks (on MRI).

Have you considered using detach and attach APIs alongside threads?

[Marc-André Cournoyer]

mmm, I'd be careful with that recommendation. Rack::Lock prevents concurrent execution when thin is in threaded mode, which is a good thing if that is what your app requires. In my experience, many users seem to think that eventmachine has some kind of magic concurrency model that just ushers away all need for concurrency control.

Why would use threaded mode w/ Rack::Lock? Threaded mode only calls the app in a thread. That's like putting a mutex around the whole thread.

 

Using fibers by default is fraught with problems generally. They interact badly with thread locals, and have limited size stacks (on MRI).

The scope of the Fiber is very small in the commit I made, one yield in #each. I doubt that could make the 4kb stack explode.

MRI's Enumerator are also using fibers internally, my implementation is simply not using an exception to stop.

 

Have you considered using detach and attach APIs alongside threads?

But that would pull the socket out of the event loop and require blocking operations, right?

--

M-A

http://macournoyer.com

[James Tucker]

On Feb 3, 2013, at 6:14 PM, Marc-André Cournoyer <macou...@gmail.com> wrote:

mmm, I'd be careful with that recommendation. Rack::Lock prevents concurrent execution when thin is in threaded mode, which is a good thing if that is what your app requires. In my experience, many users seem to think that eventmachine has some kind of magic concurrency model that just ushers away all need for concurrency control.

Why would use threaded mode w/ Rack::Lock? Threaded mode only calls the app in a thread. That's like putting a mutex around the whole thread.

Correct. Most of the time it's a bad idea, however consider the following:

map 'http://api.whatever.com/' do

  run MySinatraAPI

end

map '/' do

  run Rails::Application

end

Using fibers by default is fraught with problems generally. They interact badly with thread locals, and have limited size stacks (on MRI).

The scope of the Fiber is very small in the commit I made, one yield in #each. I doubt that could make the 4kb stack explode.

Depends, what if someone passed a body that was actually an iterative wrapper around ERB, and that ERB had a variable binding that was something like unexecuted arel queries. The subsequent code execution as a result of lazy evaluation has undefined depth, quite likely plenty deep.

MRI's Enumerator are also using fibers internally, my implementation is simply not using an exception to stop.

Understood :-)

Have you considered using detach and attach APIs alongside threads?

But that would pull the socket out of the event loop and require blocking operations, right?

Correct, but that's exactly what we need, in the rails context, under Rack::Lock. A normal rails request blocks the reactor today anyway, that's exactly why streaming isn't working. You can't release the reactor unless you perform a soft-lock on the acceptor / receive_data (pipeline) calls - for concurrency unsafe apps at least. Please also note, that I'm saying concurrency unsafe, which is genuinely less well defined in community code than "thread safe".

[Marc-André Cournoyer]

Sorry for the late reply on this.

Why would use threaded mode w/ Rack::Lock? Threaded mode only calls the app in a thread. That's like putting a mutex around the whole thread.

Correct. Most of the time it's a bad idea, however consider the following:

map 'http://api.whatever.com/' do

  run MySinatraAPI

end

map '/' do

  run Rails::Application

end

I'm playing around, refactoring some features (like threading and streaming) inside middlewares to make them optional. So you could do

map '/my_long_and_slow_request' do

  use Thin::Threaded

end

map '/stream' do

  use Thin::Streamed

end

I think that would solve the problem. What do you think?

Correct, but that's exactly what we need, in the rails context, under Rack::Lock. A normal rails request blocks the reactor today anyway, that's exactly why streaming isn't working. You can't release the reactor unless you perform a soft-lock on the acceptor / receive_data (pipeline) calls - for concurrency unsafe apps at least. Please also note, that I'm saying concurrency unsafe, which is genuinely less well defined in community code than "thread safe".

Right. I agree this would work and make it do what people expect it to do (send data right away). But one of the big advantage I see w/ streaming on Thin is that other stuff (receiving, sending data) could append while you're streaming the body. And that would just kill that feature.

I think because of that it would make sense to say streaming will only work if your body#each code is thread safe. But has to be optional for sure.

[James Tucker]

Ok, so it only works under threaded mode? I think that's ok.

If it is to work in single-threaded mode, then it's not ok, "fiber safe" is not the same as "thread safe", if you're thinking about cooperative stack concurrency models (unfortunately).

[Robert Poor]

All:

I'm late to the party here, but I'm trying to get `thin start --threaded` to serve up server side events.  When run in --trace mode, I can see periodic messages being sent out, but the client (in this case, curl --no-buffer locahost:3000/...) receives them all at once when the connection is closed.  I'm not sure if this is an issue with my thin configuration or something else.

I'm using a 'modern' rails (4.1.8) and thin (1.6.3).

Full details at http://stackoverflow.com/questions/27464994/how-to-get-rails-4-actioncontrollerlive-streaming-working-with-thin-and-ruby-2

Any pointers welcome.

[Robert Poor]

Here's what `rake about` has to say about my middleware:

Middleware                Rack::Sendfile, ActionDispatch::Static, Rack::Lock, #<ActiveSupport::Cache::Strategy::LocalCache::Middleware:0x007febbd6ba458>, Rack::Runtime, Rack::MethodOverride, ActionDispatch::RequestId, Rails::Rack::Logger, ActionDispatch::ShowExceptions, ActionDispatch::DebugExceptions, ActionDispatch::RemoteIp, ActionDispatch::Reloader, ActionDispatch::Callbacks, ActiveRecord::Migration::CheckPending, ActiveRecord::ConnectionAdapters::ConnectionManagement, ActiveRecord::QueryCache, ActionDispatch::Cookies, ActionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDispatch::ParamsParser, Rack::Head, Rack::ConditionalGet, Rack::ETag

I'm not yet fluent in middleware, so what sane techniques exist for tracing the lifecycle of a message through the stack?  For example, is there a way to turn on tracing?  

On Mon Dec 15 2014 at 11:02:58 AM James Tucker <jftu...@gmail.com> wrote:

Probably somethign in your middleware stack is buffering the whole response.

[Robert Poor]

Let me try to answer my own question, but if I'm headed for the rocks, I hope someone will chime in!

I can create a debugging middleware object that simply prints outgoing messages as they are received, and then insert this object in various points in the rack stack.  If I find them being printed periodically at one level and not at a lower level, then I can pinpoint which rack module is doing the buffering.

[Robert Poor]

Naive question: is ActionController::Live necessary to get SSE?  

Better question: Are there any "best practice" examples of using thin for SSE?

On Wed Dec 17 2014 at 1:49:59 PM James Tucker <jftu...@gmail.com> wrote:

SSE should be possible without using hijack or the throw hack, but if I remember correctly ActionController::Live is using a weird approach at hijack, so yes, it bypasses the stack on the way back. See rack/SPEC for the hijack spec. Unfortunately, when I say AC::Live is weird, I mean it, so you're going to have to go digging into that code to actually understand it. I almost rewrote it for someone once, because they had some problems, but instead I just gave them a small standalone rack app, as it was far far less effort.

On Wed Dec 17 2014 at 4:09:37 PM Robert Poor <rdp...@gmail.com> wrote:

I created a "RackSniffer" module that simply prints messages to $stdout before passing them to the next level.  I inserted them at the very top and the very bottom of Rack.  None of the sniffers reported any activity until the stream was closed.  

Does this mean that thin bypasses rack somehow?  If not, where the heck are the messages getting buffered?

FWIW, here's my stack with the RackSniffers installed:

$ rake middleware

use RackSniffer

use Rack::Sendfile

use RackSniffer

use ActionDispatch::Static

use Rack::Lock

use #<ActiveSupport::Cache::Strategy::LocalCache::Middleware:0x007ff4ca36ada0>

use Rack::Runtime

use Rack::MethodOverride

use ActionDispatch::RequestId

use Rails::Rack::Logger

use ActionDispatch::ShowExceptions

use ActionDispatch::DebugExceptions

use ActionDispatch::RemoteIp

use ActionDispatch::Reloader

use ActionDispatch::Callbacks

use ActiveRecord::Migration::CheckPending

use ActiveRecord::ConnectionAdapters::ConnectionManagement

use ActiveRecord::QueryCache

use ActionDispatch::Cookies

use ActionDispatch::Session::CookieStore

use ActionDispatch::Flash

use ActionDispatch::ParamsParser

use Rack::Head

use Rack::ConditionalGet

use Rack::ETag

use RackSniffer

run ServerSideEvents::Application.routes