If so, should I do anything to enable it?
I tried this gem just after posting this question but realized it doesn't work transparently, specially on JRuby. I'll check the other options I missed in the README, thanks!
--
You received this message because you are subscribed to a topic in the Google Groups "Roda" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ruby-roda/tLwAGEzH0Pc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ruby-roda+...@googlegroups.com.
To post to this group, send email to ruby...@googlegroups.com.
Visit this group at https://groups.google.com/group/ruby-roda.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-roda/3f9d817c-1f56-48ec-bf3e-c871ccba133a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "Roda" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ruby-roda+...@googlegroups.com.
To post to this group, send email to ruby...@googlegroups.com.
Visit this group at https://groups.google.com/group/ruby-roda.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-roda/CAGmv%2BwKovBXTnQT9xsqknNe%3D-_KFfZ_J_Vmov15ZqjnumhEQhA%40mail.gmail.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ruby-roda/6DFFF666-8754-4766-83FE-024406D2795A%40gmail.com.
Em 13-07-2016 21:01, Jeremy Evans escreveu:
On Wednesday, July 13, 2016 at 2:47:02 PM UTC-7, Rodrigo Rosenfeld Rosas wrote:If so, should I do anything to enable it?
Nothing ships with Roda itself, but Roda works with a number of reloaders: http://roda.jeremyevans.net/rdoc/files/README_rdoc.html#label-Reloading
If you want an example of reloading using Roda and Sequel with rack-unreloader for reloading: https://github.com/jeremyevans/roda-sequel-stack
Thanks,Jeremy
Hi Jeremy,
Please take my comments here not as complaints but as a review from what I've found so far after looking at the list of reloaders in that README.
First, I believe code reloading is a very important feature for any framework running on whatever language, even though it's often bugged when handling some edge cases or have their caveats (I remember in Java one would have to restart the application eventually after reload happening many times due to memory leaks).
Also, I recognize that even though it seems to just work and work great in Rails getting it implemented correctly seems to be pretty hard and Ruby doesn't seem to be an exception. I haven't digged on how ActiveSupport::Dependencies handle it yet, but I'd probably use it if it didn't monkey patched some Ruby core classes.
But I think we can't simply avoid talking about it just because it's hard. Even if the reloader can't support some edge situations, it's preferred to have a somewhat broken reloader than not having a reloader at all while in the development environment. But Roda seems to give reloading little relevance since it simply says "hey, if you want auto-reloading you can use any reloader that works with Rack, so we don't need to provide one" and give a list of several reloaders one would be able to choose. And that's it, next topic. I find this topic should be better covered in the README and I can try to help expanding this section to better explain the current state of code reloading in Rack applications.
So, let's please discuss the issues with each option in that README:
rack-unreloader
It doesn't work transparently on JRuby before version 9.1.
Also, with the current implementation one has to require all app files with Unreloader.require(path) before running Unreloader. It would be good if it was possible to set up Unreloader mechanism, like it's possible with ActiveSupport::Dependencies.mechanism = :require (or :load). It could be any API, as long as it would be possible to make Unreloader.require be implemented as a regular require for test and production environments. This way, we wouldn't have to require all dependencies upfront before running the Rack application and would instead use Unreloader.require on each file just like we would call require_dependency with ActiveSupport.
It's a complex implementation which seems to try to be very efficent on what it does by tracking which files have been loaded. However this may not always be desired, specially if we are not interested in reloading external gem codes or stdlib libraries like JSON and so on. It would be good if we
had the option to specify a list of globs specifying which files should be reloaded when changed in a similar way ActiveSupport::Dependencies handles it.
On the other side, there are times where it is helpful to reload such libraries, in case you want to debug some gem code and it would be great to keep allowing such feature, which is something ActiveSupport::Dependencies doesn't support as far as I know.
rack-unreloader seems to be the best option among those listed in the README and should be probably the recommended option, while explaining its caveats. It's also very complex, so I guess it may be possible for an application to be bitten by some bugs related to the autoreloading implementation in this gem.
Also, it could possibly perform better if it used some evented file watcher rather than scanning all loaded files to see whether they changed or not. This could be noticeable I guess if you load several gems from app.rb and not before setting up Unreloader. A file watcher in specific directories would be probably faster.
Maybe it would be a good idea to advise developers to not change any sources while a request is in progress, specially in a multi-threaded environment, even though I don't think this is a common practice ;)
Conclusion:
Currently, for someone giving a glance on Roda's README it makes one think that auto-reloading is not an issue with Roda at all since there are 5 available options one could choose, as if all of them would actually suite the development of big applications. Specially if someone is coming from Rails which provides a great auto-reloader, this is probably something that should be better explained in that section.
We should probably better describe the options so that each developer interested in Roda, like me, wouldn't have to do that work and read each source to try to figure out what they do and what they don't. I also think that Roda should actively recommend rack-unloader as it seems like the only option which is really suitable for large code bases. It also supports MRI, JRuby 9.1+ and Windows I guess.
I'll give rack-unloader a try as I'm starting to port a big Rails app to Roda and I intend to redirect some routes to the Roda app and only fully replace Rails with Roda once everything is ported. If I find any bigger issues with rack-unloader in the process I may consider trying to create yet another reloader, that would work more similar to ActiveSupport::Dependencies, without the monkey patches and using an evented file watcher if possible. But I think it's a better idea to give rack-unloader a shot first.
I've ported successfully a JRuby on Rails application to Roda yesterday so that it could give me an idea on how it feels and I liked it. It was a simple application with just two actions exporting some data to Excel (old XLS format) using Apache POI, one of them using streaming. It was pretty forward to replace that app with Roda since most of the work for sharing the session with the main app has been done 2 days ago, when I worked on this gem I released yesterday morning:
https://github.com/rosenfeld/rails_compatible_cookies_utils
Both actions were GET ones so I didn't have to worry about CSRF and implementing a compatible protection mechanism. But in case someone is interested in sharing some encrypted or signed cookies values with a Rails app running in another process, they might be interested in this gem. Once I start moving some parts of the main MRI app to Roda I may add some support for forgery protection that would be compatible with how Rails handles them.
So far, I'm excited about Roda and I hope I'll be able to completely replace Rails with Roda at some point.
On Thursday, July 14, 2016 at 8:53:18 AM UTC-7, Rodrigo Rosenfeld Rosas wrote:Em 13-07-2016 21:01, Jeremy Evans escreveu:
On Wednesday, July 13, 2016 at 2:47:02 PM UTC-7, Rodrigo Rosenfeld Rosas wrote:If so, should I do anything to enable it?
Nothing ships with Roda itself, but Roda works with a number of reloaders: http://roda.jeremyevans.net/rdoc/files/README_rdoc.html#label-Reloading
If you want an example of reloading using Roda and Sequel with rack-unreloader for reloading: https://github.com/jeremyevans/roda-sequel-stack
Thanks,Jeremy
Hi Jeremy,
Please take my comments here not as complaints but as a review from what I've found so far after looking at the list of reloaders in that README.
Rodrigo,
First, thank you for taking the time to write a detailed response going over the various reloading options and their issues.First, I believe code reloading is a very important feature for any framework running on whatever language, even though it's often bugged when handling some edge cases or have their caveats (I remember in Java one would have to restart the application eventually after reload happening many times due to memory leaks).
I agree that automatic code reloading is useful. That being said, when I used Sinatra I never used a reloader, just manually restarted the application as necessary, and I didn't find it arduous. Automatic code reloading is nicer in my experience, but not strict requirement for me. One issue with automatic code reloading is it brings with it the tendency to throw code at a wall and see what sticks, as opposed to trying to understand what is actually happening. That being said, I use automatic code reloading in most of my production apps.
Also, I recognize that even though it seems to just work and work great in Rails getting it implemented correctly seems to be pretty hard and Ruby doesn't seem to be an exception. I haven't digged on how ActiveSupport::Dependencies handle it yet, but I'd probably use it if it didn't monkey patched some Ruby core classes.
Correct, automatic code reloading is not an easy thing to implement correctly. I don't think I would have tried to implement it from scratch, rack-unreloader was actually forked from Padrino's reloader.
ActiveSupport::Dependencies is a good solution if you buy into The Rails Way (monkey patch all the things!). It works by overriding const_missing and require, which is pretty heavy handed if you aren't using Rails. I don't think there is an approach that will give you the same result as ActiveSupport::Dependencies without monkey patching const_missing and require.
But I think we can't simply avoid talking about it just because it's hard. Even if the reloader can't support some edge situations, it's preferred to have a somewhat broken reloader than not having a reloader at all while in the development environment. But Roda seems to give reloading little relevance since it simply says "hey, if you want auto-reloading you can use any reloader that works with Rack, so we don't need to provide one" and give a list of several reloaders one would be able to choose. And that's it, next topic. I find this topic should be better covered in the README and I can try to help expanding this section to better explain the current state of code reloading in Rack applications.
It's not so much that Roda gives little relevance to the issue, it's just not a problem that Roda itself tries to solve, so Roda just gives pointers to possible options. Roda is not a full stack framework, it's a toolkit/library. Setting up reloading automatically would mean specifying where things go (routes, models, etc.), which is something that should be specified at the framework level.
I disagree that it is Roda's place to describe the ruby reloading ecosystem. Roda lists rack-unreloader first, but that's as far as it goes in terms of recommendations. For one thing, as the maintainer of both Roda and rack-unreloader, I'm biased and I don't think it's fair for me to make an explicit recommendation.
So, let's please discuss the issues with each option in that README:
rack-unreloader
It doesn't work transparently on JRuby before version 9.1.
Also, with the current implementation one has to require all app files with Unreloader.require(path) before running Unreloader. It would be good if it was possible to set up Unreloader mechanism, like it's possible with ActiveSupport::Dependencies.mechanism = :require (or :load). It could be any API, as long as it would be possible to make Unreloader.require be implemented as a regular require for test and production environments. This way, we wouldn't have to require all dependencies upfront before running the Rack application and would instead use Unreloader.require on each file just like we would call require_dependency with ActiveSupport.
I don't think it's possible for reloading to work transparently with no code changes without monkey patching. Until Ruby itself offers such a feature, you'll have to choose a transparent/monkey patching approach or an explicit/no monkey patching approach.
It's a complex implementation which seems to try to be very efficent on what it does by tracking which files have been loaded. However this may not always be desired, specially if we are not interested in reloading external gem codes or stdlib libraries like JSON and so on. It would be good if we
had the option to specify a list of globs specifying which files should be reloaded when changed in a similar way ActiveSupport::Dependencies handles it.
I believe you can use rack-unreloader to specify which files should be reloaded, even if they are in gems or stdlib. You just provide a directory to rack-unreloader and it will handle all ruby files in all subdirectories.
On the other side, there are times where it is helpful to reload such libraries, in case you want to debug some gem code and it would be great to keep allowing such feature, which is something ActiveSupport::Dependencies doesn't support as far as I know.
Correct. rack-unreloader also supports other advanced features that I'm not sure ActiveSupport::Dependencies handles, such as classes split across multiple files, but my knowledge of ActiveSupport::Dependencies is not perfect, so I could be wrong..rack-unreloader seems to be the best option among those listed in the README and should be probably the recommended option, while explaining its caveats. It's also very complex, so I guess it may be possible for an application to be bitten by some bugs related to the autoreloading implementation in this gem.
I mentioned above why I don't want to give an explicit recommendation. rack-unreloader's README mentions caveats when using it. To be fair, I believe many of the issues are endemic to any similar approach to reloading in ruby.Also, it could possibly perform better if it used some evented file watcher rather than scanning all loaded files to see whether they changed or not. This could be noticeable I guess if you load several gems from app.rb and not before setting up Unreloader. A file watcher in specific directories would be probably faster.
It's certainly possible this could speed up large apps. However, I haven't needed it even in my largest app, so it's not something I've worked on (my largest app is only medium sized, 35 routing subtrees and 118 models). Evented file watchers tend to be operating system specific too, and all the world is not Linux and Mac OS X. That being said, it's certainly something I'd consider if submitted via a pull request as an optional feature.
Maybe it would be a good idea to advise developers to not change any sources while a request is in progress, specially in a multi-threaded environment, even though I don't think this is a common practice ;)
This probably isn't a big deal, as editors are generally going to write out the entire file, and a partially written file would just result in a SyntaxError.
Conclusion:
Currently, for someone giving a glance on Roda's README it makes one think that auto-reloading is not an issue with Roda at all since there are 5 available options one could choose, as if all of them would actually suite the development of big applications. Specially if someone is coming from Rails which provides a great auto-reloader, this is probably something that should be better explained in that section.
Roda makes no indication that any of the reloading options will suit your needs, it just mentions the options available. It's up to the user to do their own due diligence and evaluate the options.
We should probably better describe the options so that each developer interested in Roda, like me, wouldn't have to do that work and read each source to try to figure out what they do and what they don't. I also think that Roda should actively recommend rack-unloader as it seems like the only option which is really suitable for large code bases. It also supports MRI, JRuby 9.1+ and Windows I guess.
Just because rack-unreloader is the best solution for your code bases and mine does not necessarily make it the best solution for all code bases. There are a lot of environments where a forking reloader may be the best solution. For one, forking reloaders offer what appears to be transparent reloading without any monkey patching. As long as your app loads quickly and uses MRI, they can be a good solution.
I'll give rack-unloader a try as I'm starting to port a big Rails app to Roda and I intend to redirect some routes to the Roda app and only fully replace Rails with Roda once everything is ported. If I find any bigger issues with rack-unloader in the process I may consider trying to create yet another reloader, that would work more similar to ActiveSupport::Dependencies, without the monkey patches and using an evented file watcher if possible. But I think it's a better idea to give rack-unloader a shot first.
I'm certainly open to patches/extensions to rack-unreloader, so if you like how it works in general but want to support additional features, open an issue/pull request in rack-unreloader and we can discuss the issue there.
I've ported successfully a JRuby on Rails application to Roda yesterday so that it could give me an idea on how it feels and I liked it. It was a simple application with just two actions exporting some data to Excel (old XLS format) using Apache POI, one of them using streaming. It was pretty forward to replace that app with Roda since most of the work for sharing the session with the main app has been done 2 days ago, when I worked on this gem I released yesterday morning:
https://github.com/rosenfeld/rails_compatible_cookies_utils
Both actions were GET ones so I didn't have to worry about CSRF and implementing a compatible protection mechanism. But in case someone is interested in sharing some encrypted or signed cookies values with a Rails app running in another process, they might be interested in this gem. Once I start moving some parts of the main MRI app to Roda I may add some support for forgery protection that would be compatible with how Rails handles them.
You may want to look at https://github.com/jeremyevans/roda-rails, as it provides some useful integration code until you can fully replace Rails with Roda.
So far, I'm excited about Roda and I hope I'll be able to completely replace Rails with Roda at some point.
That's great. As always, if you have any questions or need any help, please ask here.
Em 14-07-2016 15:30, Jeremy Evans escreveu:
ActiveSupport::Dependencies is a good solution if you buy into The Rails Way (monkey patch all the things!). It works by overriding const_missing and require, which is pretty heavy handed if you aren't using Rails. I don't think there is an approach that will give you the same result as ActiveSupport::Dependencies without monkey patching const_missing and require.
Actually const_missing is not really required to implement a reloader. Rails does that because it supports auto-loading based on the missing constant name, which I don't intend to support. In my Rails apps I always require my dependencies with require_dependency so I guess const_missing never reaches my app's code. I think patching require is necessary though, but only in development mode, which I think it should be fine for the benefits we get. ActiveSupport::Dependencies will also patch in production mode but it shouldn't be required.
It's not so much that Roda gives little relevance to the issue, it's just not a problem that Roda itself tries to solve, so Roda just gives pointers to possible options. Roda is not a full stack framework, it's a toolkit/library. Setting up reloading automatically would mean specifying where things go (routes, models, etc.), which is something that should be specified at the framework level.
That's not quite true. You can describe an auto-reloading mechanism and explain how to configure it to work with the user's app. No need to set any conventions for that.
I disagree that it is Roda's place to describe the ruby reloading ecosystem. Roda lists rack-unreloader first, but that's as far as it goes in terms of recommendations. For one thing, as the maintainer of both Roda and rack-unreloader, I'm biased and I don't think it's fair for me to make an explicit recommendation.
So, let me provide you some feedback from a real user of Roda. As a Roda user, I'd like to focus on creating web apps. Auto-reloading should simply work seamless in my opinion if I'm using some web framework no matter what language I use. I see lots of potential in Roda, but most people won't consider it as a serious development environment if they can't get auto-reloading to mostly work. At least this is my opinion as an advanced Ruby user but I guess regular or beginner Ruby users would think of this as a blocker issue. I think Roda should be more concerned about this. As you probably know, I'm a long time Sequel user and the main reason is that it just works. And it works great. It's expected that an ORM shouldn't be concerned about auto-reloading. But I think it's an expected feature for a web framework. The same way I evangelize for Sequel, I'd love to do the same for Roda, but I won't be able to until I figure out a fix for the auto-reloading problem. Of course, I don't want auto-reloading to work just to evangelize it but because I value it very much for my own usage.
So, let's please discuss the issues with each option in that README:
rack-unreloader
It doesn't work transparently on JRuby before version 9.1.
Also, with the current implementation one has to require all app files with Unreloader.require(path) before running Unreloader. It would be good if it was possible to set up Unreloader mechanism, like it's possible with ActiveSupport::Dependencies.mechanism = :require (or :load). It could be any API, as long as it would be possible to make Unreloader.require be implemented as a regular require for test and production environments. This way, we wouldn't have to require all dependencies upfront before running the Rack application and would instead use Unreloader.require on each file just like we would call require_dependency with ActiveSupport.
I don't think it's possible for reloading to work transparently with no code changes without monkey patching. Until Ruby itself offers such a feature, you'll have to choose a transparent/monkey patching approach or an explicit/no monkey patching approach.
Yes, I'm fine with basically monkey patching "require" just in development mode. Also, I think we could try to find other people interested in that feature (most of us maybe) and ask for such a feature that helps implementing auto-reloading in an efficient way in the Ruby-core mailing list or maybe we could create a new issue in Redmine there. It would be great if we could register some callback each time require is called that would provide us every new created constant both in the top level object as well as inside modules or classes.
I believe you can use rack-unreloader to specify which files should be reloaded, even if they are in gems or stdlib. You just provide a directory to rack-unreloader and it will handle all ruby files in all subdirectories.
I don't think I understood what you have suggested... Since you don't override require, if a watched file is changed but it requires another file outside of that directory, you'd try to unload constants defined by those files, right? This has the potential of causing issues with JRuby and imported Java libraries among others. My goal is to specify explicitly what is reloadable and what is not. Anything required with require is not reloadable for example. I can't implement that without overriding "require".
Maybe it would be a good idea to advise developers to not change any sources while a request is in progress, specially in a multi-threaded environment, even though I don't think this is a common practice ;)
This probably isn't a big deal, as editors are generally going to write out the entire file, and a partially written file would just result in a SyntaxError.
This is not what I meant. Requiring a different code in the middle of other request using the changed classes could cause some issues, although I agree this would be pretty rare and we shouldn't be concerned about it. But I find it good to know about it and avoid changing code while some request is still being handled or at least know that this could produce side effects on that request... Considering a threaded web server of course, like Puma.
Roda makes no indication that any of the reloading options will suit your needs, it just mentions the options available. It's up to the user to do their own due diligence and evaluate the options.
Yes, exactly, but that's really bad from a user's point of view in my opinion.
We should probably better describe the options so that each developer interested in Roda, like me, wouldn't have to do that work and read each source to try to figure out what they do and what they don't. I also think that Roda should actively recommend rack-unloader as it seems like the only option which is really suitable for large code bases. It also supports MRI, JRuby 9.1+ and Windows I guess.
Just because rack-unreloader is the best solution for your code bases and mine does not necessarily make it the best solution for all code bases. There are a lot of environments where a forking reloader may be the best solution. For one, forking reloaders offer what appears to be transparent reloading without any monkey patching. As long as your app loads quickly and uses MRI, they can be a good solution.
Yes, that's true, but the main problem is that I find that such statements should be included in the README. Something like, "What reloader should you use?". Then you can explain that the forked approach used by A and B provide transparent reloading on MRI and Linux but will reload the full application which could be slow if your application takes much time to boot. Those are simple statements that the user can understand while choosing a reloader without having to dig on each option's README or source.
I'm certainly open to patches/extensions to rack-unreloader, so if you like how it works in general but want to support additional features, open an issue/pull request in rack-unreloader and we can discuss the issue there.
Thanks. I think I'll try to create a proof-of-concept of how I think the reloader should work and if that doesn't go well then I'll try to see how to improve rack-unreloader so that it suites better my app's needs.
I've ported successfully a JRuby on Rails application to Roda yesterday so that it could give me an idea on how it feels and I liked it. It was a simple application with just two actions exporting some data to Excel (old XLS format) using Apache POI, one of them using streaming. It was pretty forward to replace that app with Roda since most of the work for sharing the session with the main app has been done 2 days ago, when I worked on this gem I released yesterday morning:
https://github.com/rosenfeld/rails_compatible_cookies_utils
Both actions were GET ones so I didn't have to worry about CSRF and implementing a compatible protection mechanism. But in case someone is interested in sharing some encrypted or signed cookies values with a Rails app running in another process, they might be interested in this gem. Once I start moving some parts of the main MRI app to Roda I may add some support for forgery protection that would be compatible with how Rails handles them.
You may want to look at https://github.com/jeremyevans/roda-rails, as it provides some useful integration code until you can fully replace Rails with Roda.
Yes, I took already a look at it a few weeks ago, but it's actually quite simple for me to integrate with Rails. But since my final goal is to move totally out from Rails, I'd like to be sure I have everything which is required to achieve that goal. That's why I prefer to implement such utility methods so that I know I have no more hard dependencies on Rails. We have already moved from Sprockets to Webpack several months ago. We use Sequel for several years now. I got rid of js-routes this week. And finally implemented the cookies utilities that allow me keep the current cookies if I want to even if as a fallback for a little while. The missing bits I can think of are getting rid of Devise, implementing the CSRF protection mechanism (maybe there's something for that which is generic for Rack, but during the transition I'll have to keep a compatible implementation) and converting some controller tests to Rack tests. But it shouldn't be too complicated to move from Rails to Roda I guess, although it may require quite some time as there's quite a bit of Ruby code in the controllers...
On Thursday, July 14, 2016 at 1:27:05 PM UTC-7, Rodrigo Rosenfeld Rosas wrote:Em 14-07-2016 15:30, Jeremy Evans escreveu:
ActiveSupport::Dependencies is a good solution if you buy into The Rails Way (monkey patch all the things!). It works by overriding const_missing and require, which is pretty heavy handed if you aren't using Rails. I don't think there is an approach that will give you the same result as ActiveSupport::Dependencies without monkey patching const_missing and require.
Actually const_missing is not really required to implement a reloader. Rails does that because it supports auto-loading based on the missing constant name, which I don't intend to support. In my Rails apps I always require my dependencies with require_dependency so I guess const_missing never reaches my app's code. I think patching require is necessary though, but only in development mode, which I think it should be fine for the benefits we get. ActiveSupport::Dependencies will also patch in production mode but it shouldn't be required.
Using require_dependency is not transparent, it's basically just a global method that works like Unreloader.require.
If you want a transparent solution (i.e. drop into an existing ruby codebase that uses require), you would have to monkey patch require.
And to get constants autoloaded that haven't been required, you would have to override const_missing.
It's not so much that Roda gives little relevance to the issue, it's just not a problem that Roda itself tries to solve, so Roda just gives pointers to possible options. Roda is not a full stack framework, it's a toolkit/library. Setting up reloading automatically would mean specifying where things go (routes, models, etc.), which is something that should be specified at the framework level.
That's not quite true. You can describe an auto-reloading mechanism and explain how to configure it to work with the user's app. No need to set any conventions for that.
While you are correct that I could describe how rack-unreloader works in Roda's README, as I mentioned, I don't think that is Roda's place.
I disagree that it is Roda's place to describe the ruby reloading ecosystem. Roda lists rack-unreloader first, but that's as far as it goes in terms of recommendations. For one thing, as the maintainer of both Roda and rack-unreloader, I'm biased and I don't think it's fair for me to make an explicit recommendation.
So, let me provide you some feedback from a real user of Roda. As a Roda user, I'd like to focus on creating web apps. Auto-reloading should simply work seamless in my opinion if I'm using some web framework no matter what language I use. I see lots of potential in Roda, but most people won't consider it as a serious development environment if they can't get auto-reloading to mostly work. At least this is my opinion as an advanced Ruby user but I guess regular or beginner Ruby users would think of this as a blocker issue. I think Roda should be more concerned about this. As you probably know, I'm a long time Sequel user and the main reason is that it just works. And it works great. It's expected that an ORM shouldn't be concerned about auto-reloading. But I think it's an expected feature for a web framework. The same way I evangelize for Sequel, I'd love to do the same for Roda, but I won't be able to until I figure out a fix for the auto-reloading problem. Of course, I don't want auto-reloading to work just to evangelize it but because I value it very much for my own usage.
Remember, Roda is a toolkit/library, not a framework. If you want a framework in which to build an application, that's where you would use something like: https://github.com/jeremyevans/roda-sequel-stack, which sets things up for you.
I appreciate where you are coming from, and I'm not trying to say that autoreloading isn't important or useful. However, Roda and rack-unreloader are orthogonal. You can use Roda with other reloaders, and you can use rack-unreloader with other web libraries. As a counterpoint to your issue, Sinatra does not ship with a reloader or mention reloading in the README (other than template reloading, which Roda also does without a reloader in development mode), and is by the 2nd most popular ruby web library after Rails (there is a reloader in sinatra-contrib, which is a separate library).
So, let's please discuss the issues with each option in that README:
rack-unreloader
It doesn't work transparently on JRuby before version 9.1.
Also, with the current implementation one has to require all app files with Unreloader.require(path) before running Unreloader. It would be good if it was possible to set up Unreloader mechanism, like it's possible with ActiveSupport::Dependencies.mechanism = :require (or :load). It could be any API, as long as it would be possible to make Unreloader.require be implemented as a regular require for test and production environments. This way, we wouldn't have to require all dependencies upfront before running the Rack application and would instead use Unreloader.require on each file just like we would call require_dependency with ActiveSupport.
I don't think it's possible for reloading to work transparently with no code changes without monkey patching. Until Ruby itself offers such a feature, you'll have to choose a transparent/monkey patching approach or an explicit/no monkey patching approach.
Yes, I'm fine with basically monkey patching "require" just in development mode. Also, I think we could try to find other people interested in that feature (most of us maybe) and ask for such a feature that helps implementing auto-reloading in an efficient way in the Ruby-core mailing list or maybe we could create a new issue in Redmine there. It would be great if we could register some callback each time require is called that would provide us every new created constant both in the top level object as well as inside modules or classes.
I think having ruby natively support code reloading would be very helpful. I'm not sure there is a general solution that would handle all cases correctly, though.
I believe you can use rack-unreloader to specify which files should be reloaded, even if they are in gems or stdlib. You just provide a directory to rack-unreloader and it will handle all ruby files in all subdirectories.
I don't think I understood what you have suggested... Since you don't override require, if a watched file is changed but it requires another file outside of that directory, you'd try to unload constants defined by those files, right? This has the potential of causing issues with JRuby and imported Java libraries among others. My goal is to specify explicitly what is reloadable and what is not. Anything required with require is not reloadable for example. I can't implement that without overriding "require".
I think it shouldn't be difficult to extend rack-unreloader so that if you do:
Unreloader.require('a.rb')
and a.rb requires b.rb, there is a way to specify to only reload a.rb and not b.rb. Of course, if you do this, you would need to specify which constants were defined in a.rb. I think that's something ActiveSupport::Dependencies can handle automatically, but only because it overrides require.
Maybe it would be a good idea to advise developers to not change any sources while a request is in progress, specially in a multi-threaded environment, even though I don't think this is a common practice ;)
This probably isn't a big deal, as editors are generally going to write out the entire file, and a partially written file would just result in a SyntaxError.
This is not what I meant. Requiring a different code in the middle of other request using the changed classes could cause some issues, although I agree this would be pretty rare and we shouldn't be concerned about it. But I find it good to know about it and avoid changing code while some request is still being handled or at least know that this could produce side effects on that request... Considering a threaded web server of course, like Puma.
Gotcha. I would think in development mode, you would probably want to run single threaded (or use Rack::Lock).
Roda makes no indication that any of the reloading options will suit your needs, it just mentions the options available. It's up to the user to do their own due diligence and evaluate the options.
Yes, exactly, but that's really bad from a user's point of view in my opinion.
Well, I understand that it is suboptimal. But I don't think it's ethical for me to recommend something else I maintain even if I think it is better in some cases. As I mentioned, there are cases where I think a forking reloader is a better choice.
We should probably better describe the options so that each developer interested in Roda, like me, wouldn't have to do that work and read each source to try to figure out what they do and what they don't. I also think that Roda should actively recommend rack-unloader as it seems like the only option which is really suitable for large code bases. It also supports MRI, JRuby 9.1+ and Windows I guess.
Just because rack-unreloader is the best solution for your code bases and mine does not necessarily make it the best solution for all code bases. There are a lot of environments where a forking reloader may be the best solution. For one, forking reloaders offer what appears to be transparent reloading without any monkey patching. As long as your app loads quickly and uses MRI, they can be a good solution.
Yes, that's true, but the main problem is that I find that such statements should be included in the README. Something like, "What reloader should you use?". Then you can explain that the forked approach used by A and B provide transparent reloading on MRI and Linux but will reload the full application which could be slow if your application takes much time to boot. Those are simple statements that the user can understand while choosing a reloader without having to dig on each option's README or source.
OK, I'll agree with you there. Would it be possible for you to submit a pull request that does that, just describing briefly the type of reloader and possible tradeoffs, without making a recommendation? I would feel better if such a pull request was submitted, instead of me doing so, to avoid ethical issues.
I'm certainly open to patches/extensions to rack-unreloader, so if you like how it works in general but want to support additional features, open an issue/pull request in rack-unreloader and we can discuss the issue there.
Thanks. I think I'll try to create a proof-of-concept of how I think the reloader should work and if that doesn't go well then I'll try to see how to improve rack-unreloader so that it suites better my app's needs.
Sounds good.I've ported successfully a JRuby on Rails application to Roda yesterday so that it could give me an idea on how it feels and I liked it. It was a simple application with just two actions exporting some data to Excel (old XLS format) using Apache POI, one of them using streaming. It was pretty forward to replace that app with Roda since most of the work for sharing the session with the main app has been done 2 days ago, when I worked on this gem I released yesterday morning:
https://github.com/rosenfeld/rails_compatible_cookies_utils
Both actions were GET ones so I didn't have to worry about CSRF and implementing a compatible protection mechanism. But in case someone is interested in sharing some encrypted or signed cookies values with a Rails app running in another process, they might be interested in this gem. Once I start moving some parts of the main MRI app to Roda I may add some support for forgery protection that would be compatible with how Rails handles them.
You may want to look at https://github.com/jeremyevans/roda-rails, as it provides some useful integration code until you can fully replace Rails with Roda.
Yes, I took already a look at it a few weeks ago, but it's actually quite simple for me to integrate with Rails. But since my final goal is to move totally out from Rails, I'd like to be sure I have everything which is required to achieve that goal. That's why I prefer to implement such utility methods so that I know I have no more hard dependencies on Rails. We have already moved from Sprockets to Webpack several months ago. We use Sequel for several years now. I got rid of js-routes this week. And finally implemented the cookies utilities that allow me keep the current cookies if I want to even if as a fallback for a little while. The missing bits I can think of are getting rid of Devise, implementing the CSRF protection mechanism (maybe there's something for that which is generic for Rack, but during the transition I'll have to keep a compatible implementation) and converting some controller tests to Rack tests. But it shouldn't be too complicated to move from Rails to Roda I guess, although it may require quite some time as there's quite a bit of Ruby code in the controllers...
roda-rails only provides a couple of things:
* Use Rails flash in Roda* Use Rails CSRF protection in Roda
So when you are using Roda and Rails together, you just use the rails42 plugin provided by roda-rails. As soon as you are using just Roda, remove that plugin and use Roda's flash and csrf plugins, which should be API compatible.
Em 14-07-2016 18:36, Jeremy Evans escreveu:
That's not really a framework either, just a skeleton :)Remember, Roda is a toolkit/library, not a framework. If you want a framework in which to build an application, that's where you would use something like: https://github.com/jeremyevans/roda-sequel-stack, which sets things up for you.
I appreciate where you are coming from, and I'm not trying to say that autoreloading isn't important or useful. However, Roda and rack-unreloader are orthogonal. You can use Roda with other reloaders, and you can use rack-unreloader with other web libraries. As a counterpoint to your issue, Sinatra does not ship with a reloader or mention reloading in the README (other than template reloading, which Roda also does without a reloader in development mode), and is by the 2nd most popular ruby web library after Rails (there is a reloader in sinatra-contrib, which is a separate library).
And yet the main reason I haven't moved from Rails yet is exactly the auto-reloading feature which I don't find in other frameworks. I have a few aside apps that would be better for me to create them in something simpler like Sinatra or Padrino (Roda didn't exist by that time) but since auto-reloading wasn't enabled by default I found it easier to simply create other Rails apps due to this sole reason. I don't think it's a good idea for Roda to get inspiration from Sinatra. And even though auto-reloading is handled in sinatra-contrib, it's maintained by the same team as far as I can tell and I guess most people using Sinatra are also using its reloader. Roda doesn't provide any official contrib package offering an auto-reloading solution, so it's not really fair to compare it with Sinatra with that respect. I see Roda (and Sinatra, Padrino, etc) as tools to help me create Ruby web applications, but I can't imagine myself writing web apps restarting the application after every change. Or reloading the full app after every change. If you have several Sequel models it will have to issue several queries to get their metadata and it can take quite a while just to load the models.
Gotcha. I would think in development mode, you would probably want to run single threaded (or use Rack::Lock).
Not if you are serving websockets in the same process or if you are serving assets from the Rack app or if you are testing an application using long-polling among other features that require multi-thread to be enabled (I already had to test with multi-threading enabled even in development mode while performing some performance-optimizations in the client-side code).
OK, I'll agree with you there. Would it be possible for you to submit a pull request that does that, just describing briefly the type of reloader and possible tradeoffs, without making a recommendation? I would feel better if such a pull request was submitted, instead of me doing so, to avoid ethical issues.
Yes, I can work on such PR, but I don't think the description would be short (except for the forked model). Describing both rack-unreloader and ActiveSupport::Dependencies would take a lot of space. Maybe it would be better to create a new documentation on auto-reloading and link to that in the README.
On Thursday, July 14, 2016 at 5:33:22 PM UTC-7, Rodrigo Rosenfeld Rosas wrote:Em 14-07-2016 18:36, Jeremy Evans escreveu:
That's not really a framework either, just a skeleton :)Remember, Roda is a toolkit/library, not a framework. If you want a framework in which to build an application, that's where you would use something like: https://github.com/jeremyevans/roda-sequel-stack, which sets things up for you.
Fair enough. It's really more of an example app that is easy to clone and get modify, but it serves a similar purpose as rails new.
I appreciate where you are coming from, and I'm not trying to say that autoreloading isn't important or useful. However, Roda and rack-unreloader are orthogonal. You can use Roda with other reloaders, and you can use rack-unreloader with other web libraries. As a counterpoint to your issue, Sinatra does not ship with a reloader or mention reloading in the README (other than template reloading, which Roda also does without a reloader in development mode), and is by the 2nd most popular ruby web library after Rails (there is a reloader in sinatra-contrib, which is a separate library).
And yet the main reason I haven't moved from Rails yet is exactly the auto-reloading feature which I don't find in other frameworks. I have a few aside apps that would be better for me to create them in something simpler like Sinatra or Padrino (Roda didn't exist by that time) but since auto-reloading wasn't enabled by default I found it easier to simply create other Rails apps due to this sole reason. I don't think it's a good idea for Roda to get inspiration from Sinatra. And even though auto-reloading is handled in sinatra-contrib, it's maintained by the same team as far as I can tell and I guess most people using Sinatra are also using its reloader. Roda doesn't provide any official contrib package offering an auto-reloading solution, so it's not really fair to compare it with Sinatra with that respect. I see Roda (and Sinatra, Padrino, etc) as tools to help me create Ruby web applications, but I can't imagine myself writing web apps restarting the application after every change. Or reloading the full app after every change. If you have several Sequel models it will have to issue several queries to get their metadata and it can take quite a while just to load the models.
Roda gets half of its inspiration from Sinatra, and the other half from Rum/Cuba. I happen to think Sinatra as a fine source of inspiration. Actually, if you compare Sinatra to Rails in terms of influence of programming libraries in general (not just ruby libraries), I think you'll find that Sinatra has more influence than Rails.
There are Sinatra-like libraries for practically every programming language. There are certainly some Rails-like libraries, but not nearly as many.
It is true that Roda does not offer an official contrib package, but that's because it ships with a large number of plugins, so there isn't a need.
I do take your point about reloading being must have for some users,
and I agree we should do more in terms of explaining the situation.Gotcha. I would think in development mode, you would probably want to run single threaded (or use Rack::Lock).
Not if you are serving websockets in the same process or if you are serving assets from the Rack app or if you are testing an application using long-polling among other features that require multi-thread to be enabled (I already had to test with multi-threading enabled even in development mode while performing some performance-optimizations in the client-side code).
Fair enough, that's something I hadn't considered, and you are right that you'd need a threaded server to deal with that.OK, I'll agree with you there. Would it be possible for you to submit a pull request that does that, just describing briefly the type of reloader and possible tradeoffs, without making a recommendation? I would feel better if such a pull request was submitted, instead of me doing so, to avoid ethical issues.
Yes, I can work on such PR, but I don't think the description would be short (except for the forked model). Describing both rack-unreloader and ActiveSupport::Dependencies would take a lot of space. Maybe it would be better to create a new documentation on auto-reloading and link to that in the README.
Well, I really don't want a long description of different reloading practices. There's no need to mention ActiveSupport::Dependencies, since it's not compatible with Roda anyway.
Let me try to come up with some basic verbiage that at least tries to be objective, and I'll see what you think about it.
Em 14-07-2016 23:25, Jeremy Evans escreveu:
Roda gets half of its inspiration from Sinatra, and the other half from Rum/Cuba. I happen to think Sinatra as a fine source of inspiration. Actually, if you compare Sinatra to Rails in terms of influence of programming libraries in general (not just ruby libraries), I think you'll find that Sinatra has more influence than Rails.
This is not really fair since it's much simpler to create Sinatra-like API clones for other languages than to build a full-featured Rails alternative. I'm not suggesting that this is the main reason and I understand several people will prefer a micro framework approach, but we can't be sure whether the reason for many Sinatra clones is due to its simplicity (it's much quicker to build such a clone) or because they prefer the Sinatra API over a full-featured web framework like Rails :)
There are Sinatra-like libraries for practically every programming language. There are certainly some Rails-like libraries, but not nearly as many.
It is true that Roda does not offer an official contrib package, but that's because it ships with a large number of plugins, so there isn't a need.
Yeah, but no plugin is there to handle autoreloading ;)
Well, I really don't want a long description of different reloading practices. There's no need to mention ActiveSupport::Dependencies, since it's not compatible with Roda anyway.
Why do you think so? It's actually quite simple to use AS::Dependencies with Roda and I think it should be mentioned among the other options as well. Here's a sample app demonstrating how easy it is to use AS auto-reloader with Roda:
https://github.com/rosenfeld/roda-as-reloader
Em 14-07-2016 23:44, Jeremy Evans escreveu:Here's what I came up with: https://github.com/jeremyevans/roda/commit/cc0651e575926f2abe74dfd009d3da87456ee987
Please let me know what you think.
I found the description on rack-unloader insufficient. It should be better described either in this README or in the project's README. It needs lots of examples to proper explain its gotchas so that one would have an idea with a quick glance while evaluating many web frameworks or libraries in Ruby. They won't want to dig into the implementation details until they were able to have a good overall view. In that aspect, we should make it easier for them to understand the current state of auto-reloading since this is a must have for most users in my opinion and a critical piece for someone looking for a Rails alternative for example.
It's worth mentioning that rack-unreloader provides two basic options. You can either use regular "require" for non-reloadable code and manually specify every constant that should be removed from the reloadable requires which would be performed by Unreloader.require or; you would have to require all libraries upfront in config.ru. For example, this would break:
config.ru:
Unloader.require './app' # without specifying the { 'App' } block
app.rb:
require 'some_gem' # This would add SomeGem constant
SomeGem.do_something # After reloading this would raise an exception that SomeGem is not defined because it was unloaded
...
This is a big deal to me as I don't want to be forced to specify all non-loadable requires before requiring 'app' and I find it weird to have to think about which constants are added by each required file so that I would specify them on each Unloader.require block...
Also, how would your description about rack-unreloader be different from ActiveSupport::Dependencies? It worths mentioning that ActiveSupport::Dependencies will monkey patch some Ruby core classes, including Object.require and const_missing, while rack-unloader will not. And that ActiveSupport::Dependencies supports auto-loading by following some filenames and class names conventions while rack-unloader does not. Also, AS::Dependencies will use load instead of require in the development environment so that it doesn't have to change $LOADED_FEATURES. Also, rack-unloader will reload only changed files while AS::Dependencies will reload all auto-loaded files as well as those loaded with require_dependency.
For the rerun and shotgun description, I would add that they can't be used with JRuby apps.
"If you are unsure where to start, it may be best to start with rerun or shotgun,"
I'd add a parenthesis before comma: "(unless you're running on JRuby or Windows)"
https://github.com/rosenfeld/roda-as-reloader
Ah, I wasn't aware it was that easy to use. I agree, it should be mentioned as option. I'll try to update the description to include it.
Just to let you know, I've written two articles on the subject. A review of the code reloaders and another one about my own reloader:
http://rosenfeld.herokuapp.com/en/articles/ruby-rails/2016-07-18-autoreloader-a-transparent-automatic-code-reloader-for-ruby
http://rosenfeld.herokuapp.com/en/articles/ruby-rails/2016-07-18-a-review-of-code-reloaders-for-ruby
I'll start porting some code to Roda this week. Once I'm comfortable with the new auto-reloader implementation I'll send a PR to include it in Roda's README.