Controller vs Request Specs in Rails 5

[Jack Royal-Gordon]

In Rails 5, the team changed controller tests to effectively be integration tests (use URLs instead of actions), don’t test controller instance variables, don’t test what template is rendered, etc. Instead, you should test the contents of the page for specifics that indicate the request completed as expected.

With RSpec (at least with 3.1), controller tests in Rails 5 are still based on the old unit test (Test::Case). So I’m wondering where I should go here?

It should be noted that I’m just "passing through” Rails 5 on my way to Rails 6 (or at least 5.2), as that may affect the answer to my question.

Here are some of the alternatives I can think of:

1) Stay with current software, use the "rails-controller-testing” to restore template and assigns testing, and leave it alone for a future release.

2) Stay with current software, rewrite all of the controller specs as request specs (and abandon the controller specs).

3) Upgrade to a newer version of RSpec that treats controller specs as request specs (does this even exist?) and rework the controller specs to follow the rules of request specs.


There are a couple questions that inform this answer, and perhaps deserve to be answered in their own right:

1) When all is said and done, what is the position of the RSpec team regarding controller specs vs. request specs?

2) Since it seems that, for at least awhile, RSpec and Rails were a bit at odds regarding the nature of these tests, what minor versions of RSpec 3 align with (are intended to be used with) which versions of Rails 5?


It seems to me that these are relatively fundamental questions, and most probably have been discussed/debated at great length, so if you can point me to such a discussion that describes the official opinion of the RSpec team (if there is one), that would be great. (I don’t need someone to write all that up again if it already exists! I’ve seen a lot of blogs giving people’s opinoins on controller unit specs vs. request specs vs. feature specs and I don’t really want to touch the request spec vs. feature spec argument here.)

[Phil Pirozhkov]

In Rails 5, the team changed controller tests to effectively be integration tests (use URLs instead of actions), don’t test controller instance variables, don’t test what template is rendered, etc. Instead, you should test the contents of the page for specifics that indicate the request completed as expected.

With RSpec (at least with 3.1), controller tests in Rails 5 are still based on the old unit test (Test::Case). So I’m wondering where I should go here?

As far as I remember, they still are even in rspec-rails 4.0.

 

It should be noted that I’m just "passing through” Rails 5 on my way to Rails 6 (or at least 5.2), as that may affect the answer to my question.

Here are some of the alternatives I can think of:
1) Stay with current software, use the "rails-controller-testing” to restore template and assigns testing, and leave it alone for a future release.
2) Stay with current software, rewrite all of the controller specs as request specs (and abandon the controller specs).
3) Upgrade to a newer version of RSpec that treats controller specs as request specs (does this even exist?) and rework the controller specs to follow the rules of request specs.

Depends on how frequently `assigns` and `assert_template`.

As you have a different goal, and `rails-controller-testing` is extracted and pretty well maintained, I guess you have better investment for your time than to rewrite specs that work.

version of RSpec that treats controller specs as request specs (does this even exist?)

It doesn't to my best knowledge.

There are a couple questions that inform this answer, and perhaps deserve to be answered in their own right:

1) When all is said and done, what is the position of the RSpec team regarding controller specs vs. request specs?

Controller specs are sometimes used as requests specs, and this lets you easily shot you in your foot. I've just recently seen several `get` calls in a single example, and memoization in the controller that remained between those requests.

Rails core team worked hard to make request specs fast. I don't see any reason not to use them.

 

2) Since it seems that, for at least awhile, RSpec and Rails were a bit at odds regarding the nature of these tests, what minor versions of RSpec 3 align with (are intended to be used with) which versions of Rails 5?

`rspec-rails` 4.x supports Rails 5 (and even 4.2).

There was a more reliable table than that in `rspec-rails` source somewhere, but just by checking README.md of different maintenance branches of `rspec-rails` 3.x

`rspec-rails` 3.0-3.5 Rails 3&4

`rspec-rails` 3.5-3.9 Rails 3&4&5

It seems to me that these are relatively fundamental questions, and most probably have been discussed/debated at great length, so if you can point me to such a discussion that describes the official opinion of the RSpec team (if there is one), that would be great. (I don’t need someone to write all that up again if it already exists! I’ve seen a lot of blogs giving people’s opinoins on controller unit specs vs. request specs vs. feature specs and I don’t really want to touch the request spec vs. feature spec argument here.)

For a start, there's an interesting discussion here https://github.com/rspec/rspec-rails/issues/2373 and here https://github.com/rubocop-hq/rspec-style-guide/issues/113

[Jack Royal-Gordon]

Thanks, Phil, for the detailed and well thought-out answer. I appreciate the effort.

There was at least one comment in there that the Rails team intends to completely remove functional tests at some point in the future (I think the exact phrasing was on the order of “deprecating and eventually removing”. So that makes it sound like at some point I’m going to have to rewrite all my controller tests as request specs.

But it also sounds like there is not universal agreement on the removal of controller tests (understatement?), and that there is a discrepancy between the concept that request specs should test all client-facing behavior, and the desire to test certain non-client-facing(?) behavior; after all, the goal of testing is to work towards bug-free code, and if you don’t test something you won’t know that it is bug-free.

So I think, for me, the correct approach is to continue with the controller tests as-is (in other words, adding the ‘rails-controller-testing” gem, while putting new tests of client-facing behavior into request specs and non-client-facing tests into controller tests. Eventually, Rails will remove controller tests and when they do, they will have done the best they can for migrating controller tests to request specs. I did upgrade to rspec-rails 3.9 - no serious issues.

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/CAAk5Ok-GjtsSQfyFGreEbLnhC_b53iJg15%2B6YKMiw7_Gm9mT0A%40mail.gmail.com.

[Jon Rowe]

Some clarifications that might help your deliberations:

`rspec-rails` maintains controller specs as a parallel to controller tests, with the intention that if you wish to continue to use them you bring in the `rails-controller-test` gem.

We recommend request specs rather than controller specs for the same reasons the Rails team does, thus it is likely rspec-rails will remove support for controller specs at the same time Rails does.

Controller specs were always a bit weird, pretending to be unit tests when in fact they were a set of limited integration tests with introspection into the internals of a controller, I would never recommend asserting on assigns in a controller.

Cheers

Jon

----------------

ma...@jonrowe.co.uk

https://jonrowe.co.uk

On 30 November 2020 at 17:05, Phil Pirozhkov wrote:

In Rails 5, the team changed controller tests to effectively be integration tests (use URLs instead of actions), don’t test controller instance variables, don’t test what template is rendered, etc. Instead, you should test the contents of the page for specifics that indicate the request completed as expected.

With RSpec (at least with 3.1), controller tests in Rails 5 are still based on the old unit test (Test::Case). So I’m wondering where I should go here?

As far as I remember, they still are even in rspec-rails 4.0.

 

It should be noted that I’m just "passing through” Rails 5 on my way to Rails 6 (or at least 5.2), as that may affect the answer to my question.

Here are some of the alternatives I can think of:

1) Stay with current software, use the "rails-controller-testing” to restore template and assigns testing, and leave it alone for a future release.

2) Stay with current software, rewrite all of the controller specs as request specs (and abandon the controller specs).

3) Upgrade to a newer version of RSpec that treats controller specs as request specs (does this even exist?) and rework the controller specs to follow the rules of request specs.

Depends on how frequently `assigns` and `assert_template`.

As you have a different goal, and `rails-controller-testing` is extracted and pretty well maintained, I guess you have better investment for your time than to rewrite specs that work.

version of RSpec that treats controller specs as request specs (does this even exist?)

It doesn't to my best knowledge.

There are a couple questions that inform this answer, and perhaps deserve to be answered in their own right:

1) When all is said and done, what is the position of the RSpec team regarding controller specs vs. request specs?

Controller specs are sometimes used as requests specs, and this lets you easily shot you in your foot. I've just recently seen several `get` calls in a single example, and memoization in the controller that remained between those requests.

Rails core team worked hard to make request specs fast. I don't see any reason not to use them.

 

2) Since it seems that, for at least awhile, RSpec and Rails were a bit at odds regarding the nature of these tests, what minor versions of RSpec 3 align with (are intended to be used with) which versions of Rails 5?

`rspec-rails` 4.x supports Rails 5 (and even 4.2).

There was a more reliable table than that in `rspec-rails` source somewhere, but just by checking README.md of different maintenance branches of `rspec-rails` 3.x

`rspec-rails` 3.0-3.5 Rails 3&4

`rspec-rails` 3.5-3.9 Rails 3&4&5

[Jack Royal-Gordon]

If it’s that black-and-white that Request specs are to be preferred over Controller Specs, you should consider reflecting that in the documentation on RSpec Controller Specs. Right now, Controller Specs are documented very much the way they would have been before this debate appeared (and certainly before the moves were made by Rails to move away from controller tests). In fact, two of the four examples in the introduction to Controller Specs are about rendered templates and controller instance variables. (BTW, the Request Spec overview also mentions #render_template.)

Personally, I really drank the koolaid on controller tests. What I have done with Controller Specs is to focus on making sure that I test every piece of code that is in the Controller to ensure that they do what is expected of them, and the existence of robust controller specs has allowed me to discover and resolve a number of breaking changes when I upgraded Ruby and/or Rails (which, if you’ve been following my saga, is both numerous and ongoing — I started at Ruby 2.0 and Rails 3.2, and have made it up to Ruby 2.4/Rails 5.1 with a target of 2/7/6.1). While a request spec will likely find a problem in your program’s behavior (if you’ve figured out enough of the logic to drive all the different paths through the code, which is not trivial when you’re focusing on the big picture of the request), the controller specs were excellent at pinpointing exactly the source of a problem, and therefore led to a quick fix. In Controller tests, I stub out almost everything not in the controller, so that it really is like a controller unit test.

Summarizing my approach, I have built unit tests for just about every class (including miscellaneous code in the lib folder and initializers). Then I built (or will build) system specs that focus on both testing the JS and the end-to-end functioning of the pages. I’d prefer to test the JS in more of a unit-test fashion, but there really isn’t a way to do that (that I know of).

But if you’re saying that the writing is on the wall about Controller tests, I guess I will start moving towards request specs in their place. Since most of the controller instance variables exist to affect the template behavior, I will turn tests involving assigns and rendered templates into tests of exactly what was rendered in the page, even though that is an indirect test of the code that establishes the value of the instance variable.

This is one of those places where Rails being opinionated isn’t a good thing for me - I happen to disagree with the opinion. ;)

Regards,

Jack

--
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/dejalu-217-3700ca81-2903-4a43-aa70-87b831d42325%40jonrowe.co.uk.

[Jon Rowe]

If it’s that black-and-white that Request specs are to be preferred over Controller Specs, you should consider reflecting that in the documentation on RSpec Controller Specs. Right now, Controller Specs are documented very much the way they would have been before this debate appeared (and certainly before the moves were made by Rails to move away from controller tests). In fact, two of the four examples in the introduction to Controller Specs are about rendered templates and controller instance variables. (BTW, the Request Spec overview also mentions #render_template.)

Thats because the documentation was likely written when controller specs where the only option, and we as a team haven't gotten around to updating them, nor am I likely to any time soon, but I'd happily review contributions on this front.

What I have done with Controller Specs is to focus on making sure that I test every piece of code that is in the Controller to ensure that they do what is expected of them, and the existence of robust controller specs has allowed me to discover and resolve a number of breaking changes

All of which would have been apparent with request specs too.

While a request spec will likely find a problem in your program’s behavior (if you’ve figured out enough of the logic to drive all the different paths through the code, which is not trivial when you’re focusing on the big picture of the request),

A controller always executes in the context of the request, a controller spec just skips parts of the stack, but not all of the stack, and can actually introduce bugs where you pass in things which are not actually how rack parses them.

In Controller tests, I stub out almost everything not in the controller, so that it really is like a controller unit test.

You can also stub out like that in a request spec, however thats not what I mean by "controller specs are integration tests" a controller spec always has part of the rack stack wrapped around it, and you don't er, control, the initialisation of the controller. So it is never a unit test, you are always integrating parts of Rails with your controller.

I actually wrote proper controller unit tests as part of an experimental setup, with an eye to improving controller specs in rspec-rails, however its really difficult and involved a large knowledge of rails setup, thus its far better to give in and accept the truth, that they are integration tests, and treat them as such.

You can still mock out your service layer in that context, (or go down the DI route to allow control of your inner stack) but the Rails part is so coupled to the router and rack stack you simply have to accept it (or, I guess PRs welcome 😂)

Hope that helps

Jon

[Jack Royal-Gordon]

As an observation, I’ve noticed that my unit specs tend to find breaking differences when I upgrade Ruby/Rails, while my request specs, feature specs, and system specs tend to find changes in how those test environments function from release to release. Of course, that’s once I have a stable version of my code. When making changes, I rely on the request/feature/system tests to determine the the code works as intended.

Maybe the reason that I’m so focused on unit tests is because my focus is on upgrading Ruby and Rails, rather than changes to my app’s functionality.

On Dec 6, 2020, at 10:10 AM, Jon Rowe <ma...@jonrowe.co.uk> wrote:

If it’s that black-and-white that Request specs are to be preferred over Controller Specs, you should consider reflecting that in the documentation on RSpec Controller Specs. Right now, Controller Specs are documented very much the way they would have been before this debate appeared (and certainly before the moves were made by Rails to move away from controller tests). In fact, two of the four examples in the introduction to Controller Specs are about rendered templates and controller instance variables. (BTW, the Request Spec overview also mentions #render_template.)

Thats because the documentation was likely written when controller specs where the only option, and we as a team haven't gotten around to updating them, nor am I likely to any time soon, but I'd happily review contributions on this front.

I shall work on that, as I feel like if I had read what I’m envisioning there, I might have done things differently starting last summer (not placing blame -- I get that you can’t do everything, and  you focused on functionality, and that without functionality there is no purpose for documentation).

What I have done with Controller Specs is to focus on making sure that I test every piece of code that is in the Controller to ensure that they do what is expected of them, and the existence of robust controller specs has allowed me to discover and resolve a number of breaking changes

All of which would have been apparent with request specs too.

I’m under the impression that testing which template was rendered and which variables assigned is considered verboten with request specs. Aren’t you supposed to test things like request status and actually rendered response (not how the response got rendered)?

While a request spec will likely find a problem in your program’s behavior (if you’ve figured out enough of the logic to drive all the different paths through the code, which is not trivial when you’re focusing on the big picture of the request),

A controller always executes in the context of the request, a controller spec just skips parts of the stack, but not all of the stack, and can actually introduce bugs where you pass in things which are not actually how rack parses them.

When you’re testing a model, you’re not just testing the code you wrote in that model, you’re testing the part of Rails (AR/AM) that the code interacts with. So the argument that you aren’t just testing your controller code, you’re testing it’s interaction with Rack, AC, etc. seems specious to  me.

Further, to my mind, the mark of a good unit test is that it tests a bite-sized piece of code. When you write a request spec that tests the “entire stack”, you’re testing the templates, helpers, and models as well as the portions of the stack that Rails provides. And that would be fine if a bug in any of that will always reflect in the output. but sometimes you have a bug that hides another bug, and that will only be found by unit testing. Further, request Specs  don't really test the entire stack, because they leave out the browser and JS, so it’s neither full-stack nor unit test.

I feel like we need unit tests for virtually every piece of code and system tests (acceptance tests) to test that all the code when put together achieves the high-level goal.

In Controller tests, I stub out almost everything not in the controller, so that it really is like a controller unit test.

You can also stub out like that in a request spec, however thats not what I mean by "controller specs are integration tests" a controller spec always has part of the rack stack wrapped around it, and you don't er, control, the initialisation of the controller. So it is never a unit test, you are always integrating parts of Rails with your controller.

While it’s no doubt true that you can, the documentation says you should not stub in request specs.

I actually wrote proper controller unit tests as part of an experimental setup, with an eye to improving controller specs in rspec-rails, however its really difficult and involved a large knowledge of rails setup, thus its far better to give in and accept the truth, that they are integration tests, and treat them as such.

Just as writing tests for your models requires knowledge of how the model works.

You can still mock out your service layer in that context, (or go down the DI route to allow control of your inner stack) but the Rails part is so coupled to the router and rack stack you simply have to accept it (or, I guess PRs welcome 😂)

I’m not going to tilt at THAT windmill! 

Hope that helps

Jon

Thanks for taking the time to respond. It must get tiring sometimes, defending yourself to folks who are second-guessing your decisions.

Regards,

Jack

-- 
You received this message because you are subscribed to the Google Groups "rspec" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rspec+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rspec/f1320e23-d12c-4e47-90ea-d80f6f598077%40www.fastmail.com.