error running integration specs for radiant 1.0.1 app

[Craig Ambrose]

Hi everyone,

I have an app I have upgraded from 0.8.6 to 1.0.1, and all the specs
are working fine, except for the integration specs. Running them
generates the following error immediately:

no such file to load -- file_not_found_page (MissingSourceFile)
/Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/activesupport-2.3.14/
lib/active_support/dependencies.rb:184:in `require'
/Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/activesupport-2.3.14/
lib/active_support/dependencies.rb:184:in `require'
/Users/craig/.rvm/gems/ruby-1.8.7-p334@nzffa/gems/radiant-1.0.1/spec/
datasets/file_not_found_dataset.rb:1

This is true if I run "bundle exec rake spec:integration", or if they
run as part of a full spec run.

Although my app is an upgrade, I've disabled all extensions and still
get this issue.

Line 1 of radiant-1.0.1/spec/datasets/file_not_found_dataset.rb does
indeed try to require 'file_not_found_page.rb'. If I require this file
from within the console of my app, it works fine. So, possibly this is
an issue related to include paths. Potentially this may relate to
which exact version of cucumber I'm using. My Gemfile.lock includes:

cucumber (1.1.9)
builder (>= 2.1.2)
diff-lcs (>= 1.1.2)
gherkin (~> 2.9.0)
json (>= 1.4.6)
term-ansicolor (>= 1.0.6)
cucumber-rails (0.3.2)
cucumber (>= 0.8.0)
database_cleaner (0.6.7)

Does anyone have any leads on this?

cheers,

Craig Ambrose

[Jim Gay]

Can you paste the full stack trace somewhere?

--
Write intention revealing code #=> http://www.clean-ruby.com

Jim Gay
Saturn Flyer LLC
571-403-0338

[Craig Ambrose]

Hi Jim,

At some point in my experimentation, the error changed to something like a failure to find the feature directory. I was attempting to proceed in very slow and measured steps, but I missed noticing exactly how and when that error change occurred. To fix that later error, I added an empty cucumber directory structure to my base app. Now, when it tries to run the integration tests (by which it seems to mean cucumber scenarios) it finds zero cucumber features and so passes. That means that my "rake spec" is passing again, so it seems good.

Is that what I should be expecting? Does refinery come with any cucumber scenarios of it's own, or is it fine to be passing with zero specs?

regards,

Craig

-- 

Craig Ambrose

[Jim Gay]

On Mon, Apr 30, 2012 at 8:28 PM, Craig Ambrose <cr...@craigambrose.com> wrote:
> Hi Jim,
>
> At some point in my experimentation, the error changed to something like a
> failure to find the feature directory. I was attempting to proceed in very
> slow and measured steps, but I missed noticing exactly how and when that
> error change occurred. To fix that later error, I added an empty cucumber
> directory structure to my base app. Now, when it tries to run the
> integration tests (by which it seems to mean cucumber scenarios) it finds
> zero cucumber features and so passes. That means that my "rake spec" is
> passing again, so it seems good.
>
> Is that what I should be expecting? Does refinery come with any cucumber
> scenarios of it's own, or is it fine to be passing with zero specs?

Ha ha. I don't know if refinery does, but Radiant has cucumber features ;-P
I don't recall if the task runs the Radiant specs along with your own,
but I'll try to dig into this sometime this week if I can.

-Jim

[Craig Ambrose]

Sorry Jim, I'm working on both Radiant and Refinery projects at the moment, and getting a little confused by the "R" words. I meant "does radiant come with cucumber scenarios", but I think you got that.

It's kind of strange testing my Radiant app. When I run rake spec, it runs *everything*. Heaps of specs from Radiant itself, for models, controllers, helpers, etc. It runs specs for all of the extensions. Interestingly, the entire process fails if any of the radiant specs fail, but spec failures in the extension suites don't cause the "rake spec" process to return failure as a whole. The whole suite takes ages, over five minutes.

When I add extensions, they inevitably monkey patch Radiant, and then it's really hard to keep the specs passing. Extensions that do things like reorder pages inevitably cause radiant specs to fail (since it expects a different order). This is to be expected I imagine. It's sort of odd to run the tests for Radiant and all the extensions on one big app. How can they all be expected to pass? Since I'm now to the Radiant app I'm working with, I've carefully kept them all passing, and it took days. I've now got all the extensions I need installed, and a couple of the extensions have failing specs but Radiant doesn't, and so the process as a whole passes. 

More sensible would be if only my application itself ran specs using the app code. Radiant specs should run without my app, or extensions loaded. Extension specs should run with radiant loaded, but not anything else. I shouldn't typically run Radiant or extension specs when I'm developing my app, only if I'm working on radiant itself, or an extension. I should be able to test extensions from within their own repositories without having to put them into an app.

Refinery (and I do mean refinery this time) does this by using the rather crude approach of putting a dummy rails app into the spec directory of Refinery itself. So, it doesn't copy that into the gem, but it is used for running the specs. I'm a bit less sure how extension specs work.

It seems to me like poor testing processes are a struggle for Radiant. It's a more feature rich CMS than Radiant, with more useful extensions available. However, the extensions typically get orphaned, or change without a proper upgrade path. My experience of upgrading a refinery 0.8 app, with custom extensions, to a 1.0.1 app has led me through a huge number of github forks of extensions, and shown some pretty hideous code in some of the extensions. I've seen work by two separate consultancies doing Radiant work professionally and yet not writing tests, perhaps because it seems too hard.

Anyway, pretty easy to pick on things from the outside. Obviously a huge amount of work has gone into things, and I don't want to be too critical. I hope the above thoughts are of some constructive use to folks.

-- 

Craig Ambrose

[Jim Gay]

Thanks Craig.

This is helpful feedback.

Radiant is in the process of some major changes to the structure. Much
of it is code built upon code built upon code and there are plenty of
pieces that need a clean sweep. That takes time, of course. Radiant is
the oldest Rails CMS and carries a lot of the baggage of that.

[Alex Switzer]

Hi. How do I get myself removed from this group. I can't seem to get it right. Thanks. 

From: Craig Ambrose <cr...@craigambrose.com>
Reply-To: <radia...@googlegroups.com>
Date: Tuesday, 1 May 2012 1:00 PM
To: <radia...@googlegroups.com>
Subject: Re: [Radiant] error running integration specs for radiant 1.0.1 app