Spree extension testing

[Chris]

tldr; How can I run the tests for the spree_store_credits extension
(or any official extension). I need step by step testing newbie
instructions.

I'm having problems figuring out how to run the various tests for
extensions in Spree.

First off, I have never written or run any ruby/rails tests before, so
from a testing perspective I'm a total newbie so please be kind.

I have read through the docs on testing Spree found here:
http://edgeguides.spreecommerce.com/testing.html

The documentation seems to be directed at someone running with the
gem. I'm actually working with the latest edge code (in this case the
namespace branch) and can't figure out how to make the commands shown
in the documentation work from a clone of the repository.

The documentation really isn't very good at explaining how to test for
someone who is new to Ruby/Rails testing, and the normal Rails testing
docs that I find seem to be centered around testing the app rather
than the extensions.

In the section "Testing Standalone Extensions" it says to:
a) spree extension foofah
b) rake test_app
c) bundle exec rspec spec

It doesn't say anything about the relative directory that you are
running these commands from. The first command gives me an error
about not being able to find Spree. So how do I tell it to use my
cloned copy?

If I first generate a generic (rails new appname) app and then add
Spree to the Gemfile, and then run: "bundle exec spree extension
foofah" I get an extension created inside the main app directory as
expected. Of course this is counter intuitive since step (b) says
that I'm creating a test app, so why did I need to create an
additional app?

But then if I try step (b) from the same main app directory I get:
rake aborted!
Don't know how to build task 'test_app'

If I run command (b) from inside the spree_foofah extension directory
then I get:
no such file to load --
../../spree_master_namespaced/lib/generators/spree/test_app_generator

The path to Spree in the above error is correct, but there is no
/lib/generators/spree/test_app_generator.

Searching through the Spree code I can't find a test_app_generator
file/folder anywhere.

So what am I missing?

I figure it's long past time that I start creating tests for my
extensions, but I can't even figure out how to properly run the tests
for the existing official extensions, much less create and run my own.

If somebody would be so kind as to give me step by step instruction
starting from an empty folder on the filesystem to being able to test
the spree_store_credits extension
(https://github.com/spree/spree_store_credits), or any other official
extension I would REALLY appreciate it.

I figure once I can figure out how to do it with one of the official
extensions then I can figure out how to do it for my own.

By the way, I know that the spree_store_credits extension won't work
with the namespace branch of Spree. That's the point. I'm using that
extension, and I want to get all the extensions that I'm using
working with the namespace branch. I want to be able to run the tests
to ensure that I have all the pieces working properly before I submit
any pull requests.

Thanks in advance,
Chris

[Tomek "Tomash" Stachewicz]

Hi,

I have a few notes from the times of making a spec-suite for my Spree extensions, also to put them on travis-ci (so the test suite had to be self-contained).

I need to reorganize the notes, will turn them into a blog post and possibly a pull request (some of the generated files should look differently), at most tomorrow.

(These three extensions might be a beginning of a "put your Spree extension on Travis-CI, too!" project, Spree team seems to be also enthusiastic about it)

For the time being, please take a look at the Spree extensions I've added test suites for:

https://github.com/tomash/globalize-spree

https://github.com/tomash/spree-pp-website-standard

https://github.com/pero-ict-solutions/spree-homepager

The files you're interested in are:

Gemfile, gemspec

Rakefile (you might want to comment-out Cucumber-related stuff if you'll be writing only specs)

spec/spec_helper.rb

That should get you going as for now :)

Tomash

[Chris]

Thanks for the links.
These lines have been helpful.

> Gemfile, gemspec
> Rakefile (you might want to comment-out Cucumber-related stuff if you'll be
> writing only specs)
> spec/spec_helper.rb

I'm still a long way from being able to run the tests though so I'm
looking forward to your writeup.

How is anyone from the core Spree team testing the official extensions
against the namespaced branch?

For that matter, how are you even testing the namespaced branch itself?

Thanks,
Chris

[Ryan Bigg]

How is anyone from the core Spree team testing the official extensions
against the namespaced branch?

We're going to be merging the namespace branch into master and then we will work on updating the official extensions to be compatible with these new changes. As far as I am aware, we will not release a new (minor) version until these official extensions have been updated. 

To clarify: by official extensions I mean the ones listed on http://github.com/spree.

I'll talk with with BDQ and Sean when I next can.

For that matter, how are you even testing the namespaced branch itself?

The namespace branch, just like master, has tests within each part of Spree, which currently consist of specs and features. We would also hope to test the namespace branch on real Spree application once we have completed the merge.

--
You received this message because you are subscribed to the Google Groups "Spree" group.
To post to this group, send email to spree...@googlegroups.com.
To unsubscribe from this group, send email to spree-user+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/spree-user?hl=en.

[Chris]

Ryan,
I appreciate the response, but it really doesn't help me. My goal
in this is to learn how to test my code which is always being written
against the edge version of Spree. In this case the edge is the
namespaced branch.

I think I have figured out how to test the individual spree portions
now (core, auth, etc). I thought they could be done from the top
spree folder, but apparently I need to be inside each of the
individual section folders to run the tests. At least that's the only
way I can seem to get them to run. These are the kinds of details
that seem to be missing from the documentation. Part of the problem
is with the wording on the documentation page:

[quote]
Spree consists of several different gems (see the Source Code Guide
for more details.) Each of these gems has its own test suite which can
be found in the spec directory. Since these gems are also Rails
engines, they can’t really be tested in complete isolation – they need
to be tested within the context of a Rails application.
[/quote]

So, let's step back one step then. How do I test ANY official
extension using the master branch of Spree?

Because when I try with the spree_store_credits extension
(https://github.com/spree/spree_store_credits) the steps listed here
don't seem to work.
http://edgeguides.spreecommerce.com/testing.html

I can't be the only one that can't seem to figure out how to do this.
Of course this is exactly why I don't currently have any tests. I'm
always under the gun to get the actual code written and can't spend
weeks trying to figure out how to make the testing framework
functional.

I've also tried with Tomash's extensions and what I get is:

spree-homepage$ bundle exec rake test_app
You cannot specify the same gem twice with different version
requirements. You specified: rspec-rails (= 2.6.1) and rspec-rails (>=
0)
spree-homepager$ bundle exec rspec spec
You cannot specify the same gem twice with different version
requirements. You specified: rspec-rails (= 2.6.1) and rspec-rails (>=
0)

I'm sure that I am just missing some little thing that once I figure
it out I'll have no clue why it was so hard, but I'm obviously blind
or stupid or both, because I can't seem to figure it out.

Any help is truly appreciated.
Chris

[Ryan Bigg]

Chris: I would really love to help you right now but I need to make my way back to Australia from Kiev, which will take me about 30 hours.

I hope Tomash can advise you on how to do this testing.

[Chris]

Ryan,
No worries mate :-) Thanks for all your help.

Have a safe trip.

Chris

[Tomek "Tomash" Stachewicz]

I've also tried with Tomash's extensions and what I get is:

spree-homepage$ bundle exec rake test_app
You cannot specify the same gem twice with different version
requirements. You specified: rspec-rails (= 2.6.1) and rspec-rails (>=
0)
spree-homepager$ bundle exec rspec spec
You cannot specify the same gem twice with different version
requirements. You specified: rspec-rails (= 2.6.1) and rspec-rails (>=
0)

I'm sure that I am just missing some little thing that once I figure
it out I'll have no clue why it was so hard, but I'm obviously blind
or stupid or both, because I can't seem to figure it out.

Ayyup. Check for mismatches betweet .gemspec and Gemfile, and shall the need arise, force them to be same versions.

(or try to just comment-out add_development_dependency "rspec-rails" in gemspec).

[Brian Quinn]

@Chris, here's the steps I just did to create a new testable extension:

gem install spree --version="0.70.1"

spree extension foofah

cd foofah

bundle install

bundle exec rake test_app

#create a spec inside spec/whatever/some_spec.rb

bundle exec rake spec

While extension testing has always been a bit of an issue, the recent changes to the extension generator has definitely helped set us up for better testing of extensions. Most official extensions are still using the old approach to running tests which requires a local cloned version of Spree (and a ENV variable pointing to the clone). The new extension generator creates a dummy application within the spec directory using the gem version of Spree, not a locally cloned version. This seems to be adopted standard for testing Rails engines.

We do have an job to do to clean the test situation for official extensions, and I think this will be covered as part of the namespacing updates (as all the extensions will need to be revisited). Generally when we've got a major change / new version of Spree on the master branch we'll start to update the Rails Dog Radio demo site which includes numerous official extensions. This workflow allows us to iron out any problems with new changes (and create upgrade instructions) before we push a release -  it also helps us get started updating extensions. You're a little earlier to the namespace branch party so you're feeling a lot of the pain we are as we're doing the upgrade.

On the testing front, LBRapid has been working on replacing all our cucumber testing with rspec request specs so that will also help simplify our testing setup (and instructions). 

Tomash - I'm keen to see any improvements / suggestions you have to extension testing in general.

Later,

Brian Quinn

-------------------------------------------

Rails Dog LLC

2 Wisconsin Circle, Suite 700

Chevy Chase, MD 20815

voice: (301)560-2000

-------------------------------------------

--

You received this message because you are subscribed to the Google Groups "Spree" group.

To view this discussion on the web visit https://groups.google.com/d/msg/spree-user/-/EVJeuzNGsXwJ.

[Chris]

Brian,
Thanks for the steps. I guess the one that I was missing is the
fact that it needs to be run from the gem, not my cloned copy.

So what do you do to test the extensions against the latest Spree code
before an official gem is built? Or does that not happen? Do you
just build and install a local gem (I'm not exactly sure how to do
that, but I seem to remember that it's possible)?

I played with the SPREE_PATH environment variable too, but it still
wasn't working. So why exactly can it be run from a gem but not from
the cloned copy? That doesn't really make sense to me.

Old testing method vs. new testing method. Rspec, cucumber,
....(insert your favorite test env here)..., it seems like the
favorite way of testing changes even faster than the Spree code
itself, which already is running at a breakneck speed.

So if I understand it correctly:
1) All the official extensions are currently testing with a method
that is going away in the near future, so don't use them as an
example.

2) The latest edge code (the namespace branch) can only be tested via
the individual components (core, auth, promo, etc).

I've got a ton of extensions to update to work with the namespace
changes, so once Ryan said he was ready to make the merge and asked
people to play around with it I figured it was a good time to start
making the changes. I knew it wasn't going to be pain free, but I did
think running tests against it would be easier.

My personal project has gotten so big that every time Spree is updated
it's a major undertaking to make sure everything is still working.
With as big a change as the namespacing is and the amount of work it's
going to take to update all my code I figured it was as good of a time
as any to start writing some tests.

At this point I figure I'll just hold off until everything settles way
down before dealing with the tests. No point in writing a bunch of
tests against the current Spree gem when they are just going to all
break anyway with the namespace changes.

Chris

[Chris]

Tomash,
I guess that makes sense. I've never really looked at the gemspec
files since i've always run everything from a cloned source and the
Gemfile was specifying the full version. I didn't realize that both
files were being analyzed. Good to know.

Thanks again,
Chris

> --
> You received this message because you are subscribed to the Google Groups
> "Spree" group.

> To view this discussion on the web visit

> https://groups.google.com/d/msg/spree-user/-/Jc06uLzVP1YJ.