Improve Behaviour Testing Framework and Test Suite
SilverStripe has good support for unit testing, and the framework is generally well tested. But those tests are developer centric, and don’t describe high level user goals very well. Even worse, a lot of high impact interface bugs are slipping through the cracks because of this unit-focused approach. A more behaviour-driven testing will make our interface more specified and solid in a sustainable fashion.
Goals:
- Extend the existing proof-of-concept based on Cuke4PHP and PHP Webdriver
- Connect existing database fixture logic in SilverStripe’s testing framework to the setup/teardown logic of Cuke4PHP
- Create reusable set of assertions (“Given … when … then”) fitting the problem space of a content management interface
- Specify existing CMS behaviour based on examples
- Start with high-level goals, e.g. “Given I am logged in as a CMS author, when I open a page, enter ‘new content’ and save the page, the website shows ‘new content’”
- Document how to use the Cuke4PHP system, how the assertions are structured and how they can be used. The goal is to create a living specification, so a top priority should be spreading knowledge about it to other maintainers.
Requirements:
- Ability to quickly grasp relatively complex user interactions by using an interface, and describing them in a clear way through examples
- Intermediate PHP skills to write the test assertions
- Beginner SilverStripe skills in order to create framework-specific extensions to the library
My name is Sayak Sarkar and I am a 2nd semester M.Sc.(Computer
Applications) student at Symbiosis Institute Of Computer Studies &
Research, Pune.
I am a regular FOSS contributor and also do some Bengali translations
and debugging for Mozilla and Wikimedia.
Also I am proficient with web content management systems such as
Drupal and Wordpress, and quite familiar with PHP and MySQL and also
know the basics of module creation in Drupal.
I've also got a good grasp on the basic functioning of Content
Management Systems, and have experimented with creating mini Content
Management Systems.
I take a keen interest in exploring through open source content
management systems and going through their functioning.
I'm interested in taking up the "Improve Behaviour Testing Framework
and Test Suite" project as it seems quite appealing to me. However, I
haven't worked on any such Testing frameworks in the past and as such
I'm a bit nervous about the project.
However, I am certain that with some guidance I will be able to grasp
the concept quickly. As stated above I have some intermediate php
skills, and I've also been going through SilverStripe for quite a few
days now.
I would be very grateful if you could give me some suggestions on how
to get started with Cuke4PHP and PHP Webdriver as those are areas
where I need to improve upon.
Awaiting your suggestion,
Sayak
> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/i_s8GtlypUcJ.
> To post to this group, send email to silverst...@googlegroups.com.
> To unsubscribe from this group, send email to
> silverstripe-d...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/silverstripe-dev?hl=en.
--
About Me:http://about.me/sayak_sarkar
Twitter: http://twitter.com/sayak_sarkar
Blog: http://sayaksarkar.wordpress.com
Thanks for the update, I've been working on understanding PHP BDD and
CUKE4PHP and your links are really helpful, will get back to you as
soon as I finish going through all of it. :-)
Regards,
Sayak
> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/QfNtSizzHJkJ.
In the meantime I've already forked the repositories and am trying to
work a around a little bit on it. Will give you a detailed update by
Sunday night. :)
Regards,
Sayak
On Sat, Mar 31, 2012 at 8:37 PM, Ingo Schommer <ingo.s...@gmail.com> wrote:
> Hello Sayak, how you're getting on with Cuke? Anything I can help with? 6
> days until the application deadline :) - All the best to India, Ingo
>
> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/CtUWLqTt7bAJ.
Thanks
Ingo
On Sunday, 1 April 2012 at 10:13 PM, Michał Ochman wrote:
> Hello!
>
> My name is Michał and I'll be applying to GSoC for this project. I'm
> in my last year of MSc studies in Computer Science and since I have
> been collaborating with Wojtek Szkutnik's Imagomedia on SilverStripe
> projects for over a year, I have decided to help out in open source
> development before it's too late for me ;-)
>
> For the last 5+ years I was involved in PHP development for some big
> companies such as Betfair, Ladbrokes and Bwin.com (http://Bwin.com). While we did mostly
> WordPress and custom PHP development, but in my free time I've been
> enjoying playing around with SilverStripe. I also did lots of
> JavaScript development so I'm comfortable with working on both backend
> and frontend. One of our last projects was Agile-driven and since we
> have used all best practices of TDD development such as unit testing,
> acceptance tests and other various testing techniques, I feel quite
> comfortable with developing the testing framework. I have done most of
> my TDD projects in Python, which involved various testing libraries
> (mostly PyUnit), techniques (using mocks, factories, testing all
> layers of the apps from frontend to http communication) and
> technologies (lettuce, splinter /a python open source library for
> webdriver/), as well as handling CI intergration (Jenkins).
>
> I'll try to do the code test today and have a closer look at the state
> of current SilverStripe unit tests - maybe I'll find some place for
> improvements for the core tests as well.
>
> Best Regards,
> Michał Ochman
>
> > > > To post to this group, send email to silverst...@googlegroups.com (mailto:silverst...@googlegroups.com).
> > > > To unsubscribe from this group, send email to
> > > > silverstripe-d...@googlegroups.com (mailto:silverstripe-d...@googlegroups.com).
> > > > For more options, visit this group at
> > > > http://groups.google.com/group/silverstripe-dev?hl=en.
> > >
> >
> >
> >
> > > --
> > > About Me:http://about.me/sayak_sarkar
> > > Twitter:http://twitter.com/sayak_sarkar
> > > Blog:http://sayaksarkar.wordpress.com
> >
> >
> >
> > --
> > About Me:http://about.me/sayak_sarkar
> > Twitter:http://twitter.com/sayak_sarkar
> > Blog:http://sayaksarkar.wordpress.com
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
> To post to this group, send email to silverst...@googlegroups.com (mailto:silverst...@googlegroups.com).
> To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com (mailto:silverstripe-d...@googlegroups.com).
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
As for the docs, I know it's just work in progress but since there's a lot of linux users out there, it would be very handy to have the installation instructions for both linux and OS X - just to make sure we don't discourage any potential contributors ;-) Also, as a side note - obviously, the setup also requires curl, which isn't currently mentioned in the spec ;-).
My last 0.02$ about the doc is that we should probably strongly encourage using the ChromeDriver. I have worked with both firefox and chrome drivers, and only the latter provide support for some advanced CSS/JavaScript methods/attributes/states. For example, in one of our recent projects we were unable to test a JavaScript autosuggestion feature using the firefox drivers, but it worked just fine with ChromeDriver.
Also, I'll have to check how it works with cuke4php, but I know that in bigger testing environments it's handy to separate testing modules and include only the really needed functions - when I did similiar things in Python, we had a common_steps file with the most popular steps and module-specific step libraries. From what I see, there is a lot of features to introduce - form handling, all kinds of JS stuff etc - looks like an interesting job to do!
One last question - do core SilverStripe developers use some kind of CI environment right now? If not, it would be very handy to set up a test runner which would execute unit tests and selenium tests on every commit (I would suggest jenkins). We'd just need to set up a headless environment for selenium tests (they need some kind of X simulation to run the "browser tests").
As a last remark, it would be very nice to easily integrate with possible selenium tests included by module/widget developers - this way the runner would include not only the core tests but also tests introduced by installed plugins.
--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
On 12/04/2012, at 10:37 AM, Michał Ochman wrote:
> Moving the discussion from Google Melange here :)
> I think that there are some cases where some kinds of Factories may come very useful. Obviously, we don't have to decide on this until I'm proved right or wrong during the development phase, but I believe that we'll end up using both, anyway.
Agreed, we're actually already using crude fixture generators in our projects, usually just custom code in SapphireTest->setUp().
There's really no way around this once you get into non-trivial fixtures (lots of repetition or large dependency graph).
On a recent project, we generated half-completed orders based on certain product criteria,
to simulate taking them through the booking process.
We also dynamically generated SOAP XML fixtures based on the data submitted to the mock -
both quite bespoke generators which would be hard to generalise into a factory class, right?
>
>
> Since implementing the use of Phactory or any other Factory library would be pretty straightforward, I think that this isn't much of a problem - if we decide that Factories would be useful in the testing process, I can implement them as part of my GSoC task, and if for some reason we decide to struggle with a fixture-only testing system, we can just go on with what we have. My point is that factories are very easy to implement and would be probably a huge boost to the testing process, while fixtures would be still very useful for pre-filling the database with some standard records which are not test-specific.
Can you provide a specific example where testing core CMS functionality would become unmaintainable with YAML fixtures?
I guess something like ModelAdmin pagination would need 20+ records to work, and you'd need a bit of variation to test the effectiveness of search.
I'd suggest that we timebox this to a couple of days proof of concept - I don't want us to get stuck in something that's only marginally related to the GSOC project.
Specifically, how much value we can get out of Phactory against using something more bespoke, but closer to our ORM definitions. What do you think?
Ingo