GSOC 2012 - Student wanted for "Improve Testing Framework"

Showing 1-18 of 18 messages
GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 3/26/12 2:30 AM
Hey guys,

I'm still looking for a student to take on a fairly central piece in
the future development of the CMS interface: The behaviour testing framework.
Its a great opportunity to work with cutting edge tech in the PHP space,
as well as adding a useful skill to your portfolio - employers
love a documented personal focus on software quality.
When starting with this, I've also found it quite rewarding as a programmer
to shift gears from lowlevel implementation tests to describing business
functionality - it gets you more into a client-focused mode of thinking,
again a great asset for your career.

Alright, enough sales pitch - anybody keen? ;)
Ingo

P.S.: Ryan Dao, if you can still be swayed to switch from your ecommerce
project idea, let me know :)
 

Improve Behaviour Testing Framework and Test Suite

SilverStripe has good support for unit testing, and the framework is generally well tested. But those tests are developer centric, and don’t describe high level user goals very well. Even worse, a lot of high impact interface bugs are slipping through the cracks because of this unit-focused approach. A more behaviour-driven testing will make our interface more specified and solid in a sustainable fashion.

Goals:

  • Extend the existing proof-of-concept based on Cuke4PHP and PHP Webdriver
  • Connect existing database fixture logic in SilverStripe’s testing framework to the setup/teardown logic of Cuke4PHP
  • Create reusable set of assertions (“Given … when … then”) fitting the problem space of a content management interface
  • Specify existing CMS behaviour based on examples
  • Start with high-level goals, e.g. “Given I am logged in as a CMS author, when I open a page, enter ‘new content’ and save the page, the website shows ‘new content’”
  • Document how to use the Cuke4PHP system, how the assertions are structured and how they can be used. The goal is to create a living specification, so a top priority should be spreading knowledge about it to other maintainers.

Requirements:

  • Ability to quickly grasp relatively complex user interactions by using an interface, and describing them in a clear way through examples
  • Intermediate PHP skills to write the test assertions
  • Beginner SilverStripe skills in order to create framework-specific extensions to the library
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Sayak Sarkar 3/26/12 2:43 AM
Hey Ingo,

My name is Sayak Sarkar and I am a 2nd semester M.Sc.(Computer
Applications) student at Symbiosis Institute Of Computer Studies &
Research, Pune.

I am a regular FOSS contributor and also do some Bengali translations
and debugging for Mozilla and Wikimedia.

Also I am proficient with web content management systems such as
Drupal and Wordpress, and quite familiar with PHP and MySQL and also
know the basics of module creation in Drupal.

I've also got a good grasp on the basic functioning of Content
Management Systems, and have experimented with creating mini Content
Management Systems.

I take a keen interest in exploring through open source content
management systems and going through their functioning.

I'm interested in taking up the "Improve Behaviour Testing Framework
and Test Suite" project as it seems quite appealing to me. However, I
haven't worked on any such Testing frameworks in the past and as such
I'm a bit nervous about the project.

However, I am certain that with some guidance I will be able to grasp
the concept quickly. As stated above I have some intermediate php
skills, and I've also been going through SilverStripe for quite a few
days now.

I would be very grateful if you could give me some suggestions on how
to get started with Cuke4PHP and PHP Webdriver as those are areas
where I need to improve upon.

Awaiting your suggestion,
Sayak

> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/i_s8GtlypUcJ.
> To post to this group, send email to silverst...@googlegroups.com.
> To unsubscribe from this group, send email to
> silverstripe-d...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/silverstripe-dev?hl=en.

--
About Me:http://about.me/sayak_sarkar
Twitter: http://twitter.com/sayak_sarkar
Blog: http://sayaksarkar.wordpress.com

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 3/26/12 3:04 AM
Hello Sayak, thanks for your interest! :) 
In terms of getting started, I'd suggest you check out this blog post,
which guides you through the various steps required:
(haven't followed them myself, so not sure how accurate they are)

In terms of both you and us gaining confidence that the project is a good fit,
it would be ideal if you get to a point where you can log in to the CMS
via PHP-Webdriver controlled by Cuke4PHP, and maybe save a page.
Do you think that's realistic to achieve during the application period?

We have gotten this far already, so its more of an excercise for you.
The code required is currently not open source, but thats just a formality
(it would form the basis for your project work). Here's a gist to give
you an idea of our approach: https://gist.github.com/2204261

Ingo
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 3/26/12 4:08 PM
Hello again, we've just opensourced the project I talked about:
Its more of a proof of concept, and due to heavy development on SS3
most of the actual CMS behaviour tests currently fail - this could
be a nice area for you to get started as well: Try to fix those tests,
and submit us pull requests :)
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Sayak Sarkar 3/26/12 4:13 PM
Hey Ingo,

Thanks for the update, I've been working on understanding PHP BDD and
CUKE4PHP and your links are really helpful, will get back to you as
soon as I finish going through all of it. :-)

Regards,
Sayak

> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/QfNtSizzHJkJ.

>
> To post to this group, send email to silverst...@googlegroups.com.
> To unsubscribe from this group, send email to
> silverstripe-d...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/silverstripe-dev?hl=en.

--
About Me:http://about.me/sayak_sarkar
Twitter: http://twitter.com/sayak_sarkar
Blog: http://sayaksarkar.wordpress.com

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 3/31/12 8:07 AM
Hello Sayak, how you're getting on with Cuke? Anything I can help with? 6 days until the application deadline :) - All the best to India, Ingo
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Sayak Sarkar 3/31/12 12:44 PM
Hey Ingo,
Sorry, I couldn't update my status with Cuke earlier. My laptop's hard
disk crashed so had been a bit busy with recovery and stuff. I did go
through Cuke, and hope to submit a draft within Monday evening, there
are some areas which I am still going through and trying to smooth out
before submitting a plan, hence the delay.

In the meantime I've already forked the repositories and am trying to
work a around a little bit on it. Will give you a detailed update by
Sunday night. :)
Regards,
Sayak

On Sat, Mar 31, 2012 at 8:37 PM, Ingo Schommer <ingo.s...@gmail.com> wrote:
> Hello Sayak, how you're getting on with Cuke? Anything I can help with? 6
> days until the application deadline :) - All the best to India, Ingo
>
> --
> You received this message because you are subscribed to the Google Groups
> "SilverStripe Core Development" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/silverstripe-dev/-/CtUWLqTt7bAJ.

>
> To post to this group, send email to silverst...@googlegroups.com.
> To unsubscribe from this group, send email to
> silverstripe-d...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/silverstripe-dev?hl=en.

--
About Me:http://about.me/sayak_sarkar
Twitter: http://twitter.com/sayak_sarkar
Blog: http://sayaksarkar.wordpress.com

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Sayak Sarkar 4/1/12 10:37 AM
Hey Ingo,
Still trying to get around the examples, though I have to confess, I'm
still not very comfortable with cucumber implementation.though I have
understood the basic idea behind creating testing frameworks.
I'm trying to try out some sample test cases, but haven't yet been
quite successful yet. Once, I sort out the the problem I would draft a
proposal and submit it. Of course if I'm unable to sort out the issues
within a couple of days, I would ask for your help. ;-)
Regards,
Sayak
Re: GSOC 2012 - Student wanted for "Improve Testing Framework" Michał Ochman 4/1/12 1:13 PM
Hello!

My name is Michał and I'll be applying to GSoC for this project. I'm
in my last year of MSc studies in Computer Science and since I have
been collaborating with Wojtek Szkutnik's Imagomedia on SilverStripe
projects for over a year, I have decided to help out in open source
development before it's too late for me ;-)

For the last 5+ years I was involved in PHP development for some big
companies such as Betfair, Ladbrokes and Bwin.com. While we did mostly
WordPress and custom PHP development, but in my free time I've been
enjoying playing around with SilverStripe. I also did lots of
JavaScript development so I'm comfortable with working on both backend
and frontend. One of our last projects was Agile-driven and since we
have used all best practices of TDD development such as unit testing,
acceptance tests and other various testing techniques, I feel quite
comfortable with developing the testing framework. I have done most of
my TDD projects in Python, which involved various testing libraries
(mostly PyUnit), techniques (using mocks, factories, testing all
layers of the apps from frontend to http communication) and
technologies (lettuce, splinter /a python open source library for
webdriver/), as well as handling CI intergration (Jenkins).

I'll try to do the code test today and have a closer look at the state
of current SilverStripe unit tests - maybe I'll find some place for
improvements for the core tests as well.

Best Regards,
Michał Ochman

On Apr 1, 7:37 pm, Sayak Sarkar <sayak.bugsm...@gmail.com> wrote:
> Hey Ingo,
> Still trying to get around the examples, though I have to confess, I'm
> still not very comfortable with cucumber implementation.though I have
> understood the basic idea behind creating testing frameworks.
> I'm trying to try out some sample test cases, but haven't yet been
> quite successful yet. Once, I sort out the the problem I would draft a
> proposal and submit it. Of course if I'm unable to sort out the issues
> within a couple of days, I would ask for your help. ;-)
> Regards,
> Sayak
>
>
>
>
>
>
>
>
>
> On Sun, Apr 1, 2012 at 1:14 AM, Sayak Sarkar <sayak.bugsm...@gmail.com> wrote:
> > Hey Ingo,
> > Sorry, I couldn't update my status with Cuke earlier. My laptop's hard
> > disk crashed so had been a bit busy with recovery and stuff. I did go
> > through Cuke, and hope to submit a draft within Monday evening, there
> > are some areas which I am still going through and trying to smooth out
> > before submitting a plan, hence the delay.
>
> > In the meantime I've already forked the repositories and am trying to
> > work a around a little bit on it. Will give you a detailed update by
> > Sunday night. :)
> > Regards,
> > Sayak
>
Re: [silverstripe-dev] Re: GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 4/2/12 1:07 AM
Hey Michal, that sounds very promising!  
Same advice as to Sayak, please try to get
our proof of concept implementation going on your machine as a first step :)
Just to clarify, this project is about high-level behaviour (automated) testing
rather than unit testing, although I'm sure there's going to be some overlap
and duplication, so looking at the unit tests won't hurt.

Thanks
Ingo

On Sunday, 1 April 2012 at 10:13 PM, Michał Ochman wrote:

> Hello!
>  
> My name is Michał and I'll be applying to GSoC for this project. I'm
> in my last year of MSc studies in Computer Science and since I have
> been collaborating with Wojtek Szkutnik's Imagomedia on SilverStripe
> projects for over a year, I have decided to help out in open source
> development before it's too late for me ;-)
>  
> For the last 5+ years I was involved in PHP development for some big
> companies such as Betfair, Ladbrokes and Bwin.com (http://Bwin.com). While we did mostly

> WordPress and custom PHP development, but in my free time I've been
> enjoying playing around with SilverStripe. I also did lots of
> JavaScript development so I'm comfortable with working on both backend
> and frontend. One of our last projects was Agile-driven and since we
> have used all best practices of TDD development such as unit testing,
> acceptance tests and other various testing techniques, I feel quite
> comfortable with developing the testing framework. I have done most of
> my TDD projects in Python, which involved various testing libraries
> (mostly PyUnit), techniques (using mocks, factories, testing all
> layers of the apps from frontend to http communication) and
> technologies (lettuce, splinter /a python open source library for
> webdriver/), as well as handling CI intergration (Jenkins).
>  
> I'll try to do the code test today and have a closer look at the state
> of current SilverStripe unit tests - maybe I'll find some place for
> improvements for the core tests as well.
>  
> Best Regards,
> Michał Ochman
>  
> On Apr 1, 7:37 pm, Sayak Sarkar <sayak.bugsm...@gmail.com (http://gmail.com)> wrote:
> > Hey Ingo,
> > Still trying to get around the examples, though I have to confess, I'm
> > still not very comfortable with cucumber implementation.though I have
> > understood the basic idea behind creating testing frameworks.
> > I'm trying to try out some sample test cases, but haven't yet been
> > quite successful yet. Once, I sort out the the problem I would draft a
> > proposal and submit it. Of course if I'm unable to sort out the issues
> > within a couple of days, I would ask for your help. ;-)
> > Regards,
> > Sayak
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> >  
> > On Sun, Apr 1, 2012 at 1:14 AM, Sayak Sarkar <sayak.bugsm...@gmail.com (http://gmail.com)> wrote:
> > > Hey Ingo,
> > > Sorry, I couldn't update my status with Cuke earlier. My laptop's hard
> > > disk crashed so had been a bit busy with recovery and stuff. I did go
> > > through Cuke, and hope to submit a draft within Monday evening, there
> > > are some areas which I am still going through and trying to smooth out
> > > before submitting a plan, hence the delay.
> >  
> >  
> >  
> > > In the meantime I've already forked the repositories and am trying to
> > > work a around a little bit on it. Will give you a detailed update by
> > > Sunday night. :)
> > > Regards,
> > > Sayak
> >  
> >  
> >  
> > > On Sat, Mar 31, 2012 at 8:37 PM, Ingo Schommer <ingo.schom...@gmail.com (http://gmail.com)> wrote:
> > > > Hello Sayak, how you're getting on with Cuke? Anything I can help with? 6
> > > > days until the application deadline :) - All the best to India, Ingo
> > >  
> >  
> >  
> >  
> > > > --
> > > > You received this message because you are subscribed to the Google Groups
> > > > "SilverStripe Core Development" group.
> > > > To view this discussion on the web visit
> > > > https://groups.google.com/d/msg/silverstripe-dev/-/CtUWLqTt7bAJ.
> > >  
> >  
> >  
> >  
> > > > To post to this group, send email to silverst...@googlegroups.com (mailto:silverst...@googlegroups.com).

> > > > To unsubscribe from this group, send email to
> > > > silverstripe-d...@googlegroups.com (mailto:silverstripe-dev+unsubscribe@googlegroups.com).

> > > > For more options, visit this group at
> > > > http://groups.google.com/group/silverstripe-dev?hl=en.
> > >  
> >  
> >  
> >  
> > > --
> > > About Me:http://about.me/sayak_sarkar
> > > Twitter:http://twitter.com/sayak_sarkar
> > > Blog:http://sayaksarkar.wordpress.com
> >  
> >  
> >  
> > --
> > About Me:http://about.me/sayak_sarkar
> > Twitter:http://twitter.com/sayak_sarkar
> > Blog:http://sayaksarkar.wordpress.com
>  
>  
>  
> --  
> You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
> To post to this group, send email to silverst...@googlegroups.com (mailto:silverst...@googlegroups.com).
> To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com (mailto:silverstripe-dev+unsubscribe@googlegroups.com).

> For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.

Re: [silverstripe-dev] Re: GSOC 2012 - Student wanted for "Improve Testing Framework" Michał Ochman 4/4/12 9:59 PM
Hello,

Obviously it's about high level behaviour, I was just pointing out that I will have a look at the unit tests since it's one of the best ways of getting to know the code base.

I have managed to set up the testing environment and have a few remarks:

First of all, as a general note - in an ideal world, we would have a separate command to set up a test environment, then run the selenium server, then execute the tests - this way we could simulate the production setup and make sure that all will run well after deploying the code. Unfortunately, since it's almost not doable in PHP, we'll have to make the best out of it and just make a simple script for checking/running the selenium server and then running the tests - to make it a one-click operation. 

As for the docs, I know it's just work in progress but since there's a lot of linux users out there, it would be very handy to have the installation instructions for both linux and OS X - just to make sure we don't discourage any potential contributors ;-) Also, as a side note - obviously, the setup also requires curl, which isn't currently mentioned in the spec ;-). My last 0.02$ about the doc is that we should probably strongly encourage using the ChromeDriver. I have worked with both firefox and chrome drivers, and only the latter provide support for some advanced CSS/JavaScript methods/attributes/states. For example, in one of our recent projects we were unable to test a JavaScript autosuggestion feature using the firefox drivers, but it worked just fine with ChromeDriver. 

Also, I'll have to check how it works with cuke4php, but I know that in bigger testing environments it's handy to separate testing modules and include only the really needed functions - when I did similiar things in Python, we had a common_steps file with the most popular steps and module-specific step libraries. From what I see, there is a lot of features to introduce - form handling, all kinds of JS stuff etc - looks like an interesting job to do!

One last question - do core SilverStripe developers use some kind of CI environment right now? If not, it would be very handy to set up a test runner which would execute unit tests and selenium tests on every commit (I would suggest jenkins). We'd just need to set up a headless environment for selenium tests (they need some kind of X simulation to run the "browser tests"). 

As a last remark, it would be very nice to easily integrate with possible selenium tests included by module/widget developers - this way the runner would include not only the core tests but also tests introduced by installed plugins. 

Sorry for my disordered train of thought, I was just writing down all things that came to my mind to keep them in mind on a later stage ;-) I will start polishing my GSoC application right now to make sure to get it to you on time. If you have any remarks, please let me know - I'll be glad to iterate over my ideas to make the summer project as exciting as possible. 

Looking forward to getting involved in SilverStripe development!

Best,
Michał

W dniu 2 kwietnia 2012 10:07 użytkownik Ingo Schommer <in...@silverstripe.com> napisał:
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.


Re: [silverstripe-dev] Re: GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 4/5/12 1:54 AM
Hey Michal

On 5/04/2012, at 6:59 AM, Michał Ochman wrote:


As for the docs, I know it's just work in progress but since there's a lot of linux users out there, it would be very handy to have the installation instructions for both linux and OS X - just to make sure we don't discourage any potential contributors ;-) Also, as a side note - obviously, the setup also requires curl, which isn't currently mentioned in the spec ;-).
Agreed that it should be "one click" if possible, and that we need improved install docs- Pull requests welcome :)
SilverStripe the company is 99% on OSX for dev machines, so our preferences tend to show in these early stage projects.

My last 0.02$ about the doc is that we should probably strongly encourage using the ChromeDriver. I have worked with both firefox and chrome drivers, and only the latter provide support for some advanced CSS/JavaScript methods/attributes/states. For example, in one of our recent projects we were unable to test a JavaScript autosuggestion feature using the firefox drivers, but it worked just fine with ChromeDriver. 
Ideally its not an either-or decision, one motivation of using WebDriver is its cross browser support.
We'll have to see how realistic it is to use a behaviour test suite as cross browser smoke tests,
vs. how much work it is to deal with the various quirks (as you mentioned).

Also, I'll have to check how it works with cuke4php, but I know that in bigger testing environments it's handy to separate testing modules and include only the really needed functions - when I did similiar things in Python, we had a common_steps file with the most popular steps and module-specific step libraries. From what I see, there is a lot of features to introduce - form handling, all kinds of JS stuff etc - looks like an interesting job to do!
If you look at the class structure, we're already trying to build layers of "common steps",
and I think its crucial for developer adoption to have a clearly structured library of reuseable steps.
Otherwise we'll have devs implementing the same steps all over again, leading to a perceived
lack of productivity on the tool.

One last question - do core SilverStripe developers use some kind of CI environment right now? If not, it would be very handy to set up a test runner which would execute unit tests and selenium tests on every commit (I would suggest jenkins). We'd just need to set up a headless environment for selenium tests (they need some kind of X simulation to run the "browser tests"). 
We're using buildbot (buildbot.silverstripe.org), but mostly migrated to TeamCity (http://teamcity.silverstripe.com, there's a guest login for the open source stuff).
The idea is to run all our CI on the same environment, including the behavioural tests. Lower priority for the GSOC project iself, although it would be handy.
Headless environments: Not sure if its worth the trouble, as I imagine debugging problems (DOM/JS/CSS) gets quite difficult. Do you have
any experience using headless browsers in non-trivial app tests?

As a last remark, it would be very nice to easily integrate with possible selenium tests included by module/widget developers - this way the runner would include not only the core tests but also tests introduced by installed plugins.
Yeah, we follow that philisophy with our PHPUnit runs as well, so its just natural to extend it to behavioural tests as well.

All the best
Ingo
Re: [silverstripe-dev] Re: GSOC 2012 - Student wanted for "Improve Testing Framework" Michał Ochman 4/12/12 1:37 AM
Moving the discussion from Google Melange here :)

Ingo wrote: Completely agree with the lack of fixture support in the current proof of concept, that'll be one of the first things to fix in the GSOC project to enable meaningful testing. Have you looked at the way we generate fixtures through YAML in SilverStripe? Good find with Phactory, but I think we're 2/3 there with the SapphireTest+YamlFixture classes already. One of the base assumptions in the proof of concept is that the steps can use the underlying SilverStripe PHP framework, so we can generate fixtures programmatically, and even share them between PHPUnit and Cuke tests.

I went through the YAML fixtures feature and gave it some more thought during the last few days. I agree that YAML fixtures are very useful in some cases, but from what I see and understand, you always have to generate them first. It's not a problem when you test something like things described at http://doc.silverstripe.org/sapphire/en/topics/testing/create-sapphire-test , but I think that there are some cases where some kinds of Factories may come very useful. Obviously, we don't have to decide on this until I'm proved right or wrong during the development phase, but I believe that we'll end up using both, anyway. I wrote down a couple of reasons for why Factories may be better than Fixtures for testing:

1. Flexibility
After all, fixtures are pre-defined sets of data. You can import some parts of them but in general, one fixture fits only one purpose. A Factory is much more versatile - if you want to generate a hundred objects and don't care about their properties, you just do ObjectFactory::create() a hundred times. However, if you want to generate objects with specific attributes, you don't have to use another factory - you just need to specify an argument like in ObjectFactory::create(color=yellow) to get a hundred of yellow objects. This way, you can test every behaviour related to an object, using just one tool and don't have to generate fixtures on a per-case basis

2. Code readability
Using fixtures moves part of the testing implementation to an external file. This means, that to fully understand the tests you need to browser through the test files and then to fixtures files. The purpose of tests is to test specific behaviour, so that when something breaks we can browse the tests code and see what's happening there as fast as possible. Factories provide clean code integrated with tests, while fixtures require extra attention from developers. 

3. Maintainability 
Imagine that you want to change some part of the tests. If you re-use fixtures in multiple tests, it's highly possible that some of the tests will break when you change a specific fixture. Factories provide a solution which make tests fully independent from other tests. Also, if there is more than one developer working on tests using fixtures, there is no easy way to know whether a fixture is used in just one test or a hundred. Again, factories fix that ;-)  

Since implementing the use of Phactory or any other Factory library would be pretty straightforward, I think that this isn't much of a problem - if we decide that Factories would be useful in the testing process, I can implement them as part of my GSoC task, and if for some reason we decide to struggle with a fixture-only testing system, we can just go on with what we have. My point is that factories are very easy to implement and would be probably a huge boost to the testing process, while fixtures would be still very useful for pre-filling the database with some standard records which are not test-specific.

Best Regards,
Michał 


W dniu 5 kwietnia 2012 10:54 użytkownik Ingo Schommer <in...@silverstripe.com> napisał:

--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 4/12/12 7:33 AM
Hello Michal,

On 12/04/2012, at 10:37 AM, Michał Ochman wrote:

> Moving the discussion from Google Melange here :)
> I think that there are some cases where some kinds of Factories may come very useful. Obviously, we don't have to decide on this until I'm proved right or wrong during the development phase, but I believe that we'll end up using both, anyway.
Agreed, we're actually already using crude fixture generators in our projects, usually just custom code in SapphireTest->setUp().
There's really no way around this once you get into non-trivial fixtures (lots of repetition or large dependency graph).
On a recent project, we generated half-completed orders based on certain product criteria,
to simulate taking them through the booking process.
We also dynamically generated SOAP XML fixtures based on the data submitted to the mock -
both quite bespoke generators which would be hard to generalise into a factory class, right?
>

>
> Since implementing the use of Phactory or any other Factory library would be pretty straightforward, I think that this isn't much of a problem - if we decide that Factories would be useful in the testing process, I can implement them as part of my GSoC task, and if for some reason we decide to struggle with a fixture-only testing system, we can just go on with what we have. My point is that factories are very easy to implement and would be probably a huge boost to the testing process, while fixtures would be still very useful for pre-filling the database with some standard records which are not test-specific.
Can you provide a specific example where testing core CMS functionality would become unmaintainable with YAML fixtures?
I guess something like ModelAdmin pagination would need 20+ records to work, and you'd need a bit of variation to test the effectiveness of search.
I'd suggest that we timebox this to a couple of days proof of concept - I don't want us to get stuck in something that's only marginally related to the GSOC project.
Specifically, how much value we can get out of Phactory against using something more bespoke, but closer to our ORM definitions. What do you think?

Ingo

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Michał Ochman 4/13/12 3:31 AM
Hello!

Actually, I don't think that there is anything that you can do with fixtures that you can't do with factories. In the worst case scenario, you can use factories the same way that you would use fixtures, but from inside the code. Factories handle dependencies real well and you can both use sub-factories and not have to worry about the details, or specify custom dependencies to make sure that you get exactly what you need.

As for maintainability, the whole idea of YAML fixtures seems like kind of a workaround rather than a proper solution - maybe I'm just used to factories, but it seems like the simpliest way to handle multiple object generation. There is a lot of discussions about similiar issues in Rails and most of them speak in favour of factories. I think that I'll just take a day or two to make a proof of concept at the beginning of GSoC or sometime in May and if it doesn't take off - well, it won't make much harm. 

Best Regards,
Michał
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Ingo Schommer 4/13/12 5:41 AM
Hey Michal,

If you look in our core unit tests, the fixtures are usually quite small,
and for the amount of characters used quite expressive, particularly
when it comes to relation aliasing (Parent: =>SiteTree.my-parent).
Even if you ignore any PHP APIs, just using PHP's array() notation
will make this less readable than YAML. This is one of our longer ones:
Even at this length and level of repetition, I find it fairly easy to follow.

I think there's a tension between readability and maintainability here,
but my point is that it needs to be decided based on project requirements.
Then again, I've never worked with a generic factory API.
Had a look at a popular Rails one, and can definitely see its appeal:

Ingo

Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Sam Minnée 4/13/12 9:43 PM
Ingo, I know that for projects with more complex domain models (say a booking engine, as a hypothetical example ;-) ) Factories can help abstract away the fixture plumbing. Mark Stephens might also have some opinions base on his current project.

I don't think we have a sufficiently complex domain model in core to make much of a difference, but as a tool in the arsenal it could be useful.
--
You received this message because you are subscribed to the Google Groups "SilverStripe Core Development" group.
To post to this group, send email to silverst...@googlegroups.com.
To unsubscribe from this group, send email to silverstripe-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/silverstripe-dev?hl=en.
Re: [silverstripe-dev] GSOC 2012 - Student wanted for "Improve Testing Framework" Michał Ochman 4/18/12 1:07 AM
Introducing some basic factories will not be much of a hassle so it won't interfere with the project and I will be able to implement some basic proof of concept. I already started going through Phactory docs to have a head start.

Best Regards,
Michał

W dniu 14 kwietnia 2012 06:43 użytkownik Sam Minnée <s...@silverstripe.com> napisał: