Testing newbie

[Joss Ingram]

Often after an upgrade or some other change I like to make sure all page types can be added without errors or issues, so I manually add a page of each model type to the admin.

I'd like to automate this, I've done a simple test for the homepage type to start

from django.test import TestCase

from foxsite.models import HomePage

class HomePageTestCase(TestCase):

    def setUp(self):

        HomePage.objects.create(title="Test Homepage", body="<p>Welcome</p>",path="00010002",keywords="home,page,test",slug="home",url_path="/home-page/",depth=2,numchild=0,show_in_menus=True,live=True,seo_title="",search_description="")

        

    def test_homepage_created(self):

        home = HomePage.objects.get(slug="home")

        self.assertEqual(home.title, 'Test Homepage')

Is this method ok, am I likely to run into issues as the content gets more complicated, for example I'm not entirely sure how the path value is derived?

Should I be using fixtures to achieve this? I don't want test data to remain in the db though, as I understand it the data from django tests gets removed straight after the test.

Thanks

Joss

[Brett Grace]

As a preface, I would steer away from doing automated testing against a live, production database (if that is what you are describing), since tests can do bad things and computers are really fast. The Django test framework generally expects to run against an empty database, which is destroyed at the end of the test.

I don't have "the answer" but here are some general thoughts that might help:

- Fixtures have their place, but they are going to need to be updated whenever you migrate the database, which can be fiddly and annoying. They also bypass form validation, so changes you make that affect how the admin area behaves (blank=False) might not show up when running these tests.

- You can create pages dynamically in your tests, similar to how you are doing above (but more idiomatically, read on), with similar limitations to fixtures. There is nothing preventing you from saving a model in code that would fail with the UI, so it doesn't answer, "does the admin UI work?" because testing this way bypasses form validation (blank=False), authentication, permissions, and business logic constraints (e.g., allowed parent/child page types).

There's definitely a place for unit testing your page models but to be honest I hardly every write "can I save a model to the database?" type tests. It's more useful for testing things like, "does 'get_blog_posts' only return the posts that are live, children of the current page, and have a publication date in the past?"

For the actual mechanics of unit testing page models, you're right that setting the path directly is going to get challenging. The page path is basically used to create the "tree" of pages, imagine if your computer filesystem was not hierarchical (no folders). You could still simulate folders with a naming convention. One way would be to use fixed width names, so you would be able to tell, for example, that file "000100990023.pdf" would correspond to a path like "/0001/0099/0023.pdf". Using digits allows you to sort things efficiently. During your tests you want to make sure that the tree doesn't become invalid, so it's best to use the Treebeard API to manage the path for you.

The best way to learn this is to follow the example of the tests in Wagtail itself (see wagtailcore/tests/test_page_model.py) (you can read up on Treebeard here: https://tabo.pe/projects/django-treebeard/docs/tip/api.html)

A highly abbreviated example (not actually tested, more for the sake of illustration):

root_page = Page.objects.get(depth=1)  # there is always One Root Page at the top of the tree
home_page = HomePage(title="Home", body="Foo") # Create a new page in memory, it's not saved yet
home_page.baz = "Bar"  # we can assign properties, don't need to stuff everything into the constructor
root_page.add_child(home_page)  # Treebeard API call saves it to the database and sets the path appropriately

home_page.save_revision().publish()  # Appease the Wagtail page revision gods, I always do this superstitiously

If I read your intention correctly your primary goal is to confirm that the user workflow didn't break, so you might consider a higher level integration test that simulates browser behavior. A couple of ways to do this:

- Instead of creating the model directly, try instantiating the corresponding ModelForm and simulating a request through the Wagtail admin code. The tests in wagtailadmin do this, take a look at TestPageCreation in wagtailadmin/tests/test_pages_views.py

- Use Selenium to create a script that simulates user interaction. I haven't used it but they do have a graphical IDE you can use to record and play back behavior. If you decide to go this route I wold be interested to know if it works well.

Happy hunting.

[Joss Ingram]

Wow, Thanks for that. it's really helpful. 

I was interested in running tests just locally not in production. I think using the treebeard method might be the best fit for us, but I'll also look at Selenium. You were correct about my primary intention, but I'm planing to write some tests for some of our custom views also.

[Joss Ingram]

Hi Brett,

I'm trying to do the higher level testing through the wagtailadmin, so I'm trying to adapt some of the code you referenced from the wagtail admin tests, I'm a bit confused as to whether I need to build a separate 'tests' app as part of my project, which reflects my models like the wagtailadmin.tests but has its own test data?

I want to use something like this from the TestPageCreation class (below) but to test my existing models.

def test_create_simplepage(self):
	response = self.client.get(reverse('wagtailadmin_pages:add', args=('tests', 'simplepage', self.root_page.id)))
	self.assertEqual(response.status_code, 200)
	self.assertContains(response, '<a href="#content" class="active">Content</a>')
	self.assertContains(response, '<a href="#promote" class="">Promote</a>')

I've tried changing it too

def test_create_homepage(self):

            response = self.client.get(reverse('wagtailadmin_pages:add', args=('foxsite', 'homepage', self.root_page.id)))

            self.assertEqual(response.status_code, 200)

            self.assertContains(response, '<a href="#content" class="active">Content</a>')

            self.assertContains(response, '<a href="#promote" class="">Promote</a>')

but i get the error below as I don't have the setup in my urls.py, and I'm not sure how to? 

NoReverseMatch: u'wagtailadmin_pages' is not a registered namespace

On Wednesday, 22 July 2015 20:12:49 UTC+1, Brett Grace wrote:

[Joss Ingram]

Using your code sample with a slight change I've added the code below to my tests, and it works well so thanks for that. I'd like to get it testing through simulating a request through the admin code though ideally.

                

    def test_homepage_created(self):

        root_page = Page.objects.get(depth=1)  

        

        home_page = HomePage(title="Test Homepage", body="<p>Welcome</p>")

        home_page.slug = "home" 

        home_page.keywords = "home,page,test"

        home_page.show_in_menus = True

        home_page.live = True

        home_page.seo_title = ""

        home_page.search_description= ""

        

        root_page.add_child(instance=home_page)  

        home_page.save_revision().publish() 

        

        home = HomePage.objects.get(slug="home")

        self.assertEqual(home.title, 'Test Homepage')

On Wednesday, 22 July 2015 20:12:49 UTC+1, Brett Grace wrote:

[Brett Grace]

I haven't executed this testing strategy, so I can't give you the solution, but I think this is in the right direction: http://stackoverflow.com/questions/19678747/django-reusable-app-url-namespace-issue

If I have a chance in the next day or two I'll see if I can get something working.

I don't believe you would need a separate test app to make this work. It looks like you have the right function signature for wagtailadmin.views.pages.create (if your own code isn't running the latest Wagtail you might have to use a different signature, from what I can tell it was reorganized between 0.8 and 1.0 — which I suppose raises an interesting contra vs. unit testing and testing with Selenium. Writing tests against the admin app means your own tests become dependent on some of the internal, private implementation details of Wagtail. I don't think it's terrible since it should be easy to track those changes, but it's not negligible either).

[Brett Grace]

Cool. A couple of points from my experience

- At some point you might need to assign your HomePage to a Site object. For testing I think it's okay, but I'm not sure, so if you get strange results it's something to look at. For example I'm not sure if using url reverse will work if you pages aren't assigned to a Site.

- If you do set the home page on a site you can get it back with "Site.objects.get(is_default_site).root_page" rather than by slug, which might be important because...

- A gotcha with slugs is that they don't have to be unique. So you could have more than one page with the slug "home" or "test" or whatever.

- In general you can find pages by "walking the tree" rather than by slug or title or something else. Page extends Treebeard's Node so you can use API calls as in your queryset, so you can do things like:

root_page.get_children().get(slug="home")

which will return a generic "Page" object, or the following:

HomePage.objects.child_of(root_page).get(slug="home")

[Joss Ingram]

Brett,

We have Selenium working now which was a bit tricky inside vagrant, one simple test, the next step is to use it to add a wagtail page, currently it just tests a search form.

I will post details of how it goes soon hopefully.

Joss

[Joss Ingram]

Although I've managed to get selenium to add a basic page through the admin, any fields that are within an expandable GUI section of the page, as for inline orderable elements, can't be found using Selenium's find element by name function, input fields like the one below seem invisible. I notice those sections are within <script> tags too. Any ideas on how I could refer to those input fields on the page?

<input id="id_chunk_items-__prefix__-chunk_title" maxlength="255" name="chunk_items-__prefix__-chunk_title" type="text" />

[Brett Grace]

What steps are you using to test? I think that you have to drive a click event to the "reveal" button, wait for the content to be added to the DOM, and then run your test. Something like this maybe?

http://stackoverflow.com/questions/12692172/test-dynamically-loaded-content-with-selenium-web-driver

Sorry if I'm misunderstanding the scenario.

[Joss Ingram]

It could be a waiting issue, I am clicking the 5th H2 tag which opens the panel I need. Here is my code below:

from django.test import TestCase

from django.test import LiveServerTestCase

from selenium.webdriver.firefox.webdriver import WebDriver

from selenium.webdriver.common.keys import Keys

from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

from pyvirtualdisplay import Display

class AddPagesTestCase(LiveServerTestCase):      

    @classmethod

    def setUpClass(cls):

        cls.display = Display(visible=0, size=(1024, 768))

        cls.display.start()

        cls.driver = WebDriver()

        super(AddPagesTestCase, cls).setUpClass()

    @classmethod

    def tearDownClass(cls):

        cls.driver.quit()

        cls.display.stop()

        super(AddPagesTestCase, cls).tearDownClass()

               

    def test_add_article(self):

        

        self.driver.get("http://localhost:8000/admin/")

        assert "Wagtail - Sign in" in self.driver.title

        

        elem = self.driver.find_element_by_name("username")

        elem.send_keys("selenium")

        elem = self.driver.find_element_by_name("password")

        elem.send_keys("password")

        elem.send_keys(Keys.RETURN)

        assert "Wagtail - Dashboard" in self.driver.title

        

        self.driver.get("http://localhost:8000/admin/pages/new/foxsite/articlepage/3/")

        assert "New Article Page" in self.driver.title

        elem = self.driver.find_element_by_name("title")

        elem.send_keys("Test Article")

        element2 = self.driver.find_elements_by_xpath("//h2")

        chunks = element2[5]

        chunks.click()

        assert "chunk_items-__prefix__-chunk_title" in self.driver.page_source

        

        (FAILS HERE, Can't find the elements)

        elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_title")

        elem.send_keys("Chunk Title")

        elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_row")

        elem.select_by_visible_text('1')

        elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_colour")

        elem.select_by_visible_text('Hessian')

        

        self.driver.find_element_by_css_selector(".icon-arrow-up").click()

        self.driver.find_element_by_name("action-publish").click()

        assert "Test Article" in self.driver.page_source

[Matthew Westcott]

On 28 Jul 2015, at 10:59, Joss Ingram <joss....@gmail.com> wrote:

> (FAILS HERE, Can't find the elements)
> elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_title")
> elem.send_keys("Chunk Title")
> elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_row")
> elem.select_by_visible_text('1')
> elem = self.driver.find_element_by_name("chunk_items-__prefix__-chunk_colour")
> elem.select_by_visible_text('Hessian')

Hi Joss,
These elements (the ones that exist within <script> tags in the HTML source) are not accessible as page elements - we use <script> tags to contain fragments of HTML that will be inserted into the page later on. (Using script tags for this is a bit of an esoteric trick - the browser sees <script type="text/template">, assumes this is some kind of unknown non-Javascript scripting language, and skips over it.) In this case, the HTML fragment in question is the snippet that gets added when you click 'add new chunk item', and at that point, the '__prefix__' in the name gets replaced with a unique numeric ID.

I haven't used Selenium, but it sounds like the fix is one of two things:
- if the field you're wanting to test is one that exists at the time the page is loaded, then the <script> block is a red herring - you should look elsewhere in the HTML for a field named something like "chunk_items-0-chunk_title"
- if you're trying to recreate the action of adding a new object and filling it in, you'll need to get Selenium to trigger a click on the 'add new chunk item' button (which will probably have the ID "id_chunk_items-ADD"), and *then* look for a suitably-named field with the __prefix__ replaced, such as "chunk_items-2-chunk_title".

Cheers,
- Matt

[Joss Ingram]

Matt,

Ok, thanks. I'll give a try.

Joss

[Joss Ingram]

I figured out I was clicking the wrong element to open the interface for the hidden items, so i used

self.driver.find_element(By.ID,value="id_chunk_items-ADD")

I get around the prefix bits by using       

 elem = self.driver.find_element_by_xpath("//*[contains(@id, 'chunk_row')]")        

all the fields work fine when I send_keys with the exception of any fields that are the hallo.js editor, it won't find the textarea element, again I guess there is some JS here that's actually doing stuff. I've tried clicking the container div too.

Any ideas?

Thanks

Joss

[Joss Ingram]

I used the execute_script method to get some content in there, seems to work

elem = self.driver.find_element_by_xpath("//*[contains(@id, 'chunk_body')]")

self.driver.execute_script('arguments[0].innerHTML="<p>Chunk Body</p>";', elem) 

[Brett Grace]

Do you have to send keys into the corresponding contenteditable div that comes immediately before the texture in order for Hallo.js to copy them into the textarea? I think that's how it works in the real UI. 

[Joss Ingram]

I think I tried that, though I could be wrong. It's working with the JS anyway now but thanks

I'm onto testing an image upload with Selenium and that's proving impossible due to the multiple image uploader issue, I think for a single upload it would be fine.

[Joss Ingram]

Hi, continuing my testing voyage of discovery with Selenium and Wagtail

I'm getting 'Message: Element is no longer attached to the DOM'

when doing this to try and test an image upload

fu = self.driver.find_element_by_id("fileupload")

      

fu.send_keys("/home/Users/joss/Desktop/test.jpg")

fu.send_keys(Keys.RETURN)

any ideas?

[Brett Grace]

Were you able to work past this? I'm sorry I can't help you with some of this stuff even though I pointed you in the direction of Selenium. Many thanks, though, for documenting some of your obstacles and solutions. It will definitely help get my get up to speed if I try to add such testing to one of my own projects.

[Joss Ingram]

No I'm afraid not, I've left any image upload testing for now.

I have got a few basic tests running using normal django unit tests and Selenium. The plan is to build them up as we go, will probably document some of the more esoteric Selenium / wagtail UI bits in a blog post soon.

Thanks for your past help btw