Chuck's Automated Smoke Tester Experiment

[Charles Severance]

Hi all,

I have had an idea in the back of my mind to build a “zero-configuration” smoke testing system that would put a web tool like Sakai or Tsugi through its paces without the need for any pre-built functionality tests.

The idea is to treat the site as a “site” and crawl every link from every page - like a web crawler would do and note any pages that cause errors for later verification.  Eventually this tool could run on every version and see if something regressed or got fixed.  It does not fill out forms or submit data - it just click on every url and loads every referenced file until It has hit every page in the system.

I have a really simple MVP working at:

https://github.com/csev/smoker/blob/master/README.md

It can go after a Sakai on port 8080 or a web site like http://www.tsugicloud.org - it has some special login stuff for Sakai that I need to factor into a plugin once I have it working.

If you point it at a Sakai on localhost - it will run thousands of clicks and find a few errors.  See the README for more details.

This is early days and just a nugget of an idea that I have to build something that can make both Sakai and Tsugi (and any web site) better.  As an example it seems to like www.sakailms.com pretty well.

Have fun and watch this space.

/Chuck

$ rm smoker.sqlite 

$ python3 smoker.py 

Enter web url or enter: https://www.sakailms.org

['https://www.sakailms.org']

How many pages:10000

1 https://www.sakailms.org (576885) 200 13

2 https://www.sakailms.org/sakai-lms-features (477907) 200 14

3 https://www.sakailms.org/sakai-lms-community (452946) 200 17

4 https://www.sakailms.org/try-sakai-lms-free (432680) 200 12

5 https://www.sakailms.org/sakai-news (743034) 200 95

...

42 https://www.sakailms.org/sakai-news/search/.hash.lampcamp2019 (488736) 200 15

43 https://www.sakailms.org/_partials/santa/1.11206.0/node_modules/santa-wix-code/dist/wixcode-worker.js Note non text/html page

44 https://www.sakailms.org/sakai-campfire-stories-whoo-attended-camp-this-year (2866) 404 0

45 https://www.sakailms.org/https://www.sakailms.org/sakai-campfire-stories-location-location-location (2866) 404 0

46 https://www.sakailms.org/https://www.sakailms.org/sakai-campfire-stories-dining-and-other-adventures (2866) 404 0

No unretrieved HTML pages found

$ python3 broken.py 

https://www.sakailms.org/_partials/santa/1.11206.0/node_modules/santa-wix-code/dist/wixcode-worker.js

Linked from: https://www.sakailms.org/post/sakai-campfire-stories-dining-and-other-adventures

403 211 application/xml

https://www.sakailms.org/sakai-campfire-stories-whoo-attended-camp-this-year

Linked from: https://www.sakailms.org/post/sakai-campfire-stories-heigh-ho-heigh-ho-it-s-off-to-work-we-go

404 2866 text/html;charset=utf-8

https://www.sakailms.org/https://www.sakailms.org/sakai-campfire-stories-location-location-location

Linked from: https://www.sakailms.org/post/sakai-campfire-stories-heigh-ho-heigh-ho-it-s-off-to-work-we-go

404 2866 text/html;charset=utf-8

https://www.sakailms.org/https://www.sakailms.org/sakai-campfire-stories-dining-and-other-adventures

Linked from: https://www.sakailms.org/post/sakai-campfire-stories-heigh-ho-heigh-ho-it-s-off-to-work-we-go

404 2866 text/html;charset=utf-8

4 rows.

[Michael Greene]

Unfortunately, i'm not going to have time to dig into this until after the freeze and start of term. This does seem similar to something I've been thinking about - running cyprus.io and lighthouse-ci on pull request for automated qa/a11y testing. Do you think it'd be possible to hard code some URLS for testing so you could target the tests? Like a "this PR affects the assignments project, so only run tests on the assignment pages, not the entire build/webapp" kind of thing.

Looks pretty cool.

MG