Tox Unittests
Sebastian Kral
I know there are a couple of questions surrounding Maya unittests out there but I didn't find a best practices thread. I am trying to find out what the current best way of automating unittests (which rely on pymel, cmds or any other Maya internal API) is? I would love to use tox to automate these tests but a quick look suggested it's not as easy as I thought.
I see two possible ways of implementing this right now please let me know if you see any other.
1. Getting mayapy to play nice with a virtual environment. Probably need to write some custom script which sets up the virtual environment for tox to use. I tried to tell tox to use mayapy which resulted in some missing file errors when tox tried to create the virtual environment.
2. Compiling a Python version which matches mayapy and can be used to create a proper virtual environment. In the end we will still want to "import maya.standalone" to make the unittests work. Need to compile Python and write custom virtual environment creator to create it properly to make all the dependencies work.
Using tox would make the whole process very flexible and people know how to use it. I hope someone already invested some time and brain power and can share his process with the world.
Cheers,
Sebastian
Marcus Ottosson
Hi Sebastian,
I have no experience with tox per-se, but I do have experience unittesting with Maya and can share with you my approach.
Firstly, I’m glad to see interest in this. For such an important thing, it’s both surprisingly invisible and unknown. Even in large production facilities who rely largely on code for day-to-day work. To simply test ones work under a variety of circumstances.
I took a brief glance at what tox is about, and I take it you use it for its ability to run tests in virtual environments, one per version of Maya. In that case, this approach should suit you as well.
The overall recipe is this.
- Setup a Docker image per environment, including Python and/or Maya versions and dependencies
- Run each test per Docker image
- Optionally setup continuous integration
Docker
If you aren’t familiar with Docker yet, you can think of it like virtual environments within which you run tests. In this case, each virtual environment is really a full OS, but without the overhead of a full virtual machine. Much like a regular Python virtual environment in terms of use and performance.
For Maya, I’ve setup a series of images for most versions that you run like this.
$ docker run -ti --rm mottosso/maya
Following this command, Docker will download the image and put you in a “virtual environment” with the latest version of Maya available, and mayapy added to PATH
$ mayapy --version
Python 2.7.6
This particular environment looks like this.
Which is built on-top of this environment.
You could run tests non-interactively like this.
$ cd my_tests
$ docker run --rm -v $(pwd):/root mottosso/maya nosetests --verbose
In this case, my tests are located in the my_tests directory, which is mounted into the virtual environment. Once up and running, it’s running nosetests as though you were inside of the interactive terminal typing nosetests --verbose yourself.
Depending on your Docker image, you could have pytest or any other testing mechanism installed of course, I just happen to use nose here.
I mentioned I had many versions of Maya available, you can run those too by specifying their version explicitly.
$ docker run -ti --rm mottosso/maya:2016sp1
Available versions range from 2013sp1 to 2016sp1. Some of the latest ones, such as 2016ext2 are on the way as well.
Continuous Integration
When I set this up, I had in my sights an environment within which I could have tests run automatically, like I do with regular Python projects on GitHub with AppVeyor or Travis-CI. So I developed a small Travis clone that I could run locally or on my own server, something suited for DCC testing of this nature.
What it does is, similar to Travis, whenever a commit is made, a webhook is sent to the server in which this Flask application is running. The Flask application then subprocess.Popen one or more Docker images as specified in the configuration file and provides a simple web page with results.
The back-end is documented in the wiki.
The front-end isn’t public at the moment, but as a plain Flask application you could simply run app.py as-is to see what it looks like. It isn’t particularly fancy and simply provides an interface to the console output of each build, the highlight is how it does it independently and isolated per build. Like Travis.
At this point, if you were to head down this path, you would have an automated testing environment set up in which you could run tests in any number of Python versions, Python bundled versions and various permutations of Maya and plug-ins and other DCCs at each commit to any related project.
Good luck, hope it helps, and let me know if you have any questions!
Best,
Marcus
--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/89d07928-e674-4cd4-abb6-94cee650e203%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Cesar Saez
Andres Weber
Sebastian Kral
--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/a2978b9a-5d4e-4bcb-81a1-04f5e0f94d6b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Sent from mobile device
Marcus Ottosson
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/CAFM1w9BXnnx3vzst0noZEZDkDecojjPTNePpLWQocV53ejBz5A%40mail.gmail.com.