.. title: Package and Distribute Your Test Suite .. slug: package-and-distribute-your-test-suite .. date: 2016-02-23 22:26:33 UTC .. updated: 2016-02-23 22:26:33 UTC .. tags: testing .. category: .. link: .. description: .. type: text Automated test suites must be packaged for installation and distributed just like your other build artifacts. They are not some special things that live in a code repository (repo) separate from the rest of your code. My team was writing tests and keeping them all in a git repo. The repo would be cloned and then PyTest would run these tests. All users of tests - from developers to Continuous Integration (CI) and everyone else in between - would run steps akin to:: git clone git@github.com:username/testrepo.git cd testrepo git checkout main pip install -r requirements.txt py.test tests/ This worked for a while but was not a good fit as time progressed for these reasons: .. TEASER_END: Read more * All users had to muck around with git * There was no way to release the test suite at release time * Only the current state of tests could be used with no good option to rollback to a previous state My team set out to create a process more like: * Release automated test suite as an installable artifact from CI * All users do a ``pip install --upgrade testsuite`` and execute script ``run_test_suite`` When this new process was implemented it gave us many benefits: * Tests are released as installable Python packages multiple times a day * Each project - production code or tests - is treated the same and goes through the same CI pipeline; no special snowflakes * The test suite itself is run through CI more easily and accepted or rejected * Teams can leverage CI to create custom test suites * During product release tests get released as well * It's easy to pick and choose the state of test suite to use based on release versioning * Users can install and uninstall test suites just like any other software; less cognitive load Based on my experience I highly recommend your teams to write automated test suites to be packaged, released, distributed, and installed like any other software component. P.S. I'm planning to write a more technical post (over on Code Ghar) on how we release and use our automated test suite. I'll link to it here when it's written.