Tests in Tilt
With the Tilt
test primitive, you can run your tests from Tilt itself. Take advantage of Tilt’s file-watching and responsiveness, and the power and customizability of Starlark, to integrate testing with the rest of your Tilt workflow.
How to Use It
Define tests in your Tiltfile with the
test('cart-tests', 'go test ./pkg/cart', deps=['./pkg/cart']) test('py-tests', 'pytest app/', deps=['app/']) test('slow-integration-test', 'tests/integration.sh', trigger_mode=TRIGGER_MODE_MANUAL, auto_init=False)
test supports all the same arguments as
local_resource see API reference. The only notable difference today is that for tests, parallelism is on by default (i.e. the default of
Your tests will appear in their own section of the overview grid, and alerts will be reflected on the test cards and in the status bar, so that you know at a glance when something is wrong.
Just like any other resource managed by Tilt, you can drill into a test and see logs and other details.
In the log view, you can filter the sidebar by item type (if, say, you want to see only tests, or you want to filter tests out and focus on your app in the cluster). You can also toggle “Alerts On Top” sorting, to make it even more visible when something has gone wrong and needs your attention
When Tests Run
Just as with a
local_resource, you can associate files with a test via the
deps=[...] parameter. When any of these file dependencies change, Tilt will re-execute the test:
# A test depending on an entire directory test('cart-tests', 'go test ./pkg/cart', deps=['./pkg/cart']) # A test depending on specific relevant files test('unit-test-remittance', 'pytest remittance_test.py', deps=['app/remittance.py', 'app/remittance_test.py', 'app/payment_utils.py'])
(More on how to intelligently set your dependencies below.)
Sometimes, you don’t want your test(s) to execute automatically. Maybe you want to iterate on a few tests and disable the rest, or prevent an expensive integration test from running all the time. Use the “auto” slider in the UI to toggle automatic execution on or off for a given test.
You can also control this behavior via the Tiltfile, using the
Programmatically Registering Tests
Part of what’s great about Starlark (the dialect of Python that Tiltfiles are written in) is that it’s a programming language, which makes Tiltfiles extremely flexible. Here are some examples of different ways you can register your tests to Tilt.
Your entire unit test suite
If your whole unit test suite runs relatively quickly/has caching (like Go tests), you can just run the whole thing when relevant files change:
def all_go_files(path): return str(local('find . -type f -name "*.go"')).split("\n") test('go-tests', 'go test ./... -timeout 30s', deps=all_go_files())
Go tests by package
Here, your tests are split into different cards (one per Go package). This approach gives you both more control over when certain tests run (in the snippet below, tests for package
foo run only when a file in directory
foo changes), and more visibility into what is going wrong (because tests are broken up more, so it’s easier to see what failed).
The trade-off for speed and visibility here is looser matching between files and the tests they affect (the above example will rerun tests for package
foo when its files or anything it depends on change).
CWD = os.getcwd() def all_go_package_dirs(): pkgs_raw = str(local('go list -f "" ./...')).rstrip().split("\n") pkgs =  for pkg in pkgs_raw: cleaned = pkg.strip() if cleaned: pkgs.append(cleaned) return pkgs def pretty_name(s): return s.replace(CWD, '').lstrip('/') for pkg in all_go_package_dirs(): name = pretty_name(pkg) test(name, 'go test %s' % pkg, deps=[pkg])
JS tests by test file
Alternately, you can split your tests up even further, into one card per test file.
This snippet naively matches by prefix–e.g.
foo.test.tsx will run on changes to
foo.test.tsx. Note that it’s written for a flat file hierarchy (i.e. all the JS files and tests live at the root of
web/src), but is easy to modify to fit your directory structure.
web_src_files = [os.path.basename(f) for f in listdir('web/src')] test_files = [f for f in web_src_files if f.endswith('test.tsx')] for tf in test_files: cmd = "cd web && yarn test --watchAll=false %s" % tf slug = tf.replace('.test.tsx', '') deps = [os.path.join('web/src/', f) for f in web_src_files if f.startswith(slug)] test(slug, cmd, deps=deps)
There’s a lot more we want to do with this feature, but above all, we want to know what our users want. Let us know what you think!