Incremental Validation
If you haven't read the tutorial on Validators, read it first, then come back to this tutorial.
What you'll achieve
By the end of this tutorial, you'll understand incremental validation and will run incremental SQL and Content Validation against your Looker instance.
What is incremental validation?
Incremental validation means that Spectacles only shows you errors that are unique to your branch and don't already exist in production. Running incrementally means that a) you'll often see fewer errors and b) for some validators, you'll get results faster.
For example, let's say we're writing some additional LookML on a development branch called feature/period-over-period. If our Looker instance already has a lot of Content Errors on the production branch, running the Content Validator on our development branch probably won't be very helpful.
Why not? It will be difficult to tell which of the many Content Errors are a direct result of our changes and which Content Errors are unrelated, already existing in production.
Running the Content Validator in incremental mode fixes this: Spectacles will only show us the Content Errors that are unique to our branch, hiding all of the errors that already exist in production.
You can enable incremental validation for the SQL and Content validators, but not the LookML or Assert validators. Let's try running some incremental validations.
Validate content incrementally
Create a new branch off production called spectacles-tutorial and make a change that introduces a Content Error. The easiest way to do this is to rename a dimension you know some content depends on.
Commit the change and push it to remote.
Next, run the following command to start an incremental Content Validation against your branch. You'll need to substitute your config filename and Looker project name in the command.
spectacles content \
--config-file $CONFIG_FILE \
--project $LOOKER_PROJECT \
--branch spectacles-tutorial \
--incremental
Spectacles should identify the Content Errors associated with the LookML change you made, but shouldn't show you any additional, unrelated errors, even if your Production branch has errors.
Change the target branch
By default, Spectacles compares your desired branch or commit to production. What if you wanted to compare to another branch instead of production?
In Looker, branch off of spectacles-tutorial and create a new branch called spectacles-tutorial-more. Make a different change to introduce a new Content Error.
When comparing our -more branch to our first branch incrementally, we would only expect to see one error identified by Spectacles: the newest error that we just created.
Let's try it. To compare to a non-production state, pass --target with our old branch.
spectacles content \
--config-file $CONFIG_FILE \
--project $LOOKER_PROJECT \
--branch spectacles-tutorial-more \
--incremental \
--target spectacles-tutorial
Validate SQL incrementally
You can also validate SQL changes incrementally. This is especially beneficial because Spectacles will only test Explores that have changed, which often makes validation faster for large Looker instances.
For example, to test dimensions on our branch where SQL that has changed, we can run this command:
spectacles sql \
--config-file $CONFIG_FILE \
--project $LOOKER_PROJECT \
--branch spectacles-tutorial \
--incremental \
Spectacles will show any Explores where the SQL has not changed as skipped:
===================== Testing 1 explore [concurrency = 10] =====================
✓ ecommerce.users skipped
✓ ecommerce.sessions passed
Completed SQL validation in 56 seconds.
Like the Content Validator, you can also change the target for incremental SQL validation.
spectacles sql \
--config-file $CONFIG_FILE \
--project $LOOKER_PROJECT \
--branch spectacles-tutorial-more \
--incremental \
--target spectacles-tutorial