Incremental Validation
note
If you haven't read the tutorial on Validators, read it first, then come back to this tutorial.
What you'll achieve
By the end of this tutorial, you'll understand incremental validation and will run incremental SQL and Content Validation against your Looker instance.
What is incremental validation?
Incremental validation means that Spectacles only shows you errors that are unique to your branch and don't already exist in production. Running incrementally means that a) you'll often see fewer errors and b) for some validators, you'll get results faster.
info
For example, let's say we're writing some additional LookML on a development branch called feature/period-over-period
. If our Looker instance already has a lot of Content Errors on the production branch, running the Content Validator on our development branch probably won't be very helpful.
Why not? It will be difficult to tell which of the many Content Errors are a direct result of our changes and which Content Errors are unrelated, already existing in production.
Running the Content Validator in incremental mode fixes this: Spectacles will only show us the Content Errors that are unique to our branch, hiding all of the errors that already exist in production.
You can enable incremental validation for the SQL and Content validators, but not the LookML or Assert validators. Let's try running some incremental validations.
Validate content incrementally
In Looker, create a new branch off production called spectacles-tutorial
and make a change that introduces a Content Error. The easiest way to do this is to rename a dimension you know some content depends on. Commit the change and push it to remote.
Next, in Spectacles, start an incremental Content Validation against your tutorial branch.
In the top menu, click Suites
Click New Suite
Name your Suite "Incremental Validation"
Disable all validators except the Content Validator
Configure the Content Validator section with the following options:
- Only incremental errors: Check the box
Click Create Suite
On the Suites page, next to the Suite you just created, click Create Run
- Under Workspace, choose Dev Mode
- Under Branch, choose
spectacles-tutorial
- Click Start
Spectacles will start a new run using incremental Content Validation.
Spectacles should identify the Content Errors associated with the LookML change you made, but shouldn't show you any additional, unrelated errors, even if your Production branch has errors.
Change the target branch
By default, Spectacles compares your desired branch or commit to production. What if you wanted to compare to another branch instead of to production?
In Looker, branch off of spectacles-tutorial
and create a new branch called spectacles-tutorial-more
. Make a different change to introduce a new Content Error.
When comparing our -more
branch to our first branch incrementally, we would only expect to see one error identified by Spectacles: the newest error that we just created.
Let's try it. In Spectacles, we'll start an incremental Content Validation against the spectacles-tutorial
branch instead of production.
In the top menu, click Suites
On the Suites page, next to the Suite you created in the previous step, click Create Run
- Under Workspace, choose Dev Mode
- Under Branch, choose
spectacles-tutorial-more
- Under Incremental Validation > Target Ref, enter
spectacles-tutorial
- Click Start
Validate SQL incrementally
You can also validate SQL changes incrementally. This is especially beneficial because Spectacles will only test Explores that have changed, which often makes validation faster for large Looker instances.
First, let's introduce some SQL errors. On the spectacles-tutorial
branch, introduce some errors to the sql
field of some of your dimensions and commit the change.
Next, let's edit our Suite to perform incremental SQL validation in addition to Content Validation.
In the top menu, click Suites
Click the Suite named Incremental Validation
Enable the SQL Validator
Configure the SQL Validator section with the following options:
- Only incremental errors: Check the box
Click Update Suite
On the Suites page, next to the Incremental Validation Suite, click Create Run
- Under Workspace, choose Dev Mode
- Under Branch, choose
spectacles-tutorial
- Click Start
Spectacles will start a new run using incremental SQL and Content Validation.
Spectacles should identify the errors associated with the LookML change you made, but shouldn't show you any additional, unrelated errors, even if your Production branch has errors.
Spectacles will mark any Explores where the SQL has not changed as Unmodified
, indicating that those Explores were skipped and not tested.
Like the Content Validator, you can also change the target for incremental SQL validation.