Skip to main content

Incremental Validation

note

If you haven't read the tutorial on Validators, read it first, then come back to this tutorial.

What you'll achieve​

By the end of this tutorial, you'll understand incremental validation and will run incremental SQL and Content Validation against your Looker instance.

What is incremental validation?​

Incremental validation means that Spectacles only shows you errors that are unique to your branch and don't already exist in production. Running incrementally means that a) you'll often see fewer errors and b) for some validators, you'll get results faster.

info

For example, let's say we're writing some additional LookML on a development branch called feature/period-over-period. If our Looker instance already has a lot of Content Errors on the production branch, running the Content Validator on our development branch probably won't be very helpful.

Why not? It will be difficult to tell which of the many Content Errors are a direct result of our changes and which Content Errors are unrelated, already existing in production.

Running the Content Validator in incremental mode fixes this: Spectacles will only show us the Content Errors that are unique to our branch, hiding all of the errors that already exist in production.

You can enable incremental validation for the SQL and Content validators, but not the LookML or Assert validators. Let's try running some incremental validations.

Validate content incrementally​

In Looker, create a new branch off production called spectacles-tutorial and make a change that introduces a Content Error. The easiest way to do this is to rename a dimension you know some content depends on. Commit the change and push it to remote.

Next, in Spectacles, start an incremental Content Validation against your tutorial branch.

  1. In the top menu, click Suites

  2. Click New Suite

  3. Name your Suite "Incremental Validation"

  4. Disable all validators except the Content Validator

  5. Configure the Content Validator section with the following options:

    • Only incremental errors: Check the box

  6. Click Create Suite

  7. On the Suites page, next to the Suite you just created, click Create Run

    • Under Workspace, choose Dev Mode
    • Under Branch, choose spectacles-tutorial
    • Click Start

Spectacles will start a new run using incremental Content Validation.

Spectacles should identify the Content Errors associated with the LookML change you made, but shouldn't show you any additional, unrelated errors, even if your Production branch has errors.

Change the target branch​

By default, Spectacles compares your desired branch or commit to production. What if you wanted to compare to another branch instead of to production?

In Looker, branch off of spectacles-tutorial and create a new branch called spectacles-tutorial-more. Make a different change to introduce a new Content Error.

When comparing our -more branch to our first branch incrementally, we would only expect to see one error identified by Spectacles: the newest error that we just created.

Let's try it. In Spectacles, we'll start an incremental Content Validation against the spectacles-tutorial branch instead of production.

  1. In the top menu, click Suites

  2. On the Suites page, next to the Suite you created in the previous step, click Create Run

    • Under Workspace, choose Dev Mode
    • Under Branch, choose spectacles-tutorial-more
    • Under Incremental Validation > Target Ref, enter spectacles-tutorial
    • Click Start

Validate SQL incrementally​

You can also validate SQL changes incrementally. This is especially beneficial because Spectacles will only test Explores that have changed, which often makes validation faster for large Looker instances.

First, let's introduce some SQL errors. On the spectacles-tutorial branch, introduce some errors to the sql field of some of your dimensions and commit the change.

Next, let's edit our Suite to perform incremental SQL validation in addition to Content Validation.

  1. In the top menu, click Suites

  2. Click the Suite named Incremental Validation

  3. Enable the SQL Validator

  4. Configure the SQL Validator section with the following options:

    • Only incremental errors: Check the box

  5. Click Update Suite

  6. On the Suites page, next to the Incremental Validation Suite, click Create Run

    • Under Workspace, choose Dev Mode
    • Under Branch, choose spectacles-tutorial
    • Click Start

Spectacles will start a new run using incremental SQL and Content Validation.

Spectacles should identify the errors associated with the LookML change you made, but shouldn't show you any additional, unrelated errors, even if your Production branch has errors.

Spectacles will mark any Explores where the SQL has not changed as Unmodified, indicating that those Explores were skipped and not tested.

Like the Content Validator, you can also change the target for incremental SQL validation.