Limiting Resource Consumption

The Spectacles SQL Validator is designed to be lightweight and avoid consuming excess resources within Looker and within your data warehouse. We do this by limiting query concurrency and with pseudo dry-run queries.

Query concurrency

Spectacles caps the number of queries it runs simultaneously in your data warehouse. By default, we cap concurrency at 10 simultaneous queries. On modern data warehouses, you can comfortably increase this number to 20-50 concurrent queries.

If you notice a slowdown in Looker while running SQL validation, consider decreasing this value to see if it helps.

Pseudo dry-run queries

All Spectacles queries include a LIMIT 0 and WHERE 1=2 clause. These clauses effectively instruct the query planner in your data warehouse not to process data but to check the validity of the SQL. These queries are similar to running a dry run query.

For BigQuery users, LIMIT 0 queries don't even scan data, so you shouldn't be charged for the queries that Spectacles runs.