Aggressive Savings
Low cost, higher risk of queuing during bursts.
Where to focus your FinOps efforts today.
v… | Released …
An enterprise-grade diagnostic and simulation suite for Google Cloud BigQuery costs. Analyzes historical telemetry, query workloads, and storage configurations across GCP organizations to maximize cost efficiency.
Note: This analysis evaluates physical vs logical billing specifically for BASE TABLE storage (excluding materialized views and external tables).
| Project | Dataset | Current Model | Recommended Model | Monthly savings ($) | Savings (%) | Action |
|---|
Partitioning and clustering recommendations from two complementary engines: Google Active Assist (compute-optimized) and our Static Schema Auditor (bytes-scanned-optimized).
Optimizes for Compute (Slot-ms). Analyzes the last 30 days of query history to calculate the net compute savings of clustering/partitioning.
Optimizes for Storage & On-Demand Costs (Bytes Scanned). Performs a structural analysis of tables >1GB to enforce clustering/partitioning best practices.
These partitioning and clustering recommendations are generated by Google's native AI engine based on your query history telemetry over the last 30 days.
| Project | Dataset | Table | Recommendation | Suggested Columns | Partition Column | Est. On-Demand Savings | Est. Editions Savings |
|---|
This module statically audits your database schemas to proactively detect massive, unclustered, or unpartitioned tables before they are queried and incur high costs.
| Project | Dataset | Table | Rows | Logical Size | Partitioned | Clustered | Suggested Columns | Risk Status |
|---|
This analyzer profiles your historical BigQuery jobs to determine their most cost-effective billing model (On-Demand vs. Editions/Autoscaler). It is designed to act as an Advanced Workload Router, helping you identify structural "slot hogs" (better for On-Demand) and "data hogs" (better for Editions).
Pro-Tip for Spikey Workloads: For extremely sparse workloads with high peaks, it is often cheaper to move those specific projects back to On-Demand billing (paying per TB processed) instead of Editions. This eliminates the cost of idle slots during cool downs entirely. This analyzer is designed to identify exactly these candidates!
To identify the specific inefficiencies of individual queries, this tool evaluates each job in isolation. In reality, BigQuery Editions highly benefit from concurrency (sharing idle slots across many overlapping jobs). Because blending queries together hides individual bad actors, we use a Smart Job-Centric approach, applying advanced heuristics to approximate how an isolated job behaves against the BigQuery Autoscaler:
This scatter plot visualizes jobs based on their projected costs. Jobs above the dotted line are more expensive on Editions (CPU-heavy). Jobs below the dotted line are more expensive on On-Demand (IO-heavy).
| Project ID | On-Demand Cost ($) | Editions Cost ($) | Error Tax ($) | Savings ($) |
|---|
| Project | Job ID | Current Model | Recommended Model | Profile | Savings ($) | Savings (%) | Action |
|---|
BigQuery slots are virtual compute units used to execute jobs. This tool analyzes slot utilization at a high-resolution, second-by-second granularity using INFORMATION_SCHEMA.JOBS_TIMELINE to help you size reservations effectively. While Google Cloud's Slot Estimator provides recommendations based on hourly averages, this high-resolution approach identifies short-lived peaks to ensure you have enough capacity to avoid slot contention during bursty workloads. It provides the following metrics:
Note: All recommendations are rounded to the nearest 50 slots, matching BigQuery's scaling increments.
Scope Note: The chart shows all query usage across your organization (including On-Demand queries). However, the Recommendations table only analyzes jobs that ran within a reservation.
This view answers two critical questions for capacity planning:
Descriptive: Based on configuration logs of what was actually provisioned and billed.
Why this matters: BigQuery Enterprise slot capacity is highly dynamic. Autoscaling fluctuates continuously based on workload, and capacity commitments can change at any time. To understand your true cost, we must calculate the exact "area under the curve" (time x slots) down to the second. This view helps you understand your compute overages and optimize commitments.
Note: These metrics represent the compute capacity provisioned and billed by Google Cloud, not the actual slots consumed by your queries.
What percentage of our compute hours were spent on autoscaling vs our baseline?
Descriptive: Shows the proportion of provisioned capacity types.
| Reservation ID | Recommendation | Action |
|---|
BigQuery's Reservation Predictability features allow you to control how your reservations scale beyond their baseline slot capacity using different scaling modes. These modes determine the order and types of additional slots (idle or autoscaler) that can be accessed, up to a configured max_slots limit.
ALL_SLOTS (Predictable Performance & Maximum Potential)
This mode aims to provide a balance of performance and cost-efficiency by utilizing all available slot sources. After the reservation's baseline slots (slot_capacity) are fully consumed, it will:
max_slots.Use Case: Good for workloads that can benefit from opportunistic use of idle capacity but also require the guarantee of autoscaling to meet peak demand up to a predictable limit.
IDLE_SLOTS_ONLY (Most Cost-Effective)This mode prioritizes cost savings by only using free capacity. After baseline slots are used, it will:
max_slots.Use Case: Ideal for cost-sensitive workloads that can tolerate performance variability. Depends entirely on the availability of idle slots.
AUTOSCALE_ONLY (Dedicated Scaling)This mode ensures that the reservation scales only with dedicated resources, without relying on shared idle capacity. After baseline slots are used, it will:
max_slots.
Use Case: Suitable for workloads requiring consistent scaling behavior based purely on their own demand. Requires ignore_idle_slots to be set to true.
Prerequisite: To use any of these scaling modes with max_slots, "Reservation-based fairness" must be enabled on the administration project.
Select a risk profile to see the recommended baseline slots and estimated costs.
Low cost, higher risk of queuing during bursts.
Optimal balance of cost and performance.
Zero queuing risk, highest cost.
Based on our real query history, what size should our reservations be?
Prescriptive: Suggested optimal capacity based on actual query workload demand.
Why are these recommendations higher than the official Google Cloud Slot Estimator?
The official GCP tool optimizes for cost based on your actual billed usage, which is artificially capped by your current max_slots limit. It only sees what was provisioned, not what was queued.
This FinOps Optimizer analyzes your unconstrained query demand to optimize for performance. The delta between the two tools represents the query queueing and performance degradation your users experience during bursts.
| Reservation ID | Autoscaler baseline (P90) | Autoscaler max (P90) | Autoscaler max (P99) | Autoscaler max (Peak) |
|---|
Estimate savings from enabling Fluid Scaling (per-second billing).
This simulation estimates the savings you could achieve by enabling Fluid Scaling on your reservations, which eliminates the 60-second minimum billing floor for short queries.
According to the official Google Cloud blog, BigQuery fluid scaling helps lower costs by up to 34% on average for autoscaling workloads by scaling up resources instantly when needed and scaling back when they don't. While this tool provides a conservative estimate based on query logs, the blog provides a good benchmark for overall expected savings.
Conservative Math Assumption: The tool calculates "wasted slot hours" by multiplying the actual slots used by the query by the unused fraction of the first minute. This assumes queries can "pack" into existing capacity within the same reservation.
Cross-Reservation Sharing Limitation: Query logs do not show if slots were shared with other reservations in real-time. However, the official Google Cloud documentation states that autoscaled slots are not shared across reservations. Therefore, if a short query triggers an autoscaler event and runs in isolation within its reservation, those slots are locked and wasted for the full 60 seconds, even if other reservations could use them! This means if your workload is spiky and isolated, your actual waste is much larger, and savings could be up to 35x higher than displayed.
Lookback Window: Savings are calculated for the specific days selected in parameters. Projecting this over a year or across more queries will reveal higher true scale value!
| Reservation ID | Status | Legacy Slot-Hrs | Fluid Slot-Hrs | Total Used Slot-Hrs | Recoverable Slot-Hrs | % Savings | $ Window | $ Monthly | $ Annual |
|---|
Top 100 (query pattern × reservation) combinations ranked by estimated savings impact. The same pattern may appear once per reservation it runs on.
| Pattern | Sample Job ID | Workload Type | Reservation | Job Count | Avg Duration (s) | Avg Slots/Job | Exposure Score | Indicative Savings (Window) |
|---|
This simulator helps you find the most cost-effective baseline by balancing the cost of idle slots (waste) against the cost of autoscaling (bursts). It tests every baseline size and calculates the total bill for different commitment tiers. The turning point with the lowest 1-Year Commit cost will be highlighted.
This matrix simulates costs across different Baseline choices. For each baseline level, we calculate how much capacity would be covered by the fixed baseline vs. how much would spill over into variable Autoscale capacity.
In Buckets
Shows frequency distribution. E.g., how many minutes your total usage was within that specific slot range.
Utilization %
Measures how efficiently that baseline is used. 100% means no slots wasted; lower percentages imply idle capacity.
Autoscale
The volume of compute (in hours or months) that exceeded the baseline and required dynamic scaling.
Total Monthly Cost
The final calculated sum of (Autoscale + Baseline) projected to a standard 730-hour billing month.
Tip: Look for the row that minimizes the Total Monthly Cost for your target commitment tier to find your optimal efficiency point.
Resolution Note: This simulation aggregates slot usage to 1-minute averages to balance performance and accuracy. This acts as a good approximation for Fluid Scaling, but may smooth out sub-minute spikes.
| Commitment Type | Autoscale Slots | Recommended Baseline | Estimated Monthly Cost |
|---|
| In Buckets | Below Baseline | Autoscale | Autoscale Cost | Baseline Cost | Total Monthly Cost | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bucket | Minutes | Slots | Utilization % | Slot-Hours | Slot-Month | EE PAYG | EE PAYG | EE 1 Year | EE 3 Years | EE PAYG | EE 1 Year | EE 3 Years |
Google Cloud billing does not attribute BigQuery Editions costs to the specific projects that ran the queries. Following hybrid cost attribution model is helping with that:
| Project ID | Reservation ID | Direct Usage Cost ($) | Allocated Waste ($) | Total Cost Attribution ($) |
|---|
| Project ID | Total Slot Hours |
|---|
This tool identifies reservations experiencing a "continuous trickle" of small, frequent queries. Documentation and expert guidelines recommend specific strategies to optimize these workloads and avoid unnecessary autoscaler activation overhead.
Setting a Baseline > 0 is recommended to trade a small fixed cost for lower query latency. The slots are already allocated, allowing light interactive queries to start instantly without waking up the autoscaler.
Pro-Tip (Slot Sharing): To reduce waste, place multiple reservations on the same Admin Project. Idle baseline slots are automatically shared with other workloads!
Since these "trickle" queries usually scan very little data (low I/O) but are penalized by the 1-minute slot minimum on Editions, they are often much cheaper on the On-Demand model (which charges purely per TB processed).
Implementation: You can mix pricing models in your organization. Simply bind the projects running these small queries to a dedicated Billing Project configured for On-Demand.
BigQuery's Advanced Runtime can automatically optimize short queries by consolidating execution into a single stage and skipping the shuffle layer. This reduces slot usage and latency dramatically.
Tip: To leverage this, keep query working sets small and result sizes small. Ensure you have enabled the Advanced Runtime for your project. Read more in the Google Cloud Blog.
Compare the cost of the job under both models: calculate the On-Demand cost based on total_bytes_billed and compare it against the Editions cost based on total_slot_ms (factoring in the 1-minute minimum for bursts). Use the Query cost optimizer tab in this tool to see these specific cost comparisons for individual jobs!
Uses global settings for Project, Region, and Lookback Window.
Are we paying for idle slots due to a continuous trickle of small queries?
Prescriptive: Identifies reservations with high frequency of low-slot queries that might benefit from a fixed baseline.
| Reservation ID | Total Flagged Hours | Peak Hourly Queries | Top Offending Projects | Recommendation |
|---|
Grid showing when the trickle happens across the week (Hour of Day vs Day of Week).
| Hour | Sun | Mon | Tue | Wed | Thu | Fri | Sat |
|---|
Are these queries causing inefficient scale-ups? This table reveals the most frequent small queries that run during periods of high query counts but low slot usage. Look for queries with high frequency but very low (or 0 MB) data processed (like transactions or literal inserts).
Why 0 MB? These queries are the perfect example of what the blog post calls "short" or "trickle" queries. They process no data but still consume slot time and trigger overhead. They are excellent candidates for Short Query Optimizations or for being routed to On-Demand pricing (where a 0 MB query costs nothing or at most the 10MB minimum charge, avoiding the 1-minute slot penalty on Editions).
| Query Text | Project | Example Job ID | Frequency | Avg Slot Hours | Avg Duration (s) | Avg Bytes Processed | Recommendation |
|---|
This view helps you identify which users are running the most queries, scanning the most data, and consuming the most slot hours. Use this information to optimize workloads, educate users, or allocate costs.
Status: The current implementation uses a direct usage calculation (Data Billed × On-Demand Rate) and (Slot Hours × Editions Rate).
Complexity: In a shared reservation environment, attributing exact costs to users is hard because idle slots are shared and bursts are unpredictable. True cost might require a proportional allocation of reservation waste.
Uses global settings for Project, Region, and Lookback Window.
| User Email | Query Count | Total Data Billed | Total Slot Hours | Est. On-Demand Cost | Est. Editions Cost |
|---|
BigQuery's History-Based Optimization (HBO) is a native feature that uses telemetry from completed executions of similar queries to apply additional optimizations and improve query performance (both slot time and latency). History-Based Optimization is not for reservations only. It applies to all queries in BigQuery, including those running on the On-Demand pricing model.
When you run a query repeatedly, BigQuery learns from each run and applies optimizations like semi-join reduction, join commutation, and parallelism adjustment. This continuous learning continues until no more optimizations can be applied. Optimizations are only used if there is high confidence of benefit, and are revoked if they fail to improve performance.
Since HBO is enabled by default and cannot be easily toggled off for A/B testing in some environments, this tool helps you estimate its impact. We compare optimized queries with their most recent unoptimized runs to show you the percentage of execution time saved.
Note: If HBO has been disabled for a project or organization, you can use the provided DDL in the status panel to re-enable it.
Uses global settings for Project, Region, and Lookback Window by default, or use the overrides below. (Project Scope Only)
| Job ID | % Time Saved | New Elapsed (ms) | Original Elapsed (ms) |
|---|
While the existing Storage Cost Optimizer tab handles the actual billing costs (should we pay for compressed or uncompressed data?), this Storage Hygiene Auditor tab handles architectural waste (are we burning money by leaving default settings on tables with high churn?).
Identify datasets with high churn rates where reducing the default 7-day Time Travel window to 2 days can instantly cut storage costs.
Detect tables that are fully dropped and recreated daily, highlighting opportunities to switch to incremental MERGE or partitioned updates.
Uses global settings for Project and Region.
| Project | Dataset | Table Name | Current Size (GiB) | Time Travel Size (GiB) | Churn Ratio | Health Status |
|---|
This scanner helps identify structural and pipeline inefficiencies that waste compute and storage costs.
User Story: As a Data Architect, I want the app to detect jobs using the legacy tabledata.insertAll API or high-frequency SQL INSERTs, so I can migrate them to the Storage Write API and cut streaming costs by 50%.
User Story: As a FinOps Manager, I want to identify Materialized Views (MVs) where the background refresh cost (Slots consumed) is actually higher than the compute saved during querying, so I can pause auto-refresh if it's wasting money.
Applying a LIMIT clause to a SELECT * query does not affect the amount of data read. You are billed for reading all bytes in the entire table, and the query counts against your free tier quota.
Best practice: LIMIT speeds up performance, but doesn’t reduce costs. For data exploration, consider using BigQuery’s (free) table preview option instead.
This table identifies expensive queries (>100 GB scanned) using inefficient patterns. Each row shows the impact, root cause, and suggested fix.
| User Email | Project ID | Job ID | Billed (GB) | Abuse Type | Estimated Waste ($) | Query Snippet | Suggested Fix |
|---|
Audits data ingestion pipelines for compute waste. Identifies service accounts running high-frequency SQL INSERT statements. Migrating these to the modern Storage Write API consumes zero query slots and cuts streaming costs by 50%.
| User Email | Project ID | Insert Count (Last 1D) | Wasted Slot Hours | Estimated Waste ($) | Recommendation |
|---|
This table shows the compute cost (slot hours) consumed by background refreshes for each Materialized View. High refresh costs with low query usage indicate waste.
| Project ID | Dataset | MV Name | Refresh Count | Total Slot Hours |
|---|
This table lists jobs where a single worker node took 10x longer than average to process data, indicating unbalanced joins or grouping keys.
| Project ID | Job ID | User Email | Stage Name | Avg Compute (ms) | Max Compute (ms) | Skew Ratio |
|---|
This table lists heavy queries running with INTERACTIVE priority that are candidates for BATCH priority.
| Project ID | Job ID | User Email | Reason | Duration (Min) | Total Slot MS |
|---|
Temporary, dev, and sandbox datasets without expiration rules are one of the biggest sources of passive storage waste.
| Project ID | Dataset ID | Status |
|---|
If "Require Partition Filter" is off, users will accidentally run full-table scans on massive tables instead of filtering by date.
| Project ID | Dataset ID | Table Name | Partition Type |
|---|
This table lists queries where a Materialized View was considered but rejected by the optimizer.
| User Email | Job ID | MV Name | Rejected Reason |
|---|
This table lists queries that triggered resource warnings (e.g., approaching limits).
| User Email | Job ID | Warning |
|---|
BigQuery analyzes query execution stages and publishes performance telemetry. This dashboard identifies queries affected by performance bottlenecks, including slot contention, insufficient shuffle quota, and significant changes in input data volume.
Uses global settings by default, or use the overrides below. (Project Scope Only)
Queries that experienced significant delays due to slot resource limitations.
| Job ID | User Email | Project ID | Stage ID |
|---|
Queries slowed down due to limits in BigQuery's memory-based shuffle storage quota.
| Job ID | User Email | Project ID | Stage ID |
|---|
Queries with a dramatic increase in input data size relative to historical executions.
| Job ID | User Email | Project ID | Data Increase % |
|---|
While usually viewed purely as a performance feature, BI Engine is a primary FinOps cost-saving tool.
bytes_billed to zero.slot_ms consumed, saving money in your reservation.Dashboards are often slow or expensive because queries silently fail to use BI Engine (e.g., due to unsupported SQL functions). This tool parses the specific failure reasons in the logs to tell you exactly why your dashboards are scanning disk instead of RAM.
Note: The ROI calculation assumes On-Demand pricing of $6.25 per TB for the bytes saved.
Uses global settings for Project and Region.
| User Email | Job ID | Processed (GiB) | Billed (GiB) | Mode | Failure Reasons |
|---|
Enabling schema context requires the executing service account/identity to hold:
- bigquery.tables.get
- bigquery.tables.list
- bigquery.jobs.listAll (if auditing other users)
Admin Note: Treat the shared service account as having broad metadata-read access across the project and scope permissions accordingly.
This feature uses BigQuery's native Vertex AI integration (AI.GENERATE_TEXT) to analyze your top expensive queries.
roles/aiplatform.user) role assigned on the project.This analysis sends table schemas (DDLs) to Vertex AI to verify partitioning/clustering. Row-level data is NEVER sent.
Audits your most expensive queries (Top 5, 10, or 20) based on slot usage. The system dynamically over-fetches the highest-cost jobs from BigQuery, automatically filtering out queries containing deleted tables or system views that lack sufficient schema context, ensuring you only receive the precise number of valid, actionable recommendations you requested. Gemini then audits the final list for performance anti-patterns, hiding perfectly optimized queries.
A semantic, serverless audit of your most resource-hungry SQL queries. The system analyzes raw query telemetry, retrieves referenced table schemas (DDLs) from the metadata catalog, and leverages Gemini to highlight performance anti-patterns with tailored refactoring advice.
| Job ID | User Email | Total Slot MS | Query SQL | Schema Coverage | AI Optimization Advice |
|---|
⚠️ Schema Context Will Be Sent to Vertex AI
To provide deeper, metadata-aware optimization advice (like checking partition and cluster keys), this tool reads the DDL (table definitions) of tables referenced in your queries and includes them in prompts sent to Agent Platform.
DDLs may contain sensitive information, including:
OPTIONS)Reassurance: Row-level data is NEVER read, processed, or transmitted. Only structural metadata and query text are analyzed.
By proceeding, you confirm you are authorized to send this metadata to Agent Platform. Do not run this on datasets governed by data-residency, sovereignty, or contractual restrictions without prior security review.