# Real-Time Analytics: When You Need It and When You Don't

> Not every metric needs real-time data. Here's how to decide what needs sub-second freshness, what can be cached, and how pre-aggregation handles both.

"We need real-time analytics" is one of the most common requests in data engineering. It's also one of the most misunderstood. When the VP of Sales says "real-time," they usually mean "faster than the dashboard that refreshes overnight." When the CTO says it, they might mean sub-second event streaming. The gap between those two definitions is a 6-month infrastructure project.

Most teams don't need true real-time. They need fast enough. And "fast enough" is achievable with pre-aggregation caching at a fraction of the complexity and cost of a streaming architecture.

## What is real-time analytics?

Real-time analytics means querying data with minimal latency between when an event happens and when it's visible in your analytics. The spectrum:

| Freshness | Latency | Architecture | Use case |
|-----------|---------|-------------|----------|
| **True real-time** | < 1 second | Event streaming (Kafka, Flink) | Fraud detection, stock trading, live monitoring |
| **Near real-time** | 1-60 seconds | Micro-batch or streaming | Operational dashboards, alerting |
| **Frequent refresh** | 1-60 minutes | Scheduled refresh + caching | KPI dashboards, AI agent queries |
| **Batch** | Hours to daily | Scheduled ETL | Board reports, monthly summaries |

Most analytics use cases fall in the "frequent refresh" category. Revenue by region doesn't need sub-second freshness. Active users in the last hour doesn't need event streaming. A pre-aggregation cache that refreshes every 15 minutes covers 90% of what teams call "real-time."

## When you actually need real-time

True real-time analytics (sub-second latency from event to query result) is worth the infrastructure investment when:

- **Fraud detection.** Every second of delay is potential fraud that slips through.
- **Live monitoring.** Server health, API error rates, active user counts for live products.
- **Trading and pricing.** Financial instruments where stale data means wrong prices.
- **Live events.** Streaming metrics during a product launch, marketing campaign, or live broadcast.

If you're in one of these categories, you need an event streaming architecture: Kafka, Flink, Materialize, or similar. A semantic layer complements this by governing the metric definitions, but the freshness comes from the streaming pipeline.

## When you don't (and what to use instead)

Most business analytics doesn't need sub-second freshness. It needs:

1. **Fast query response** (milliseconds, not seconds)
2. **Reasonably fresh data** (minutes or hours, not days)
3. **Consistent numbers** (same answer everywhere)

[Pre-aggregation](/glossary/pre-aggregation) gives you all three without a streaming architecture.

### How pre-aggregation works

The [semantic layer](/glossary/semantic-layer) pre-computes your most common queries and caches the results. When a dashboard or AI agent asks for "revenue by region this month," the query hits the cache instead of scanning raw warehouse tables.

```yaml
cubes:
  - name: orders
    sql_table: public.orders
    measures:
      - name: total_revenue
        sql: "CASE WHEN status != 'refunded' THEN amount ELSE 0 END"
        type: sum
      - name: order_count
        type: count
    dimensions:
      - name: region
        sql: region
        type: string
      - name: created_at
        sql: created_at
        type: time

    pre_aggregations:
      - name: revenue_by_region
        measures:
          - total_revenue
          - order_count
        dimensions:
          - region
        time_dimension: created_at
        granularity: day
        refresh_key:
          every: 15 minutes
```

The cache rebuilds every 15 minutes. Queries that match the pre-aggregation return in single-digit milliseconds. Queries that don't match fall through to the warehouse (seconds).

**The result:** your dashboard loads in under a second. Your AI agent gets responses in milliseconds. Your data is at most 15 minutes old. For 90% of analytics use cases, this is indistinguishable from "real-time."

### Different refresh rates for different metrics

Not all metrics need the same freshness. Configure per rollup:

```yaml
    pre_aggregations:
      # Operational: refresh every 5 minutes
      - name: active_users_live
        measures: [active_users]
        time_dimension: last_seen_at
        granularity: minute
        refresh_key:
          every: 5 minutes

      # Strategic: refresh every hour
      - name: daily_revenue
        measures: [total_revenue, mrr]
        dimensions: [plan, region]
        time_dimension: created_at
        granularity: day
        refresh_key:
          every: 1 hour

      # Historical: refresh daily
      - name: monthly_cohorts
        measures: [retention_rate, ltv]
        dimensions: [cohort_month, plan]
        time_dimension: created_at
        granularity: month
        refresh_key:
          every: 24 hours
```

Operational metrics refresh every 5 minutes. Revenue refreshes hourly. Historical cohorts refresh daily. Each metric gets the freshness it needs without over-engineering.

## Real-time analytics for B2B products

If you're shipping analytics to B2B customers, "real-time" has a different meaning. Your customers expect their dashboards to load fast and show recent data. They don't expect sub-second event streaming.

Pre-aggregation handles this well:

- **Fast load times.** Cached queries return in milliseconds. Customer dashboards feel instant.
- **Multi-tenant caching.** The cache respects tenant isolation. Customer A's cached rollup doesn't include Customer B's data. Security context filters apply at cache build time.
- **Cost control.** On usage-based warehouses ([BigQuery](/integrations/bigquery), [Snowflake](/integrations/snowflake)), fewer raw queries means lower costs. Serving from cache is essentially free.
- **AI agent compatibility.** Agents make multiple queries per interaction. Pre-aggregation keeps response times low even with agent-scale query volumes.

## Real-time analytics architecture compared

| Approach | Freshness | Query speed | Complexity | Cost | Best for |
|----------|-----------|------------|-----------|------|----------|
| **Event streaming** (Kafka + Flink) | Sub-second | Fast (materialized views) | Very high | High | Fraud, trading, live monitoring |
| **Streaming DB** (Materialize, RisingWave) | Seconds | Fast | High | Medium-high | Continuous queries, CDC |
| **Warehouse + pre-aggregation** | Minutes | Milliseconds (cached) | Low | Low-medium | Dashboards, AI agents, embedded analytics |
| **Warehouse direct** | Minutes-hours (ETL dependent) | Seconds | Low | Medium | Ad-hoc analysis, batch reports |
| **Spreadsheet** | Manual | N/A | None | Free | One-off analysis |

For most teams, the warehouse + pre-aggregation approach covers the use case. Add streaming only for the metrics that genuinely need sub-second freshness.

## Getting started

Once your queries return fast enough, agents become a consumer, and an agent that fetches rows still has to chart them. Add a `visualize` tool to your MCP server so it can:

```bash
npm install @bonnard/mcp-charts
```

```typescript
import { addCharts } from "@bonnard/mcp-charts";

addCharts(server, { runSql });
```

That registers the `visualize` tool on your server. The agent calls it with a query, your `runSql` returns the rows (from a cached rollup or the warehouse), and Bonnard renders an interactive chart inside Claude or ChatGPT. Adapters ship for Postgres, BigQuery, Snowflake, Databricks, and DuckDB, or pass your own `runSql`.

Full walkthrough: [Add Interactive Charts to Your MCP Server](/blog/mcp-charts). For the design choices behind the tool: [How Bonnard Builds Agent-Friendly MCPs](/blog/how-bonnard-builds-agent-friendly-mcps).

Source is on [GitHub](https://github.com/bonnard-data/mcp-charts).

## Frequently asked questions

### What is real-time analytics?

Real-time analytics means querying data with minimal delay between when events happen and when they appear in your analytics. True real-time (sub-second) requires event streaming. Near-real-time (minutes) is achievable with pre-aggregation caching on top of a data warehouse.

### Do I need real-time analytics?

Most business analytics doesn't need sub-second freshness. If your use case is dashboards, AI agent queries, or [embedded analytics](/glossary/embedded-analytics), a pre-aggregation cache refreshing every 5-60 minutes provides "real-time" performance at a fraction of the streaming complexity. True real-time is needed for fraud detection, live monitoring, and financial trading.

### What is the difference between real-time and batch analytics?

Batch analytics processes data on a schedule (hourly, daily). Real-time analytics processes data continuously with minimal delay. Pre-aggregation caching sits in between: the cache rebuilds on a schedule (every N minutes), but queries against the cache return in milliseconds. It's batch refresh with real-time query performance.

### How does pre-aggregation make analytics feel real-time?

Pre-aggregation pre-computes your most common queries and caches the results. When a user or AI agent queries "revenue by region," the response comes from the cache in single-digit milliseconds instead of scanning raw warehouse tables (which takes seconds). The data is minutes old, but the query feels instant.

### What is streaming analytics?

Streaming analytics processes events as they arrive, maintaining continuously updated query results. Technologies include Kafka, Flink, Materialize, and RisingWave. It's the right choice for sub-second freshness requirements. For most business analytics, pre-aggregation caching provides comparable user experience at lower cost and complexity.