Every packet decision, DNS query, and application-layer event Neurowall makes can be streamed into ClickHouse — a column-store database purpose-built for exactly this kind of high-volume, append-only analytical data. The result: dashboards and reports that answer questions your live firewall state can't.
A firewall's operational database is built for fast rule lookups and verdicts — not for answering "which source IPs generated the most blocked traffic last quarter" or "how did DNS query volume change after we onboarded a new tenant." Those are analytical questions over historical event volume, and row-store operational databases are the wrong tool for them.
Rule and threat tables are optimized for point lookups and small transactional writes. Aggregating millions of historical events across them is slow and risks contending with live traffic decisions.
Firewall events, DNS queries, and application-layer (L7) events are naturally captured by different subsystems. Without a shared store, correlating them (e.g. "which DNS query preceded this blocked connection") means manually joining logs by hand.
Without a queryable history, questions about growth, seasonality, or attack trend lines require re-deriving them from raw logs every time — not a repeatable reporting process.
Auditors and compliance reviewers want historical evidence — not just "the policy is X today," but "the policy enforced Y over the review period." That needs a queryable event history.
Neurowall's telemetry sidecar, Ramen, ingests firewall (L3/L4), application-layer (Vaanvil), DNS (dnstap), and network traffic (Darkstat) events through webhooks and durable NATS JetStream subjects, batches them, and writes them into ClickHouse. Because ClickHouse is column-oriented, aggregate queries — counts, sums, distinct-value cardinality — over months of event history run in milliseconds to seconds, not minutes.
Ingestion runs through NATS JetStream, so events are durably queued even if ClickHouse or Ramen is briefly unavailable — no silent data loss during restarts or maintenance windows.
| Use Case | Description |
|---|---|
| Traffic trend reporting | Query historical allow/block/drop volume by source, destination, protocol, or time window — daily, weekly, quarterly |
| Top-talkers / top-blocked analysis | Rank source IPs, destinations, or rules by event volume across any historical range |
| DNS query analytics | Aggregate dnstap records to find query volume by domain, client, or resolver — useful for both security and capacity planning |
| Cross-layer correlation | Join L3/L4 firewall events with L7 (Vaanvil) events in the same store to trace a request end-to-end |
| Compliance evidence | Produce historical enforcement reports for audit and rule-review cycles, backed by real event data rather than point-in-time snapshots |
| Capacity & growth planning | Track traffic growth over time per gateway or per tenant to inform scaling decisions |
Without a dedicated analytics pipeline, most teams answer "what does our traffic history say" the same expensive way every single time it's asked.
| Common approach | The recurring cost |
|---|---|
| Manually grepping raw log files | Every new question means writing a new ad-hoc script against unindexed text — slow, error-prone, and the analysis dies with the person who wrote it |
| Exporting to a spreadsheet each time | Manual export-and-pivot doesn't scale past a few thousand rows, and the "report" is a one-off artifact that's stale the moment it's generated |
| A generic BI tool bolted onto raw logs | Without a columnar backing store, the BI tool itself becomes slow at exactly the aggregate queries it exists to answer quickly |
| No historical reporting at all | Questions about trends, growth, or attack patterns over time simply go unanswered — decisions get made on gut feel instead of data that already exists somewhere in the logs |
The analytics pipeline exists to make exactly these questions fast — see ClickHouse Database for why the underlying store is suited to this workload specifically.
Reports are written in SQL against a durable event store — repeatable and version-controllable, not a one-off spreadsheet export that goes stale immediately.
Firewall, DNS, and application events share the same store and time axis, so cross-source questions don't require stitching together separate tools by hand.
Because ingestion is continuous and durable, the data needed for next quarter's report is already being collected today — no scramble when leadership or an auditor asks.