In part 1 we covered why LogSeam is built on a data lake and how the pipeline writes every log once, as open Parquet in your bucket, in a shape a search can skip through. This part covers the compute that reads it: how the platform runs and scales itself, the two kinds of nodes, where the searching actually happens and the four shapes every query takes.
One system, managed by the platform
LogSeam runs as one environment per customer, with no shared services. We built it that way for security, performance and scale. It runs on-prem, in your cloud account or in ours, and every deployment is isolated and built to the customer's requirements.
Compute comes in two layers: ingest nodes, which process every log on the way in, and worker nodes, which run everything else, from search and aggregation to AI agents and scheduled jobs. One master, the controller for the whole system, schedules thousands of jobs across both, each matched to the compute and storage it needs. The master that picks a query's shape is the same one that adds and removes nodes.
Scaling compute usually means running Kubernetes or another orchestrator on top of the platform. LogSeam scales itself instead. You define scale groups, the instance shapes you're willing to pay for, and the platform draws on them: when pressure builds, a node boots in a few seconds, joins and starts on the jobs assigned to it; when pressure eases, it goes back. The always-on floor keeps every job running in between. Ingest and worker nodes both scale this way, while object storage, the application database and the master stay put.
Pick your shapes
Most of the time, a SOC needs an economical car: great mileage, low maintenance, 100% reliable. Every now and then it needs to go really fast, for the moments when someone has to look through a lot of data. Most platforms make you buy one car and drive it everywhere. LogSeam gives you both, and you decide how many of each.
Having both is what lets LogSeam scale two ways at once: more nodes for more questions, and more nodes on one big question. Scale-out alone adds nodes, but one heavy question still runs on one of them. Fan-out alone splits the question across a cluster sized months ago, so under load everyone queues. Here a heavy question fans out across new nodes the moment they join, and when it's quiet they go back: speed when you ask, and you pay for it only then.
You pick the shapes as well as the counts. Ingest and search nodes stay small, 8 to 32 cores, so capacity grows by adding nodes: each one boots in seconds and carries only a small slice of the work. Aggregation nodes are the big ones, 32 cores and up, with the memory and local NVMe to merge partials and run the heaviest CTEs and window functions. There's no ceiling on either pool; your budget is the limit. Spot is where the fast part gets cheap: an 8-core general-purpose Arm node has recently run about $0.20 an hour on spot against $0.39 on demand (source). If AWS reclaims a spot node mid-job, the master reassigns its slice to another node; nothing on a dynamic node needs to survive it.
Where the searching happens
Search falls off fast with age. Splunk's own documentation says 97% of searches look back 24 hours or less (source). With a year of retention, that day is 0.3% of the data you keep. Facebook saw the same shape in its monitoring system: at least 85% of queries hit data from the past 26 hours (source).
Nearly all searching lands on a sliver of the data, and that sliver stays small at any ingest rate. The other 3% are the hunts and investigations that reach back weeks or months. They're rare, but each one can read 30 to 365 times the data of a 24-hour search, so they're where the heavy lifting is.
That split is why the compute is shaped the way it is: the daily driver keeps the recent window close and answers the everyday searches, and the race car comes in only for the deep ones. Everything else sits in object storage at object-storage prices. Paying for hot, indexed disk to hold a year of data that 3% of searches touch is the cost this design avoids.
Many small nodes, not one big cluster
The engines usually put on a lake (Spark, Trino, Presto) were built for big jobs on a cluster sized in advance, with a long-lived JVM to tune on every node, and they shuffle data all-to-all across the network between stages. That's the right shape for huge joins and rewriting whole tables. It's the wrong shape for a SOC, where the load is many concurrent searches and aggregations, mostly small and arriving in bursts. AI amplifies this pattern: it runs hundreds of small queries rather than a few sweeping searches over long time ranges.
So LogSeam runs many small instances, each taking a small-to-medium slice of the work. There's no node-to-node shuffle, and only merged partials move.
The master's query planner shapes each search at query time. It decides more than how to split a query across nodes: it weighs the state of the whole system (the catalog, Rosetta, the health and shapes of the compute, and the AI asking the question) and builds a plan in milliseconds. AI works with the planner directly, so it learns good queries from bad ones and gets results that fit its context window. Everything starts with the plan, and the plan covers more than the SQL: it's the shape of the whole system.
We keep extending the planner. Some SQL query shapes can't be distributed in our architecture; today it distributes 85 to 90% of them, and we're working toward 95 to 98%. Every plan runs in one of four shapes, keeping work on the node that read the data:
Each node carries its own vectorized, columnar SQL engine. It reads Parquet from object storage, uses every core on the box, and spills to local NVMe when a job outgrows memory. There's no JVM, no warm-up and nothing on the node that can't be lost, so a dynamic node can start, take a slice, return its partial result and leave.
Every worker node can also run anything else the system needs. Our AI lets every worker node join the swarm, so a single AI question can fan out across many of them. The whole system is composable: work, engines and workflows are all managed centrally and scale as needed.
Caching works at more than one level and across every node. Results and parts of results, catalogs, Parquet footers, key values and indicators are all cached, so work done once doesn't have to be done again, and the platform manages and prunes it continuously.
The cache is more than a fast lookup. It links threat intelligence to the attack patterns and attack chains the system recognizes, forming a knowledge mesh that every part of LogSeam draws on: queries, the AI and the work, from rules and triage to cases and hunts.
Closing the six gaps
That brings us back to the six gaps from part 1. Here's what closes each one:
Next: what it costs
Part 3 puts numbers on all of this: the build most customers start with, the full bill at 1 TB a day, how it compares with Microsoft Sentinel and why your cloud discount reaches every node and every byte.