4 posts tagged with "security-data"

Legacy SIEMs are due for retirement​

The world's most popular SIEM was built over 20 years ago. It hasn't changed much since.

Unfortunately, neither has the competition.

If you look at the Gartner Magic Quadrant for SIEM, every vendor present is either running on a variation of the same legacy infrastructure, or on a data warehouse that was never built for logs to begin with.

The rest of the world, however, has changed a lot in 20 years. And with GenAI adoption ramping, the rate of change will accelerate.

Four trends will define the future of the SIEM​

At IceGuard, we're betting that four trends will define the future of SIEM:

1. Log volumes will grow exponentially. Logs are generated by actions in your environment. Agents easily take >10x more actions per day than humans, and humans often run multiple agents in parallel, 24/7 in many cases. As a result, log volumes will grow by >10x as well.

2. Query volumes against the SIEM will grow exponentially. Queries used to require a human to formulate the question, craft the query, and interpret the results. With AI, hundreds of queries can be generated by a single prompt. This means the total query volume against your SIEM and other SecOps data stores will grow by multiple orders of magnitude.

3. Attacks will get more frequent, faster, and more creative. AI lowers the barriers for attackers to develop and launch sophisticated attacks against a large number of targets. Advanced techniques such as zero-day exploits, spear phishing, and EDR evasion are becoming commoditized, and attacks can complete in minutes.

4. Security logs will become a highly sensitive data asset. AI traces are one of the most important log sources for security teams to monitor. However, AI traces contain highly sensitive data such as your IP, business processes, credentials, and confidential activity of privileged employees. Unlike traditional audit logs, AI traces must be safeguarded like any other crown-jewel asset. This means they can't be spread across SaaS services and point solutions, but require centralized storage with governed access in your own environment.

If you agree with that list, it's clear that the legacy SIEMs are the wrong solution. They were already too slow and expensive for a pre-AI world.

In an AI world, they'll be obsolete.

It's time we raise the bar.​

Unlike what most security vendors will tell you, 10 TB/day is not big data. And it should not require a million-dollar SIEM budget or a decoupled architecture to handle efficiently.

In fact, we believe most security organizations should have the majority of their security data in a single, centralized data store. We call that data platform a Modern SIEM, and it needs to meet the following requirements:

1. Complete visibility. It provides unified, governed access to all your data, without relying on cold stores or federation. You can search months of data and get answers in seconds.

2. Continuous detection. It can run stateful detection logic in near real time, while easily incorporating behavioral baselines to minimize manual tuning. It can seamlessly aggregate, correlate, and enrich detection findings when they're created, so you don't need to spend your investigation time on ETL.

3. Unrivaled efficiency. It can ingest >10x the log volume and run >10x the query volume of your legacy SIEM without throttling or blowing up your budget.

4. Sovereign data. Your data remains in your account, stored in open formats. Your vendor does not have standing data access, and data can be retained indefinitely at object store rates.

Legacy SIEM architectures can't deliver any of those requirements. Neither can general-purpose data warehouses or a security data mesh!

We should stop accepting mediocre incumbents that have stopped innovating, and new entrants who reduce the scope and ambition of the SIEM.

Let's raise the bar!

FAQ​

Cost confusion​

There is a prevailing belief in the SecOps community that there is too much security data to store in one system.

We think that's wrong.

In fact, we think a single system is a far better place to store it than any form of decoupled architecture.

The confusion comes from SIEM incumbents overcharging and underdelivering, not from fundamental limits of technology. To understand that point, it's worth looking at some numbers.

On a modern system, ingesting, processing, and detecting against one raw GB of data should cost less than $0.02. Storing it for 90 days should cost less than $0.02 in an object store. A Modern SIEM ingesting 1 TB per day should therefore cost less than $15,000 per year in hardware. The vendor needs to make money too, of course. Assume an 8x markup over the hardware cost, totaling $0.30 per GB. That brings the annual cost to $120,000.¹

This is a small fraction of the economic activity required to produce those logs.

For example, an employee laptop produces 0.1 to 0.3 GB per day if you log aggressively. A production server may produce 100 times that. Now you're looking at a logging bill of less than $35 per laptop and less than $3,500 per production server per year.² Apply this math across your systems and the logging bill is usually a tiny fraction of the cost of the activity you're monitoring. With sensible logging practices, ingesting the logs into a reasonably priced SIEM should be a no-brainer for almost any organization that cares about security.

It's pricing, not physics​

So why are people complaining about SIEM pricing?

Because incumbent SIEMs aren't charging $0.30 per GB. They're charging between $3 and $6 per GB, a markup of more than 100x.³

No wonder users are struggling with the cost.

The problem is the pricing, not the physics.

Let's fix the root cause​

So what happens if we fix the ingestion price?

The workarounds disappear.

You no longer need a pipeline to route your data because all your logs go directly into the SIEM. Since the SIEM stores everything, you can ditch your cold store and federation engine as well. If your SIEM offers fully managed ingestion, which it should, the management overhead disappears too. Point your log source at the SIEM, or vice versa, and let the logs flow.

No need for additional complexity!

At current SIEM prices, it's understandable that users don't ingest everything. But that's because SIEM vendors are overcharging, not because there is too much data.

It's time we fixed the root cause and stopped peddling workarounds.

FAQ​

Footnotes​

1. Hardware: ($0.02 compute + $0.02 storage) × 1,000 GB × 365 days = $14,600 per year. With an 8x markup: $14,600 × 8 = $116,800, or about $0.32 per GB. We round to $0.30 per GB and $120,000 per year. ↩

2. Laptop: 0.3 GB × 365 days × $0.30 per GB = $32.85. Production server: $32.85 × 100 = $3,285. ↩

3. These prices apply to the 1 to 10 TB per day range. Above 10 TB per day, SIEM vendors might discount below these amounts. ↩

To decouple, or not to decouple...​

As legacy SIEMs continue to disappoint, the market remains on the hunt for options. One of the more popular ideas is the "decoupled SIEM." The idea is seductive: decompose the ingestion, storage, querying, and detection logic into separate systems, then purchase "best-of-breed" options in each category.

No more "vendor lock-in" or "ingestion tax," just the sweet sound of logs flowing efficiently through your decoupled system.

Unfortunately, reality is a lot more complicated.

In software engineering, coupling is the degree of interdependence between software modules. Decoupling - the act of reducing dependencies between systems - is often considered a good thing.

But it has a cost: In a decoupled system, you lose the ability to optimize the system as a whole.

This is why most high-performance, complex systems are not decoupled. Your laptop, phone, car, and relational database are all built using tightly integrated components.

We believe a SIEM should be built the same way.

The cost of decoupling​

The SIEM's purpose is to provide visibility and near real-time detection across your entire environment. This requires ingesting, normalizing, storing, detecting, and querying over massive amounts of data with minimal latency, effortless scalability, and at an affordable price. This is hard enough to accomplish in a single system. It's almost impossible in a decoupled one.

Let's dig into a few examples.

Detection latency​

Attacks are getting faster, and a 30-minute MTTD is no longer going to cut it. However, in a decoupled SIEM, it's hard to push detection latency and investigation time down. Each component will buffer and batch independently, quickly adding minutes to event arrival and detection time. Moreover, unless your detection engine supports efficient near real-time detection,¹ you'll be running your detections in 15-minute batches instead of near real time. Added up, you're usually 20 to 30 minutes behind on complex, stateful detections such as thresholds or sequences.

Investigation experience​

In a decoupled SIEM, your data is spread across systems: cold stores, warehouses, vendor APIs, and one or more SIEMs. Analysts find themselves jumping between tabs, APIs, and search languages. Federated search can help, but it is far too slow for interactive use and rarely scales beyond POCs and simple use cases. If you don't believe us, run a load test yourself.

As a result, investigations are slow and tedious.

Operational complexity​

Each solution must be bought, set up, tuned, and governed separately. This means multiple access mechanisms, permission systems, monitoring systems, query languages, detection frameworks, APIs, support teams, and procurement cycles.

If something breaks, good luck.

The integration between systems becomes its own ongoing project. Schemas, enrichment, detection logic, indexing strategy, retention, and access mechanisms need to be coordinated across each component. Since a simple change in one part of the chain can have far-reaching side effects in downstream systems, you need comprehensive end-to-end testing to make changes safely.

We could go on and on.

Let's keep it coupled​

The key insight is that the quality of the user experience is directly related to the degree of coupling between each component. To achieve low-latency detection, fast search, and minimal operational overhead at an affordable cost, each component must be designed to work together as a system.

Decoupling makes that harder.

The call for a decoupled SIEM is a rational response to the poor technical execution and exorbitant pricing of SIEM incumbents. But that doesn't make it the right long-term architecture for the SIEM.

Let's keep the SIEM coupled.

FAQ​

Footnotes​

1. Note that some pipeline vendors do "detection in the stream." Assuming that works (you should test that thoroughly), the detection capabilities are no longer "decoupled." Good engineering decision, but awkward for their product marketing teams. ↩

Unloved, but not unnecessary​

Few categories carry as many negative connotations as the SIEM: slow, expensive, hard to operate, and painful to migrate from. The exorbitant pricing has even become a meme.

Unsurprisingly, the startup world has been mulling over what's next: a fabric? A mesh? An agentic cyber defense platform?

Word salads aside, the problem is real. You need full visibility into your environment, and legacy SIEMs are too slow and expensive to deliver. No wonder you're shopping around for options.

Our hot take: you don't want a mesh, fabric, or agentic defense platform. You want a SIEM.

Why SIEMs matter​

The core problem you need to solve is cross-source visibility and near real-time detection. You have tens of thousands of identities, assets, and applications producing billions of events daily. On top of that sit 10-50 security point solutions that monitor slices of that activity. Detections, investigations, and reporting frequently need to span multiple sources, usually under time pressure. Without a unified access mechanism coupled with low-latency detection capabilities, this is painful and time-consuming. At AI scale, it becomes impossible.

But cross-source visibility and near real-time detection are exactly the problems SIEMs were meant to solve.

Why do we need to invent a new category?

Why people don't like their SIEM​

The underlying issue is that the SIEM incumbents have been unable to deliver on the promise of SIEM.

And they don't have the capabilities or incentives to improve.

On the capability side, their core technology is so old and bloated that they cannot make meaningful improvements to their platforms' performance, scalability, or efficiency.

On the incentive side, they are making bank by keeping prices high and ingest volumes low. As long as their customers keep ingesting more data and remain unable to migrate, why change?

The SIEM is not dead!​

AI adoption is making SIEMs more important than ever: more systems to monitor, higher activity volumes, and faster attacks. Full visibility and near real-time detection across the entire environment is becoming table stakes for a modern SOC. Forget the mesh - only a SIEM can deliver that.

The SIEM is not dead. We just need better SIEMs.

FAQ​