ELK is dead, long live ...?

In 2013 you set up centralised logging by installing Elasticsearch, Logstash and Kibana, and the reason that was a good decision is that it was not really a decision. Ten years on, every letter has multiple credible answers, the licence under the first one has moved, and the integration work is yours. Here is what happened, what it costs, and the four questions I now use to pick a stack.

What the acronym was worth

The 2013 stack worked. Logstash parsed the lines, Elasticsearch indexed them, Kibana drew the graph, and for most of what a team needed at the time that was the end of the problem. It asked for memory and it asked for care, but it did the job it said it would do.

The part that is easy to undervalue in hindsight is that there was nothing to choose. One answer, and everybody had it. The post you found at 3am assumed your components. Someone had already published the upgrade order. Grok patterns got traded like recipes. The next person you hired had run the same three things at their last job, and the meeting about what to use never happened.

That arrangement was already coming apart when Elastic named the Elastic Stack in 2016, for the honest reason that Beats had joined and there was no letter for it. The acronym stopped describing the product seven years ago. Most of us kept saying it anyway, which is the clearest sign of what we were actually attached to. Not the software. The default.

The default is measurably gone

This is not nostalgia talking. Two things you can check:

Grafana Labs surveyed observability practitioners in March 2023 and found 52% of respondents running six or more observability tools, and 11% running sixteen or more. That is not a field with a standard stack in it. That is a field where everyone has assembled something different and is maintaining it.

And the two descendants of the E now benchmark against each other in public. Elastic’s comparison, published last month, concludes that Elasticsearch is 40% to 140% faster than OpenSearch. The methodology is contested, as vendor benchmarks against a fork usually are. Take whichever side you like. The relevant fact for someone choosing a log store is that the two candidates closest to the old default are now parties in a dispute, and there is no neutral authority above them to settle it.

How each letter came apart

The L split in two directions at once. Logstash was doing two jobs, moving lines and reshaping them, and both left. The reshaping went into the engine: Elasticsearch 5.0 added ingest pipelines in 2016, so grok, date, rename, convert and drop could run inside the cluster with no separate process to operate. The moving went out to something small enough to sit on every host. Filebeat arrived in late 2015 as the supported replacement for logstash-forwarder, and then the rest of the field turned up: Fluentd, a CNCF project graduated in 2019, with a plugin for every destination anyone has ever had. Fluent Bit, written in C for embedded targets, which is why it fits in single-digit megabytes per node. Vector, in Rust, open sourced by Timber.io under MPL 2.0 and acquired by Datadog in February 2021.

Logstash was never deprecated and is still maintained. It stopped being the thing you reach for first, which for a component whose value was being the default comes to much the same thing. What survived the split is the vocabulary: open a Fluent Bit config written this year and you will find Logstash_Format On, which is how you ask a shipper released in 2023 to write daily indices in the naming scheme of the tool it replaced.

The E stopped being open source. On 14 January 2021 Elastic announced that Elasticsearch and Kibana would move off Apache 2.0 onto a dual SSPL and Elastic License 2.0 arrangement, effective with 7.11. A week later AWS said it would fork. The fork came off 7.10.2, the last Apache 2.0 release, was named OpenSearch that April and shipped 1.0 on 12 July 2021, Apache 2.0 throughout, including the security, alerting, SQL and anomaly detection plugins that had been proprietary or paid. We wrote about the relicensing at the time and built our managed service on the fork the year after.

Then the split reached application code. From 7.14, Elastic’s official clients check for an X-Elastic-Product header on the response and refuse to talk to a server that does not send it. Your client library is now a choice of side. Libraries and shippers forked to match, which is why Fluent Bit ships a separate opensearch output rather than pointing its Elasticsearch one at a forked server.

Something quieter happened alongside the licensing: people started arguing with the index model itself. Elasticsearch and OpenSearch index everything by default, which is what makes an arbitrary field query fast and what makes the storage bill what it is. Loki indexes labels only and leaves the bodies in object storage, so ingest is cheap and a query with no useful label is a brute-force scan. ClickHouse stores logs as columns, compresses them hard, and beats an inverted index at aggregates over enormous ranges, provided you bring a schema.

And the K became a question too. Kibana went across under the same terms in 7.11. OpenSearch Dashboards forked from Kibana 7.10.2 and stayed Apache 2.0. Grafana, AGPLv3 since April 2021, will point at any of them through a datasource, which is the quiet reason it keeps taking this slot: the team already has it open for metrics, and a dashboard you are looking at anyway beats one you have to go and find.

What that costs, in practice

Each of those moves is defensible on its own. Together they hand you a job that did not exist in 2013: you are the integrator now.

Nothing ships as a set. The shipper, the store and the interface come from different projects on different release cadences, and no document anywhere tells you which versions belong together or in what order to upgrade them. You work it out, you write it down, and then you own it. Three examples from configs I run:

• A Fluent Bit output writing to OpenSearch usually carries Suppress_Type_Name On. That flag exists because mapping types were removed on one side of a fork while the shipper still spoke the older dialect. It is a line in a config file whose entire job is to paper over a difference between two projects that used to be one.
• Create the index template before the first document arrives, never after. Dynamic mapping picks field types from whatever it sees first and the choice is fixed for the life of the index, so a status code that turned up once as a string is a keyword you cannot aggregate as a number until you reindex.
• Every pairing has its own authentication story, and none of them is the one in the tutorial you found. Whether the shipper does TLS to the store, how credentials reach a DaemonSet, what the interface authenticates against: three separate decisions that used to arrive as one.

None of this is hard. All of it is yours to know, and there are a few dozen more like it. That is precisely the tax the default used to pay on your behalf.

Is OpenTelemetry the new default?

It is the only real candidate, and it may well get there. It has the vendors, a CNCF home, and semantic conventions that would make a log line from one service comparable to a log line from another, which is more than ELK ever standardised.

Two things to be clear about today. Logs are the youngest of its three signals, behind traces in both specification maturity and language support, and the collector’s components carry mixed stability levels, which is a polite way of saying read each README before you depend on it. More fundamentally, OpenTelemetry stops at the door of the backend on purpose. It standardises how telemetry is described, collected and transmitted. It says nothing about where the data lands, how it is indexed, what you query it with, or what you look at it in.

So when it lands properly it settles the shipper question and the format question, and hands you the store question unchanged. That one is the expensive one. Planning to wait for OpenTelemetry is planning to answer the easy half later.

The four questions I ask instead

Since nobody is going to hand the decision back, I stopped looking for a stack and started asking the same four things of any combination.

Is anyone allowed to run the engine? This is the licence question, and it is upstream of everything else because it decides how many suppliers you can choose between and what happens if you fall out with one of them. Source you can read is not the same as source you may use.

Who decides what it does next? A permissive licence does not tell you who merges the pull requests or sets the roadmap. Governance is a separate question and people routinely answer the first one and assume it covered the second.

What does the index model refuse to answer? Every store buys its speed somewhere. Index everything and you pay per gigabyte. Index labels and you cannot search what you did not label. Store columns and you need to know the shape of the data in advance. Pick the one whose weak spot you can live with, because you will meet it during an incident.

Can I get the data out? Open format, an export that runs on your schedule, and a restore somebody has actually tested. This is the difference between changing supplier and rewriting your logging.

Against those questions, this autumn:

Stack	Licence	Roadmap	Weak spot
OpenSearch, Dashboards	Apache 2.0	AWS	Index size per GB
Elasticsearch, Kibana	SSPL, ELv2	Elastic	Choice of operator
Loki, Grafana	AGPLv3	Grafana Labs	Unlabelled search
ClickHouse, Grafana	Apache 2.0	ClickHouse Inc	Schema and shipper

Where I land

OpenSearch with OpenSearch Dashboards, and Fluent Bit in front of it.

On the licence, it is Apache 2.0 the whole way through, security plugin included, so nobody needs permission to run it for you. On the index model, it indexes everything, which is the expensive answer and also the one that does not make you guess in advance what you will need to search during an outage. On getting out, the data is JSON on an open format with snapshot and restore into an S3 bucket you control. Two and a half years past the fork it is plainly its own project rather than a rebadged 7.10: 2.9 shipped in July with search pipelines, an ML framework and further vector search work, none of which has a counterpart in the 7.10 it forked from. Dashboards is Apache 2.0 too, which is why it is included on every tier we sell rather than being the upsell.

On governance, I would rather give you the honest answer: this is the weakest of the four. The roadmap and the trademark sit with one company, there is no foundation behind the project, and a permissive licence does not substitute for one. It is the change I most want to see, and we will say so here when it comes.

Fluent Bit answers the shipping question by having won it on the merits. One C binary per node, a Kubernetes filter that tags each line with its namespace, pod and container before it leaves the machine, and a native OpenSearch output. It went from one billion downloads in March 2022 to three billion that October, and it is what the major managed Kubernetes services ship in their own logging paths. If any component of this stack is a new default, it is this one.

The whole integration is one output stanza:

[OUTPUT]
    Name               opensearch
    Match              *
    Host               ${OPENSEARCH_HOST}
    Port               443
    tls                On
    HTTP_User          ${OPENSEARCH_USER}
    HTTP_Passwd        ${OPENSEARCH_PASSWORD}
    Logstash_Format    On
    Logstash_Prefix    fluent-bit
    Suppress_Type_Name On

Our centralised logging guide has the rest of it end to end: the index template to create before anything writes, then that same stanza driven from a plain Linux host, from Docker Compose through the Fluentd log driver, and from a Kubernetes DaemonSet with the credentials in a Secret.

Three components rather than one product, and a decision rather than a default. That is what 2023 gives us to work with, and the best you can do with it is pick parts that will still be yours to run in five years.

If you would rather not operate the store half of that, launch a cluster and point Fluent Bit at it.