Logging from Kubernetes into OpenSearch often starts with a username and password stored in a Secret. That is straightforward to set up, but the credential spreads across nodes and clusters and almost never gets rotated. Meanwhile, Kubernetes can issue short-lived, signed identities to pods, and OpenSearch can validate them. This article walks through connecting those pieces, then solving the less obvious problem of keeping clusters isolated.
We sent 494,175 numbered log lines through Fluent Bit and Vector into OpenSearch and counted every one that arrived. With stock settings, one shipper lost 62 percent of a busy pod's logs and the other lost 85 percent, both without logging a single error. Five defaults were behind it, and each one is a one-line fix.
We shipped exactly 100 MB of logs into OpenSearch, once with Fluent Bit and once with Vector, and measured every byte that landed on disk. One copy came to about 1.5 times the raw size, the replica doubled that, and the search index, not the logs, turned out to be the biggest part of the bill.