Search:

Open-Source Observability Transformation on AWS

From Datadog to a Platform-Owned, Vendor-Agnostic Observability Stack

Group 1

About Lumo


A California-based agricultural technology company that builds cloud-based solutions to help growers monitor, control, and optimize water usage at scale.
LUMO leverages modern cloud infrastructure to deliver reliable, data-driven irrigation management, supporting growers in addressing water scarcity and improving operational efficiency.


Problem

LUMO was using a vendor-based observability solution to collect logs, metrics, and traces across all AWS Kubernetes (EKS) environments.
As platform usage grew, the observability model began to introduce operational and financial challenges.

The Main Issues

- Usage-based pricing led to unpredictable and increasing observability costs

- Log ingestion costs created end-of-month billing uncertainty

- Strong dependency on a single vendor reduced architectural flexibility

- Scaling observability became harder without scaling costs proportionally


Technical Challenges

- Observability had to remain fully available during any transition

- Applications were already instrumented and running in production

- Telemetry needed centralized governance and consistent enrichment

- Retention and data growth required tighter control without licensing constraints

This created a clear mandate: Move from Datadog to a sustainable, cost-predictable, open-source observability platform without losing visibility at any stage.

Client: Lumo
Project type: Observability on AWS
Website: www.lumo.ag

Solution

LUMO designed and implemented a centralized, open-source observability platform on AWS, replacing the vendor-based model without disrupting production workloads.

The solution focused on ownership, predictability, and platform-level observability. 

Here's How It's Built

- A dedicated central observability backend on Amazon EKS, isolated from application workloads

- Grafana Loki for centralized log storage

- Grafana Tempo for distributed tracing

- MinIO as the object storage backend

- OpenTelemetry Collector DaemonSets deployed in each environment for telemetry collection

- Manual OpenTelemetry SDK instrumentation in Go

- OpenTelemetry log bridge to standardize application logs

- Telemetry enrichment with environment, cluster, namespace, and infrastructure context

- Retention policies defined by business needs rather than licensing limits

Metrics stored in AWS Managed Prometheus and visualized via AWS Managed Grafana

AWS Managed Grafana

 

kloia-lumo-case-study-central-observability-backend-on-Amazon-EKS
Frame 1

Results

The transformation established observability as a platform-owned capability, rather than a per-application or vendor-managed service.
- Zero observability gaps during migration
- Full removal of vendor-specific agents
- Predictable and controllable observability costs
- Centralized governance with distributed telemetry collection
- Improved operational clarity and faster incident response

Monthly costs are now fixed at 475 dollars. We're no longer dependent on Datadog, we removed all the agents. The best part? We didn't have to change a single line of application code.

Observability is now ours to own. We built a system around a central backend using the OpenTelemetry community, with consistent troubleshooting across all environments. We keep logs for 15 days and traces for 7 days, it's sufficient and controlled.

In the end, we migrated to a completely open-source infrastructure. Whatever changes in the future, we don't have to worry about rewriting code and going back to Datadog. We're free.

Some of our results:

  • 68%
    Cost Saving
  • 99%
    Telemetry delivery rate

Contact