The complexity and escalating architecture of modern applications, coupled with a proliferation of technologies, make troubleshooting and problem-solving increasingly challenging for developers. However, observability offers some respite as it enables analysis of a system's internal operations. Nevertheless, it also presents its own set of inevitable challenges. Let's explore this topic further throughout this blog.

What is Observability?

Observability is the capability to infer the internal state of a system through telemetry data, which is the behavior-related data emitted by the system.

Telemetry Data: Types

Metrics: Metrics are quantitative data that measure the performance of an infrastructure or application over time. This includes CPU utilization, request rate, and system error rate.

Logs: Logs are timestamped events that occur within your system. They can be errors, warnings, or informational messages and are instrumental in diagnosing issues.

Traces: Traces document the path taken by requests through a system. They are useful for debugging, providing insights into where or at which component a system issue occurs.

What is OpenTelemetry?

OpenTelemetry is an open-source framework for observability. It facilitates the creation, management, and dissemination of telemetry data through APIs, SDKs, and tools. These components help in instrumenting, generating, collecting, and exporting telemetry data, thus offering developers insights into application performance.

Challenges:

Instrumentation: This is the process of integrating telemetry data extraction within a codebase. This task can be complex, particularly if the programming language used does not support many automatic tools, requiring numerous code modifications and the addition of APIs and SDKs.

Collectors: Configuring OpenTelemetry collectors involves defining extensive YAML configurations, which has a steep learning curve.

Performance Overhead: Automatic instrumentation may impose significant performance overhead on applications, deterring organizations from adopting open-source projects like OpenTelemetry.

Overcoming Challenges: Using Odigos

Odigos is an open-source tool that generates distributed traces for any application without modifying the codebase. It automatically instruments data from an application, producing distributed traces, logs, and metrics, and dynamically deploys and scales collectors based on application traffic.

Odigos: Features

Language Detection: Odigos automatically identifies the programming language of each application within a cluster and conducts automatic instrumentation accordingly.

Languages supported by Odigos: Java, Python, .NET, Node.js, and Go. Given the challenge of instrumenting compiled languages, such as Go, without altering the code, Odigos leverages eBPF to address this issue.

Observability Tool Compatibility: Odigos instruments data in the OpenTelemetry format, making it compatible with any observability tool that supports the OpenTelemetry Protocol (OTLP).

Collectors Management: Odigos automatically scales OpenTelemetry collectors based on the volume of observability data.

Odigos Pipeline:

Instrumentations: These are the code segments that capture OpenTelemetry signals from your application. They can be written in the application’s native programming language or through eBPF programs that tap into the Linux kernel for data.

OpenTelemetry SDK: This library operates within each instrumented process, receiving OpenTelemetry data from instrumentations and exporting it beyond the process boundary.

Node Collector: Deployed as a single instance on each node within a cluster, this OpenTelemetry collector processes and forwards telemetry data to the Cluster Gateway Collector.

Cluster Gateway Collector: Positioned as a Kubernetes deployment, it receives telemetry data from Node Collectors, processes it, and then exports it.

Conclusion:

Identifying and resolving issues within applications can be daunting and time-consuming, necessitating the right tools and knowledge of the system. While observability provides valuable insights for enhancing system comprehension and optimization, it's crucial to be aware of its limitations and potential blind spots. Recognizing these aspects allows us to adopt a balanced view toward observability.

Shoutout to Odigos for collaborating with me on this blog.