In the world of observability, context is king. For years, Elastic APM has provided dedicated views and capabilities for understanding the health of your applications and services. When you need to know how your checkout service is performing, you can go straight to its dedicated page, view key metrics like latency and throughput, and directly access related transactions and errors. This entity-centric view is invaluable for targeted monitoring and diagnostics.
But what happens when the problem isn't neatly confined to a single service? What if you need to ask more complex, exploratory questions that span across your entire dataset? Questions like:
-
Show me all traces where a specific user experienced a latency of over two seconds, and correlate it with any frontend errors that occurred at the same time.
-
Are there any slow database queries happening only for customers on our premium plan?
-
Which specific RPC call is the common source of failure across three different microservices?
Historically, answering these questions has been effective, but it required navigating different UIs and manually piecing together clues, leading to a less-than-seamless experience, a common challenge across various observability platforms.
Today, we're excited to announce a key improvement for trace search and analytics. We are bringing native support for Traces into Discover, complete with an integrated trace waterfall view. You can now apply the full capabilities of ad-hoc data exploration and ES|QL to your tracing data.
From Curated Views to Broader Data Exploration
Discover is the primary interface for data exploration in the Elastic Stack. It's the workbench where you can freely explore, filter, and correlate all of your indexed data. By integrating traces into this environment, you can now move beyond APM's curated views and conduct more flexible investigations, searching by any trace attribute.
You can now easily search for individual spans or errors, filter by OpenTelemetry resource attributes and span attributes, and analyze complex scenarios, all without leaving the Discover interface you know and love.
A Practical Scenario: Unraveling a Slow API
Imagine a critical frontend API to place orders is experiencing intermittent slowdowns. Your team has the APM service view, which confirms the high latency, but the root cause isn't immediately obvious. The slowdown seems to be happening deep within a complex chain of microservice calls.
This is where the new Discover functionality is particularly useful.
Your investigation can now start directly in Discover with a broad ES|QL query to find the slowest transactions for that specific endpoint. Example below is using OpenTelemetry demo, which you can try yourself on OpenTelemetry demo
ES|QL
FROM traces-*
| WHERE span.name == "oteldemo.CheckoutService/PlaceOrder"
| SORT span.duration.us DESC
This simple query reveals the most problematic transactions. From the results table, a single click on any trace opens a detailed, end-to-end trace waterfall view—right there in Discover. No context switching, no new browser tabs.
The waterfall reveals that a downstream currency service is taking a long time to respond. But why? You can now refine your ESQL query to ask a more sophisticated question, digging into the span attributes to find the specific downstream service call that is causing the bottleneck:
ES|QL
FROM traces-*
| WHERE service.name == "currency" and span.name == "Currency/Convert"
| SORT span.duration.us DESC
With this query, you’ve instantly found the exact spans within the
Workflow now utilizes a single tool, Discover, for an iterative process of discovery and refinement.
Benefits of a Unified Experience
These new capabilities simplify complex workflows that are otherwise difficult to achieve:
- Correlate Everything: Combine trace filters with log messages, infrastructure metrics, or any other data you have in Elasticsearch. Find a slow trace and immediately see the corresponding logs from the affected pod, all in a single view.
- Enhanced Flexibility: Go beyond pre-defined filters. Use the full power of ES|QL to group, aggregate, and filter your trace data based on any attribute, providing comprehensive data analysis options.
- Integrated Experience: Move from a high-level ES|QL query to a detailed trace waterfall without ever breaking your investigative flow.
Looking Ahead: Investigation in Discover
Traces in Discover, powered by ES|QL, delivers a flexible and potent investigative toolset. This is just the beginning, with more to come, including improved correlation between spans, logs, and exceptions within Discover, additional ES|QL commands and powerful UI to make analysis of complex traces easier.
We invite you to dive in and experience it for yourself. Bring your most complex questions and your trickiest bugs. You can now find answers more directly in Discover. This functionality is already available on Serverless. Existing users hosting Elastic themselves will need to upgrade to 8.19+ or 9.1+ to access it.
Try it out today on Elastic Cloud or the OpenTelemetry demo. We look forward to hearing your feedback as you explore traces in Discover!