The future of observability: Trends and predictions business leaders should plan for in 2023 and beyond

If the past year has taught us anything, it’s that the more things change, the more things stay the same. The whiplash and pivot from the go-go economy post-pandemic to a belt-tightening macroeconomic environment induced by higher inflation and interest rates has been seen before, but rarely this quickly. Technology leaders have always had to do more with less, but this slowdown may be unpredictable, longer, and more pronounced than expected. So let’s take a look into our cloud and observability crystal ball to identify some key trends and predictions that we see impacting the industry this coming year and beyond.

1. Tightening macroeconomic conditions drive focus on TCO

With tightening macroeconomic conditions, organizations will continue to focus on accelerating cloud adoption for enabling digital transformation and delivering a differentiated customer experience. Observability will play a crucial role in ensuring these digital transformation efforts are successful.

Observability will be increasingly critical to providing a view into the performance of applications and infrastructure as they become more distributed in hybrid and multi-cloud environments. As a result, your observability solution’s ability to integrate with cloud services, along with the ability to scale to large amounts of data, becomes increasingly important. Full-stack, cloud-native observability will be critical in helping organizations track the adoption of cloud services and transformation projects while staying on time and within budget. 

Additionally, in the earlier days of cloud adoption and stronger economic conditions, cloud spend was a concern but behind in priority to the pace of quicker cloud adoption. With tightening macroeconomic conditions, however, companies are looking for ways to improve compute efficiencies on cloud providers such as AWS, Azure, and Google Cloud. This goal requires a detailed understanding of the performance of applications and infrastructure. Zero instrumentation, whole-system visibility via continuous profiling helps identify performance bottlenecks, leading to optimized resource usage and an increased green focus.

[Related article: Key strategies for CIOs to safely and smartly move to the cloud and speed digital transformation]

2. Increased focus on correlating operational and business data makes observability business-critical

In the data center world, where infrastructure and apps were treated as pets, digitization was limited and change was infrequent; monitoring usage and consumption helped identify bottlenecks when business-critical applications were being impacted. However, in the cloud-native world, where the norm is multiple code deployments in a day, distributed applications using microservices on ephemeral infrastructure present a significantly different challenge. 

While a picture is worth a thousand words, monitoring just infrastructure and application performance in today’s complex environments is often insufficient to understand the impact on the business. For SRE and operations teams to succeed, they need to be able to correlate business and operational KPIs and proactively detect anomalies to ensure that SLOs and SLAs are being met. This increased accountability requires an observability solution that can ingest and correlate all telemetry data and relevant business metrics and alert on error budget and SLO/SLI violations.

3. Tools consolidation continues to gain traction

Economic and operational efficiency continues to drive executive-level focus on tool consolidation. Organizations are looking for solutions that help them ingest all their telemetry data (metrics, logs, and traces) in a single platform that can deliver proactive insights and the ability to understand the unknown unknowns. A single unified observability platform also enables teams to accelerate business transformation and deliver efficiencies by developing and adopting best practices, as well as improved workflows and automation.

In addition, it is becoming increasingly clear to organizations that observability doesn’t end with the collection of metric, log, and trace-related data. They are also interested in leveraging observability to understand the impact of application and infrastructure performance and availability on customer experience and business KPIs — for example, abandoned carts, customer wait time, and delayed transactions. 

Increasingly, CIOs, CFOs, and business leaders are looking at the impact of IT systems on the business and how it can be leveraged to make critical decisions proactively. The ability to connect these dots rests in consolidating all the information on a unified platform — this means shifting away from using swivel-chair methods across siloed tools that are unnecessarily complex, take more time to find the root cause, and slow problem resolution while often not providing visibility into the business impact.

4. Adoption of cloud-native technologies like Kubernetes (k8s) and serverless in production environments accelerates

Application teams are increasingly adopting technologies that help them abstract hardware and software layers, making it easier for the infrastructure to dynamically scale to the requirements and load of the applications. This automation reduces operational costs while providing teams the agility to deploy code and feature updates more quickly. 

And while these technologies operationally help modern applications scale easily, their complexity and self-healing nature makes troubleshooting and performance optimization challenging without a unified observability solution. Kubernetes and serverless are becoming the de facto standard for modern applications, and monitoring and management tools must evolve with these newer architectural and deployment options. 

5. Machine learning and analytics capabilities continue to evolve and enable increased automation with AIOps

The volume of data continues to increase exponentially, and teams are often forced to be reactive or search through an avalanche of data to find meaningful insights. Increasingly, operations teams are looking to gain proactive actionable insights from the data. Although there has been a lot of hype related to machine learning (ML), the need for machine learning frameworks and algorithms is real and will continue to grow and become more critical for observability. 

Teams will require out-of-the-box, domain-specific ML capabilities that don’t require data science knowledge while also having the flexibility to customize and bring your own ML models for organizations with in-house data science expertise. The ability to run analytics and machine learning to surface anomalies and automate IT Ops workflows will continue to be a key driver for modern observability solutions.

6. Open standards become critical decision criteria when choosing an observability platform

Currently, a number of observability vendors claim support for open standards (such as OpenTelemetry), but often, the support for open standards is lightweight and primarily for marketing purposes. 

Customers and organizations will start to demand full support for open standards as they continue to avoid vendor lock-in. Observability vendors will look to materially increase their support for commonly used open-source technologies to meet market expectations. As a result, native open standards support will be expected within any observability offering. The goal: allow organizations to seamlessly integrate these widely available technologies into their workflows with their existing vendor tools with minimal work.  

7. Observability Centers of Excellence become a must-have

With the complexity of deployment and environments increasing, and multiple development and operations teams wanting to get visibility and insights, organizations are increasingly realizing the need for an observability Center of Excellence (COE). An observability COE helps drive standards across multiple teams and ensures adherence to best practices (e.g., using the right labels, properly instrumenting code, not leaving debug enabled), along with implementing pipeline automation capabilities. 

In addition, these COEs also understand observability-related challenges, bring best-of-breed solutions, and drive standardized processes and workflows across the entire organization, making sure that expected ROI and value are delivered. COEs represent a substantial step forward, in terms of observability maturity, for most organizations.

8. Pushback on traditional observability pricing models ramps up

Another impact of observability becoming business-critical is the pushback on traditional pricing models for observability that are inherently complex, unintuitive, and often accompanied by surprises. Operations and development teams need to be focused on ensuring the availability and performance of applications and infrastructure. Not worrying about the observability bill allows teams to focus on adopting observability and gaining complete visibility across their environment, leading to faster issue resolution. 

While observability pricing will continue to have some level of complexity due to the inherent complexity of the underlying applications and infrastructure supported, there will be an increased focus on value-driven pricing with tiers and add-ons. Customers will look to understand the pricing models and favor transparent and predictable options that enable them to deliver on the business outcomes: accelerating cloud adoption, alerting noise reduction, delivering on SLAs, and improving teamwork and collaboration.

Observability is key to navigating your organization through economic uncertainty

Enterprise technologies and IT operations have always been about driving efficiencies and bringing new capabilities to teams for improving business operations. And in today’s cost-constrained financial environment, organizations that are able to contain costs without sacrificing the future of the business will emerge from this downturn in much better shape than their competition. 

Observability will continue to evolve quickly and help organizations better understand where they can optimize and improve their IT operations, as well as cloud adoption and utilization. This will enable executives to walk that fine line where operational budgets are met while still executing on long-term, strategic goals.

Learn more about cloud and observability trends for the upcoming year in our webinar: Emerging trends in observability for today’s challenging economic conditions. In this recorded webinar, guest speakers from Forrester and AWS provide insights on how your peers are approaching cloud adoption and observability in a cost-constrained environment.