Brewing Observability Insights: What LinkedIn, Netflix, and eBay Taught Me at Monitorama 2024

A Senior Product Manager's perspective on Portland's premier monitoring conference

The Monitorama Experience: More Than Just Tech Talks

There's something magical about the first sip of artisanal coffee served by a live barista at 8 AM while chatting with fellow observability enthusiasts from around the globe. That's how each day of Monitorama 2024 began for me this June in Portland, Oregon—a perfect blend of caffeine, community, and technical curiosity that set the tone for what would be three incredible days of learning.

As a Senior Product Manager for Microsoft Azure, I found myself surrounded by practitioners and vendors from across the observability landscape. The Gerding Theater at The Armory provided a beautiful historic backdrop for the single-track conference format that has become Monitorama's hallmark.

What makes Monitorama special:

• Single-track format ensures everyone shares the same experience
• Generous 30-minute breaks between talk pairs encourage meaningful conversations
• Nearly two-hour lunches provide ample networking opportunities
• Live barista coffee creates natural gathering points for discussions
• Historic Gerding Theater venue adds character to the experience

Three Standout Presentations from Industry Giants

What makes Monitorama particularly valuable is hearing directly from companies operating at unprecedented scale. Their challenges today often become everyone else's challenges tomorrow. This year, three presentations stood out for their depth of insight and practical applications.

Note: All Monitorama 2024 presentations mentioned in this article are available on the official Monitorama YouTube channel. I highly recommend watching the full talks to get all the details and see the impressive visualizations that these companies shared.

eBay: The Reality of Distributed Tracing at Massive Scale

\ Vijay Samuel's presentation "Distributed Tracing - All the Warning Signs Were Out There!" offered a refreshingly candid look at eBay's journey implementing distributed tracing across approximately 5,000 microservices and hundreds of databases.

While distributed tracing has long been heralded as an observability silver bullet, Samuel detailed the harsh realities they encountered: unmanageably large trace waterfalls that became difficult to interpret, confidence-eroding issues with mis-instrumented applications, and the challenge of making tracing actually deliver on its promise of reduced time to resolution.

Key takeaways from eBay's presentation:

• Scaling distributed tracing to 5,000+ microservices reveals challenges invisible at smaller scales
• Trace visualization becomes overwhelming with extremely large trace waterfalls
• Mis-instrumented applications quickly erode confidence in the entire tracing system
• Traditional head-based sampling misses important anomalies and errors
• Tail-based sampling (decisions made after seeing complete traces) helped capture valuable outliers
• Implementation matters more than the theoretical benefits of tracing

LinkedIn: Engineering Observability for Professional Networks

\ LinkedIn's presentation dove deep into how they've architected their observability platform to support their massive professional network with hundreds of millions of users and complex interconnected services.

What made LinkedIn's approach fascinating was their emphasis on context-preservation across their observability pipeline. Rather than treating metrics, logs, and traces as separate data streams, they've invested heavily in maintaining correlation between these signals to provide their engineers with coherent views of system behavior.

Key takeaways from LinkedIn's presentation:

• Context preservation across different telemetry types is crucial for meaningful insights
• Two-phase sampling strategy balances cost control with comprehensive visibility
• Initial sampling decisions at the edge based on simple criteria
• "Rescue sampling" preserves traces that become interesting after initial sampling
• Correlation IDs consistently applied across all telemetry types enable powerful cross-signal analysis
• Custom visualization tools designed specifically for their unique network structure

Netflix: Observability Culture as Competitive Advantage

\ Netflix's presentation stood out not just for technical content but for demonstrating how observability has become central to their engineering culture and competitive advantage.

Rather than focusing solely on tools and techniques, their speaker emphasized how Netflix has woven observability requirements into their entire software development lifecycle. Every new service at Netflix is built with comprehensive instrumentation from day one, with clear ownership of the telemetry quality.

Key takeaways from Netflix's presentation:

• Observability treated as a product feature, not an operational afterthought
• "Observability as code" approach keeps monitoring definitions in the same repositories as application code
• Clear ownership model for telemetry quality with regular reviews
• Custom tooling to validate instrumentation quality automatically
• Engineering culture emphasizes "if it's not observable, it's not production-ready"
• Sophisticated automation for anomaly detection that learns from operator feedback

Why These Large-Scale Lessons Matter

These presentations from eBay, LinkedIn, and Netflix are particularly valuable because they operate at scales that push the boundaries of what's possible in observability. Their challenges today are likely to become common challenges for the broader industry tomorrow.

Industry-wide themes from these giants:

• Traditional observability approaches break down at extreme scale
• Cost management becomes critical as data volumes explode
• Integration across telemetry types yields more powerful insights than siloed approaches
• Organizational factors often matter more than technical solutions
• Custom visualization tailored to specific business domains enhances usability
• Sampling strategies must evolve beyond simple probabilistic approaches

Key Takeaways for Observability Practitioners

After three days immersed in Portland's monitoring community, these were the most valuable insights:

1. Tail sampling is becoming essential: As distributed systems grow more complex, intelligently selecting which traces to retain based on their complete context is becoming a standard practice rather than an advanced technique.
2. Aggregation is your friend: The traditional guidance to store everything is being replaced by more nuanced approaches that strategically aggregate data at different points in the telemetry pipeline.
3. Organizational adoption requires investment: Successfully implementing advanced observability across an enterprise depends as much on organizational factors (training, tooling, advocacy) as technical ones.
4. The vendor landscape is evolving rapidly: The emergence of specialized observability pipeline tools indicates a maturing market with increasingly sophisticated options for enterprises.
5. Community remains the heart of innovation: Despite the commercial interests represented at the conference, the spirit of open-source collaboration and community knowledge-sharing remains the driving force behind observability advances.

The Monitorama Magic

As I boarded my train home, laptop bag filled with stickers and t-shirts from various vendors, I reflected on what makes Monitorama special. It's not just the technical content, though that's excellent. It's not just the Portland setting, though the food, coffee, and atmosphere are wonderful.

What creates the Monitorama experience:

• Deliberate creation of spaces for human connection
• Single-track format ensuring shared context for discussions
• Real-world case studies from companies at the cutting edge
• Coffee, food, and social events that facilitate natural networking
• Community-driven ethos that values practitioner experience over vendor pitches

What makes Monitorama magical is the deliberate creation of space for human connection around a shared technical passion. In a world of virtual conferences and remote work, there's immense value in sitting across from another engineer over coffee, sketching architecture diagrams on napkins, and debating the merits of different sampling approaches.

With the announcement that Monitorama is taking a hiatus for the remainder of 2025, these memories feel even more precious. The organizers' decision to take a break and reimagine the event speaks to their commitment to quality over quantity. I'm already eagerly anticipating what the refreshed Monitorama will look like when it returns next year, and I look forward to reconnecting with this amazing community of practitioners who are pushing the boundaries of what's possible in monitoring and observability.