Principal Software Engineer - Observability

Job description

Job Posting Title: Principal Software Engineer - Observability Req ID: 10157430 Job Description: Disney Entertainment and ESPN Product & Technology Technology is at the heart of Disney’s past, present, and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more – all working to build and advance the technological backbone for Disney’s media business globally. The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for our businesses. We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Company’s media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world. Here are a few reasons why we think you’d love working here: Building the future of Disney’s media: Our Technologists are designing and building the products and platforms that will power our media, advertising, and distribution businesses for years to come. Reach, Scale & Impact: More than ever, Disney’s technology and products serve as a signature doorway for fans’ connections with the company’s brands and stories. Disney+. Hulu. ESPN. ABC. ABC News…and many more. These products and brands matter to millions of people globally. Innovation: We develop and implement groundbreaking products and techniques that shape industry norms and solve complex and distinctive technical problems. Product Engineering is a unified team responsible for the engineering of Disney Entertainment & ESPN digital and streaming products and platforms. This includes product engineering, media engineering, quality assurance, and the engineering behind personalization, commerce, lifecycle, and identity. The Observability & Insights group ensures that Disney Streaming’s distributed systems are reliable, performant, and transparent. We build the telemetry, dashboards, alerting, insights pipelines, and developer experience tooling that enable engineers across the organization to understand system health and take action quickly. Job Summary As a Principal Software Engineer, you are a forward-thinking, highly technical, hands-on-keyboard builder. You work across unfamiliar codebases, teams, and subject matters to accelerate software engineering velocity with AI, shipping production code for our products and applications across the enterprise in a fast-paced, AI-native engineering environment, with a strong understanding of application observability. You will design and build intelligent systems that improve the reliability and performance of Disney’s large-scale streaming ecosystem, including fully autonomous agentic systems and real-time pipelines that turn telemetry, logs, and user signals into automated detection, root cause analysis, and proactive insights across Disney+, Hulu, and ESPN. You operate with independence and a strong innovation bias, scoping your own work, making sound tradeoff decisions, and tying everything you build to clear business value and security approvals. You use frontier AI models as a force multiplier to deliver well above typical engineering velocity while keeping token and compute costs rational through optimization and smart model routing. You have a strong desire to work as a technical thought leader setting technical direction and raising the engineering bar through design and code reviews. Responsibilities Design and operate intelligent, production-grade systems that use real-time signals and AI-driven detection to improve the health of streaming platforms, critical services, and customer experience. Build and scale fully autonomous agentic systems powered by modern frontier models (e.g. Anthropic, OpenAI, Google, Meta) that reason over complex system behavior, decide and act with minimal human intervention, and drive faster detection and resolution. Design intelligent harnesses, memory, and multi-agent orchestration patterns – including prompting and context strategies, task decomposition, retrieval, tool routing, and error recovery – that maximize model performance and accuracy and reduce hallucinations across AI-generated context and actions. Develop end-to-end systems across front end and back end, including data and decisioning pipelines, and scalable APIs that deliver predictive signals, explainability, and insights to engineering teams and product stakeholders. Keep AI usage cost-effective through prompt and context optimization, caching, batching, and smart model routing across frontier APIs and self-hosted open-source models. Apply software engineering best practices end to end: clean, well-tested code, thorough code reviews, and mature CI/CD, owning components of production systems. Drop into brand-new or unfamiliar codebases, rapidly build a working mental model, and modernize them with AI, partnering cross-functionally to embed intelligence into workflows such as incident response, release validation, and customer insights and to drive innovation in observability, reliability, and developer productivity. Basic Qualifications Bachelor’s degree in computer science, Engineering, or equivalent experience. 10+ years of software engineering experience, including building AI-powered or data-driven applications and scalable APIs (e.g. FastAPI, Flask) and deploying production systems at scale. Proven ability to ramp quickly on unfamiliar codebases and rebuild or modernize systems with AI at significantly higher productivity than a typical engineer, while maintaining quality. Tangible examples where your application of AI has led to 2x to 4x times productivity gains are a strong plus. Demonstrated expertise in model prompting, context engineering and harnessing, and the design of orchestration patterns and fully autonomous agentic workflows. Strong AI/ML engineering experience, including orchestrating foundation models (e.g. Claude, OpenAI, Qwen) using frameworks like LangChain or LangGraph. Track record building and deploying high-quality systems across both front end and back end on hyperscaler cloud platforms (AWS, Azure, or GCP), and integrating frontier model APIs (Anthropic, OpenAI). Fluency with AI-assisted development tools (e.g., Cursor, Claude Code) to accelerate engineering velocity, and demonstrated experience optimizing AI usage for cost and performance through prompt optimization, caching, and smart model routing. Experience with modern development practices, including version control (GitHub), containerization (Docker), cloud-native deployments (AWS/EKS), and mature CI/CD pipelines. Strong understanding of API design, microservices architecture, and standard SDLC workflows. Track record of working with extreme independence and innovation, scoping and delivering high-impact work that maps directly to business outcomes, and the ability to set technical direction and mentor other engineers. Strong analytical and technical skills to troubleshoot issues, iterate rapidly, and quickly arrive at viable solutions. Strong collaboration and communication skills, with the ability to work cross-functionally and clearly explain complex technical concepts to technical and non-technical stakeholders. Preferred Qualifications Experience building SaaS solutions in the observability space and handling high-volume telemetry data (e.g., Datadog, Grafana, Conviva). Familiarity with OpenTelemetry (OTel) is a bonus. Experience integrating with enterprise AI services such as AWS Bedrock, including model invocation, routing, and governance integration. Experience deploying and serving open-source models hosted locally or in private infrastructure, including inference optimization and cost/performance tuning. Familiarity with large-scale data platforms and distributed data processing tools (e.g., PySpark, Pandas, Databricks, Snowflake). Knowledge of prompt design, model evaluation, and fine-tuning foundation models (e.g., Claude, GPT). Experience implementing production-grade systems at scale within a fast-paced, distributed environment. About Disney Entertainment and ESPN Product & Technology At Disney Entertainment and ESPN Product & Technology, we’re blending imagination and innovation to reimagine the ways people experience and engage with the world’s most beloved stories and products. We create amazing experiences, transform the future of media, and build products and platforms that enable the connection between people everywhere and the stories and sports they love. The hiring range for this position in Glendale, CA is $184,300 to $247,100 per year and in New York, NY is $193,100 to $258,900 per year. The base pay actually offered will take into account internal equity and also may vary depending on the candidate’s geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered. Job Posting Segment: PE - Sports, News & Entertainment, Tech Enablement Job Posting Primary Business: PE - Sports, News & Entertainment, Enablement - Insights & Observability Primary Job Posting Category: Software Engineer Employment Type: Full time Primary City, State, Region, Postal Code: New York, NY, USA Alternate City, State, Region, Postal Code: USA - CA - 1200 Grand Central Ave Date Posted: 2026-08-13