Comcast Lead Site Reliability Engineer in Virtual, Pennsylvania
As a Lead Site Reliability Engineer, you will be responsible for the performance, reliability, and security of revenue-critical systems serving millions of QPS with latency thresholds of 100ms.
You will own and manage the infrastructure that underpins our bidding system and will work with individual engineering teams to ensure that their systems are highly available, highly monitored and easily deployable.
Our core product is an easy to use, massive scale and high availability advertising platform. We offer our customers the most extensible and transparent advertising system in the world.
Ideal candidates will have
At least 10 years of experience in the infrastructure space
Experience in large scale cloud environments (thousands of servers)
Experience in designing, analyzing and troubleshooting large-scale distributed systems
Strong experience in Observability (monitoring and alerting) at large scale
Hands-on experience with AWS
Experience with high performance networking
Experience with low level performance tuning (kernel, knowing how to use ebpf tools to understand what to tune, etc)
“What type of projects would I do in the first three months?”
Here's an example of what's coming up for us:
Designing our next generation of observability tooling, taking us from what we have now to something capable of tying together metrics, logs, and traces.
Overhauling our (very large!) scale load balancers, exploring envoy and other technologies for the second generation of these.
“How about the first year?”
We’ve no end to projects the SRE team would love to work on:
Improve our Kubernetes platform, exploring additional technologies like service meshes, improved load balancing, autoscaling improvements, taking advantage of spot instance fleets.
Work with other engineers to rearchitect and rebuild some of the core services their teams rely on to be more efficient and cost effective.
Standardizing our approaches to observability so it’s easy for a developer to do the right thing.
Move our terraform automation to be backed by terraform cloud
These are just some of the things we plan to work on over the next year, and we hope you’ll bring your knowledge and expertise and propose projects of your own to help us continually improve.
The qualities of our team culture include
An ethic of service and a belief in putting the customer first
A powerful sense of pragmatism to figure out what needs to be done right versus right now
A curiosity about technology and a desire to use it to solve problems in all sorts of domains
An openness to feedback and more than just the spelling skills to know that there’s no I in Team
An appreciation of repeatability, observability, and operational simplicity
If this speaks to you, we want to hear from you!
Employees at all levels are expected to
Understand our Operating Principles; make them the guidelines for how you do your job.
Own the customer experience - think and act in ways that put our customers first, give them seamless digital options at every touchpoint, and make them promoters of our products and services.
Know your stuff - be enthusiastic learners, users and advocates of our game-changing technology, products and services, especially our digital tools and experiences.
Win as a team - make big things happen by working together and being open to new ideas.
Be an active part of the Net Promoter System - a way of working that brings more employee and customer feedback into the company - by joining huddles, making call backs and helping us elevate opportunities to do better for our customers.
Drive results and growth.
Respect and promote inclusion & diversity.
Do what's right for each other, our customers, investors and our communities.
- This information has been designed to indicate the general nature and level of work performed by employees in this role. It is not designed to contain or be interpreted as a comprehensive inventory of all duties, responsibilities and qualifications.
Comcast is an EOE/Veterans/Disabled/LGBT employer.
Relevant Work Experience
10 Years +
Base pay is one part of the Total Rewards that Comcast provides to compensate and recognize employees for their work. Most sales positions are eligible for a Commission under the terms of an applicable plan, while most non-sales positions are eligible for a Bonus. Additionally, Comcast provides best-in-class Benefits. We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That’s why we provide an array of options, expert guidance and always-on tools, that are personalized to meet the needs of your reality – to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the compensation and benefits summary (https://jobs.comcast.com/life-at-comcast/benefits) on our careers site for more details.