Staff Software Engineer, ML Training Platform

Job Description:

Staff Software Engineer, ML Training Platform

Location:
This role is completely remote-friendly. If you happen to live close to one of our physical office locations, our doors are open for you to come into the office as often as you'd like.

Who We Are:
The Machine Learning Platform team at Reddit is a high-impact team that owns the infrastructure that powers recommendations, content discovery, user and content quantification, while directly impacting other teams such as Growth, Ads, Feeds, and Core Machine Learning teams.

What You’ll Do:
As a Staff Software Engineer, ML Training Platform, this person will be instrumental in architecting, implementing, and maintaining foundational Machine Learning Training infrastructure that powers Feeds Ranking, Content Understanding, Recommendations and much more to fulfill Reddit’s mission of bringing community and belonging to everyone in the world. You will oversee GPU optimization in AI/ML batch workloads and debug performance bottlenecks in GPU workloads. You will build and own systems and tools that enable MLEs and data scientists, and continuously improve the ML software development lifecycle.

• Optimize model training on GPUs
• Lead the building, testing, and maintenance of ML infrastructure at Reddit
• Propose, design, and implement high-performance ML platform solutions that significantly advance the deployment of models that serve millions of redditors a seamless experience for MLEs
• Play a pivotal role in designing, building, and optimizing the infrastructure and tooling required to support large-scale machine learning workflows
• Analyze bottlenecks in distributed systems and optimize for performance and cost-efficiency
• Work with management on team goal setting, planning, and de-risk project execution
• Mentor other team members in adopting a rigorous DevOps approach to maintain and/or improve ML platform components and services health and quality

Who You Might Be:

• 8+ years of work experience in a production software development environment or building data systems
• Experience with XLA for Tensorflow or torch.inductor for pytorch for kernel fusion during training
• Experience with optimization of data workloads using collosal.AI or Deepspeed
• Experience with distributed Training optimization using deepspeed, horovod or collosalAI
• Experience with design and architecture of large scale ML Systems
• Experience with training workflows, hyperparameter tuning, and resource optimization on CPU and GPU
• Experience with MLOps practices and tools such as Ray and MLFlow
• Hands-on experience with Kubernetes, Docker, or other container orchestration systems

Benefits:

• Comprehensive Healthcare Benefits and Income Replacement Programs
• 401k Match
• Family Planning Support
• Gender-Affirming Care
• Mental Health & Coaching Benefits
• Flexible Vacation & Reddit Global Days off
• Generous paid Parental Leave
• Paid Volunteer time off

In addition to base salary, this job is eligible to receive equity in the form of restricted stock units, and depending on the position offered, it may also be eligible to receive a commission. Additionally, Reddit offers a wide range of benefits to U.S.-based employees, including medical, dental, and vision insurance, 401(k) program with employer match, generous time off for vacation, and parental leave.

The base pay range for this position is:

$230,000 - $322,000 USD