Talent.com
Senior Data Engineer - Community Discovery ML
Senior Data Engineer - Community Discovery MLTwitch • San Francisco, CA
Senior Data Engineer - Community Discovery ML

Senior Data Engineer - Community Discovery ML

Twitch • San Francisco, CA
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

About Us

Twitch is the world’s biggest live streaming service, with global communities built around gaming, entertainment, music, sports, cooking, and more. It is where thousands of communities come together for whatever, every day.

We’re about community, inside and out. You’ll find coworkers who are eager to team up, collaborate, and smash (or elegantly solve) problems together. We’re on a quest to empower live communities, so if this sounds good to you, see what we’re up to on and , and discover the projects we’re solving on our . Be sure to explore our to learn how to ace our interview process.

About the Role

Community Discovery ML team focus on providing personalized, relevant experience for Twitch users through Recommendation and Search. We are looking for a senior data engineer to join us. You will be the first data engineer hired in a hybrid team of ML engineers and scientists and working on data challenges related to ML models and products. You will extend, design and build new capabilities in our data systems to ensure fast ML model development and productionization. You will impact cross teams by defining expectations for data usage patterns and data quality.

You will report to an Engineering Manager and work in San Francisco / Bay Area.

You Will :

  • Oversee team data architecture to meet ML use cases in production.
  • Design and build scalable data pipelines to support personalization models.
  • Develop and maintain low-latency, large-scale streaming and batch data processing systems.
  • Collaborate with applied scientists and ML engineers to integrate data into production models.
  • Optimize data workflows for performance and cost efficiency.
  • Implement best practices for data governance and security.
  • Troubleshoot and resolve data-related issues, with a focus on identifying and solving data quality problems.
  • Mentor others in the team in data related solutions and skills.

You Have :

  • 6+ years of experience as a data engineer or in a similar role.
  • Proficiency in SQL, Python, or Scala.
  • Experience with building batch and streaming data pipelines with high throughput and low latency.
  • Strong understanding of data architecture and data modeling principles.
  • Experience analyzing large datasets to identify gaps and inconsistencies, provide data insights, and promote effective product solutions
  • Hands-on experience with cloud platforms (AWS, GCP, or Azure) and their data services.
  • Familiarity with ETL tools and data warehousing solutions.
  • Experience with distributed data processing technologies such as Apache Spark, Flink, and Kafka.
  • Experience working with cross-functional roles like ML engineers and scientists.
  • Bonus Points

  • Experience with AWS data ecosystems like Redshift, Kinesis and Glue.
  • Understand data requirements for ML production systems.
  • Extensive experience with mature and large-scale production data systems and capable of defining a strong North Star and making increments progress towards that.
  • Perks

  • Medical, Dental, Vision & Disability Insurance
  • 401(k)
  • Maternity & Parental Leave
  • Flexible PTO
  • Amazon Employee Discount
  • [job_alerts.create_a_job]

    Senior Data Engineer Community Discovery ML • San Francisco, CA

    [internal_linking.similar_jobs]
    Senior Machine Learning Engineer - Personalization

    Senior Machine Learning Engineer - Personalization

    Warner Bros. Discovery • San Francisco, CA, United States
    [job_card.full_time]
    Senior MLE, Personalization, Content Discovery.Discovery the stuff dreams are made of.When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and supe...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Data Engineer, Analytics

    Senior Data Engineer, Analytics

    Discord • San Francisco, CA, United States
    [job_card.full_time]
    Discord is used by over 200 million people every month for many different reasons, but there’s one thing that nearly everyone does on our platform : . Over 90% of our users play games, spending a comb...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Remote Senior ML Engineer – LLMs, DL & MLOps

    Remote Senior ML Engineer – LLMs, DL & MLOps

    Zendesk • San Francisco, CA, United States
    [filters.remote]
    [job_card.full_time]
    A customer experience software company is seeking a Machine Learning Engineer to join their Enterprise Machine Learning team. This role emphasizes the development of advanced ML solutions, particula...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Data & Infra Engineer

    Senior ML Data & Infra Engineer

    Homeward • San Mateo, CA, United States
    [job_card.full_time]
    A progressive healthcare organization in California seeks an experienced professional to transform healthcare delivery through data and machine learning algorithms. The role involves managing the en...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer, Perception & Sensor Pipelines Lead

    Senior ML Engineer, Perception & Sensor Pipelines Lead

    Waymo • San Francisco, CA, United States
    [job_card.full_time]
    A leading autonomous driving technology company is seeking an experienced Machine Learning Engineer to lead the development of robust data platforms for large-scale system evaluations.You'll collab...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer (MLE)

    Senior Machine Learning Engineer (MLE)

    Equinix • Redwood City, CA, United States
    [job_card.full_time]
    Senior Machine Learning Engineer.Equinix is the world's digital infrastructure company, shortening the path to connectivity to enable the innovations that enrich our work, life and planet.A place w...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Machine Learning Engineer - Ads

    Senior Machine Learning Engineer - Ads

    Uber • San Francisco, CA, United States
    [job_card.full_time]
    The Ads Machine Learning (Ads ML) team at Uber is responsible for providing relevant ad recommendations to the users across the different applications within the Uber ecosystem.We focus on building...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Data Engineer

    Senior Data Engineer

    Alembic Technologies • San Francisco, CA, United States
    [job_card.full_time]
    About The Role - Senior Data Engineer.As a Senior Data Engineer at Alembic, you'll tackle some of the hardest technical challenges in marketing analytics while building systems that Fortune 100 com...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Senior ML Engineer for Ad Tech Growth & Bidding

    Senior ML Engineer for Ad Tech Growth & Bidding

    UNITY • San Francisco, CA, United States
    [job_card.full_time]
    A leading ad-tech company seeking a Senior Machine Learning Engineer in San Francisco to enhance ads delivery systems.The role involves optimizing large-scale distributed systems, developing biddin...[show_more]
    [last_updated.last_updated_less] • [promoted] • [new]
    Senior ML Engineer AI-Driven Cloud Infra

    Senior ML Engineer AI-Driven Cloud Infra

    Salesforce.Com Inc • San Francisco, CA, United States
    [job_card.full_time]
    A leading cloud-based software company is seeking an experienced Software Engineer in San Francisco.You'll design and maintain automated cloud infrastructure, applying AI tools to tackle complex ch...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior ML Engineer, Discovery & Generative AI

    Senior ML Engineer, Discovery & Generative AI

    Scribd, Inc. • San Francisco, CA, United States
    [job_card.full_time]
    A leading content platform is seeking a Machine Learning Engineer to design, build, and optimize scalable ML systems.The role involves working through the entire ML lifecycle, including data ingest...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer

    Senior Machine Learning Engineer

    Axon • San Francisco, CA, United States
    [job_card.full_time]
    Join Axon and be a Force for Good.At Axon, we’re on a mission to Protect Life.We’re explorers, pursuing society’s most critical safety and justice issues with our ecosystem of devices and cloud sof...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Platform Engineer

    Senior Machine Learning Platform Engineer

    Poshmark, Inc. • Redwood City, CA, United States
    [job_card.full_time]
    A leading resale marketplace firm in Redwood City is seeking a Machine Learning expert to drive data science initiatives, manage the entire machine learning lifecycle from data collection to deploy...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Research Engineer, LLM Data

    Senior Research Engineer, LLM Data

    MLabs • San Francisco, CA, United States
    [job_card.full_time]
    Our client is a research lab that provides post-training data and RL environments to foundation model labs and frontier applied AI companies. They have raised significant funding from top-tier VCs a...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer - Data & Recommendations

    Senior Machine Learning Engineer - Data & Recommendations

    Digital Technologies, LLC • San Francisco, CA, United States
    [job_card.full_time]
    A technology firm based in San Francisco is looking for a Senior Software Engineer specializing in Machine Learning.The ideal candidate will have 7-10 years of software development experience, incl...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Data Engineer : ML Pipelines & Signal Processing

    Senior Data Engineer : ML Pipelines & Signal Processing

    Zendar • Berkeley, CA, United States
    [job_card.full_time]
    An innovative tech firm in Berkeley seeks a Senior Data Engineer to manage complex data engineering pipelines.You will ensure data quality, support ML engineers across locations, and establish infr...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer - Ranking and Recommendations

    Senior Machine Learning Engineer - Ranking and Recommendations

    San Francisco Staffing • San Francisco, CA, United States
    [job_card.full_time]
    Senior Machine Learning Engineer.The Shopping Ranking Team mission is enabling eaters to effortlessly make shopping decisions and find what they need. We pursue this mission via an ML-driven algorit...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Senior Machine Learning Engineer - GenAI Platform

    Senior Machine Learning Engineer - GenAI Platform

    Databricks Inc. • San Francisco, CA, United States
    [job_card.full_time]
    Founded in late 2020 by a small group of machine learning engineers and researchers, Mosaic AI enables companies to securely fine-tune, train and deploy custom AI models on their own data, for maxi...[show_more]
    [last_updated.last_updated_30] • [promoted]