Software Engineer - ML/LLM InferenceAlldus • San Francisco, CA, United States

Software Engineer - ML / LLM Inference

Alldus • San Francisco, CA, United States

[job_card.30_days_ago]

[job_preview.job_type]

[job_card.full_time]

[job_card.job_description]

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Alldus

Principal Recruitment Consultant | AI & Machine Learning | Co-organizer of the AI in Action Podcast

My client is searching for a talented engineer to work on ML / LLM inference and serving. They specialize in developing next-gen LLM fine-tuning and inference engines.

We are seeking a talented and motivated Software Engineer specializing in Machine Learning (ML) and Large Language Model (LLM) inference to join our dynamic ML Inference team. In this role, you will bridge the gap between AI / ML research and systems programming to build and enhance our next-generation LLM Inference Engine. You will play a crucial role in optimizing the performance, scalability, and efficiency of our LLM serving systems.

Key Responsibilities :

Develop and Enhance Inference Engine :

Design, implement, and optimize the next-generation LLM Inference Engine.
Integrate the latest LLM inference techniques from research to enhance latency and throughput.

Performance Optimization :

Conduct deep performance optimizations across multiple layers of the technology stack, including PyTorch, C++, and CUDA.

Analyze and improve system performance to meet the demands of various use cases.

Work closely with customers to understand specific performance requirements and optimize solutions accordingly.

Provide technical expertise and support to ensure successful deployment and operation of inference systems.

Technical Leadership :

Define the roadmap and technical vision for the inference stack.

Lead initiatives to drive innovation and maintain the competitive edge of our inference technologies.

Infrastructure Development :

Collaborate with partner teams to build and maintain scalable, multi-replica serving infrastructure.

Ensure the reliability and scalability of LLM serving systems to handle increasing workloads.

Qualifications : Technical Skills :

Proficiency in systems programming languages such as C++.

Strong experience with machine learning frameworks, particularly PyTorch.

Expertise in GPU programming and CUDA for performance optimization.

Solid understanding of AI / ML concepts, especially related to large language models.

Experience :

Proven experience in developing and optimizing ML / LLM inference systems.

Demonstrated ability to integrate research advancements into production systems.

Experience with performance tuning and profiling across various technology stacks.

Experience with vLLM

Seniority level

Mid-Senior level

Employment type

Full-time

Job function

Industries

Staffing and Recruiting and Software Development

Referrals increase your chances of interviewing at Alldus by 2x

Inferred from the description for this job

San Francisco, CA $130,000.00-$238,000.00 3 days ago

San Francisco, CA $40,000.00-$70,000.00 2 weeks ago

San Francisco, CA $145,000.00-$230,000.00 5 days ago

Full-Stack Software Engineer (Jr / Mid level)

San Francisco, CA $220,000.00-$350,000.00 4 hours ago

San Francisco, CA $150,000.00-$230,000.00 2 months ago

San Francisco, CA $150,000.00-$176,000.00 2 months ago

San Francisco, CA $99,500.00-$200,000.00 1 day ago

San Francisco, CA $130,000.00-$140,000.00 2 days ago

San Francisco, CA $120,000.00-$190,000.00 8 months ago

San Francisco, CA $125,000.00-$175,000.00 1 month ago

Software Engineer, Frontend (All Levels)

San Francisco, CA $150,000.00-$220,000.00 1 hour ago

San Francisco, CA $56.25-$173,000.00 2 weeks ago

San Francisco, CA $176,000.00-$250,000.00 2 weeks ago

Alameda, CA $130,000.00-$160,000.00 4 weeks ago

San Francisco, CA $150,000.00-$283,000.00 2 weeks ago

San Francisco, CA $150,000.00-$300,000.00 5 days ago

San Francisco, CA $165,000.00-$165,000.00 2 years ago

San Francisco, CA $140,000.00-$280,000.00 7 months ago

San Francisco, CA $140,000.00-$180,000.00 1 month ago

San Francisco, CA $130,000.00-$185,000.00 2 months ago

San Francisco, CA $99,500.00-$200,000.00 1 day ago

San Francisco, CA $150,500.00-$269,200.00 2 days ago

San Francisco, CA $100,000.00-$200,000.00 1 year ago

San Francisco, CA $120,000.00-$200,000.00 2 years ago

San Francisco, CA $150,000.00-$250,000.00 9 months ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

[job_alerts.create_a_job]

Software Engineer • San Francisco, CA, United States

[internal_linking.related_jobs]

ML Systems Engineer : Distributed LLM Training & Inference

Scale AI • San Francisco, CA, United States

[job_card.full_time]

A leading AI technology company in San Francisco seeks a team member to build and optimize a machine learning framework for large language models. Candidates should have system optimization experien...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

AI / ML Software Engineer : Inference on Custom Accelerators

Amazon • San Francisco, CA, United States

[job_card.full_time]

A leading e-commerce platform in San Francisco is seeking a Software Development Engineer to develop and optimize machine learning models for custom hardware accelerators.This role involves perform...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

ML Engineer

Catalyst Labs • Menlo Park, CA, US

[job_card.full_time]

Is a rapidly growing Tier 1 VC backed startup based in New York with $60 million in funding revolutionizing how outside sales and service teams work. Their AI technology captures and analyzes real-w...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Sr. Software Engineer, ML

Relyance AI • San Francisco, CA, United States

[job_card.full_time]

NLP for information extraction from legal documents, ML / NLP for information extraction from code and general ML in code analysis, and overall AI backend initiatives. You will partner with cross-func...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

ML Inference Engineer — Scalable AI Systems

Together • San Francisco, CA, United States

[job_card.full_time]

A pioneering AI company in San Francisco seeks a Machine Learning Engineer to join their Inference Engine team.This role involves optimizing AI inference systems, developing high-performance servic...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Senior Software Engineer, Machine Learning

Planet Labs PBC • San Francisco, CA, United States

[job_card.full_time]

We believe in using space to help life on Earth.Planet designs, builds, and operates the largest constellation of imaging satellites in history. This constellation delivers an unprecedented dataset ...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

ML Systems Engineer

Genmo • San Francisco, CA, US

[job_card.full_time]

We are Genmo, a research lab dedicated to building open, state-of-the-art models for video generation towards unlocking the right brain of AGI. Join us in shaping the future of AI and pushing the bo...[show_more]

[last_updated.last_updated_30] • [promoted]

AIML - Sr. Software Engineer, ML Platform Technologies (MLPT)

Apple Inc. • San Francisco, CA, United States

[job_card.full_time]

Software Engineer, ML Platform Technologies (MLPT).San Francisco Bay Area, California, United States Machine Learning and AI. Want to build the platform that enables the next generation of intellige...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Senior Software Engineer - ML Infrastructure

Plaid • San Francisco, CA, US

[job_card.full_time]

Plaid is evolving into an AI-first company, where data and machine learning are the key enablers of smarter, more secure insight products built on top of Plaid’s vast financial data network.T...[show_more]

[last_updated.last_updated_30] • [promoted]

Machine Learning Engineer, Training Infrastructure

IntelliPro Group Inc. • San Francisco, CA, US

[job_card.full_time]

Machine Learning Engineer, Training Infrastructure.We are looking for an ML Engineer with 3+ YOE in high-performance computing systems to manage and optimize our computational infrastruct...[show_more]

[last_updated.last_updated_30] • [promoted]

Distributed ML Systems Engineer- Inference

Together AI • San Francisco, CA, United States

[job_card.full_time]

Together AI is seeking a Distributed ML Systems Engineer to design and build scalable machine learning systems that power our accelerated AI initiatives. This role involves developing large-scale, f...[show_more]

[last_updated.last_updated_30] • [promoted]

ML / AI Software Engineer - Metrics Frameworks

General Motors of Canada • San Francisco, CA, United States

[job_card.full_time]

As a n AI / ML Engineer on the Metrics Frameworks team, part of the Simulation, Evaluation, and Data organization, you will be an individual contributor focused on developing and optimizing infrastru...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Software Engineer, ML Developer Experience

Cerebras • San Francisco, CA, United States

[job_card.full_time]

At Anyscale, we are on a mission to democratize distributed computing and make it accessible to software developers of all skill levels. We are commercializing Ray, a popular open-source project tha...[show_more]

[last_updated.last_updated_30] • [promoted]

ML Research Engineer, ML Systems

Scale AI, Inc. • San Francisco, CA, United States

[job_card.full_time]

Scale's ML platform (RLXF) team builds our internal distributed framework for large language model training and inference. The platform has been powering MLEs, researchers, data scientists and opera...[show_more]

[last_updated.last_updated_30] • [promoted]

LLM Inference Performance Engineer

Baseten • San Francisco, CA, United States

[job_card.full_time]

A dynamic AI startup in San Francisco is seeking a Software Engineer focused on ML performance.This role involves optimizing large language models, debugging and enhancing ML solutions, and produci...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Software Engineer - ML & Platform

Rhizome • San Francisco, CA, United States

[job_card.full_time]

A changing climate demands Resilience by Design.We like solving hard problems with creativity, tenacity, and empathy for our customers. At the same time, we believe that being better stewards in our...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

ML / AI Software Engineer - Metrics Frameworks

General Motors • San Francisco, CA, United States

[job_card.full_time]

As an AI / ML Engineer on the Metrics Frameworks team, part of the Simulation, Evaluation, and Data organization, you will be an individual contributor focused on developing and optimizing infrastruc...[show_more]

[last_updated.last_updated_variable_days] • [promoted]

Software Engineer- L2

Twilio • San Francisco, CA, United States

[job_card.full_time]

At Twilio, we're shaping the future of communications, all from the comfort of our homes.We deliver innovative solutions to. As we continue to revolutionize how the world interacts, we're acquiring ...[show_more]

[last_updated.last_updated_variable_days] • [promoted]