About the Team
Our Inference team brings OpenAI’s most capable research and technology to the world through our products. We empower consumers, enterprise and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they’ve never been able to before. We focus on performant and efficient model inference, as well as accelerating research progression via model inference.
About the Role
We are looking for an engineer who wants to take the world's largest and most capable AI models and optimize them for use in a high-volume, low-latency, and high-availability production and research environment.
In this role, you will:
Work alongside machine learning researchers, engineers, and product managers to bring our latest technologies into production.
Work alongside researchers to enable advanced research through awesome engineering.
Introduce new techniques, tools, and architecture that improve the performance, latency, throughput, and efficiency of our model inference stack.
Build tools to give us visibility into our bottlenecks and sources of instability and then design and implement solutions to address the highest priority issues.
Optimize our code and fleet of Azure VMs to utilize every FLOP and every GB of GPU RAM of our hardware.
You might thrive in this role if you:
Have an understanding of modern ML architectures and an intuition for how to optimize their performance, particularly for inference.
Own problems end-to-end, and are willing to pick up whatever knowledge you're missing to get the job done.
Have at least 5 years of professional software engineering experience.
Have or can quickly gain familiarity with PyTorch, NVidia GPUs and the software stacks that optimize them (e.g. NCCL, CUDA), as well as HPC technologies such as InfiniBand, MPI, NVLink, etc.
Have experience architecting, building, observing, and debugging production distributed systems. Bonus point if worked on performance-critical distributed systems.
Have needed to rebuild or substantially refactor production systems several times over due to rapidly increasing scale.
Are self-directed and enjoy figuring out the most important problem to work on.
Have a humble attitude, an eagerness to help your colleagues, and a desire to do whatever it takes to make the team succeed.
Software Engineer, Model Inference • San Francisco
A leading data and AI company is seeking a Senior Software Engineer, Model Serving to design and implement core systems that ensure scalability and operational excellence.You will drive architectur...[internal_linking.show_more]
A pioneering technology firm in San Francisco is seeking a Founding Engineer for ML Inference.This highly technical role focuses on optimizing real-time generative media models.You'll design novel ...[internal_linking.show_more]
A tech company in San Francisco is looking for skilled engineers to optimize AI systems.Candidates should have significant software engineering experience, ideally in distributed systems, and a kee...[internal_linking.show_more]
Anthropic's mission is to create reliable, interpretable, and steerable AI systems.We want AI to be safe and beneficial for our users and for society as a whole.Our team is a quickly growing group ...[internal_linking.show_more]
At Fluidstack, we're building the infrastructure for abundant intelligence.We partner with top AI labs, governments, and enterprises - including Mistral, Poolside, Black Forest Labs, Meta, and more...[internal_linking.show_more]
Software Engineer (Model Evaluation & Benchmarking).San Francisco, California (Hybrid).Equity + Healthcare + 401(k) + PTO.Are you a Software Engineer interested in working on the systems that measu...[internal_linking.show_more]
L3Harris is dedicated to recruiting and developing high-performing talent who are passionate about what they do.Our employees are unified in a shared dedication to our customers’ mission and quest ...[internal_linking.show_more]
Thinking Machines Lab's mission is to empower humanity through advancing collaborative general intelligence.We're building a future where everyone has access to the knowledge and tools to make AI w...[internal_linking.show_more]
Confidence can sometimes hold us back from applying for a job.Here's a secret: there's no such thing as a "perfect" candidate.Poshmark is looking for exceptional people who want to make a positive ...[internal_linking.show_more]
At Nudge, our mission is to develop the best technology for interfacing with the brain to improve people's lives.We're starting with an approach that we believe can help the most people the fastest...[internal_linking.show_more]
Software Engineer, Model Inference.Our Inference team brings OpenAI’s most capable research and technology to the world through our products.We empower consumers, enterprise and developers alike to...[internal_linking.show_more]
R & D Engineer 5 (0443) Job 83887 - Berkeley Wireless Research Center (BWRC).At the University of California, Berkeley, we are dedicated to fostering a community where everyone feels welcome and ca...[internal_linking.show_more]
Hover helps people design, improve, and protect the properties they love.With proprietary AI built on over a decade of real property data, Hover answers age-old questions like “What will it look li...[internal_linking.show_more]
Employer Industry :AI Development Tools Why consider this job opportunity :Flexible time off to promote work-life balance Medical, Dental, and Vision benefits for employees and family coverage Remo...[internal_linking.show_more]
Pulse is tackling one of the most persistent challenges in data infrastructure: extracting accurate, structured information from complex documents at scale.We have a breakthrough approach to docume...[internal_linking.show_more]
Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all.Figma's platform helps teams bring ideas to life-whether you're brainstorming, creating ...[internal_linking.show_more]
Senior Software Engineer, Model Inference.San Francisco Bay Area, California, United States Software and Services.Join Apple Maps to help build the best map in the world.In this role on ML Platform...[internal_linking.show_more]
Our mission is to build the next generation of AI: ubiquitous, interactive intelligence that runs wherever you are.Today, not even the best models can continuously process and reason over a year-lo...[internal_linking.show_more]
