turbopuffer
US / Canada
AI Infrastructure
Posted 5 hours ago
Last seen open on Nov 16, 2026 ยท posted today
turbopuffer is looking for a software engineer who wants to build inference for searching +1T documents.
what you'll do:
work closely with our inference providers to make sure that the p99 inference latency stays as low as possible
write code to improve how the database interacts with our inference providers
integrate new embedding & reranking models into turbopuffer
automate yourself out of a job
must have:
engineering
you've worked in a systems language (Rust, C++, Java, Go, etc.)
experience in inference engineering, e.g. managing a production deployment of an LLM/embedding/reranking model
worked on handling/processing large datasets
you love shipping things to prod
domain expertise
you understand the tradeoff between latency and throughput for model deployments and you know how to think about improving model performance
keeping up with state-of-the-art embedding & reranker models and their evaluation
youโre familiar with the most common inference engines like vLLM and SGLang
you know how to manage the infra of model deployments and how to reason about GPU autoscaling.
typist - you can write well and can explain complex ideas simply
human - you build trust and admit what you donโt know
nice to have:
an interest in embedding & reranker model training
prior experience in vector and full-text search
knowledge about in-depth areas of inference engines & GPU engineering
tpuf is hardcore & whimsical. we value:
overstep > understep
show > tell
complexity is earned
showing up as part of our customerโs team
making stuff easy to react to
Get alerted to similar roles
Get new AI Infrastructure jobs in your inbox
Weekly, free, unsubscribe anytime
This role is classified as AI Infrastructure - see the AI Infrastructure hub for the rest of the market.
$220k - $277k is the middle 50% of disclosed salaries, measured from 208 live AI Infrastructure postings on this board. Roughly two thirds of postings disclose nothing, so this describes the ones that do, not the whole market.
Hiring for a role like this?
Reach AI professionals browsing the board - your listing goes live instantly.