Your brand on every pageBecome a sponsor โ†’
This role is no longer accepting applications. The posting has closed or expired - but there are other AI roles hiring now below.
turbopuffer logo

software eng, inference

turbopuffer

US / Canada

AI Infrastructure

Posted 5 hours ago

midunknown

Last seen open on Nov 16, 2026 ยท posted today

Job Description

turbopuffer is looking for a software engineer who wants to build inference for searching +1T documents.

what you'll do:

  • work closely with our inference providers to make sure that the p99 inference latency stays as low as possible

  • write code to improve how the database interacts with our inference providers

  • integrate new embedding & reranking models into turbopuffer

  • automate yourself out of a job

must have:

  • engineering

    • you've worked in a systems language (Rust, C++, Java, Go, etc.)

    • experience in inference engineering, e.g. managing a production deployment of an LLM/embedding/reranking model

    • worked on handling/processing large datasets

    • you love shipping things to prod

  • domain expertise

    • you understand the tradeoff between latency and throughput for model deployments and you know how to think about improving model performance

    • keeping up with state-of-the-art embedding & reranker models and their evaluation

    • youโ€™re familiar with the most common inference engines like vLLM and SGLang

    • you know how to manage the infra of model deployments and how to reason about GPU autoscaling.

  • typist - you can write well and can explain complex ideas simply

  • human - you build trust and admit what you donโ€™t know

nice to have:

  • an interest in embedding & reranker model training

  • prior experience in vector and full-text search

  • knowledge about in-depth areas of inference engines & GPU engineering

tpuf is hardcore & whimsical. we value:

  • overstep > understep

  • show > tell

  • complexity is earned

  • showing up as part of our customerโ€™s team

  • making stuff easy to react to

Applications closed
Share this role:WhatsAppLinkedInX

Get alerted to similar roles

Get new AI Infrastructure jobs in your inbox

Weekly, free, unsubscribe anytime

Career context

This role is classified as AI Infrastructure - see the AI Infrastructure hub for the rest of the market.

$220k - $277k is the middle 50% of disclosed salaries, measured from 208 live AI Infrastructure postings on this board. Roughly two thirds of postings disclose nothing, so this describes the ones that do, not the whole market.

Share your AI salary

Anonymous. No login, no name, no email required. It takes about 20 seconds, and it is how this board builds real pay data for AI roles instead of guesses.

Before bonus and equity.

Hiring for a role like this?

Reach AI professionals browsing the board - your listing goes live instantly.

Post a job โ†’

Other AI jobs hiring now

Stay ahead of the curve. Get the week's new AI jobs in your inbox. Free, unsubscribe anytime.