Platform Engineer, RL & Eval Data | 100% remote
About the team
The Human Data Division builds training and evaluation data for frontier AI labs: RL environments, coding and STEM evaluation tasks, and expert-annotated datasets. We are a small team that moves fast and holds a high quality bar.
The role
We have built our own evaluation, RL and annotation platforms, and they are already in use. We are looking for a platform engineer to join the platform team as a core member and play a central part in their next stage: keeping them running, making them faster and more reliable, and building the features the team and our clients need next.
You will work across all of these systems and get to know them in depth. You will work closely with the people who write our tasks, review our data and work with the labs, and help turn what they need into working software.
Responsibilities
- Maintain and improve the platforms. Learn the existing codebases in depth and keep them healthy in production: fix bugs, pay down technical debt, improve test coverage and documentation.
- Build new features. Design and ship new capabilities across the stack, from the database and backend to the APIs and internal tools.
- Scale the systems. Grow the platforms from today's volume to much larger workloads, with predictable performance and cost.
- Build RL environments and their infrastructure. Build environments that agents act in, and the infrastructure that runs them at scale: sandboxes, environment images, rollouts and reward computation.
- Keep everything reliable and secure. Monitoring, alerting, incident response, access control, safe handling of untrusted code and protection of client data.
