AI Developer Trace Task Auditor
FinanceRemote — United StatesHourly contractPosted Aug 28
Evaluate the quality and correctness of AI-assisted software-development traces used to train and evaluate a frontier AI lab's models. You'll assess end-to-end coding sessions produced with AI-assisted developer tools — judging correctness, workflow soundness, and reasoning — and provide clear, rubric-based written feedback.
Basic Qualifications
- 3+ years professional software development
- Hands-on experience with AI-assisted coding tools and agentic / spec-driven workflows (Cursor, GitHub Copilot, Claude Code, or similar)
- Strong code-reading and debugging skills across full-stack or backend systems
- Ability to evaluate multi-step coding trajectories for correctness and best practice
Preferred Qualifications
- Experience with Kiro or Amazon CodeCatalyst
- Prior work evaluating or grading AI-generated code
- Contributions to developer tooling
The application process
- 1.Short AI video interviewAbout 20 minutes on your experience; you can retake it.
- 2.ID and background checkRequired before a contract starts.
- 3.Contract or paid trialSome roles start with a paid work trial.
