South Asian Software Engineers: Coding Tasks for AI Evaluation
What we're researching
We're running a paid study on the effectiveness of coding evaluation harnesses used to test AI agents. Our team is building a comprehensive suite of programming environments designed to measure agent performance accurately. This work ensures that AI coding assistants are evaluated against realistic, high-quality development scenarios.
What you'll do
- Review proposed coding tasks and programming environments for realism
- Provide technical feedback on the structure and difficulty of evaluation harnesses
- Walk us through your approach to verifying complex code challenges
- Suggest improvements to make the test suites more robust
Compensation
$35 per hour
READY TO PARTICIPATE?
Start your paid interview now https://terac.com/interview/start/r/8a9b5977-2713-40ab-be6a-71bd59f76ca9?utm_source=ashby_listing_description
About terac
Terac is building the world's largest pool of vetted human experts for AI. Researchers, AI labs, and product teams use Terac to recruit, screen, and pay study participants across industries, languages, and skill sets.
Learn more at terac.com https://terac.com or on YouTube at @jointerac https://www.youtube.com/@jointerac.
