Remote contract combining senior software engineering with preference-data and rubric work, plus building evaluation tooling. Suits experienced engineers who can explain subtle quality issues and trade-offs clearly.
Help an AI research team turn expert engineering judgment into reliable evaluation workflows. In this contract role, you will combine software engineering with hands-on preference-data and rubric work.
What you will do
- Design and refine evaluation rubrics and preference-data criteria.
- Review examples with a high bar for quality, explaining subtle issues and tradeoffs.
- Build or maintain Python or TypeScript tooling and pipelines that support data collection and evaluation.
- Turn findings into concise, actionable write-ups for researchers and engineers.
What we are looking for
- At least five years of professional experience in Python, especially pipelines or infrastructure, or TypeScript in a large codebase.
- Hands-on experience collecting preference data or creating evaluation rubrics.
- Strong technical judgment and clear written and verbal communication.
- A current LinkedIn profile included on your resume
Engagement
- Remote contract for approximately three months, open to candidates in the United States or Canada.
- At least 20 hours per week, including four hours of daily overlap with Pacific Time.
- $100 per hour
- The selection process includes one approximately 30-minute technical and team conversation.
Listing from Braintrust's job board. Details can change; the platform's page is the final word.
