Job Overview

Join Our Team
Mercor connects elite creative and technical talent with leading AI research labs. We are headquartered in San Francisco and our investors include leading figures such as Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Healthcare AI Evaluator

• Type: Contract
• Compensation: $150–$190/hour
• Location: Remote

Responsibilities

• Write and refine prompts to guide model behavior in healthcare scenarios.
• Evaluate LLM-generated responses to healthcare queries for accuracy and completeness.
• Conduct fact-checking of medical claims using trusted sources.
• Annotate model responses highlighting strengths and improvement areas.
• Assess the tone and appropriateness of responses for real-world use.
• Apply consistent evaluation standards using detailed guidelines.

Requirements

• 5+ years of professional experience in Healthcare with an expert degree (e.g., MD, RN, PharmD, etc.).
• Experience in subdomains like Clinical Care, Specialty Medicine, Diagnostics, etc.
• Significant experience using large language models.
• Excellent written communication skills on complex medical topics.
• Attention to detail in evaluating clinical reasoning.

Preferred

• Experience with RLHF, model evaluation, or data annotation.
• Experience writing/editing healthcare content.
• Experience in clinical documentation or patient communication.
• Familiarity with evaluation rubrics or quality scoring systems.

Application Process

• Upload resume
• AI interview based on your resume
• Submit the form

For help or support, reach out via email: support@mercor.com.

Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.