Intelligence Analyst Job at micro1, San Francisco, CA

bVpNb1JLOVlUMWNPaWJYQk5PMEwvaTlPaUE9PQ==
  • micro1
  • San Francisco, CA

Job Description

Pay: $30 - $80/hour

Required Skills
Analytic rigor
Source handling
Over-refusal detection
All-Source Intelligence Analysis
About micro1
micro1 is a data lab that helps AI labs & Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM capabilities, as well as contextual evaluations to monitor and improve AI agents in enterprise settings.

Our AI recruiting agent sources and vets domain experts for our data platform that enables rapid production of high-quality training data. Our research spans high-value, specialized domains (legal, medical, finance, coding, etc.), where expert-level data is the hardest to produce and most valuable for advanced model performance. Our pipeline performance system ensures both quality and velocity.

Role Title: Intelligence Analyst

Role Type: Contractor

Location: Remote

micro1 is engaging Intelligence Analysts to contribute to a customer-facing project focused on evaluating and advancing artificial intelligence capabilities. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

Develop and map analytic cases to defined taxonomies across multiple evaluation areas, ensuring comprehensive coverage of relevant intelligence scenarios.
Author complex test cases designed to evaluate model performance on source handling, analytic rigor, and nuanced uncertainty expression.
Assess and grade AI-generated responses based on criteria such as reasoning, source use, recognition of limitations, and practical utility.
Identify and flag instances where legitimate analytic exploration is over-refused by the model, providing clear documentation and feedback.
Document edge cases and analytic disagreements, supporting continuous improvement of evaluation methodologies.
Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
Contribute insights to help calibrate analytic standards and enhance the overall quality of AI system benchmarks.

Preferred Qualifications

Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline, ideally with a civilian-IC background.
Strong proficiency in source evaluation, confidence-level assessment, and established analytic standards.
Experience in teaching, mentoring, or leading analytic reviews, with a record of fostering analytic rigor.
Exceptional written and verbal communication skills, with a focus on clarity and precision.
Ability to identify and articulate nuanced model limitations and over-refusal scenarios.
Familiarity with structured analytic techniques and intelligence tradecraft standards.
Previous participation in AI evaluation or model assessment initiatives is a plus.
Email

By clicking any of the "Continue" buttons below, you acknowledge and agree to our Terms and Privacy Policy for candidates.

Continue
or
Continue with Google
Continue with LinkedIn

Have any questions? See FAQs

Job Tags

Part time, For contractors, Remote work

Similar Jobs

Ultimate Outsourcing

Certified Tax Preparer Job at Ultimate Outsourcing

 ...: Position Overview: We are seeking a skilled and certified Tax Preparer to join our team. The Certified Tax Preparer will be responsible...  ...a fast-paced environment. Job Types: Full-time, Part-time, Seasonal, Contract Pay: $40,000.00 - $65,000.00 per year Expected... 

Panasonic Energy Corporation of North America

Information Security Analyst I Job at Panasonic Energy Corporation of North America

 ...Overview Do you want to join a team that's changing the world? Do you have a strong background as an Information Security Analyst I? Then we're looking for you! Check out the job description and apply now! Put your skills to meaningful use, gain unique experience,... 

pmX GmbH

Data Analyst (m/f/x) Job at pmX GmbH

 ...organizations unlock the full potential of their operations by combining data, digital tools, and real-world execution expertise . Our focus...  ..., Data Science, or a related field First experience (internship or up to ~2 years) in data analysis, data science, or... 

The Berliner

Bar Manager Job at The Berliner

 ...About US: The Popal Group is seeking an experienced, passionate, and hospitality-driven Bar Manager to join the opening leadership team at The Berliner, our newest concept coming soon to Washington, DC. The Berliner will celebrate the traditions of a classic German... 

Globe Life AO

__ ✓ Remote Business Development Associate | Entry-Level | Training Provided | Apply Now Job at Globe Life AO

 ...Remote Business Development Associate | Entry-Level | Training Provided | Apply Now Join our remote team in a business development-focused role where you can build communication, relationship-building, and professional business skills. This opportunity is open to entry...