AuraOne Human Data is seeking a remote contract Software AI Engineer to help assess AI-generated software outputs by examining production code, debugging traces, and developer-facing results for correctness. In this role, reviewers reproduce failures, craft minimal failing tests, and articulate fixes to guide model improvements. The position blends hands-on code inspection with principled evaluation to advance AI-assisted software development.
Responsibilities
- Reproduce candidate code outputs in a sandboxed environment for Software AI Engineer assignments, observing behavior and outcomes.
- Assess AI-driven engineering solutions for correctness, complexity, idiomatic style, and handling of edge cases.
- Write minimal failing tests that reveal bugs a model output missed, ensuring tests are focused and reproducible.
- Compare paired solutions and provide a ranking with a written rationale aligned to the evaluation rubric.
- Tag failure modes such as compile errors, runtime crashes, off-by-one errors, and security issues with severity scores.
- Document recurring code-generation failures to guide the modeling team in targeting improvements.
- Calibrate reviews against gold-standard benchmarks to maintain high inter-rater agreement.
Requirements
- Active day-to-day engineering experience with the ability to read, run, and debug unfamiliar code in the context of Software AI Engineer work.
- Ability to write concise unit tests that capture a single failure mode.
- Familiarity with at least one of these testing frameworks: pytest, Jest, JUnit, Go test, RSpec, or equivalent.
- Clear, persuasive written reasoning that can convince a senior engineer.
- Reliable asynchronous availability of at least 10 hours per week.
Technologies
- pytest
- Jest
- JUnit
- Go test
- RSpec
Why this role matters
Engineering model quality hinges on generated code that compiles, passes tests, and handles edge cases. AuraOne pairs experienced engineers with the modeling team to evaluate outputs with the rigor a human code reviewer would apply.
Example tasks
- Reproduce a model-generated engineering solution to a coding task, run its test suite, and assign a grade based on the rubric.
- Write the smallest failing test that demonstrates a model edge-case bug.
- Compare two paired solutions and rank them with a rubric-aligned written rationale.
- Triage a security issue surfaced by a model and document the patch the model should have produced.
Nice to have
- Open-source contributions or a public portfolio demonstrating production-quality code.
- Experience with the target language's standard tooling, linters, and idiomatic style guides.
- Familiarity with security-review checklists (OWASP, CWE) and AppSec patterns.
Skills
- Code review
- Debugging
- Unit testing
- Software engineering judgment
- AI engineering
- Software engineering
- AI model evaluation
Work model
Remote, US-eligible. Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from the US.
Compensation
Hourly rate is confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
