C++ Engineer for AI Training project

Job Description

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are building large-scale evaluation and training datasets to help LLMs solve realistic software engineering problems. As part of this initiative, we create verifiable software engineering tasks derived from public repository histories, using a synthetic, human-in-the-loop approach. Our goal is to expand dataset coverage across programming languages, difficulty levels, and task types. We’re looking for experienced Software Engineers ho are familiar with high-quality public GitHub repositories and can actively contribute to this effort. The role is hands-on and includes:

NOTE: In this project you will not build a project, but specifically generate data to improve model performance for one of the biggest foundational model companies.

Duration: 3 months

Commitment: 40h/week, 4h/day overlap with PST

Model: Contract, time and material

Location: 100% Remote: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico

Interview: 75 min async test

Key Responsibilities

• Review and prioritize GitHub issues from popular open-source projects
• Clone, configure, and containerize repositories (including Docker setup) for local execution
• Analyze unit test coverage and test reliability
• Run and adapt codebases locally to evaluate LLM behavior in debugging and bug-fix scenarios
• Partner closely with research teams to surface repositories and problems that meaningfully challenge LLMs
• Potentially lead small groups of junior engineers and guide collaborative project work

Requirements

• Minimum 3+ years of overall experience
• Strong experience with at least one of the following languages:  C++
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfortable running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.

Nice-to-Haves:

• Experience with dataset creation, annotation, evaluation, or ML pipelines
• Familiarity with benchmarks like SWE Bench or Terminal Bench
• Background in QA automation, DevOps, ML systems, or data engineering

Benefits

• Work in a fully remote environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.