📍 San Francisco Bay Area or New York City Area
🧠 Python | LLMs | NLP | Machine Learning | AWS
💰 $200K–$230K USD base salary + equity
🛂 Visa sponsorship available for candidates already in the US who require a visa transfer
🌱 Pre-seed AI startup | Founding engineering team
Build the intelligence layer that makes AI customer conversations accountable
Are you a Staff or Principal-level ML engineer who wants to turn cutting-edge AI into reliable, measurable, production-grade systems?
We're looking for a Founding ML Engineer to build the intelligence and compliance layer of an early-stage AI startup tackling one of the most important challenges in enterprise AI: ensuring customer-facing AI agents are accurate, safe, compliant, and accountable.
You'll work directly with the CTO, own the company's applied ML strategy, and help shape how AI-powered customer conversations are evaluated at scale. As the first dedicated ML hire, you'll have the opportunity to establish the technical foundations, build the team, and influence the product from its earliest stages.
This isn't a research-only position. You'll write production code, evaluate real-world model performance, make critical architectural decisions, and build systems where accuracy must be demonstrated — not assumed.
🧠 The opportunity
Enterprises are increasingly deploying AI agents to handle customer conversations across voice, chat, email, and SMS. These systems are already dealing with billing disputes, account changes, and other sensitive interactions.
But how can businesses be confident that their AI agents are following the rules, providing grounded answers, and handling customers appropriately?
Manual quality assurance cannot keep pace with millions of interactions. AI vendors focus on building capable agents, while enterprises need an independent way to measure their performance, identify failures, and demonstrate compliance.
The company is building an AI monitoring and assurance platform that evaluates customer interactions against applicable regulations and business requirements, helping enterprises detect issues early, reduce risk, and provide evidence of accountability.
You'll own the intelligence that makes this possible — from compliance detection and hallucination monitoring to evaluation frameworks, scoring models, and production ML observability.
🔧 What you'll own
🛡️ Compliance, hallucination & scoring architecture
Translate customer requirements and regulatory frameworks into measurable detectors for compliance, grounding, hallucinations, risk, escalation, and interaction quality.
Choose the right combination of rules, classifiers, LLM-based judges, and fallback mechanisms.
Establish confidence thresholds and promotion criteria based on explicit accuracy, latency, cost, and risk trade-offs.
📊 Evaluation frameworks & the data flywheel
Own annotation guidelines, ground-truth datasets, adjudication processes, and offline and online evaluation.
Build regression gates that prevent ungrounded outputs, unsupported compliance claims, and unmeasured accuracy from reaching production.
Turn QA reviews, customer disputes, and human-versus-model disagreements into high-quality training and evaluation data.
Use synthetic data, weak supervision, and active learning where they demonstrably improve performance.
⚙️ Customer rubric onboarding
Build the tooling, prompts, datasets, and evaluation workflows needed to translate each customer's framework into reliable production detectors.
Reduce the time required to onboard new customers without compromising measurement quality.
Develop reusable approaches while accommodating customer-specific requirements.
🎙️ Speech & conversation intelligence
Make noisy transcripts reliably scorable using robust NLP and LLM techniques.
Evaluate transcription quality and its impact on downstream compliance and scoring.
Help design the next generation of speech-processing capabilities, including automatic speech recognition (ASR), speaker diarisation, and streaming transcription.
☁️ Production ML economics & observability
Own inference cost and latency per interaction, model routing, and the trade-offs between model capability and operational efficiency.
Monitor model drift and changes in compliance detection, grounding, hallucination detection, and scoring accuracy.
Build production monitoring and alerting that surface regressions before they become customer problems.
🤝 Technical leadership & team building
Partner directly with the CTO and design partners' QA and compliance teams to understand real-world model failures.
Contribute to enterprise security and AI governance reviews.
Establish the ML engineering standards, evaluation practices, and technical direction for future hires.
Remain hands-on while mentoring engineers and building the team.
🎯 What we're looking for
Staff or Principal-level applied ML expertise: You've owned production ML, NLP, or LLM systems that other teams or customers depend on, and you've been accountable for their performance.
Rigorous evaluation experience: You've built evaluation pipelines, developed ground truth from inconsistent human labels, and made explicit decisions about acceptable error rates.
Early-stage startup experience: You've shipped products in an environment defined by ambiguity, broad ownership, and limited supporting infrastructure. This is essential.
Strong model-selection judgement: You're comfortable with prompting, retrieval, fine-tuning, distillation, classical classifiers — and recognising when a simple rule or regex is the better solution.
Hands-on Python engineering: You've trained or debugged models yourself, built and maintained production services, and worked with messy, unstructured data.
Production ML judgement: You understand inference economics, latency, model reliability, sensitive data handling, and the security requirements of enterprise customers.
Technical leadership: You can set the direction for an ML function, establish high engineering standards, and recruit the engineers who will help scale it.
Pragmatic AI-assisted development: You're comfortable using coding agents to accelerate development while maintaining rigorous standards for testing, security, and maintainability.
✨ Bonus points
Experience in any of the following would be particularly valuable:
🎙️ Production speech and audio systems, including ASR, diarisation, streaming transcription, or real-time latency optimisation.
🧪 Custom NLP or scoring-model development, including LLM-as-judge systems validated against human ground truth.
🔍 Weak supervision, programmatic labelling, or active learning in production.
🛡️ Trust and safety, fraud and risk, contact-centre quality assurance, regulatory compliance, or governance, risk and compliance (GRC).
🌟 Why consider this opportunity?
Founding-level impact: Work directly with the CTO and define the company's applied ML strategy from the ground up.
Real ownership: Take responsibility for the intelligence layer, evaluation methodology, and production performance.
A meaningful problem: Help enterprises adopt AI customer support with greater confidence, transparency, and accountability.
Real-world ML challenges: Solve difficult problems involving noisy data, model reliability, measurable accuracy, and operational constraints.
Build the team: Establish the technical standards and culture that future ML engineers will inherit.
Shape the product: Work directly with design partners and use real customer feedback to guide the platform's evolution.
📩 Interested?
If you're a hands-on ML leader who enjoys solving difficult applied AI problems, building reliable systems from first principles, and taking ownership beyond the model itself, we'd love to hear from you.
👉 Apply Now! 📩
SourceIn (sourcein.co) has been formally appointed by the client to provide recruitment and candidate pre-screening services for this role.