We are hiring for AI EVAL Engineering // Bellevue, WA (Remote)

[Vansh]

Hi, 

Hope you are doing well...!!! 

We have an opening for AI EVAL Engineering // Bellevue, WA (Remote)

 

Title: AI EVAL Engineering

Location: Bellevue, WA (Remote)

Employment Type: Contract

Please share resumes at va...@kksoftwareassociates.com

 

Required Skills
- Strong understanding of LLMs and generative AI concepts, including model behavior and output evaluation
- Experience with AI evaluation and benchmarking methodologies, including baseline creation and model comparison
- Hands-on expertise in Eval testing, creating structured test suites to measure accuracy, relevance, safety, and performance
- Ability to define and apply evaluation metrics (precisionrecall, BLEUROUGE, F1, hallucination rate, latency, cost per output)Prompt engineering and prompt testing experience across zero-shot, few-shot, and system prompt scenarios
- Python other programming languages, for automation, data analysis, batch evaluation execution, and API integration
- Experience with evaluation tools/frameworks (OpenAI Evals, HuggingFace evals, Promptfoo, Ragas, DeepEval, LM Eval Harness)
- Ability to create datasets, test cases, benchmarks, and ground truth references for consistent scoring
- Test design and test automation experience, including reproducible evaluation pipelines
- Knowledge of AI safety, bias, security testing, and hallucination analysis

Nice-to-Have
- RAG evaluation experience
- Azure OpenAI
- OpenAI
- Anthropic
- Google AI platforms
- Performance benchmarking (speed, throughput, cost)
- Domain knowledge Office apps enterprise systems networking

Regards,

Vansh Swami ||Technical Recruiter ||

Email Id: va...@kksoftwareassociates.com