AI QA Tester // Hybrid GA Locals only
Suraj Vitsus
My name is Suraj Mishra and I am a Staffing Specialist at Visionary Innovative Technology Solutions LLC. I am reaching out to you on an exciting job opportunity with one of our clients.
Role – AI Tester
Location: Atlanta, GA ( 3 days at Onsite )
Role : Ensure the quality, safety, reliability, and performance of Generative AI systems by developing and executing testing strategies that validate accuracy, fairness, security, and alignment with business and ethical standards.
Required Skills
- 8+ years of experience in QA, testing, or quality engineering, with hands‑on exposure to AI/ML or Generative AI systems
- Experience testing with chatbots, NLP‑based applications, and GenAI solutions, including prompt engineering and optimization
- Strong understanding of AI evaluation techniques, including hallucination detection, factual accuracy, bias, and output consistency
- Knowledge of Responsible AI principles, including fairness, transparency, and explainability
- Experience validating data quality with a basic understanding of statistics and AI performance metrics
- Proficient in API testing (REST, JSON) and testing AI model endpoints
- Hands‑on experience with test automation tools and scripting (Python preferred)
- Familiarity with the ML lifecycle, model versioning, and regression testing for AI systems
- Exposure to cloud-based AI platforms such as Azure OpenAI, AWS Bedrock, or Google Vertex AI
- Strong foundation in software testing methodologies, including exploratory, negative, and adversarial testing
- Ability to design test cases for non-deterministic AI systems
- Strong analytical and critical-thinking skills, with the ability to objectively assess subjective AI outputs
- Excellent documentation, communication, and collaboration skills, with experience working in Agile / DevOps, cross‑functional AI teams
Key Responsibilities
- Design and execute test strategies for GenAI systems (LLMs, RAG pipelines, AI agents, copilots)
- Validate accuracy, relevance, consistency, and factuality of AI-generated outputs
- Perform prompt testing, adversarial testing, and edge-case validation
- Test for hallucinations, bias, toxicity, and harmful content
- Validate retrieval quality in RAG-based systems (chunking, embeddings, relevance)
- Conduct regression testing for model updates, prompt changes, and data refreshes
- Collaborate with data scientists, ML engineers, product managers, and compliance teams
- Define and track AI quality metrics (BLEU, ROUGE, faithfulness, groundedness, latency)
- Automate test cases for GenAI workflows where feasible.
Thank You,
Suraj Mishra
|
Technical Recruiter |
|
|
||||
|
|
|
|||||
|
|
W: vitsus.com |
|
||||
|
Visionary Innovative Technology Solutions LLC | 276 Mulberry Street, Plantsville CT 06479 |
|||||
|
||||||
Confidentiality Notice:
This communication & accompanying documents ("this e-mail") contains confidential and/or privileged information for exclusive use of the individual to whom it is addressed. If you are not the intended recipient, please immediately notify the company & delete this e-mail. Any unauthorized use or disclosure of this e-mail is strictly prohibited. Representations in this e-mail are subject to contract. As an e-mail user please be cautious of the technical & other vulnerabilities of the internet which may result in malicious and/or unauthorized access to / use / alteration of e-mails/e-mail IDs. Thank you