PRANITHA TRIPURA
Summary
Detail-oriented AI Data Annotator and Quality Assurance professional with expertise in Large Language Model evaluation and RLHF workflows. Skilled in analyzing AI outputs, identifying performance gaps, and enhancing model reliability and alignment. Proficient in dataset validation, JSON analysis, and quality assurance processes, facilitating collaboration with engineering and research teams to drive scalable AI solutions.
Overview
19
19
years of professional experience
1
1
Certification
Work History
Technical Research Associate 3
Keywords Studios (Supporting Amazon AI Research)
10.2025 - Current
- Evaluated and optimized Large Language Models (LLMs) and agentic AI systems by executing structured prompts, analyzing model outputs, identifying performance gaps, edge cases, and failure patterns, and recommending improvements.
- Supported AI training, evaluation, and RLHF workflows through data annotation, qualitative assessment, reward signal evaluation, human-in-the-loop validation, and Chain-of-Thought (CoT) reasoning optimization to improve model alignment and reliability.
- Conducted quality assurance, spot-checking, and JSON-based verifier analysis across multiple projects, ensuring high-quality datasets, uncovering system inconsistencies, and driving process improvements that enhanced model performance and project throughput.
- Collaborated with engineering, research, and cross-functional teams to develop evaluation frameworks, refine use case scenarios, troubleshoot model behavior, mentor research associates, and support model deployment.
AI Data Quality Annotator
RWS Group
08.2023 - 09.2025
- Evaluate the quality of responses generated by LLM models in the context of Meta Ads.
- Focus on assessing the overall quality of responses, identifying areas for improvement, and ensuring that the responses align with the intended dimensions of evaluation.
- Annotate and label audio data according to predefined guidelines and specifications, ensuring high accuracy and consistency using Single Review Tool for Meta’s AI training models.
- Conduct thorough quality checks and validations on annotated data to identify and rectify errors or inconsistencies, maintaining data integrity and reliability.
- Evaluate correctness of the TTS (text-to-speech) models.
- Determine speaker’s emotions in various audio clips.
- Collaborate with a team of annotators to maintain uniformity and accuracy across all annotations and transcriptions.
- Provide detailed feedback and suggestions for improvement to fellow annotators and transcriptionists.
Data Quality Assurance Processor
Providence Facey Medical Foundation
01.2022 - 02.2022
- Review claims received and identify missing information that is necessary for processing.
- Search for missing information by matching the claim details with the available data in the EPIC EMR system.
- Update claims and submit them for re-processing.
- Handled approximately 200+ claims each day.
Manual QA Tester
Mphasis
05.2007 - 12.2009
- Involved in all meetings with business users as well as with different technical support teams.
- Built test plans resulting in a 50% increase in efficiency of the software.
- Develop and execute test plans, test cases for various software applications based on project requirements.
- Execute tests to identify defects and ensure functionality, usability, and compatibility of software system.
- Track defects in Jira (Defect Tracking tool) and collaborate with developers to resolve issues.
- Analyze test results, report findings, and provide recommendations for project improvements.
- Work with cross-functional teams, including developers, business analysts, and project managers to ensure timely delivery of quality products.
- Co-ordinate with development teams to implement a more efficient and structured test case management process, reducing testing time by 20%.
Education
MCA - Master of Computer Applications
Koneru Lakshmaiah College of Engineering, Acharya Nagarjuna University
Skills
- Competency Skills: Annotation Expertise, Labeling, Transcription, Auditing, Software Testing, RLHF, JSON Analysis
- Quality review – Content review, Verification & Validation
- Soft Skills - Collaboration & communication, Multi-task and seamlessly adapt to change
- Tools: Jira, Single Review
- Annotation Tools – CVAT, Roboflow, AWS SageMaker
- EMR Software – EPIC
Certification
- Claude 101, Claude code 101, Introduction to Claude Cowork
- Linguistic Prompt Design Training by RWS Group
- Hands-On Data Annotation: Applied Machine Learning by LinkedIn
- Tester Foundation Level 4.0: Software Testing Basics from Skillsoft
Timeline
Technical Research Associate 3
Keywords Studios (Supporting Amazon AI Research)
10.2025 - Current
AI Data Quality Annotator
RWS Group
08.2023 - 09.2025
Data Quality Assurance Processor
Providence Facey Medical Foundation
01.2022 - 02.2022
Manual QA Tester
Mphasis
05.2007 - 12.2009
MCA - Master of Computer Applications
Koneru Lakshmaiah College of Engineering, Acharya Nagarjuna University