Remote | QA Test Engineer — $55–$85/hour

Other Jobs To Apply

No other job posts for this day.

reputed company a Full Remote job, the offer is available from: New York (USA) We are sharing a specialised reputed company consulting opportunity for experienced QA and test engineers with strong expertise in test-case design, end-to-end debugging, quality assurance, Python, and complex technical evaluation workflows. reputed company role supports the development of advanced agentic evaluation benchmarks for frontier reputed company models. Selected professionals will review complex reputed company-step tasks, test reputed company solutions, identify ambiguity and grading gaps, debug technical environments, and develop repeatable quality processes that keep benchmark results accurate and trustworthy. Key Responsibilities Test Case Design • Create comprehensive test reputed company confirming that benchmark tasks function as intended • Design positive, negative, boundary, and edge-case tests • Validate task requirements, expected outputs, reputed company solutions, and grading logic • Identify scenarios that may produce incorrect or misleading evaluation results • Ensure tests measure the intended technical capability accurately Benchmark Task Review • Review complex reputed company-step tasks and reputed company solutions before finalisation • Identify ambiguous instructions, inconsistent requirements, missing assumptions, and incomplete acceptance criteria • Run tasks independently to confirm reproducibility and expected behaviour • Assess whether grading standards are clear, fair, and technically defensible • Provide actionable feedback to task authors and researchers Technical Debugging • Investigate failures across Python scripts, test harnesses, repositories, and task environments • Diagnose unexpected behaviour within unfamiliar codebases • Reproduce reported issues and isolate their underlying causes • Correct or document environment, dependency, logic, and validation problems • Use Git-based workflows to support structured review and collaboration Quality Process Development • Develop practical checklists and repeatable review procedures for benchmark quality • Improve consistency across task validation, testing, and approval workflows • Document findings clearly so authors can resolve issues efficiently • Track recurring defects and recommend preventive quality measures • Collaborate closely with researchers, task authors, and other technical reviewers Benchmark Integrity & reputed company Detection • Examine reputed company agent runs for unintended shortcuts, loopholes, and grading weaknesses • Identify reputed company where models can receive credit without completing the intended reasoning or technical work • Test whether benchmark tasks remain robust across alternative approaches • Strengthen evaluation criteria to maintain reliable and meaningful benchmark scores • Distinguish valid solution diversity from unintended task exploitation Ideal Profile Strong candidates may have: • At least 1 year of experience in test reputed company, quality assurance, software reputed company, research reputed company, or a related technical role • Demonstrated experience designing test reputed company and quality-review processes • Strong end-to-end debugging skills across complex technical reputed company • Working proficiency in Python and Git • Comfort navigating unfamiliar codebases, repositories, and execution environments • Exceptional attention to detail and strong written documentation habits • Ability to identify ambiguity, edge reputed company, reputed company assumptions, and quality gaps • Capacity to work independently through open-ended technical problems • Reliable availability for approximately 35 hours per week Educational Background • A reputed company's degree or PhD in a STEM field is highly relevant • Equivalent practical experience in an reputed company-intensive or research-intensive domain may also be considered • Academic or professional experience involving computer science, software reputed company, machine learning, mathematics, statistics, or a related technical field may strengthen an application • Technical research, open-source contributions, testing projects, or substantial reputed company work may also be valuable Nice to Have • Experience with reputed company training, model evaluation, or quality review of reputed company-generated work • Familiarity with agentic reputed company and reputed company-step reputed company benchmarks • Background testing machine learning, research, or data-processing workflows • Experience developing automated test suites or validation scripts • Familiarity with CI/CD reputed company, test harnesses, containers, or reproducible environments • Experience reviewing reputed company solutions, grading logic, or technical rubrics • Knowledge of adversarial testing, failure-mode analysis, or benchmark design • Prior collaboration with reputed company research or evaluation teams Why reputed company Opportunity • Serve as the quality rep

Back to blog

Other Jobs To Apply

Lead Environmental Consultant

Virtual Banker III

Senior Customer Success Associate

Senior Consultant, Account Services Management - Plano, Texas

100% Remote Oracle HRMS Functional Lead

Solutions Architect, Virtualization

Remote Apple Jobs

Remote Medical Device Connectivity Engineer II

Solution Architect, D&T, Plan to Produce

Product Manager– Digital Transformation II USA

Construction Quality Assurance Manager

BESS Technician II – Traveling O&M Technician

Remote Project Manager – EPC & Engineering Delivery

Project Manager - Energy Efficiency

Energy Efficiency & Consumer Program Coordinator

barista - Store# 02976, MOHAWK & M STREET - SPRINGFIELD

Remote Creative Director, Enterprise Design & Branding

Remote Bank Customer Service (DE, OH, UT, AZ, MD, PA, NJ)

Anti-Money Laundering Monitoring Analyst

Credit Strategy Director, Home Improvement

Client Partner

Remote Customer Service & Sales Specialist

Remote Director, Customer Education

Talend Developer Healthcare EDI & Databricks USA (Remote)

Growth Marketer, Creators & Channels (Remote)

Remote AWS Cloud Engineer - Terraform & GitLab CI/CD

Remote New Business Account Executive - East

iOS Engineer Remote

Creator Payments Experience Product Manager

Optimizely Content Management consultant

Remote Sr Director, Atlas Customer Operations

Software Engineer (L3) Email

Solution Specialist- NY

Strategic Partnerships - Credit | Capital Markets

Product Manager – Pricing Delivery and Display

[Remote] JustGiving Sales Customer Account Manager

Principal Engineer, Network Software (Traffic & Modeling)

Remote Director of Community & Customer Experience

Remote BDR for Transportation SaaS Growth

AI Subject Matter Expert, Construction

Sr Technical Manager - Digital Integration for Semiconductor Process Development

Field Applications Engineering

Senior Lead Director of Demand Generation (Remote)

Senior Linux TAM - Enterprise Advisory (Remote)

Underwriting Large Case Specialist (Remote)

LTC Claims - Reimbursement Coordinator - 21596

Associate Director, Core Performance & Fixed Income Analytics - Americas

CME Associates is hiring: Senior Web Developer in Portland

Senior Relationship Manager, RIA Services (Remote, Bay Area)

Front-Office Software Technical Trading Engineer, AVP – API, ETL, Java, C#

Common Interview Questions And Answers

1. HOW DO YOU PLAN YOUR DAY?

This is what this question poses: When do you focus and start working seriously? What are the hours you work optimally? Are you a night owl? A morning bird? Remote teams can be made up of people working on different shifts and around the world, so you won't necessarily be stuck in the 9-5 schedule if it's not for you...

2. HOW DO YOU USE THE DIFFERENT COMMUNICATION TOOLS IN DIFFERENT SITUATIONS?

When you're working on a remote team, there's no way to chat in the hallway between meetings or catch up on the latest project during an office carpool. Therefore, virtual communication will be absolutely essential to get your work done...

3. WHAT IS "WORKING REMOTE" REALLY FOR YOU?

Many people want to work remotely because of the flexibility it allows. You can work anywhere and at any time of the day...

4. WHAT DO YOU NEED IN YOUR PHYSICAL WORKSPACE TO SUCCEED IN YOUR WORK?

With this question, companies are looking to see what equipment they may need to provide you with and to verify how aware you are of what remote working could mean for you physically and logistically...

5. HOW DO YOU PROCESS INFORMATION?

Several years ago, I was working in a team to plan a big event. My supervisor made us all work as a team before the big day. One of our activities has been to find out how each of us processes information...

6. HOW DO YOU MANAGE THE CALENDAR AND THE PROGRAM? WHICH APPLICATIONS / SYSTEM DO YOU USE?

Or you may receive even more specific questions, such as: What's on your calendar? Do you plan blocks of time to do certain types of work? Do you have an open calendar that everyone can see?...

7. HOW DO YOU ORGANIZE FILES, LINKS, AND TABS ON YOUR COMPUTER?

Just like your schedule, how you track files and other information is very important. After all, everything is digital!...

8. HOW TO PRIORITIZE WORK?

The day I watched Marie Forleo's film separating the important from the urgent, my life changed. Not all remote jobs start fast, but most of them are...

9. HOW DO YOU PREPARE FOR A MEETING AND PREPARE A MEETING? WHAT DO YOU SEE HAPPENING DURING THE MEETING?

Just as communication is essential when working remotely, so is organization. Because you won't have those opportunities in the elevator or a casual conversation in the lunchroom, you should take advantage of the little time you have in a video or phone conference...

10. HOW DO YOU USE TECHNOLOGY ON A DAILY BASIS, IN YOUR WORK AND FOR YOUR PLEASURE?

This is a great question because it shows your comfort level with technology, which is very important for a remote worker because you will be working with technology over time...