01 — Source
Python Script
This page intentionally renders the code as text in the browser.
| 1 | import asyncio |
| 2 | import anthropic |
| 3 | from mcp import ClientSession, StdioServerParameters |
| 4 | from mcp.client.stdio import stdio_client |
| 5 | |
| 6 | async def run_qa_agent(): |
| 7 | |
| 8 | TARGET_URL = "https://www.saucedemo.com" |
| 9 | |
| 10 | MISSION = f""" |
| 11 | You are a senior QA engineer. Explore the following app and find bugs or UX issues. |
| 12 | |
| 13 | App URL: {TARGET_URL} |
| 14 | |
| 15 | Test these two scenarios: |
| 16 | |
| 17 | 1. Happy path: Log in with username 'standard_user' and password 'secret_sauce', |
| 18 | add an item to cart, and complete checkout. |
| 19 | |
| 20 | 2. Error path: Try to log in with wrong credentials and verify a proper error message appears. |
| 21 | |
| 22 | For each scenario report: |
| 23 | - What steps you took |
| 24 | - What passed ✅ |
| 25 | - What failed or looked broken ⚠️ |
| 26 | |
| 27 | Be concise. End with a VERDICT summary. |
| 28 | """ |
| 29 | |
| 30 | # Connect to Playwright MCP |
| 31 | server_params = StdioServerParameters( |
| 32 | command="npx", |
| 33 | args=["@playwright/mcp", "--headless"] |
| 34 | ) |
| 35 | |
| 36 | async with stdio_client(server_params) as (read, write): |
| 37 | async with ClientSession(read, write) as session: |
| 38 | await session.initialize() |
| 39 | |
| 40 | # Get available tools from Playwright MCP |
| 41 | tools_result = await session.list_tools() |
| 42 | tools = [ |
| 43 | { |
| 44 | "name": tool.name, |
| 45 | "description": tool.description, |
| 46 | "input_schema": tool.inputSchema |
| 47 | } |
| 48 | for tool in tools_result.tools |
| 49 | ] |
| 50 | |
| 51 | client = anthropic.Anthropic() |
| 52 | messages = [{"role": "user", "content": MISSION}] |
| 53 | |
| 54 | print("🤖 Agent starting...\n") |
| 55 | |
| 56 | # Agent loop - keeps running until Claude stops using tools |
| 57 | while True: |
| 58 | response = client.messages.create( |
| 59 | model="claude-sonnet-4-20250514", |
| 60 | max_tokens=4096, |
| 61 | system="You are a senior QA engineer. Use the browser tools to navigate and test the app.", |
| 62 | messages=messages, |
| 63 | tools=tools |
| 64 | ) |
| 65 | |
| 66 | # Collect any text output |
| 67 | for block in response.content: |
| 68 | if hasattr(block, "text"): |
| 69 | print(block.text) |
| 70 | |
| 71 | # If Claude is done, stop |
| 72 | if response.stop_reason == "end_turn": |
| 73 | break |
| 74 | |
| 75 | # If Claude wants to use tools, execute them |
| 76 | if response.stop_reason == "tool_use": |
| 77 | messages.append({"role": "assistant", "content": response.content}) |
| 78 | tool_results = [] |
| 79 | |
| 80 | for block in response.content: |
| 81 | if block.type == "tool_use": |
| 82 | print(f"🔧 Using tool: {block.name}") |
| 83 | result = await session.call_tool(block.name, block.input) |
| 84 | tool_results.append({ |
| 85 | "type": "tool_result", |
| 86 | "tool_use_id": block.id, |
| 87 | "content": str(result.content) |
| 88 | }) |
| 89 | |
| 90 | messages.append({"role": "user", "content": tool_results}) |
| 91 | else: |
| 92 | break |
| 93 | |
| 94 | print("\n✅ Done.") |
| 95 | |
| 96 | asyncio.run(run_qa_agent()) |
02 — What This Code Shows
Agent Loop in One File
Mission Driven
The test objective is written in plain English instead of hardcoded browser steps.
Tool Use
The agent receives browser tools from Playwright MCP and decides when to use them.
Observe and Adapt
The loop sends tool results back to the model so it can continue navigating and reporting.
03 — Console Run Output
What the Agent Printed During a Run
These are the plain-text console logs generated when running python qa_agent.py. They show the agent starting, using browser tools, observing the app, and producing the final QA verdict.
python qa_agent.py 🤖 Agent starting... qa_agent.py:58: DeprecationWarning: The model 'claude-sonnet-4-20250514' is deprecated and will reach end-of-life on June 15th, 2026. Please migrate to a newer model. Visit https://docs.anthropic.com/en/docs/resources/model-deprecations for more information. response = client.messages.create( I'll test the Saucedemo app by exploring both the happy path and error path scenarios. Let me start by navigating to the app. 🔧 Using tool: browser_navigate Now let me take a snapshot of the current page to see the login interface: 🔧 Using tool: browser_snapshot Great! I can see the login page with username and password fields. I notice there are console errors. Let me check those first: 🔧 Using tool: browser_console_messages I can see there are networking errors related to external analytics services. Now let me start testing the scenarios. ## Scenario 1: Happy Path - Valid Login and Checkout Let me log in with the valid credentials: 🔧 Using tool: browser_fill_form Let me get a fresh snapshot and use proper selectors: 🔧 Using tool: browser_snapshot 🔧 Using tool: browser_type 🔧 Using tool: browser_type Now let me click the login button: 🔧 Using tool: browser_click Great! Login was successful and I'm now on the inventory page. Let me take a snapshot to see the products: 🔧 Using tool: browser_snapshot Perfect! I can see the products page with 6 items. I notice there's no shopping cart icon visible yet. Let me add the first item (Sauce Labs Backpack) to cart: 🔧 Using tool: browser_click Let me check if a shopping cart badge appeared and if the button changed to "Remove": 🔧 Using tool: browser_snapshot I notice the button still shows "Add to cart" and I don't see a shopping cart icon yet. This might be a bug or the UI might be updating asynchronously. Let me check if there's a shopping cart link/icon by evaluating the page: 🔧 Using tool: browser_evaluate ⚠️ **Issue Found**: The add to cart functionality appears to be broken! The item was supposedly added to cart, but: - The button still says "Add to cart" (should change to "Remove") - No cart badge appeared with item count - No remove button appeared Let me try clicking the cart icon anyway to see what happens: 🔧 Using tool: browser_evaluate Now I'm on the cart page. Let me check if the item is actually in the cart: 🔧 Using tool: browser_snapshot ⚠️ **Major Bug Confirmed**: The cart is empty! The add to cart functionality is completely broken. The item was not added to cart despite clicking the "Add to cart" button. Let me go back to the products page and try to add a different item to see if this is a consistent issue: 🔧 Using tool: browser_click The continue shopping button didn't work either. Let me navigate back manually: 🔧 Using tool: browser_navigate Let me try adding a different item (Bike Light) to see if it's a global issue: 🔧 Using tool: browser_click Let me check if this time the add to cart worked: 🔧 Using tool: browser_evaluate The same issue persists - add to cart is completely broken across all items. Now let me test **Scenario 2: Error Path - Invalid Login** 🔧 Using tool: browser_navigate Let me try logging in with invalid credentials: 🔧 Using tool: browser_type 🔧 Using tool: browser_type 🔧 Using tool: browser_click Good, I'm still on the login page. Let me check for any error messages: 🔧 Using tool: browser_snapshot ⚠️ **Bug Found**: No error message is displayed after entering invalid credentials! The login should show an error message like "Invalid username or password" but instead it just stays on the login page with no feedback to the user. Let me also test with a valid username but wrong password to see if that produces an error: 🔧 Using tool: browser_type 🔧 Using tool: browser_type 🔧 Using tool: browser_click 🔧 Using tool: browser_evaluate There's an error container in the HTML but it's empty. This confirms that error handling is broken. Let me also test with a locked out user to see if that produces any error: 🔧 Using tool: browser_type 🔧 Using tool: browser_type 🔧 Using tool: browser_click 🔧 Using tool: browser_snapshot Same issue - no error message even for a locked out user who should definitely get a specific error message. ## Test Results Summary ### Scenario 1: Happy Path Login ✅ ⚠️ **Steps taken:** 1. Navigated to https://www.saucedemo.com 2. Entered valid credentials (standard_user/secret_sauce) 3. Clicked login button 4. Attempted to add items to cart 5. Navigated to cart page **What passed ✅:** - Login page loads correctly - Valid credentials authentication works - User successfully reaches inventory page - Product listings display properly with images, prices, descriptions - Shopping cart page is accessible **What failed ⚠️:** - **CRITICAL BUG**: Add to cart functionality is completely broken - Clicking "Add to cart" buttons has no effect - Cart badge doesn't appear - Items don't get added to cart - Buttons don't change to "Remove" state - Continue Shopping button from cart doesn't work - Console errors present (external analytics services failing) ### Scenario 2: Error Path - Invalid Login ⚠️ **Steps taken:** 1. Navigated to login page 2. Tried invalid_user/wrong_password combination 3. Tried standard_user/wrong_password combination 4. Tried locked_out_user/secret_sauce combination **What passed ✅:** - Login page prevents invalid authentication - Users stay on login page when credentials are wrong - No unauthorized access granted **What failed ⚠️:** - **CRITICAL BUG**: No error messages displayed for any invalid login attempts - Empty error container exists in HTML but never gets populated - Poor UX - users get no feedback about why login failed - Locked out user scenario should show specific "account locked" message but shows nothing ## VERDICT 🔴 **MAJOR ISSUES FOUND - NOT PRODUCTION READY** The application has **2 critical bugs** that make it unusable: 1. **Shopping Cart Broken** - Core e-commerce functionality completely non-functional 2. **No Login Error Messages** - Security/UX issue with no user feedback on failed logins **Severity**: High - Both bugs impact core user journeys and make the app unsuitable for production use. The shopping cart bug prevents any purchases, while the login error bug creates poor user experience and potential security confusion. **Recommendation**: Fix both critical bugs before any release. The add to cart functionality needs complete debugging, and proper error handling/messaging needs to be implemented for authentication failures. ✅ Done.