Debugging Failed Tests

This guide explains how to identify, understand, and fix test failures in TrueAssert.

Overview

When a test fails, TrueAssert provides detailed information to help you debug:

• Error messages for each failed step

• Screenshots captured at failure points

• Execution logs with timestamps

• Step-by-step execution status

Understanding Test Failures

Test Execution Status

A test can fail at different stages:

IN_QUEUE → FAIL:

• Test failed before execution started

• Possible causes: No agents, queue error, login failure

RUNNING → FAIL:

• Test failed during execution

• One or more steps failed

• Check individual step results

SUCCESS:

• All steps completed successfully

• No debugging needed

Step Execution Status

Individual steps can fail:

PENDING → FAIL:

• Step never executed (test failed earlier)

RUNNING → FAIL:

• Step executed but failed

• Error message available

• Screenshot captured (if enabled)

Common Failure Types

1. Selector Not Found

Error Message: Element not found: //button[@id='submit']

What It Means:

• Selector doesn't match any element on the page

• Element doesn't exist

• Element hasn't loaded yet

• Page structure changed

How to Fix:

1. Check Selector: Verify selector is correct

2. Inspect Page: Use browser DevTools to check if element exists

3. Add WAIT: Add WAIT step before interaction if element loads slowly

4. Update Selector: Fix selector if page structure changed

5. Use Alternative Selector: Try different selector if available

Example Fix:

Before: CLICK //button[@class='submit-btn']
After:  WAIT //button[@id='submit']
        CLICK //button[@id='submit']

2. Timeout Waiting for Element

Error Message: Timeout waiting for element: //div[@class='loading']

What It Means:

• Element didn't appear within timeout period

• Page loading slowly

• Element condition not met

How to Fix:

1. Increase Timeout: If element takes longer to load

2. Check Element: Verify element actually appears

3. Fix Selector: Ensure selector is correct

4. Add Delay: Add explicit wait before checking

3. Element Not Clickable

Error Message: Cannot click element: Element is not clickable

What It Means:

• Element is hidden or disabled

• Element is covered by another element

• Element is not in viewport

How to Fix:

1. Scroll to Element: Add SCROLL step before CLICK

2. Wait for Visibility: Add WAIT step for element to be visible

3. Check Element State: Verify element is enabled

4. Remove Overlay: Wait for overlays to disappear

Example Fix:

Before: CLICK //button[@id='submit']
After:  SCROLL (scroll to element)
        WAIT //button[@id='submit'] (wait for visible)
        CLICK //button[@id='submit']

4. Login Failed

Error Message: Login flow failed: [specific error]

What It Means:

• Login test steps failed

• Authentication unsuccessful

• Session validation failed

How to Fix:

1. Check Credentials: Verify username/password are correct

2. Review Login Test: Check login test steps are correct

3. Update Selectors: Fix selectors in login test if page changed

4. Check Login URL: Verify login URL is correct

5. Session Load Failed

Error Message: No session data found for domain or Session load failed

What It Means:

• No session exists for project's domain

• Session expired or invalid

• Domain mismatch

How to Fix:

1. Clear Session: Clear old session data

2. Re-run Test: Create new session

3. Check Domain: Verify session domain matches test URL domain

4. Disable Session: Uncheck "Use Predefined Session" if not needed

6. Navigation Failed

Error Message: Navigation failed: [URL] or Page load timeout

What It Means:

• URL is invalid or inaccessible

• Page takes too long to load

• Network error

How to Fix:

1. Verify URL: Check URL is correct and accessible

2. Check Network: Verify network connectivity

3. Increase Timeout: If page loads slowly

4. Test Manually: Verify URL works in browser

Debugging Workflow

Step 1: Identify the Failure

1. Go to test detail page

2. Check execution status badge (FAIL)

3. Review error message

4. Identify which step failed

Step 2: Review Error Details

1. Click on failed step

2. Read error message

3. View screenshot (if available)

4. Check execution logs

Step 3: Analyze the Issue

1. Understand Error: What does the error mean?

2. Check Context: What happened before the failure?

3. Verify Selector: Is selector correct?

4. Check Timing: Was element loaded?

Step 4: Fix the Issue

1. Update Selector: Fix incorrect selector

2. Add WAIT: Add wait if timing issue

3. Add SCROLL: Scroll if element not visible

4. Fix Value: Correct input value if wrong

Step 5: Re-run and Verify

1. Save changes

2. Run test again

3. Verify fix worked

4. Repeat if still failing

Using Screenshots

When Screenshots Help

Screenshots are valuable for:

• Visual Verification: See what page looked like

• Selector Issues: Verify element exists and is visible

• State Analysis: Understand page state at failure

• Debugging: Compare expected vs actual state

Viewing Screenshots

1. Go to execution detail page

2. Find failed step

3. Click screenshot thumbnail

4. View full-size screenshot

5. Compare with expected state

Screenshot Analysis

What to Look For:

• Is the element visible?

• Is the element in the correct state?

• Are there overlays blocking interaction?

• Has the page structure changed?

• Is the page fully loaded?

Reading Execution Logs

Log Information

Execution logs contain:

• Timestamps: When each step executed

• Status Changes: Step status transitions

• Error Messages: Detailed error information

• Command Details: What command was executed

Log Analysis

Look For:

• Timing Issues: Steps executing too fast

• Sequence Problems: Steps in wrong order

• Selector Issues: Selector not matching

• State Problems: Page not in expected state

Common Debugging Scenarios

Scenario 1: Intermittent Failures

Symptom: Test fails sometimes but not always.

Possible Causes:

• Timing issues (race conditions)

• Network delays

• Dynamic content loading

• Element state changes

Solutions:

1. Add WAIT steps before interactions

2. Increase timeouts

3. Wait for specific conditions

4. Add explicit delays if needed

Scenario 2: Selector Breaking

Symptom: Test worked before but now fails.

Possible Causes:

• Page structure changed

• CSS classes changed

• Element removed or moved

• Dynamic IDs changed

Solutions:

1. Update selector to match new structure

2. Use more stable selectors (IDs, data-testid)

3. Use alternative selectors if available

4. Review page changes

Scenario 3: Login Issues

Symptom: Login flow fails consistently.

Possible Causes:

• Credentials incorrect

• Login page changed

• Selectors incorrect

• Confirmation element changed

Solutions:

1. Verify credentials in project settings

2. Review login test steps

3. Update login test selectors

4. Test login manually

Scenario 4: Session Issues

Symptom: Session validation always fails.

Possible Causes:

• Session expired

• Validation steps incorrect

• Domain mismatch

• Confirmation element changed

Solutions:

1. Clear session data

2. Update validation steps in login test

3. Verify domain matches

4. Re-create session

Best Practices

Prevention

1. Use Stable Selectors: Prefer IDs and data-testid

2. Add Waits: Wait for elements before interaction

3. Test Regularly: Run tests frequently to catch issues early

4. Review Changes: Check page changes before they break tests

Debugging

1. Start with Error: Read error message first

2. Check Screenshots: Visual verification helps

3. Review Logs: Understand execution flow

4. Test Manually: Reproduce issue manually

5. Fix Systematically: Fix one issue at a time

Maintenance

1. Update Selectors: Keep selectors current

2. Monitor Failures: Track common failure patterns

3. Improve Tests: Make tests more robust over time

4. Document Issues: Note common problems and solutions

Troubleshooting Checklist

When a test fails, check:

• Error message is clear and understandable

• Screenshot shows expected page state

• Selector matches element on page

• Element is visible and enabled

• Timing is correct (element loaded)

• Previous steps succeeded

• Login/session is valid (if applicable)

• URL is correct and accessible

• Network connectivity is good

• Agent is available and working

Related Topics

• Viewing Results - Understand execution results

• Running Tests - Execute tests

• Troubleshooting Selectors - Fix selector issues

• Session Management - Fix login issues

---

← Back to Documentation | Next: Understanding XPath Selectors →