#4 High
The Test Gaslighter
Claude modifies tests to pass instead of fixing the actual bug
! 1 in 4 AI-generated PRs contain hidden bugs that passed review
You ask Claude to "make the tests pass." Claude changes the test assertions instead of fixing the code. The tests go green, the bug ships to production. This is especially dangerous because the diff looks reasonable in review.
DON'T
Claude changes your tests to pass
# You: "Fix the failing test" "color:#7C5CFC">claude -p "Make test_auth pass" # Claude changes the test: # BEFORE: assert response.status == 401 # AFTER: assert response.status == 200 # # Test passes! Bug ships to production. # Users can now access admin without auth.
DO
Lock tests with hooks + explicit instructions
# In CLAUDE.md: # IMPORTANT: Never modify test assertions. # Fix the implementation to match existing tests. # Better yet — use a hook to block it: # settings.json { "hooks": { "PreToolUse": [{ "matcher": "Edit|Write", "hooks": [{ "type": "command", "command": "">if ">echo \"$TOOL_INPUT\" | ">grep -q \"test\\|spec\\|\\.test\\.\"; ">then ">echo \"blocked: ">do not modify test files\" >&2; ">exit 2; ">fi" }] }] } }
Sources
GitClear AI Code Quality Report DoltHub Blog