E2E Test Bypass & Completion Visibility Tests
Implement the following plan:
E2E Test Bypass & Completion Visibility Tests
Context
We just added inline instructor-only completion summaries to PreparationChat and the assessment detail page. Now we need:
- Magic message bypass — in dev mode, a test student can send a special message to instantly complete a preparation or assessment (with a specified grade), bypassing the LLM entirely
- E2E seed endpoints — create completed/in-progress preparation sessions without LLM (assessment seed already exists)
- E2E tests — verify the completion visibility feature works, and test the magic message flow
- Cleanup fix —
preparationSessionsis missing from the e2e user cleanup
1. Magic Message Bypass in Chat Routes
Preparation chat: apps/api/src/routes/preparation-chat.ts
After saving the user message (line 145-149) and before the readiness evaluator (line 163), detect the magic message:
Magic message: __E2E_COMPLETE_PREPARATION__
Guard: process.env.NODE_ENV === "development"
When detected:
- Build a synthetic "allReady" readiness state (set all topic criteria to "ready", allReady=true)
- Update
preparationSessionswith completed status + readinessState - Stream a canned response:
"Preparation complete! (E2E bypass)" - Insert boundary system message with
{ preparationBoundary: true } - Append
\n\n---PREPARATION_COMPLETE---to stream - Return (skip LLM evaluator + discussion agent entirely)
Assessment chat: apps/api/src/routes/assessment-chat.ts
After saving the user message (line 174-179) and before the evaluator (line 193), detect the magic message:
Magic message: __E2E_COMPLETE_ASSESSMENT_<grade>__
e.g., __E2E_COMPLETE_ASSESSMENT_E__, __E2E_COMPLETE_ASSESSMENT_S__, etc.
Guard: process.env.NODE_ENV === "development"
Regex: /^__E2E_COMPLETE_ASSESSMENT_([ESNU])__$/
When detected:
- Look up the grade's points from
assessment.gradingScale - Build portion grades: all portions get the requested grade
- Insert evaluation into
assessmentEvaluationstable - Mark attempt as completed
- Stream a canned response:
"Assessment complete with grade <grade>! (E2E bypass)" - Insert boundary system message with
{ assessmentBoundary: true } - Append
\n\n---ASSESSMENT_COMPLETE---to stream - Return (skip LLM evaluator + interviewer entirely)
2. E2E Seed Endpoints: apps/api/src/routes/e2e.ts
POST /api/e2e/preparations/seed
Create a completed preparation session (parallel to existing e2e/assessments/seed):
Schema: { userId: string, preparationSlug?: string } (default: "understanding-media-ch1")
Steps:
- Load preparation config via
getPreparation(slug)to get topic structure - Create conversation with
context: "preparation" - Insert canned messages (system + assistant greeting + user response + assistant completion)
- Build a fully-ready readiness state (all criteria "ready", allReady=true)
- Create
preparationSessionsrecord withstatus: "completed", the readiness state, and completedAt - Insert boundary system message
- Return
{ sessionId, conversationId, messageIds }
POST /api/e2e/preparations/start-seed
Create an in-progress preparation session (for magic message testing):
Schema: { userId: string, preparationSlug?: string }
Steps:
- Create conversation
- Insert a canned opening assistant message
- Create
preparationSessionsrecord withstatus: "in_progress", initial readiness state - Return
{ sessionId, conversationId }
POST /api/e2e/assessments/start-seed
Create an in-progress assessment attempt (for magic message testing):
Schema: { userId: string, assessmentSlug?: string }
Steps:
- Create conversation with
context: "assessment" - Insert a canned opening assistant message
- Create
assessmentAttemptsrecord withstatus: "in_progress", initial evaluator state - Return
{ attemptId, conversationId }
3. Cleanup Fix: apps/api/src/routes/e2e.ts
Add preparationSessions deletion in both cleanup handlers (by-nonce and cleanup-all), BEFORE conversations are deleted (since prep sessions have a FK to conversations).
Import: preparationSessions from @repo/db/schema
In the cleanup loops, add:
await db.delete(preparationSessions).where(eq(preparationSessions.userId, user.id));
4. Add data-testid Attributes to Completion Summaries
apps/web/components/mdx/preparation-chat.tsx
- Outer div:
data-testid="preparation-completion-summary" - Count text:
data-testid="preparation-completion-count" - List:
data-testid="preparation-completion-list"
apps/web/app/assessments/[slug]/page.tsx
- Outer div:
data-testid="assessment-completion-summary" - Count text:
data-testid="assessment-completion-count" - List:
data-testid="assessment-completion-list"
5. E2E Tests
testing/e2e/tests/completion-visibility.spec.ts (default suite)
Uses seed endpoints (no LLM needed), tests the inline completion visibility feature.
Test: instructor sees preparation completion summary
- Create instructor + student users
- Seed a completed preparation for the student via
POST /api/e2e/preparations/seed - Instructor navigates to
/spring2026/readings/understanding-media-ch1 - Assert
preparation-completion-summaryis visible - Assert
preparation-completion-countcontains "1 student" - Assert
preparation-completion-listcontains the student's name
Test: instructor sees assessment completion summary with grade
- Create instructor + student users
- Seed a completed assessment for the student via existing
POST /api/e2e/assessments/seed - Instructor navigates to
/assessments/turing-test - Assert
assessment-completion-summaryis visible - Assert
assessment-completion-countcontains "1 student" - Assert student name and grade badge "S" are visible
Test: completion summaries hidden when viewing as student
- Create instructor user
- Seed completed preparation + assessment for another student
- Instructor enables "view as student" mode (set
viewAsStudentcookie) - Navigate to reading page → assert
preparation-completion-summarynot visible - Navigate to assessment detail → assert
assessment-completion-summarynot visible
Test: student does not see completion summaries
- Create student user + seed completed data for another student
- Navigate to reading page → assert
preparation-completion-summarynot visible - Navigate to assessment detail → assert
assessment-completion-summarynot visible
testing/e2e/tests/magic-complete.spec.ts (request-only, not in default suite)
Tests the magic message bypass through the UI. Requires the start to work (LLM needed for initial message generation). Skips gracefully if LLM not configured.
Test: magic message completes preparation
- Create student, start preparation via API (skip if fails)
- Navigate to session page, wait for chat UI
- Type
__E2E_COMPLETE_PREPARATION__and send - Wait for
preparation-boundarytestid to appear - Verify session status changed to completed
Test: magic message completes assessment with grade
- Create student, start assessment via API (skip if fails)
- Navigate to attempt page, wait for chat UI
- Type
__E2E_COMPLETE_ASSESSMENT_S__and send - Wait for
assessment-boundarytestid to appear - Verify evaluation shows grade "S"
Alternative: testing/e2e/tests/magic-complete-seeded.spec.ts (default suite)
Tests the magic message bypass using start-seed endpoints (no LLM needed at all).
Test: magic message completes seeded preparation
- Create student, start-seed a preparation via
POST /api/e2e/preparations/start-seed - Navigate to session page
- Type
__E2E_COMPLETE_PREPARATION__and send - Wait for
preparation-boundaryto appear
Test: magic message completes seeded assessment with grade
- Create student, start-seed an assessment via
POST /api/e2e/assessments/start-seed - Navigate to attempt page
- Type
__E2E_COMPLETE_ASSESSMENT_E__and send - Wait for
assessment-boundaryto appear - Verify evaluation shows grade "E"
Playwright config: testing/e2e/playwright.config.ts
Add projects:
completion-visibility— default suite (withdeps)magic-complete— request-only (no deps, likeassessment-lifecycle)magic-complete-seeded— default suite (withdeps)
Key Files to Modify
| File | Changes |
|---|---|
apps/api/src/routes/preparation-chat.ts | Magic message bypass |
apps/api/src/routes/assessment-chat.ts | Magic message bypass |
apps/api/src/routes/e2e.ts | Seed/start-seed endpoints + cleanup fix |
apps/web/components/mdx/preparation-chat.tsx | Add data-testid attrs |
apps/web/app/assessments/[slug]/page.tsx | Add data-testid attrs |
testing/e2e/playwright.config.ts | New test projects |
testing/e2e/tests/completion-visibility.spec.ts | NEW — visibility tests |
testing/e2e/tests/magic-complete.spec.ts | NEW — LLM-start magic complete |
testing/e2e/tests/magic-complete-seeded.spec.ts | NEW — seeded magic complete |
Existing Code to Reuse
getPreparation(slug)from@repo/agents— load prep config for seed endpointsgetAssessment(slug)from@repo/agents— load assessment configbuildInitialReadinessState()fromapps/api/src/services/preparation-evaluator.ts— initial state for start-seedbuildInitialEvaluatorState()fromapps/api/src/services/assessment-evaluator.ts— initial state for start-seed- Existing
e2e/assessments/seedpattern ine2e.ts— reference for preparation seed auth.fixture.tstest patterns — user creation, context management
Verification
bun run tscpassesCI=true bunx playwright test --project=completion-visibility— tests completion visibilityCI=true bunx playwright test --project=magic-complete-seeded— tests magic message bypassCI=true bunx playwright test --project=magic-complete— tests magic message with LLM start (if configured)
If you need specific details from before exiting plan mode (like exact code snippets, error messages, or content you generated), read the full transcript at: ~/.claude/projects/-Users-challen-www-usingandunderstanding-ai/62d83b9f-e7a4-47a5-bad5-b535f132c93f.jsonl
I'll start by reading all the key files I need to modify in parallel.
Now let me read the additional files I need to understand the patterns.
Let me read a few more files to understand existing patterns.
Let me check a few more things - the existing preparations test and the "view as student" mechanism.
Now I have a clear picture of all the code. Let me create tasks and start implementing.
Now let me implement the magic message bypasses and data-testid attributes in parallel.
Now let me implement the E2E seed endpoints and cleanup fix. First, let me check the getPreparation function to understand what it returns.
Now let me implement the e2e.ts changes. This is the biggest file change - I need to add the seed endpoints and fix the cleanup.
Now I'll make the edits to e2e.ts. First the import fix, then cleanup fix, then the new seed endpoints.
Now add preparationSessions deletion to the nonce-based cleanup (before conversations deletion):
Now add preparationSessions deletion to the cleanup-all handler:
Now add the three new seed endpoints before the final export. Let me add them after the assessments/seed endpoint:
Now let me check the DB schema for preparationSessions to understand the available columns, and look at the chat page to see what testids exist for the boundary messages.
Good - the boundary messages already have data-testid="preparation-boundary" and data-testid="assessment-boundary". Now let me look at the assessment-lifecycle spec to understand the pattern for magic message tests with LLM.
Now let me check what test data-testids exist for the chat input and send button:
Good. Now I have all the context needed to create the three E2E test files and update the playwright config. Let me create them all in parallel.
Now update the playwright config to add the three new test projects:
Now let me verify TypeScript compilation.
TypeScript compilation passes cleanly. Let me also quickly verify the e2e test files compile by checking for any syntax issues:
That's a pre-existing issue (not related to my changes). Let me verify the dev server is running so I can run the E2E tests:
Dev server is running. But since I changed API files, I need the API to have reloaded. Let me check that the API is responding:
API is responding (404 on root is expected). Let me run the completion-visibility tests first:
The dev database already has 10 completed preparation sessions. I need to make the assertions more resilient — check that the student name appears in the list rather than asserting exact counts.
All completion-visibility tests pass. Now let me run the magic-complete-seeded tests:
All tests pass. Let me also run the magic-complete (LLM-start) tests to verify they work or skip gracefully:
All tests pass across all three test suites. Let me mark the final task as complete.
All done. Here's a summary of what was implemented:
Magic message bypass (2 files):
apps/api/src/routes/preparation-chat.ts— After saving the user message, detects__E2E_COMPLETE_PREPARATION__in dev mode, builds an all-ready state, marks the session completed, streams a canned response with the completion signalapps/api/src/routes/assessment-chat.ts— Detects__E2E_COMPLETE_ASSESSMENT_<grade>__(E/S/N/U) in dev mode, creates evaluation with the requested grade, marks the attempt completed, streams a canned response with the completion signal
E2E seed endpoints (apps/api/src/routes/e2e.ts):
POST /api/e2e/preparations/seed— Creates a completed preparation session with canned messages and fully-ready readiness statePOST /api/e2e/preparations/start-seed— Creates an in-progress preparation session with initial state (for magic message testing)POST /api/e2e/assessments/start-seed— Creates an in-progress assessment attempt with initial state (for magic message testing)
Cleanup fix (e2e.ts): Added preparationSessions deletion in both cleanup handlers, before conversations are deleted
Data-testid attributes: Added to both preparation-chat.tsx and assessments/[slug]/page.tsx completion summary elements
E2E tests (3 new files):
completion-visibility.spec.ts— 4 tests verifying instructor sees summaries, view-as-student hides them, students don't see themmagic-complete-seeded.spec.ts— 2 tests using start-seed endpoints (no LLM needed)magic-complete.spec.ts— 2 tests using LLM start (request-only, not in default suite)
All 18 tests pass across the three suites.
Your task
First, run git status, git diff, and git log --oneline -10 to understand the current state.
Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.
All changes look clean — no debugging statements or noise. The changes split naturally into one cohesive feature. Let me commit everything together since it's a single interconnected feature:
Committed. Here's a summary:
1 commit: 8e3e13c — 10 files, 781 insertions, 10 deletions
preparation-chat.ts/assessment-chat.ts— magic message bypasse2e.ts— 3 new seed endpoints + cleanup fixpreparation-chat.tsx/assessments/[slug]/page.tsx—data-testidattrsplaywright.config.ts— 3 new test projects- 3 new test files (all 18 tests passing)
Can we make the chat window a bit easier to close on the educators page in thin mode? Maybe a close button at the top right?
Let me find the chat component used on the educators page.
I see the issue — the mobile bottom sheet only has a small drag handle to close it, which isn't very discoverable. I'll add a close button to the top-right of the sheet header.
Let me verify it looks right:
Added an X close button to the top-right of the mobile bottom sheet header. It sits next to the existing drag handle, is styled with text-foreground-muted (lightens on hover), and has an aria-label for accessibility. The whole handle bar area still closes on click too, so there are now two clear ways to dismiss the sheet.
Nice!
Glad it works well! The X button should make it much more obvious how to dismiss the chat on narrow screens.
Your task
First, run git status, git diff, and git log --oneline -10 to understand the current state.
Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.
Clean diff — no debug statements. The package.json change is just a typescript-eslint version bump (8.55.0 → 8.56.0), likely from a ncu run. These are unrelated enough to warrant separate commits.
Two commits:
267e311— Close button for mobile chat sheet6d65023— typescript-eslint 8.55.0 → 8.56.0 version bump
Are we missing another assest on the remote machine? Here's the error: @repo/api:tsc: @repo/api:tsc: $ tsc --noEmit @repo/api:lint: cache miss, executing b56d8399c4c8b34a @repo/api:lint: @repo/api:lint: $ bunx eslint --max-warnings 0 . @repo/web:tsc: app/page.tsx:10:22 - error TS2307: Cannot find module '@/app/icon-512.png' or its corresponding type declarations. @repo/web:tsc: @repo/web:tsc: 10 import logoFull from "@/app/icon-512.png";
The file is tracked and exists locally. Let me check if it's in the current branch vs main:
It exists on main too. The issue is likely that the remote machine hasn't pulled the latest, or something is off with the deploy. Let me check if maybe the import was recently added:
Your task
First, run git status, git diff, and git log --oneline -10 to understand the current state.
Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.
All code changes are formatting/style normalizations from the linter (Prettier reordering Tailwind classes, reformatting JSX/arrow functions). No logic changes, no debug statements. Let me check the package.json bumps:
Clean — all dependency bumps (@types/nodemailer 7.0.9→7.0.10, typescript-eslint 8.55.0→8.56.0). I'll split this into two commits: one for the linter/formatter normalization + close button tweak, and one for the dependency bumps.
Two commits:
984ce95— Prettier formatting normalization (Tailwind class ordering, JSX style)ba8020e—@types/nodemailer7.0.10,typescript-eslint8.56.0
Your task
First, run git status, git diff, and git log --oneline -10 to understand the current state.
Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.
Clean — just a turbo bump from 2.8.9 to 2.8.10 plus the lockfile update. No debug statements anywhere.
Done — one commit 19cd870: turbo 2.8.9 → 2.8.10.
tl