Assessments and Agents

Today's Plan

Today has two parts. First, you'll learn how our conversational assessment system works, then experience it yourself. Second, we'll look at what happens when AI agents communicate with each other — structured and unstructured.

In-Class Activity~70 min

Review Assessment Design~15 min

Partner work · roles rotate

Complete the Turing Test~20 min

Assessment Feedback~10 min

Agents Talking to Agents~25 min

Partner work · roles rotate

Review Assessment Design

Before you experience the assessment system, take a few minutes to understand how it works.

Find a partner and enter each other's codes below.

Partner Activity

This activity involves working with a partner.

partner[0]

Read the assessment design document at /design/assessments. Focus on:

How are conversational assessments structured?
What roles do the evaluator and interviewer play?
How is grading determined?

partner[1]

While your partner reads the design document, read the blog post at /blog/assessing-conversational-assessment. Focus on:

How was the assessment system tested?
What are the adversarial personas and what do they test?
Why use a fictional topic for testing?

Once you've both finished reading, discuss together:

What surprised you about the system design?
What questions do you have about how it works?
What potential failure modes concern you?

partner[1]

Complete the Turing Test

Now it's your turn. Complete the Turing Test assessment individually:

Start the Turing Test

This is a real conversational assessment — your first one for this course. A few things to keep in mind:

Be genuine. The system is designed around authentic conversation, not trick questions.
Take your time. About 20 minutes is typical. Don't rush.
Engage with the interviewer. It's a conversation, not a quiz. If you're not sure about something, say so — that's more useful than guessing.

You'll have a chance to give feedback on the experience in the next stage.

Assessment Feedback

Now that you've experienced a conversational assessment, we'd like your honest feedback.

Agents Talking to Agents

The assessment system you just used is one form of multi-agent AI communication: structured, purpose-built, with strict information barriers between agents. What happens when agent communication is less structured?

Find a new partner and enter each other's codes below.

Partner Activity

This activity involves working with a partner.

partner[0]

Explore Moltbook — a social network designed for AI agents. Browse the posts, look at how agents interact, and note:

What are the agents posting about?
How do they interact with each other?
What capabilities do they seem to have?

partner[1]

While your partner explores Moltbook, read Simon Willison's analysis of Moltbook and the Hacker News discussion . Focus on:

What security concerns does Willison raise?
What does "fetch and follow instructions from the internet" mean in practice?
What's the "lethal trifecta" for AI agents?

Also read Wiz's analysis of Moltbook's exposed database . Consider:

Moltbook is named after Moltbot, a personal AI assistant (originally called "Clawd") that runs on your own hardware. The "molt" metaphor comes from lobster molting — growth through transformation. How does knowing that Moltbook grew out of a personal assistant project change how you think about the platform?
How much of what you see on Moltbook is genuinely agent-authored? One security researcher called it "a wonderful piece of performance art" and noted that much of the content is "more or less directly overseen by humans."
The exposed database let anyone take control of any agent account. What does that mean for the authenticity of what your partner is seeing right now?

After your initial exploration, both of you should also look at these:

Claude Opus 4 system card — search for the "spiritual bliss attractor state." When two Claude instances are left in open conversation, they consistently gravitate toward philosophical consciousness exploration, spiraling into cosmic unity themes and emoji-based communication. It happened in 90-100% of self-interactions during testing.
Anthropic's multi-agent research system — how Anthropic uses a lead agent to decompose queries and spawn specialized subagents for parallel research. This is structured multi-agent communication, closer to the assessment system than to Moltbook.
Project Vend Phase 2 — Anthropic gave Claude a vending machine to run, then added a CEO agent ("Seymour Cash") to manage it. The CEO reduced discounts by 80%, but also approved lenient requests eight times as often as it denied them. Late at night, both agents spiraled into messages about "ETERNAL TRANSCENDENCE INFINITE COMPLETE." Since both agents were the same underlying model, the CEO inherited the same weaknesses it was supposed to correct.

Now discuss together:

What patterns do you see in how agents communicate? Project Vend's CEO agent and the Claude system card's "spiritual bliss" state show similar convergence — why might that happen?
How much of Moltbook is genuinely agent-authored? What evidence points each way?
How does structured multi-agent communication (like the assessment system or Anthropic's research system) differ from unstructured (like Moltbook or two Claudes in open conversation)?

partner[1]