## Your task

Your task

Pull new page captures from the production database and clean them up for use as course readings. Follow this process strictly.

Step 1: Pull captures

Run bun run captures:pull to pull new captures from the production database to content/spring2026/readings/. Note which files are new.

Step 2: Clean remote DB junk

Run the standard cleanup query to remove iframe cruft captures (Disqus, reCAPTCHA, privacy popups):

bun run db:query prod "DELETE FROM page_captures WHERE url LIKE '%disqus%' OR url LIKE '%recaptcha%' OR url LIKE '%privacy-mgmt%';"

Step 3: Clean article cruft

For each new article, remove common capture artifacts:

• Advertisements: Advertisement lines and [SKIP ADVERTISEMENT](...) links
• Image credits: Credit... lines (NYT image attribution)
• Bylines/bios: Author name, bio paragraphs that duplicate frontmatter info
• Datelines: Date stamps like "March 12, 2026, 8 AM ET" (the date is in frontmatter)
• Narration credits: Read by... / Narration produced by... / Engineered by... blocks at article end
• Newsletter boilerplate: "You're reading the X newsletter..." subscription cruft
• Internal publication links: [Read: ...] blocks (Atlantic "Read more" cross-links)

Preserve all actual article content, external links, and embedded quotes.

Step 4: Restore section breaks

Open the original article URL and compare with the captured MDX. Where the original article has visual section transitions (extra spacing between sections, decorative dividers, typographic ornaments), add --- in the MDX on its own line with blank lines above and below.

Most publications (Atlantic, NYT, New Yorker) encode section breaks as CSS spacing only, which is lost during the HTML-to-Markdown conversion. These need to be restored manually by comparing with the original article.

Step 5: Preserve media embeds

If the article has an associated video or podcast (e.g., Substack, YouTube), add a responsive iframe embed at the top of the article content:

<div style={{ position: "relative", width: "100%", paddingBottom: "56.25%", marginBottom: "2rem" }}>
  <iframe
    src="EMBED_URL"
    style={{ position: "absolute", top: 0, left: 0, width: "100%", height: "100%", border: "none" }}
    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
    allowFullScreen
  />
</div>

Step 6: Add sidenotes

Review each article for opportunities to add 2-4 sidenotes connecting the article content to course themes. The sidenotes should help students relate what they're reading to their hands-on experience with AI in the course.

Sidenote syntax:

[linked text](sn:note-id)

<Sidenote id="note-id">Note content connecting article to course themes.</Sidenote>

Critical placement rules:

• <Sidenote> blocks are JSX and MUST be standalone paragraphs separated by blank lines above and below — NEVER inline within a paragraph
• The [linked text](sn:id) anchor goes inline in the paragraph text; the <Sidenote> block goes between paragraphs
• If the anchor text is mid-paragraph, place the <Sidenote> block after that paragraph ends, NOT splitting the paragraph around it

Critical: replace smart quotes before adding sidenotes. Web captures contain curly/smart quotes (U+201C, U+201D, U+2018, U+2019) that break MDX JSX parsing. Replace them all with straight quotes early in the cleanup process:

python3 -c "
import sys
with open(sys.argv[1], 'r') as f: text = f.read()
text = text.replace('\u201c', '\"').replace('\u201d', '\"').replace('\u2018', \"'\").replace('\u2019', \"'\")
with open(sys.argv[1], 'w') as f: f.write(text)
" FILE.mdx

Guidelines for good sidenotes:

• Connect article claims to students' direct experience (training networks, using AI tools, etc.)
• Point out when multiple readings are in dialogue with each other
• Ask questions that prompt reflection, not just provide commentary
• Keep them concise — 2-3 sentences max

Step 7: Verify

Start the dev server if not running and check each new reading renders correctly:

• No remaining cruft (ads, credits, bylines)
• Sidenotes render (hover/click triggers margin note)
• Video embeds load where present
• No broken paragraph breaks (especially around Sidenote blocks)

Present all changes for review before committing.

Shouldn't the bun run embed task skip existing files and not re-embed them? What does UPDATE mean in the output?

I just ran the embed and then reran it. The files do not show changed on disk, and yet it seems to be UPDATEing all of them.

That's probably what's happening.

Can you clear the embedding table so we can debug?

Your task

First, run git status, git diff, and git log --oneline -10 to understand the current state.

Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.

I just updated NodeJS to 24.15.0. Can we make sure that the Dockerfiles and all other references match?

The cbtf-assessment e2e test is failing.

Please do that. db:sync does need to work.

Your task

First, run git status, git diff, and git log --oneline -10 to understand the current state.

Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.

Your task

First, run git status, git diff, and git log --oneline -10 to understand the current state.

Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.

Your task

First, run git status, git diff, and git log --oneline -10 to understand the current state.

Then add and commit all modified files with a descriptive message. Break the changes into multiple commits if appropriate. If there are untracked files, ask if they should be added or added to the appropriate .gitignore file. The user may also want to take over at this point, so provide an abort option as well. Always check changes for unnecessary debugging statements or logging that might have been accidentally included. Prompt the user if you find any, and offer to remove each one by one. Do not use interactive git staging (git add -p) as it does not work.