Back to Spacetimedb

Chat App Grading Checklist

tools/llm-oneshot/apps/chat-app/prompts/grading_checklist.md

2.1.06.4 KB
Original Source

Chat App Grading Checklist

Instructions: For each criterion, mark with a number:

  • [x] = Working (0 reprompts needed)
  • [1] = Working after 1 reprompt
  • [2] = Working after 2 reprompts
  • [3+] = Working after 3+ reprompts
  • [ ] = Not working / not implemented

Overall Metrics

MetricValue
Prompt Level Used********_********
Features Evaluated1-___ (max 15)
Total Feature Score_ / _ (max 45)
  • Compiles without errors
  • Runs without crashing
  • First-try success (worked without manual fixes)
MetricValue
Lines of code (backend)___
Lines of code (frontend)___
Number of files created___
External dependencies********_********

Feature 1: Basic Chat Features (Score: ___ / 3)

  • Users can set a display name (0.5)
  • Users can create chat rooms (0.5)
  • Users can join/leave rooms (0.5)
  • Users can send messages to joined rooms (0.5)
  • Online users are displayed (0.5)
  • Basic validation exists (0.5)

Feature 2: Typing Indicators (Score: ___ / 3)

  • Typing state is broadcast to other room members (1)
  • Typing indicator auto-expires after inactivity (1)
  • UI shows "User is typing..." or "Multiple users are typing..." (1)

Feature 3: Read Receipts (Score: ___ / 3)

  • System tracks which users have seen which messages (1)
  • "Seen by X, Y, Z" indicator displays under messages (1)
  • Read status updates in real-time (1)

Feature 4: Unread Message Counts (Score: ___ / 3)

  • Unread count badge shows on room list (1)
  • Count tracks last-read position per user per room (1)
  • Counts update in real-time (1)

Feature 5: Scheduled Messages (Score: ___ / 3)

  • Users can compose and schedule messages for future delivery (1)
  • Pending scheduled messages visible to author with cancel option (1)
  • Message appears in room at scheduled time (1)

Feature 6: Ephemeral/Disappearing Messages (Score: ___ / 3)

  • Users can send messages with auto-delete timer (1)
  • Countdown or disappearing indicator shown in UI (1)
  • Message is permanently deleted when timer expires (1)

Feature 7: Message Reactions (Score: ___ / 3)

  • Users can add emoji reactions to messages (0.75)
  • Reaction counts display and update in real-time (0.75)
  • Users can toggle their own reactions on/off (0.75)
  • Hover/click shows who reacted (0.75)

Feature 8: Message Editing with History (Score: ___ / 3)

  • Users can edit their own messages (1)
  • "(edited)" indicator shows on edited messages (0.5)
  • Edit history is viewable by other users (1)
  • Edits sync in real-time to all viewers (0.5)

Feature 9: Real-Time Permissions (Score: ___ / 3)

  • Room creator is admin and can kick/ban users (1)
  • Kicked users immediately lose access and stop receiving updates (1)
  • Admins can promote other users to admin (0.5)
  • Permission changes apply instantly (0.5)

Feature 10: Rich User Presence (Score: ___ / 3)

  • Users can set status: online, away, do-not-disturb, invisible (1)
  • "Last active X minutes ago" shows for offline users (0.5)
  • Status changes sync to all viewers in real-time (1)
  • Auto-set to "away" after inactivity period (0.5)

Feature 11: Message Threading (Score: ___ / 3)

  • Users can reply to specific messages, creating a thread (1)
  • Parent messages show reply count and preview (0.5)
  • Threaded view shows all replies to a message (1)
  • New replies sync in real-time to thread viewers (0.5)

Feature 12: Private Rooms & Direct Messages (Score: ___ / 3)

  • Users can create private/invite-only rooms (0.75)
  • Room creators can invite specific users by username (0.75)
  • Direct messages (DMs) between two users work (0.75)
  • Only members can see private room content and member lists (0.75)

Feature 13: Room Activity Indicators (Score: ___ / 3)

  • Activity badges show on rooms (e.g., "Active", "Hot") (1)
  • Activity level reflects recent message velocity (1)
  • Indicators update in real-time as activity changes (1)

Feature 14: Draft Sync (Score: ___ / 3)

  • Message drafts save automatically as user types (1)
  • Drafts sync across devices/sessions in real-time (1)
  • Each room maintains its own draft per user (0.5)
  • Drafts persist until sent or manually cleared (0.5)

Feature 15: Anonymous to Registered Migration (Score: ___ / 3)

  • Users can join and send messages without an account (1)
  • Anonymous identity persists for the session (0.5)
  • Registration preserves message history and identity (1)
  • Room memberships transfer to registered account (0.5)

Summary Score Sheet

FeatureMaxScoreReprompts
1. Basic Chat3______
2. Typing Indicators3______
3. Read Receipts3______
4. Unread Counts3______
5. Scheduled Messages3______
6. Ephemeral Messages3______
7. Message Reactions3______
8. Message Editing3______
9. Real-Time Permissions3______
10. Rich Presence3______
11. Message Threading3______
12. Private Rooms & DMs3______
13. Activity Indicators3______
14. Draft Sync3______
15. Anonymous Migration3______
TOTAL45______