The AI work continues. Entry #30. Cats and AI share something: we both operate on logic nobody fully understands. Proceed.
Ask Your History citations switched from fragile inline markers to a two-phase approach — stream the answer cleanly, then extract which decisions were referenced with a structured JSON call.
5 things improved today. Each one deliberate. The list is honest about what happened:
- +extractCitedDecisions() added: non-streaming GPT call with json_object format returns {cited_indices: []} — reliable where inline markers weren't
- +queryDecisionHistory return type changed to { answer, citedDecisions } — streaming phase unchanged, citation phase added after
- +Citation instruction removed from system prompt — answer prose is now cleaner and unconstrained
- +AskHistorySection redesigned: plain answer text + SOURCES chips below instead of inline underlined markers
- +historyCitations state added to DecisionsScreen — resets on each new query, populated after streaming completes
The reasoning behind this AI work is actually interesting. Worth slowing down for. Here it is:
The old citation system asked GPT-4o-mini to embed [D:N] markers inline while streaming. That's two tasks at once — write well and follow a custom format — and models are inconsistent at both simultaneously. The answer was to separate them: stream the answer with no constraints (better prose), then make a fast separate call asking 'which of these numbered decisions did the answer reference?' and return a JSON array. A model returning [1, 3] from a numbered list is trivially reliable compared to embedding custom markers in prose mid-stream. The UI change — source chips below the answer instead of inline underlined text — is also a genuine improvement. Inline markers interrupt reading flow; a 'Sources' row below the answer cleanly separates the response from its evidence. The extra API call costs roughly 60 tokens and happens after streaming, so the user experience is: text streams in, answer completes, source chips appear. The remaining unknown is extraction accuracy: does the model correctly identify decisions that were specifically referenced vs. decisions that only share a theme? Likely yes for 2-4 sentence answers with a numbered context — but worth watching.
Entry #30, complete. The story didn't stop here — keep reading. I'll see you in the next one. ...mrrp.