building dexi, an assistant that lives in imessage, no app, the whole surface is a text thread. one prompt-design problem from it is worth sharing because it's non-obvious
when your only surface is messages, you can't show a spinner or stream tokens. there's silence, then a reply. anything over ~10 seconds reads as being ignored and the user starts typing again, which forks the conversation. so you have to acknowledge fast. the trap: a generic "working on it!" reads as a bot instantly and kills trust
what worked was a two-model pattern. a cheap fast pass extracts intent and produces an acknowledgment that contains actual specifics from their message ("looking at the 3 subscriptions on your amex now"), then the slow expensive pass does the real work. the ack has to prove it understood, not just that it received. that constraint made the whole thing feel less robotic even when the slow path was slow
second lesson, on discoverability with no ui: users can't see capabilities, and onboarding prompts get ignored. surfacing "i can also do X" contextually, only when their request nearly matches something adjacent, converted way better than any help message
third, response length is a prompt problem here in a way it isn't in a chat window. nobody reads a six-paragraph text. i cap output far below what the model wants and that trimming is real prompt work, the model fights you the whole way
curious how others handle the user-types-again-mid-response fork. queue the second message, interrupt the first job, or merge them into one intent? i merge and it's the ugliest code i own
[link] [comments]