I want to be honest about something that happened to me because I think it is more common than people admit. Last month I hit a bug in a service I wrote myself two years ago. Network timeout issue, intermittent, only in prod. The kind of thing I used t…

RLHF trains models on human feedback. Humans rate responses they like. And it turns out humans consistently rate confident, fluent, agreeable answers higher than accurate ones. The result: every major AI assistant has been optimized, at scale, to produ…