Jay van Zyl @ ecosystem.Ai

Jay van Zyl @ ecosystem.Ai

5 things about AI you may have missed today: AI-linked movies raise job-loss fears, AI sparks US-China tech war, more – HT Tech

5 things about AI you may have missed today: AI-linked movies raise job-loss fears, AI sparks US-China tech war, more  HT Tech

Meta, INRIA researchers discover that explicit registers eliminate ViT attention spikes

When visualizing the inner workings of vision transformers (ViTs), researchers noticed weird spikes of attention on random background patches. This didn't make sense since the models should focus on foreground objects. By analyzing the output embed…