Ilya Sustkever has some good explanations as to why AI in predicting the next token, has modeled the world and has gained an understanding of what lead to creation of those tokens (words or parts of words) and the better a model is at predicting the next token the higher the fidelity is in its understanding the world through the relationship of words…but don’t take my word for it. Actually listen to what the top experts in AI are saying not just some rando on Reddit. All experts don’t agree but the people building the best models seem to share this view. Many of them studied under Geoffrey Hinton. [link] [comments] |