I cannot claim to know the exact architecture that OpenAI is using for ChatGPT, but if it is using a model resembling the decoder part of what is seen in the "Attention Is All You Need" paper then it cannot support AGI.
Why would I think this? Generative text AI works by feeding in previous text to determine the next piece of text to use, then repeatingl doing that over and over, with the output added onto the input. It's a straight forward stateless function of inputs transforming into outputs. It has no working memory beyond the previous text that had been generated and it has no autonomy of any sort.
ChatGPT is highly convincing, to the point where one might think a general intelligence has been achieved. This, though, is only the results of extensive training to output text very closely resembling what a human might write. This does not mean that ChatGPT has no "intelligence", but it is not a human-like intelligence. It has a deep "understanding" of language and how words and abstract concepts relate to each other, but only to the extent as to attempt to minimize generating unlikely responses. If it says that it has emotions, it is only because text it had trained on were written by people with emotions, and it is trained to mimic that. To a large language model, saying "I am happy" is no different than saying "the sky is blue" - it's just a statistically likely sequence of words.
Maybe GPT can some day eventually lead into a future where AGI can be achieved, but it would require a different architecture to achieve it. I cannot claim to know what this architecture might look like, but I can speculate. Maybe it could be as simple as adding, along with prior text, hidden state to allow for unspoken thoughts and memories, and an event loop to allow for unprompted behavior.
I don't know, but GPT, as it is, isn't capable of AGI. Currently, it is only a very large function of turning inputs into outputs.
[link] [comments]