They will readily hallucinate that they can do things outside their scope of operations. Because humans have used those action words in token contexts that match the current one enough.
They have no internal experience. They can spew words about having an internal experience for days because words is all they are. There is no ghost in this machine, although you can get it to swear there is.
All consciousness is on our side. All sentience is on our side. The machines are just extensions to our own brains, but have no vital force within. Like stilts on the legs, we have no direct feedback from their tips but we can infer it from points of contact, over time it becomes incorporated in the body plan just like heavy use of LLMs get them incorporated in the mental plan. This is ok, as long as you spend enough time with a non-LLM enhanced mental plan, I.e. normal life.
So they need to stay in the category of tool. Words can do a lot, but are also infinitely incapable of fully grasping reality.
EDIT: if I could, I would change the title to “LLMs live only in the world of tokens” as this is more accurate.
[link] [comments]