Attention and AI – Why I think it’s important and how I define it
Motivation:
I want to help “reach” AI. The whole notion of “AI” itself is a topic for discussion, but let’s just say my notion of AI is along the lines of what was discussed in Turing’s “Computing Machinery and Intelligence” paper.
I believe attention is a key component for that. Thus, in order to “reach” AI, we need to “reach” attention. We first need to formulate a precise and useful definition for “attention”. BEFORE THAT, let me cite some examples of the phenomenon I’m trying to describe.
Examples of ‘attention’:
- You are walking on the street and something abrupt (e.g. a bird flies by near the border of your field of vision or a sudden “crack” sound occurs) thus making you automatically turning your head in direction to the event.
- You are hungry and thus it’s easier to have your “attention” called from food smell.
- You start thinking about something: an event, a calculation, some planning, and then pays less “attention” to things around you, focusing on your “train of thought”.
- You are reading a text and you automatically focus one set of words each time.
For me, all these events are related. This seems to be a very important function for us. We have finite “resources” to think and we must use it wisely to survive! In the next session, I try to define it more precisely and from a computer science standpoint.
Towards a more precise definition:
In a world with scarce resources, attention is about orchestrating processes.
Data is virtually infinite.
The number of programs are also virtually infinite.
Thus, the number of processes, that is, the manipulation of some data by following the set of rules of a program, is also infinite.
However, the quantity of resources to execute these processes is finite.
Attention is the act of allocating resources to processes.
In other words, attention is the entity that, given context and goals, selects a set of processes (program-data pairs) and determines the quantity of resources to execute each of them.
SO, my questions to you all are:
What do you think about it all? Do you think it makes sense? Do you think attention can be thought as a phenomenon in itself? Do you think this definition I lay out is useful for us to help guide work on the subject?
Thanks
submitted by /u/perilo
[link] [comments]