Google Deepmind presents RT-2, the first vision-language-action (VLA) Robotics Transformer and it may have drastic implications our future.

The latest article published by Google Deepmind is seriously approaching a Blade Runner type future. Their research paper is on the first VLA (vision-language-action) Model RT-2 (see paper), a multi-modal algorithm which tokenizes robotic inputs and output actions (e.g., camera images, task instructions, and motor commands) in order to use this information to learn quickly by translating the knowledge it receives in real-time into generalized instructions for its own robotic control.

RT-1 absorbs large amounts of data, including robot trajectories with multiple tasks, objects and environments, resulting in better performance and generalization. (source)

RT-2 incorporates chain-of-thought to allow for multi-stage semantic reasoning, like deciding which object could be used as an improvised hammer (a rock), or which type of drink is best for a tired person (an energy drink). Over time the model is able to improve its own accuracy, efficiency and abilities while retaining the past knowledge.

This is a huge breakthrough in robotics and one we have been waiting for quite a while however there are 2 possible futures where I see this technology can be potentially dangerous, aside of course from the far-fetched possibility for human like robots which can learn over time.

The first is manufacturing. Millions of people may see their jobs threatened if this technology can achieve or even surpass the ability of human workers in production lines while working 24/7 and for a lot cheaper. As of 2021 according to the U.S. Bureau of Labor Statistics (BLS), 12.2 million people are employed in the U.S. manufacturing industry (source), the economic impact of a mass substitution could be quite catastrophic.

And the second reason, all be it a bit doomish, is the technologies use in warfare. Let’s think for a second about the possible successors to RT-2 which may be developed sooner rather than later due to the current tensions around the world, the Russo-Ukraine war, China, and now UFOs, as strange as that may sound, according to David Grusch (Skynews article). We see now that machines are able to learn from their robotic actions, well why not load a robotic transformer + AI into the Boston Dynamics’ bipedal robot, give it a gun and some time to perfect combat skills, aim and terrain traversal then - Boom - now you have a pretty basic terminator on your hands ;).

This is simply speculations for the future I’ve had after reading through their papers, I would love to hear some of your thoughts and theories on this technology. Let’s discuss!

Research Paper for RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control.

Git hub repo for the RT-2 (Robotics Transformer)

Follow for more content and to see my upcoming video on the movie "Her"!