We're facing a significant challenge with running LLMs (large language models) on local devices. Can it be done? Yes. But do they consume a lot of resources? Absolutely, especially if you want them to be highly functional. This is likely why Microsoft is introducing NPU (Neural Processing Unit) in newer computers, allowing AI tasks to run on NPUs instead of the usual CPU or GPU. While this is a step in the right direction, many of us are hoping for a more "Her"-like AI—a conversational AI with its own personality, capable of assisting us throughout the day, without having to wait a decade or more for hardware advancements.
This got me thinking about projects like Rabbit r1. We might soon see a push for AI cores in our homes. The Rabbit r1 and similar projects failed due to limited functionality and slow speeds. However, in theory, we could have a dedicated computer, about the size of a standard home NAS, running 24/7 as the AI's home base. Using software on our mobile devices or computers, this AI could remember, adapt, and operate similarly to Microsoft's vision, but without needing a complete hardware overhaul. This means the same AI could work across both your computer and mobile device. Moreover, it might be possible to have an AI core integrated with your local IoT devices and security system, adding extra functionality. For example, it could track which room you're in and communicate with you through local systems in that room.
While I doubt the first generation would be advanced enough to, say, take a picture of your fridge and order food automatically, I can imagine this becoming a reality in the future.
Thoughts?
[link] [comments]