~1 min read I recently spent a weekend in the Appalachian mountains with some friends, outside regular Internet access, and away from most technology. One of the rules of the weekend was to not talk about work—at all—and to leave it all behind. So things were decidedly low tech for the weekend. Lots of camp fires, board games, and talking about books and and life and stuff. Something I didn't expect, though, was so many people to still talk about AI. It came up in every conversation. A friend of mine and I carpooled up and back, and we talked about it a good bit over the 4 hours up and back about AI. He raised an interesting question: "is there any point in the average person running an LLM locally at the own home?" The question struck me a bit like the old IoT question. Is there a reason to choose between using the Amazon Echo and its ecosystem compared to, say, Hubitat? Yes, absolutely! Like this image below shows at high-level. The basic value is that it can operate without the rest of the Internet. So if you live in some place with frequent Internet outages, it can be amazing. But also, you can outlive any particular vendor in the space. I admit that I'm erasing a ton a nuance with the image, but practically, it's the good ole question of how much compute do you locally have for the kinds of problems you want to solve. And it's that second part that I return to for the question: "is there any point in the average person running an LLM locally at the own home?" I don't think so. Mostly because the use case for the average person for an LLM is mostly delegated research tasks. Basically, Google, which requires the Internet. The value most employees are seeing with AI at work is helping them doing the clear and repetitive parts of their jobs. Basically, filling out paperwork. And most people just generally don't have that much paperwork in their normal lives. And when they do have it, it's often in web portals that are designed to be hostile to them being used by fully delegated to bots. So I think there's a lot of value to the average small business owner, who has mountains of paperwork, but not necessarily the average person. — Robert This message is 100% organic human effort, so it will have typos. No, I didn't send it through an LLM to typo check it. It was made with just my fingers and my eyes. |