By Henry Ho, Founder of Geode
Founder’s Notes #3
For a long time, I assumed the future of AI would mostly belong to the cloud. That wasn’t a difficult assumption to make. The most capable models are there, backed by enormous computing power, and I use cloud AI every day myself.
But over the last couple of years, I’ve become convinced of something else. I think on-device AI is going to matter far more than many people expect, and that belief has shaped some of the biggest engineering decisions we’ve made at Geode.
In my first Founder’s Note, I wrote about why recorded audio should stay yours and why users should have a choice between local and cloud processing. Although that article focused on transcription, I never thought the idea should stop there. Once AI became part of the product, the same question came back in a different form: if users should be able to choose where their recordings are processed, shouldn’t they also be able to choose where AI runs?
That question is why we built an on-device language model into Geode.
A year or two ago, I barely thought about token usage. I’d ask AI a few questions, get an answer, and move on with my day. Today it’s completely different. AI has quietly become part of my workflow. I use it to summarize meetings, review documents, draft emails, improve writing, brainstorm ideas and ask follow-up questions throughout the day. Sometimes I don’t even notice how many times I’ve opened an AI app before lunch.
As that happened, I found myself asking a different question: does every AI task really need to go to the cloud?
For many tasks, I don’t think it does. Today’s on-device models have improved remarkably quickly, and they’re already capable of doing real work. Summarizing transcripts, organizing information and extracting key points no longer require sending everything to a remote server. Cloud models remain the best choice for many situations, but they don’t have to be the only choice.
I noticed another change as well. The more useful AI became, the more context I wanted to give it. At first it was a single document. Then it became a folder, a collection of meeting notes or several related conversations. Eventually, what I really wanted was for AI to understand how I work, not just what was in front of it.
That’s exactly what makes modern AI so powerful. The more context it has, the more helpful it becomes.
But that also made me realize something. Different situations deserve different choices. Sometimes I’m perfectly happy to send my work to a cloud model. Other times I’d much rather keep everything on my own computer. I don’t think there’s a universal answer, and I don’t think software should make that decision on behalf of its users.
Building an on-device language model wasn’t an obvious decision. Running local speech recognition is already challenging. Running a local language model well is even harder. Different computers have different capabilities, available memory is limited, and people quite reasonably expect AI to feel fast and responsive regardless of the hardware they own.
I’ll be honest: this took us much longer than I expected. Getting a model to run locally wasn’t the hard part. Making it fast enough, reliable enough and useful enough that people would actually want to use it every day was. It remains one of the hardest engineering problems we’ve worked on.
Even so, I still believe it was the right decision.
Not because I think on-device AI will replace cloud AI. I don’t. Cloud models will continue to improve, and they’ll continue to be the right answer for many kinds of work.
What I do believe is that AI is becoming part of everyday computing in the same way the internet once did. As that happens, I don’t think every task will belong in the cloud, just as I don’t think every task belongs entirely on a personal device. People will naturally use both. Sometimes they’ll want the most capable model available. Sometimes they’ll prefer AI that runs entirely on their own computer. Most of the time, they’ll simply choose whichever makes the most sense for the task in front of them.
I don’t know exactly what AI will look like five years from now.
What I do know is that I want people to have more choices, not fewer.
That’s why we bet on on-device AI.