GPT-3 in every Text Field
What if we had a text-based AI that shared context across apps?
I’ve used GitHub Copilot Technical Preview for a while now. I learned to ignore the Copilot most of the time, as the auto-completion break my thinking flow sometimes. But when the boilerplate is obvious, I’m now just waiting for it to autocomplete because I know it will get it right:
Of course, if I’m querying CoinMarketCap API and I just created a variable called sortedByVol, it means I want to sort the array by volume_24h. But now I don’t have to search in the API documentation, and I don’t have to debug the object in WebStorm to know at which depth the 24h volume is stored. It was in a.quote.USD.volume_24h. That kind of insight is as good as StackOverflow and saves time.
At the start of a new project, when it doesn’t have enough context of which libraries I intend to use, it’s hard for the Copilot to be helpful. But as the project grows, the Copilot has more context and the suggestions do get better.
It’s not yet a full replacement of Google+StackOverflow (curious how no one uses StackOverflow’s Search bar). But it’s getting there.
As I’ve mentioned before, the Copilot has evident limitations. It feels a bit like there’s a guy on the other side, which hasn’t learned to code properly, but never gets the Syntax wrong, and can pattern match very well against an enormous database. This guy is often lucky, and sometimes just gets it.
As I’ve started writing more in English, I looked for a smart grammar checker. I’m using Grammarly, and it’s an exceptional tool for a non-native speaker. I cannot recommend it enough now, using Spell-checking in 2021 is like using Emacs instead of VSCode: you can, but you’re multiplying your efforts by 0.8 with respect to any Zoomer that knows which VSCode plugins to install. The coefficient might be different if you’re a native speaker, but there will be some performance gap nonetheless.
So half of my productive life is writing code, with the Copilot suggesting lines to me even before I think of going to Stackoverflow to check how to pipe together JS libraries. The other half of my working life is writing emails and articles in English, with the Grammarly plugin suggesting how to rephrase sentences, which is much needed as often my thoughts come out in a random soup of Italian and English grammar.
I wouldn’t be able to write readable English without Grammarly. I’m not sure it’s helping me get better at it, but does it matter? If anything, I would like Grammarly to be more like the Github Copilot.
Grammarly is not as good as GPT-3, it doesn’t complete my sentences. So I tried to use my code editor to write this article and wait for it to complete my sentences. I have to put the English as JavaScript comments, but it kind of works. So, if my next article is all inside /* … */ you know why.
As a result of using the Copilot, I’m now terribly frustrated at any other Text Field. MacOS has a spell-checker that is a relic from the 90s, and its text-to-speech is nowhere near the Android voice assistant. How long will it take before every text field has GPT-3 autocompletion? Even more, we need a GTP-3 that keeps context across apps. A sort of OS-level autocomplete:
GPT-3 could just fill in my flight’s details in the Browser because it has the context of my previous search and conversation messages. I think you can go a long way with a text-only approach.
Long term, we’ll have Reinforcement Learning trained on Web UIs, that is capable of clicking around and booking a flight from a text description. But I suspect we could go a long way with just a text-only approach: every recent message, every recently opened email, every text-field input would go into a text context and the Operating System Copilot would use that context to auto-complete every Text-Field that shows up (mostly I’m thinking Browser, but why not new emails etc.).
This makes sense, as so much of our working time is wasted in carrying information from one app to the other. Just having the context of WhatsApp conversations already in the Operating System AI (OsAI) would surely provide productivity improvements. An example:
My wife remembers me on WhatsApp about the trip to Paris next month.
I open the Browser to book an Airbnb. What were the dates exactly?
OsAI should know from my previous conversation with my partner and pre-fill the search box with “Paris” and the dates my wife suggested.
It should do the same with Google Flights.
Once I’ve booked the flight, the context of those web pages should be in the memory of OsAI.
Now I book a restaurant in Paris to go out with our French friends. I send to my Fench friends the booking via WhatsApp.
OsAI could pre-write the right text message as soon as I open their name on WhatsApp. “Hi guys, I booked this table at this time”.
It feels this should be easy to train, just by recording many hours of mobile phones usage from volunteers.
What could possibly go wrong? Heheh. I feel we’ll all just end up being trained by models instead of the other way around. The history of user interface shows that people adapt to bad interfaces, and I fear the models will just train us to tolerate ever more obscure dysfunction. HHOS.