Like if I type “I have two appl…” for example, often it will suggest “apple” singular instead of plural. Just a small example, but it is really bad at predicting which variant of a word should come after the previous
Like if I type “I have two appl…” for example, often it will suggest “apple” singular instead of plural. Just a small example, but it is really bad at predicting which variant of a word should come after the previous
The kind of local/offline LLMs that would work on your phone would not be very good quality. There’s been amazing progress in quantization of LLMs to get them working on weaker GPUs with lower VRAM and CPUs, so maybe it’ll occur, but I’m not an expert.
I also don’t foresee them linking it up to a cloud-based LLM as that’d be a shit load of queries and extremely expensive.
OpenAI is probably already handling a significant amount of queries, I think for daily use the LLM should simply initialize a word map based on user history and then update it semi-occasionally, like once a week or two. Most people don’t drastically change their vocabulary in the course of a few weeks
We’re talking about orders of magnitude more queries if we start offloading predective text like that.