GPT-4 Lands, and It Can Read an Image as Well as Text
OpenAI's multimodal flagship ships straight into Stripe, Duolingo and Bing on day one.
On 14 March 2023 OpenAI released GPT-4, the successor to the model behind ChatGPT. The change a non-specialist notices first is that the model can take an image as well as text: a photograph or a screenshot can sit alongside the question, and the answer comes back in writing. In one of OpenAI's examples the model was shown a picture of a plugged-in iPhone and worked out that the adapter in it was a Lightning cable. Note the tense, though — as explained below, that input was not open to ordinary users on launch day.
The second change is competence on difficult work. OpenAI reported that GPT-4 passed a simulated bar exam at roughly the top 10% of test takers, while GPT-3.5, the model most people had used in ChatGPT until then, scored in the bottom 10%. The company was direct about where the difference is and is not visible: "In a casual conversation, the distinction between GPT-3.5 and GPT-4 can be subtle." The gap opens up as the task gets harder.
The third change is the one that matters commercially. GPT-4 did not arrive as a demo. On the day of the announcement it was already running inside Microsoft's Bing chat, at Stripe, in a new Duolingo subscription tier called Duolingo Max, in a document retrieval system for Morgan Stanley's financial analysts, in Khan Academy's automated tutoring tool, and in the Virtual Volunteer feature of Be My Eyes.
A launch that was also a set of shipped products
An announcement of this kind more often means a paper, a demo video and a wait. This one came with named deployments attached on day one. Stripe used the model to read business websites and pass a summary to its customer support staff. Be My Eyes used the image input to interpret pictures for blind and low-vision users.
Access was staged rather than open. GPT-4 answered in ChatGPT for paying Plus subscribers, subject to a usage cap. Developers who wanted it inside their own software joined a waiting list for the API. The image input was tighter still: rather than switching it on for customers generally, OpenAI began with one partner, Be My Eyes, and tested there first.
Pricing was published in a form a business can actually use: $0.03 per 1,000 prompt tokens and $0.06 per 1,000 completion tokens, with 1,000 tokens working out to roughly 750 words. That is the detail that turns a curiosity into a line item.
The limits were published too. OpenAI acknowledged that the model still asserts invented facts, sometimes confidently, and still slips on reasoning. Its knowledge stopped at September 2021, and it does not learn from being used. On safety, OpenAI put the model at 82% less likely than GPT-3.5 to answer requests for disallowed content — an improvement, not a guarantee.
What this means if you are building something
A feature you can price is a feature you can decide on. With per-token pricing published, an automatic reply that reads a page of context and writes a short answer can be costed before anyone writes code. Estimate the words going in and coming out, convert at roughly 750 words per 1,000 tokens, multiply. If the number is unacceptable, you learn it in a meeting rather than after a sprint.
The model does not know your business. September 2021 was the edge of its knowledge, and it states invented things confidently. Prices, stock, opening hours, order status — every fact specific to a client has to come out of that client's own database and be handed to the model with the question. That part is ordinary web development, and it is most of the work.
Do not scope against a feature you cannot get yet. As of this week the API sits behind a waiting list and image input is not generally available. Anything promised to a client should rest on what is switched on today, not on what was shown in a launch video.
"We will add AI later" stopped being free. Not because every site needs a chatbot, but because the deployments above were built on structured content, clean data and somewhere sensible to put generated text. A project scoped now without those is not neutral about the option — it is quietly closing it. Ask for a sane data model now: it costs nothing extra and it keeps the door open.
Sources
