OpenAI Spring Conference: New GPT-4o Multi-Modal Model Released, Real-Time Interaction and Free User Upgrades Available

OpenAI held its spring launch event at 1:00 a.m. Beijing time on Tuesday. The company was previously in the spotlight for its "Vincentian video model" Sora, but hasn't made any new big moves in a while. At the event, the company's CTO Mira Murati presented a series of updates and progress related to ChatGPT to the public.

At OpenAI's launch event, they introduced two main highlights. The first was the latest release ofGPT-4o multimodal macromodelingThis model is not only powerful, but it is also very powerful compared to theGPT-4 TurboIt's faster and cheaper.

This new model is so much more responsive that when you use it in voice mode, you'll find it responds in "real time". This means you can communicate with ChatGPT as if you were talking to a real person - you can interrupt ChatGPT at any time to ask for something new or change the subject as it responds to you. No more waiting for ChatGPT to answer your last question before continuing, which will greatly enhance your interaction with ChatGPT.

OpenAI's launch also brought a second important announcement:Free users of ChatGPT now also have access to the newly released GPT-4o model!up. Previously, they could only use GPT-3.5 version, but now they can enjoy more powerful functions brought by GPT-4o, such as data analysis, image analysis, Internet search and access to the application store and other operations. This will undoubtedly attract a huge number of new users to the GPT app store and bring greater opportunities for developers.

with regards toFor paid subscribers, they will get higher message limits, which OpenAI says is at least 5 times more than free users. When free users run out of messages, ChatGPT will automatically switch back to GPT-3.5. In addition, OpenAI plans to roll out an improved voice experience for Plus users based on GPT-4o in the next month or so. Although the GPT-4o API does not currently include voice functionality, this update will allow users to experience a more comprehensive and intelligent interaction.

Finally, there is good news for Apple computer users: theyChatGPT desktop app for macOS now available!OpenAI also revealed that a Windows version of the desktop app will be available later this year, allowing users to "photograph" content on their desktop and ask ChatGPT questions directly. The app allows users to "photograph" the contents of their desktop with a simple shortcut and then ask ChatGPT questions directly, and OpenAI has revealed that a Windows version of the desktop app will be available later this year to give Windows users the same convenience.

Review of the launch event

01:00am: The countdown ends and the live stream officially begins. Mira Murati, Chief Technology Officer, takes the stage.

At 01:01, Mira Murati shares the three main points she will be discussing today:

1) She began by emphasizing why a tool that is free and widely available is critical to OpenAI.

2) She then mentioned how the company is trying to reduce the barriers to using ChatGPT and make it easy for everyone to use it. To that end, the company is releasing a desktop version of ChatGPT.

3) Finally, she announced that the company will release a new flagship model called GPT-4o, where the "o" stands for "omni-directional", meaning that the GPT-4o is multimodal and more powerful.

At 01:05, OpenAI introduced the new desktop version of ChatGPT, an app that is not only powerful, but also has a completely new design for its user interface. For starters, macOS users will be able to use the app straight away, while Windows users will also get their version later this year.

01:08 am: GPT-4o not only has the same level of intelligence as GPT-4, but also runs faster and supports text, audio, and video functions. This is a major breakthrough for OpenAI in terms of ease of use.

01:13 am: Previously, OpenAI needed to use all three models - transcription, intelligence, and speech generation - at the same time in speech mode, which led to significant delays. ButGPT-4o works naturally with images, files and sounds for a smooth experienceThe

01:17: Due to the high efficiency of GPT-4o, OpenAI can now deliver a level of intelligence close to GPT-4 to free users.

01:19: Murati re-emphasizes.GPT-4o's high efficiency makes near-GPT-4 tools accessible to all, enjoy the convenience of smart technology.

01:20 am: There are now over 1 million GPT apps in the GPT Store. With GPT-4o free and open, the developers of these apps will have a much larger user base.

　01:22 am: Free users will also be able to upload images, analyze them, and search for real-time information using the "browser" function. GPT-4o will also support a memory function, and GPT-4o will also support an API for developers to call.Compared to GPT-4 Turbo, GPT-4o is twice as fast, cheaper 50%, and 5 times higher rate limit.

1:25 p.m.: Two research directors, Mark Chen and Barret Zoph, take the stage, where they'll demonstrate a range of new features.

01.26am: Mark Chen is exchanging pleasantries with ChatGPT, and the robot is able to understand the meaning of "nervousness" from the sound of rapid panting, and instructs him to take deep breaths.

　01:28 pm: Chen said that his demo was to illustrate that in the current speech modeNo more mechanical question-and-answer sessions with the machine.The program can be carried outReal-time communicationNo more "awkward 2-3 second waits" in real-time communication. No more "awkward 2-3 second waits" in real-time communication.

　　01:30pm: During the live demo, the presenters are constantly asking ChatGPT to change its tone of voice, including exaggerated intonation, robotic anticipation, and the use of "sing-song" storytelling.

　01:35: Barret Zoph activates his webcam and asks ChatGPT to help him solve a one-equation problem in real time.

　01:37 pm: In addition to image recognition and problem solving capabilities, ChatGPT demonstrated the ability to interact with Barret in real time based on image changes.

01:40: Barret further demonstrates ChatGPT Desktop's ability to interpret code and computer desktop (a chart of temperatures) in real time.

01:42: Murati and Mark Chen demonstrate ChatGPT for real-time cross-translation from Italian to English.

　At the end of the launch, OpenAI said that today's launch was focused on free users and that these features will be gradually pushed out to all users in the coming weeks.

　　According to the announcement on OpenAI's website, GPT-4o will be rolled out to both paid Plus and Team users, as well as free users.The message limit for Plus users will be five times higher than that of free users, and the number of available messages for Team and Enterprise users will be a bit higher. When free users reach their usage limit, ChatGPT will automatically switch to GPT-3.5 in order to continue conversations.