OpenAI, the research organization behind the popular GPT-3 language model, has announced a new update to its ChatGPT project, which aims to create a conversational agent that can chat with humans on various topics. The update adds voice and image capabilities to ChatGPT, allowing it to generate speech and visual responses based on the user’s input.
Contents
How Does ChatGPT Work?
ChatGPT is based on GPT-3, a deep neural network that can generate natural language texts on almost any topic, given some input or context. ChatGPT uses a special version of GPT-3, called DialoGPT, which is fine-tuned on a large corpus of human conversations from Reddit. This enables ChatGPT to learn how to engage in dialogues with humans, using natural language and common sense.
ChatGPT also uses a technique called retrieval-augmented generation (RAG), which allows it to access external knowledge sources, such as Wikipedia or Bing, to enrich its responses with relevant information. For example, if the user asks ChatGPT about a historical event, it can use RAG to search for facts and details from Wikipedia and incorporate them into its reply.
What’s New in ChatGPT?
The latest update to ChatGPT adds two new features: voice and image capabilities. These features allow ChatGPT to generate not only text but also speech and visual responses, depending on the user’s input and preference.
Now, ChatGPT is able to see, hear, and speak. Voice interactions with ChatGPT (on iOS and Android) and the ability to attach photos to discussions (across all platforms) will be made available to Plus customers over the course of the next two weeks:
ChatGPT can now see, hear, and speak. Rolling out over next two weeks, Plus users will be able to have voice conversations with ChatGPT (iOS & Android) and to include images in conversations (all platforms). https://t.co/uNZjgbR5Bm pic.twitter.com/paG0hMshXb
— OpenAI (@OpenAI) September 25, 2023
Voice capability
The voice capability of ChatGPT is powered by WaveNet, a neural network that can synthesize realistic human speech from text. WaveNet is trained on a large dataset of speech samples from different speakers, languages, and accents. This enables ChatGPT to produce speech responses that match the tone, style, and emotion of the text.
The voice capability of ChatGPT can be activated by the user by typing /voice before their message. For example, if the user types /voice Hello, how are you?, ChatGPT will reply with a speech response that says “Hello, I’m fine, thank you for asking”.
The user can also specify the language and accent of the speech response by adding a two-letter code after /voice. For example, /voice fr Bonjour, comment allez-vous? will generate a speech response in French with a French accent.
Take a look at some additional recently published content from us:
- Intel Core 13600K vs. 13700K: Performance, Specs, and Price
- Nvidia reveals new technology to speed up AI, launches new supercomputer
Image capability
The image capability of ChatGPT is powered by DALL-E, a neural network that can create images from text descriptions. DALL-E is trained on a large dataset of text-image pairs from the web, covering a wide range of topics and styles. This enables ChatGPT to generate images that match the content and context of the text.
The image capability of ChatGPT can be activated by the user by typing /image before their message. For example, if the user types /image a cat wearing a hat, ChatGPT will reply with an image of a cat wearing a hat. The user can also specify the style and quality of the image by adding some keywords after /image. For example, /image a cat wearing a hat cartoon high will generate a high-quality cartoon image of a cat wearing a hat.
Why is ChatGPT Important?
ChatGPT is an important project for OpenAI because it showcases the potential of GPT-3 and its extensions for creating conversational agents that can interact with humans in natural and engaging ways. By adding voice and image capabilities to ChatGPT, OpenAI aims to make the chat experience more rich and diverse, as well as more accessible and inclusive for different users.
ChatGPT is also an important project for the research community and society at large because it opens up new possibilities and challenges for natural language processing (NLP) and artificial intelligence (AI). By combining text, speech, and image generation in one system, ChatGPT poses new questions about how to evaluate, interpret, and regulate such multimodal systems. It also raises new ethical and social issues about how to ensure that such systems are fair, safe, and beneficial for all stakeholders.