Loading...

Transcript of OpenAI DevDay: Opening Keynote | JumpWatch

Features MCP Pricing FAQ

YouTube Transcript/OpenAI DevDay: Opening Keynote

Transcript of OpenAI DevDay: Opening Keynote

OpenAI

Key takeaways

GPT-4 Turbo supports context windows up to 128,000 tokens (about 300 pages) and is more accurate over long contexts.
JSON Mode guarantees that model responses are valid JSON, and a seed parameter enables reproducible outputs.
The model’s knowledge cutoff has been extended to April 2023 and can retrieve information from external documents via the new retrieval feature.
Pricing for GPT-4 Turbo is $0.01 per 1,000 prompt tokens and $0.03 per 1,000 completion tokens, making it roughly 2.75–3× cheaper than GPT‑4.
The Assistants API now provides persistent threads, built‑in retrieval, a code interpreter, and multi‑function calling to simplify building agent‑like experiences.
OpenAI introduced GPTs, customizable ChatGPT instances that combine instructions, extended knowledge, and actions, to be distributed through a forthcoming GPT Store with revenue sharing.

What this video covers

Turn YouTube into your intelligent research tool. Extract knowledge from videos instantly with AI-powered summaries and transcripts.

Follow Us

About Us

Privacy Policy
Terms of Service

Product

Features
Pricing
Chrome Extension
Web App
Blog

Free Tools

YouTube Transcript
YouTube Summary
YouTube to Text
Subtitle Downloader
Video Summarizer
Transcript Library

Support

Help Center
Contact Us
Bug Reports
Feature Requests

Made with by the JumpWatch Team

Opening and product milestones

Sam Altman recapped OpenAI’s progress, highlighting the launch of ChatGPT, GPT‑4, voice and vision capabilities, DALL‑E 3, and ChatGPT Enterprise, along with usage statistics such as 2 million developers and 100 million weekly active users.

GPT‑4 Turbo launch

The new GPT‑4 Turbo model adds a 128 k token context window, JSON Mode, reproducible outputs via seed, and upcoming logprob visibility, addressing developer feedback.

Enhanced knowledge and new modalities

Knowledge is updated to April 2023, retrieval lets the model pull in external documents, and new modalities include vision, a text‑to‑speech model, DALL‑E 3, and Whisper V3 for speech recognition.

Pricing, rate limits, and copyright shield

GPT‑4 Turbo’s pricing drops to 1¢ per 1,000 prompt tokens and 3¢ per 1,000 completion tokens, rate limits are doubled, and a copyright‑shield program protects customers from legal claims.

Customization and Microsoft partnership

OpenAI expands fine‑tuning to GPT‑4, launches a Custom Models program, and deepens its partnership with Microsoft, leveraging Azure infrastructure and integrating with GitHub Copilot.

GPTs and Assistants API

Customizable GPTs let users create purpose‑built bots with instructions, knowledge, and actions; the Assistants API adds persistent threads, retrieval, code interpreter, and advanced function calling, demonstrated with partners like Code.org, Canva, and Zapier.

Mentioned in this video

People

Sam Altman

Satya Nadella

Jessica Shieh

Romain

Alexander Graham Bell

Organizations

OpenAI

Microsoft

Azure

GitHub

Code.org

Canva

Zapier

Shopify

Discord

Snap

Coca‑Cola

Be My Eyes

Products & tools

ChatGPT

GPT-4

GPT-4 Turbo

DALL-E 3

Whisper V3

text-to-speech model

ChatGPT Enterprise

Assistants API

GPTs

Custom Models

GitHub Copilot

0:00[music]0:01-Good morning. Thank you for joining us today.0:04Please welcome to the stage, Sam Altman.0:06[music]0:13[applause]0:16-Good morning.0:18Welcome to our first-ever OpenAI DevDay.0:20We're thrilled that you're here and this energy is awesome.0:23[applause]0:28-Welcome to San Francisco.0:30San Francisco has been our home since day one.0:33The city is important to us and the tech industry in general.0:36We're looking forward to continuing to grow here.0:39We've got some great stuff to announce today,0:42but first,0:43I'd like to take a minute to talk about some of the stuff that we've done 0:46over the past year.0:48About a year ago, November 30th, we shipped ChatGPT 0:53as a "low-key research preview",0:55and that went pretty well.0:58In March,0:59we followed that up with the launch of GPT-4, still 1:02the most capable model out in the world.1:05[applause]1:10-In the last few months,1:12we launched voice and vision capabilities so that ChatGPT can now see,1:16hear, and speak.1:19[applause]1:21-There's a lot, you don't have to clap each time.1:23[laughter]1:24-More recently, we launched DALL-E 3, the world's most advanced image model.1:28You can use it of course, inside of ChatGPT.1:31For our enterprise customers,1:33we launched ChatGPT Enterprise, which offers enterprise-grade security 1:37and privacy,1:38higher speed GPT-4 access, longer context windows, a lot more.1:43Today we've got about 2 million developers building on our API 1:48for a wide variety of use cases doing amazing stuff,1:51over 92% of Fortune 500 companies building on our products,1:56and we have about a hundred million weekly active users 1:59now on ChatGPT.2:01[applause]2:05-What's incredible on that is we got there entirely 2:08through word of mouth.2:08People just find it useful and tell their friends.2:12OpenAI is the most advanced and the most widely used AI platform 2:16in the world now,2:18but numbers never tell the whole picture on something like this.2:22What's really important is how people use the products,2:24how people are using AI,2:26and so I'd like to show you a quick video.2:29-I actually wanted to write something to my dad in Tagalog.2:33I want a non-romantic way to tell my parent that I love him and I also want 2:40to tell him that he can rely on me, but in a way that still has 2:45the respect of a child-to-parent relationship 2:48that you should have in Filipino culture and in Tagalog grammar.2:52When it's translated into Tagalog, "I love you very deeply 2:55and I will be with you no matter where the path leads."2:58-I see some of the possibility, I was like,2:59"Whoa."3:00Sometimes I'm not sure about some stuff, and I feel like actually ChatGPT like,3:04hey, this is what I'm thinking about, so it kind of give it more confidence.3:07-The first thing that just blew my mind was it levels with you.3:11That's something that a lot of people struggle to do.3:15It opened my mind to just 3:18what every creative could do if they just had a person helping them out 3:23who listens.3:24-This is to represent sickling hemoglobin.3:27-You built that with ChatGPT? -ChatGPT built it with me.3:31-I started using it for daily activities like,3:34"Hey, here's a picture of my fridge.3:35Can you tell me what I'm missing?3:36Because I'm going grocery shopping, and I really need to do recipes 3:39that are following my vegan diet."3:41-As soon as we got access to Code Interpreter, I was like,3:44"Wow, this thing is awesome."3:46It could build spreadsheets.3:47It could do anything.3:49-I discovered Chatty about three months ago 3:52on my 100th birthday.3:55Chatty is very friendly, very patient,3:59very knowledgeable,4:02and very quick.4:03This has been a wonderful thing.4:05-I'm a 4.0 student, but I also have four children.4:08When I started using ChatGPT,4:10I realized I could ask ChatGPT that question.4:14Not only does it give me an answer, but it gives me an explanation.4:18Didn't need tutoring as much.4:19It gave me a life back.4:22It gave me time for my family and time for me.4:25-I have a chronic nerve thing on my whole left half of my body, I have nerve damage.4:30I had a brain surgery.4:32I have limited use of my left hand.4:34Now you can just have the integration of voice input.4:38Then the newest one where you can have the back-and-forth dialogue,4:41that's just maximum best interface for me.4:45It's here.4:47[music]4:49[applause]4:57-We love hearing the stories of how people are using the technology.5:01It's really why we do all of this.5:04Now, on to the new stuff, and we have got a lot.5:07[audience cheers]5:10-First,5:11we're going to talk about a bunch of improvements we've made,5:14and then we'll talk about where we're headed next.5:17Over the last year,5:18we spent a lot of time talking to developers around the world.5:22We've heard a lot of your feedback.5:24It's really informed what we have to show you today.5:27Today, we are launching a new model, GPT-4 Turbo.5:33[applause]5:38-GPT-4 Turbo will address many of the things 5:41that you all have asked for.5:43Let's go through what's new.5:45We've got six major things to talk about for this part.5:48Number one, context length.5:51A lot of people have tasks that require a much longer context length.5:56GPT-4 supported up to 8K and in some cases up to 32K context length,6:01but we know that isn't enough for many of you and what you want to do.6:05GPT-4 Turbo, supports up to 128,000 tokens of context.6:10[applause]6:15-That's 300 pages of a standard book, 16 times longer than our 8k context.6:20In addition to a longer context length,6:23you'll notice that the model is much more accurate over a long context.6:28Number two,6:30more control.6:32We've heard loud and clear that developers need more control 6:35over the model's responses and outputs.6:37We've addressed that in a number of ways.6:41We have a new feature called JSON Mode,6:43which ensures that the model will respond with valid JSON.6:47This has been a huge developer request.6:49It'll make calling APIs much easier.6:53The model is also much better at function calling.6:55You can now call many functions at once,6:58and it'll do better at following instructions in general.7:02We're also introducing a new feature called reproducible outputs.7:05You can pass a seed parameter, and it'll make the model return 7:08consistent outputs.7:09This, of course, gives you a higher degree of control 7:11over model behavior.7:12This rolls out in beta today.7:15[applause]7:19-In the coming weeks, we'll roll out a feature to let you view 7:22logprobs in the API.7:25[applause]7:27-All right. Number three, better world knowledge.7:31You want these models to be able to access better knowledge about the world,7:34so do we.7:36We're launching retrieval in the platform.7:38You can bring knowledge from outside documents or databases 7:41into whatever you're building.7:43We're also updating the knowledge cutoff.7:45We are just as annoyed as all of you, probably more that GPT-4's knowledge 7:49about the world ended in 2021.7:51We will try to never let it get that out of date again.7:54GPT-4 Turbo has knowledge about the world up to April of 2023,7:59and we will continue to improve that over time.8:03Number four,8:05new modalities.8:07Surprising no one,8:08DALL-E 3,8:10GPT-4 Turbo with vision,8:13and the new text-to-speech model are all going into the API today.8:17[applause]8:23-We have a handful of customers that have just started using DALL-E 3 8:27to programmatically generate images and designs.8:31Today, Coke is launching a campaign that lets its customers 8:34generate Diwali cards using DALL-E 3,8:36and of course, our safety systems help developers protect 8:39their applications against misuse.8:41Those tools are available in the API.8:44GPT-4 Turbo can now accept images as inputs via the API,8:48can generate captions, classifications, and analysis.8:52For example,8:53Be My Eyes uses this technology to help people who are blind or have low vision 8:58with their daily tasks like identifying products in front of them.9:04With our new text-to-speech model,9:06you'll be able to generate incredibly natural-sounding audio 9:10from text in the API with six preset voices to choose from.9:14I'll play an example.9:16-Did you know that Alexander Graham Bell, the eminent inventor,9:19was enchanted by the world of sounds.9:21His ingenious mind led to the creation of the graphophone,9:25which etches sounds onto wax, making voices whisper through time.9:30-This is much more natural than anything else we've heard out there.9:33Voice can make apps more natural to interact with and more accessible.9:38It also unlocks a lot of use cases like language learning,9:41and voice assistance.9:43Speaking of new modalities,9:45we're also releasing the next version 9:47of our open-source speech recognition model,9:49Whisper V3 today, and it'll be coming soon to the API.9:53It features improved performance across many languages,9:56and we think you're really going to like it.9:58Number five, customization.10:01Fine-tuning has been working really well for GPT-3.5 since we launched it 10:06a few months ago.10:07Starting today,10:08we're going to expand that to the 16K version of the model.10:12Also, starting today,10:14we're inviting active fine-tuning users to apply for the GPT-4 fine-tuning,10:18experimental access program.10:21The fine-tuning API is great for adapting our models to achieve better performance 10:25in a wide variety of applications with a relatively small amount of data,10:29but you may want a model to learn a completely new knowledge domain,10:33or to use a lot of proprietary data.10:36Today we're launching a new program called Custom Models.10:40With Custom Models,10:41our researchers will work closely with a company 10:44to help them make a great custom model, especially for them,10:48and their use case using our tools.10:50This includes modifying every step of the model training process,10:54doing additional domain-specific pre-training,10:56a custom RL post-training process tailored for specific domain, and whatever else.11:02We won't be able to do this with many companies to start.11:05It'll take a lot of work, and in the interest of expectations,11:07at least initially, it won't be cheap,11:09but if you're excited to push things as far as they can currently go.11:12Please get in touch with us,11:14and we think we can do something pretty great.11:17Number six, higher rate limits.11:20We're doubling the tokens per minute 11:22for all of our established GPT-4 customers,11:24so it's easier to do more.11:26You'll be able to request changes to further rate limits and quotas directly 11:30in your API account settings.11:32In addition to these rate limits,11:34it's important to do everything we can do to make you successful building 11:39on our platform.11:41We're introducing copyright shield.11:44Copyright shield means that we will step in and defend 11:46our customers 11:47and pay the costs incurred, if you face legal claims 11:50or on copyright infringement, and this applies both 11:53to ChatGPT Enterprise and the API.11:57Let me be clear, this is a good time to remind 11:59people do not train on data from the API or ChatGPT Enterprise ever.12:06All right.12:08There's actually one more developer request 12:10that's been even bigger than all of these and so I'd like to talk about that now 12:16and that's pricing.12:17[laughter]12:20-GPT-4 Turbo 12:22is the industry-leading model.12:24It delivers a lot of improvements that we just covered 12:27and it's a smarter model than GPT-4.12:32We've heard from developers that there are a lot of things that they want to build,12:35but GPT-4 just costs too much.12:38They've told us that if we could decrease the cost by 20%, 25%, that would be great.12:43A huge leap forward.12:46I'm super excited to announce that we worked really hard on this 12:49and GPT-4 Turbo,12:51a better model,12:52is considerably cheaper than GPT-4 by a factor of 3x for prompt tokens.12:58[applause]13:05-And 2x for completion tokens starting today.13:09[applause]13:12-The new pricing is 1¢ per 1,000 prompt tokens 13:15and 3¢ per 1,000 completion tokens.13:18For most customers,13:19that will lead to a blended rate more than 2.75 times cheaper to use 13:23for GPT-4 Turbo than GPT-4.13:26We worked super hard to make this happen.13:27We hope you're as excited about it as we are.13:30[applause]13:35-We decided to prioritize price first because we had to choose one or the other,13:39but we're going to work on speed next.13:41We know that speed is important too.13:43Soon you will notice GPT-4 Turbo becoming a lot faster.13:48We're also decreasing the cost of GPT-3.5 Turbo 16K.13:53Also, input tokens are 3x less and output tokens are 2x less.13:57Which means that GPT-3.5 16K is now cheaper 14:01than the previous GPT-3.5 4K model.14:06Running a fine-tuned GPT-3.5 Turbo 16K version 14:09is also cheaper than the old fine-tuned 4K version.14:13Okay, so we just covered a lot about the model itself.14:16We hope that these changes address your feedback.14:19We're really excited to bring all of these improvements 14:21to everybody now.14:24In all of this,14:25we're lucky to have a partner who is instrumental in making it happen.14:30I'd like to bring out a special guest, Satya Nadella, the CEO of Microsoft.14:34[audience cheers]14:37[music]14:39-Good to see you. -Thank you so much.14:41Thank you.14:42-Satya, thanks so much for coming here.14:45-It's fantastic to be here and Sam, congrats.14:48I'm really looking forward to Turbo and everything else that you have coming.14:52It's been just fantastic partnering with you guys.14:54-Awesome. Two questions.14:55I won't take too much of your time.14:56How is Microsoft thinking about the partnership currently?14:59-First-15:00[laughter]15:03--we love you guys. [laughter]15:05-Look, it's been fantastic for us.15:09In fact, I remember the first time I think you reached out 15:11and said, "Hey, do you have some Azure credits?"15:13We've come a long way from there.15:15-Thank you for those. That was great.15:18-You guys have built something magical.15:20Quite frankly, there are two things for us when it comes to the partnership.15:23The first is these workloads.15:25Even when I was listening backstage to how you're describing what's coming,15:28even, it's just so different and new.15:30I've been in this infrastructure business for three decades.15:33-No one has ever seen infrastructure like this.15:35-The workload, the pattern of the workload,15:39these training jobs are so synchronous and so large, and so data parallel.15:45The first thing that we have been doing is building in partnership with you,15:48the system, all the way from thinking from power to the DC to the rack,15:53to the accelerators, to the network.15:56Just really the shape of Azure is drastically changed 16:01and is changing rapidly in support of these models 16:04that you're building.16:05Our job, number one, is to build the best system 16:09so that you can build the best models 16:10and then make that all available to developers.16:13The other thing is we ourselves are our developers.16:16We're building products.16:17In fact, my own conviction of this entire generation 16:21of foundation models completely changed the first time I saw GitHub Copilot 16:25on GPT.16:28We want to build our GitHub Copilot all as developers on top of OpenAI APIs.16:36We are very, very committed to that.16:37What does that mean to developers?16:39Look, I always think of Microsoft as a platform company,16:43a developer company, and a partner company.16:45For example, we want to make GitHub Copilot available,16:50the Enterprise edition available to all the attendees here 16:53so that they can try it out.16:54That's awesome. We are very excited about that.16:57[applause]17:00-You can count on us to build the best infrastructure in Azure 17:05with your API support 17:07and bring it to all of you.17:09Even things like the Azure marketplace.17:11For developers who are building products out here 17:12to get to market rapidly.17:14That's really our intent here.17:17-Great. How do you think about the future, future of the partnership,17:20or future of AI, or whatever?17:23Anything you want 17:24-There are a couple of things for me that I think are going to be very,17:29very key for us.17:30One is I just described how the systems that are needed 17:36as you aggressively push forward on your roadmap 17:41requires us to be on the top of our game and we intend fully to commit 17:45ourselves deeply to making sure 17:47you all as builders of these foundation models 17:51have not only the best systems for training and inference,17:55but the most compute, so that you can keep pushing-17:57-We appreciate that.17:58--forward on the frontiers because I think that's the way 18:01we are going to make progress.18:02The second thing I think both of us care about, in fact,18:05quite frankly, the thing that excited both sides to come together is 18:09your mission and our mission.18:10Our mission is to empower every person and every organization on the planet 18:14to achieve more.18:15To me, ultimately AI is only going 18:17to be useful if it truly does empower.18:19I saw the video you played early.18:21That was fantastic to hear those voices describe what AI meant for them 18:27and what they were able to achieve.18:29Ultimately, it's about being able to get the benefits 18:31of AI broadly disseminated to everyone,18:34I think is going to be the goal for us.18:36Then the last thing is of course, we are very grounded 18:38in the fact that safety matters,18:39and safety is not something that you'd care about later,18:42but it's something we do shift left on and we are very,18:44very focused on that with you all.18:46-Great. Well, I think we have the best partnership in tech.18:48I'm excited for us to build AGI together.18:50-Oh, I'm really excited. Have a fantastic [crosstalk].18:51-Thank you very much for coming.18:52-Thank you so much.18:53-See you.18:55[applause]19:03-We have shared a lot of great updates for developers already and we got 19:07a lot more to come,19:08but even though this is developer conference,19:10we can't resist making some improvements to ChatGPT.19:15A small one, ChatGPT now uses GPT-4 Turbo with all the latest improvements,19:20including the latest knowledge cutoff, which will continue to update.19:23That's all live today.19:25It can now browse the web when it needs to, write and run code,19:28analyze data, take and generate images,19:31and much more.19:32We heard your feedback, that model picker, extremely annoying,19:34that is gone starting today.19:36You will not have to click around the dropdown menu.19:38All of this will just work together.19:41Yes.19:42[applause]19:47-ChatGPT will just know what to use and when you need it,19:51but that's not the main thing.19:54Neither was price actually the main developer request.19:58There was one that was even bigger than that.20:01I want to talk about where we're headed and the main thing we're here to talk 20:03about today.20:05We believe 20:07that if you give people better tools, they will do amazing things.20:10We know that people want AI that is smarter, more personal,20:13more customizable, can do more on your behalf.20:16Eventually, you'll just ask the computer for what you need 20:20and it'll do all of these tasks for you.20:23These capabilities are often talked in the AI field about as "agents."20:28The upsides of this are going to be tremendous.20:31At OpenAI, we really believe that gradual iterative deployment is 20:36the best way to address the safety issues, the safety challenges with AI.20:40We think it's especially important to move carefully 20:42towards this future of agents.20:44It's going to require a lot of technical work 20:47and a lot of thoughtful consideration by society.20:50Today,20:52we're taking our first small step that moves us towards this future.20:57We're thrilled to introduce GPTs.21:01GPTs are tailored versions of ChatGPT for a specific purpose.21:07You can build a GPT,21:09a customized version of ChatGPT for almost anything 21:12with instructions,21:13expanded knowledge,21:14and actions,21:16and then you can publish it for others to use.21:19Because they combine instructions, expanded knowledge, and actions,21:23they can be more helpful to you.21:25They can work better in many contexts, and they can give you better control.21:29They'll make it easier for you to accomplish all sorts of tasks 21:32or just have more fun 21:34and you'll be able to use them right within ChatGPT.21:37You can in effect program a GPT with language just by talking to it.21:42It's easy to customize the behavior so that it fits what you want.21:46This makes building them very accessible 21:48and it gives agency to everyone.21:51We're going to show you what GPTs are,21:53how to use them, how to build them,21:56and then we're going to talk about how they'll be distributed 21:58and discovered.22:00After that for developers, we're going to show you how to build 22:02these agent-like experiences into your own apps.22:05First,22:07let's look at a few examples.22:09Our partners at Code.org are working hard to expand computer science in schools.22:15They've got a curriculum that is used by tens of millions of students worldwide.22:19Code.org, crafted Lesson Planner GPT, to help teachers provide 22:24a more engaging experience for middle schoolers.22:27If a teacher asks it to explain four loops in a creative way,22:30it does just that.22:32In this case,22:33it'll do it in terms of a video game character 22:35repeatedly picking up coins.22:37Super easy to understand for an 8th-grader.22:40As you can see, this GPT brings together Code.org's,22:43extensive curriculum and expertise, and lets teachers adapt it to their needs 22:47quickly and easily.22:49Next,22:51Canva has built a GPT 22:53that lets you start designing by describing what you want 22:55in natural language.22:57If you say, "Make a poster for a DevDay reception this afternoon,23:01this evening," and you give it some details,23:04it'll generate a few options to start with by hitting Canva's APIs.23:07Now, this concept may be familiar to some of you.23:10We've evolved our plugins to be custom actions for GPTs.23:14You can keep chatting with this to see different iterations,23:17and when you see one you like, you can click through to Canva 23:20for the full design experience.23:24Now we'd like to show you a GPT Live.23:27Zapier has built a GPT that lets you perform actions 23:31across 6,000 applications to unlock all kinds of integration possibilities.23:36I'd like to introduce Jessica, one of our solutions architects,23:39who is going to drive this demo.23:40Welcome Jessica.23:42[applause] -Thank you, Sam.23:44Hello everyone.23:46Thank you all.23:49Thank you all for being here.23:51My name is Jessica Shieh.23:52I work with partners and customers to bring their product alive.23:55Today I can't wait to show you how hard we've been working on this,23:59so let's get started.24:01To start where your GPT will live is on this upper left corner.24:05I'm going to start with clicking on the Zapier AI actions 24:10and on the right-hand side you can see that's my calendar for today.24:13It's quite a day ever.24:15I've already used this before, so it's actually already connected 24:18to my calendar.24:19To start, I can ask,24:22"What's on my schedule for today?"24:24We build GPTs with security in mind.24:26Before it performs any action or share data,24:30it will ask for your permission.24:32Right here, I'm going to say allowed.24:36GPT is designed to take in your instructions, make the decision 24:41on which capability to call to perform that action,24:43and then execute that for you.24:45You can see right here, it's already connected to my calendar.24:49It pulls into my information and then I've also prompted it to identify 24:54conflicts on my calendar.24:56You can see right here it actually was able to identify that.25:01It looks like I have something coming up.25:04What if I want to let Sam know that I have to leave early?25:06Right here I say, "Let Sam know I got to go.25:11Chasing GPUs."25:15With that, I'm going to swap to my conversation with Sam 25:21and then I'm going to say, "Yes, please run that."25:26Sam,25:27did you get that?25:29-I did.25:31-Awesome.25:32[applause]25:36-This is only a glimpse of what is possible and I cannot wait to see 25:40what you all will build.25:41Thank you. Back to you, Sam.25:43[applause]25:51-Thank you, Jessica.25:52Those are three great examples.25:54In addition to these,25:56there are many more kinds of GPTs that people are creating and many,25:59many more that will be created soon.26:02We know that many people who want to build a GPT don't know how to code.26:07We've made it so that you can program a GPT just by having a conversation.26:12We believe that natural language is going to be a big part of how people use 26:15computers in the future and we think this is an interesting early example.26:19I'd like to show you how to build one.26:25All right. I want to create a GPT 26:28that helps give founders and developers advice 26:30when starting new projects.26:32I'm going to go to create a GPT here,26:36and this drops me into the GPT builder.26:40I worked with founders for years at YC and still whenever I meet developers,26:43the questions I get are always about, "How do I think about a business idea?26:47Can you give me some advice?"26:49I'm going to see if I can build a GPT to help with that.26:52To start, GPT builder asks me what I want to make,26:55and I'm going to say, "I want to help startup founders think.27:00through their business ideas 27:04and get advice.27:07After the founder has gotten some advice,27:13grill them 27:15on why they are not growing faster."27:18[laughter]27:20-All right.27:22To start off, I just tell the GPT little bit 27:23about what I want here.27:25It's going to go off and start thinking about that,27:27and it's going to write some detailed instructions for the GPT.27:31It's also going to,27:32let's see, ask me about a name.27:34How do I feel about Startup Mentor?27:36That's fine.27:37"That's good."27:39If I didn't like the name, of course, I could call it something else,27:41but it's going to try to have this conversation with me and start there.27:45You can see here on the right, in the preview mode 27:49that it's already starting to fill out the GPT.27:53Where it says what it does, it has some ideas of additional questions 27:57that I could ask.27:58[chuckles]28:01It just generated a candidate.28:03Of course, I could regenerate that or change it, but I like that.28:07I'll say "That's great."28:13You see now that the GPT is being built out a little bit more 28:15as we go.28:17Now, what I want this to do,28:19how it can interact with users, I could talk about style here.28:22What I'm going to say is,28:25"I am going to upload transcripts of some lectures 28:31about startups I have given,28:33please give 28:35advice based off of those."28:38All right.28:40Now,28:42it's going to go figure out how to do that.28:43I would like to show you the configure tab.28:45You can see some of the things that were built out here as we were going 28:49by the builder itself.28:51You can see that there's capabilities here that I can enable.28:53I could add custom actions.28:55These are all fine to leave.28:57I'm going to upload a file.29:00Here is a lecture that I picked that I gave with some startup advice,29:05and I'm going to add that here.29:07In terms of these questions,29:09this is a dumb one.29:11The rest of those are reasonable, and very much things founders often ask.29:16I'm going to add one more thing to the instructions here,29:19which is be concise and constructive with feedback.29:25All right.29:27Again, if we had more time, I'd show you a bunch of other things.29:29This is 29:31a decent start.29:32Now,29:34we can try it out over on this preview tab.29:36I will say,29:38what's a common question?29:44"What are three things to look for when hiring employees 29:47at an early-stage startup?"29:53Now, it's going to look at that document I uploaded.29:56It'll also have of course all of the background knowledge of GPT-4.30:03That's pretty good. Those are three things that I definitely have said many times.30:07Now, we could go on and it would start following 30:09the other instructions and grill me on why I'm not growing faster,30:13but in the interest of time,30:14I'm going to skip that.30:15I'm going to publish this only to me for now.30:18I can work on it later.30:20I can add more content, I can add a few actions 30:22that I think would be useful,30:24and then I can share it publicly.30:26That's what it looks like to create a GPT 30:29[applause] -Thank you.30:36By the way,30:38I always wanted to do that after all of the YC office hours,30:40I always thought, "Man, someday I'll be able 30:42to make a bot that will do this and that'll be awesome."30:44[laughter]30:46-With GPTs, we're letting people easily share and discover all the fun ways 30:51that they use ChatGPT with the world.30:55You can make private GPT like I just did,30:58or you can share your creations publicly with a link for anyone to use,31:03or if you're on ChatGPT Enterprise, you can make GPTs just for your company.31:10Later this month we're going to launch the GPT store.31:17Thank you.31:18I appreciate that.31:19[applause]31:25-You can list a GPT there and we'll be able to feature the best 31:28and the most popular GPT.31:30Of course, we'll make sure that GPTs in the store follow our policies 31:34before they're accessible.31:37Revenue sharing is important to us.31:40We're going to pay people who build the most useful and the most used GPT 31:44a portion of our revenue.31:46We're excited to foster a vibrant ecosystem with the GPT store,31:50just from what we've been building ourselves over the weekend.31:52We're confident there's going to be a lot of great stuff.31:54We're excited to share more information soon.31:58Those are GPTs 31:59and we can't wait to see what you'll build.32:02This is a developer conference, and the coolest thing about this 32:05is that we're bringing the same concept to the API.32:09[applause]32:15Many of you have already been building agent-like experiences on the API,32:20for example,32:21Shopify's Sidekick,32:23which lets you take actions on the platform.32:25Discord's Clyde,32:26lets Discord moderators create custom personalities for, and Snaps My AI,32:32a customized chatbot that can be added to group chats and make recommendations.32:36These experiences are great,32:38but they have been hard to build.32:40Sometimes taking months, teams of dozens of engineers,32:44there's a lot to handle to make this custom assistant experience.32:49Today, we're making that a lot easier with our new Assistants API.32:54[applause]32:58-The Assistants API includes persistent threads,33:01so they don't have to figure out how to deal 33:02with long conversation history,33:04built-in retrieval,33:07code interpreter, a working Python interpreter 33:09in a sandbox environment,33:12and of course the improved function calling,33:14that we talked about earlier.33:17We'd like to show you a demo of how this works.33:19Here is Romain, our head of developer experience.33:22Welcome, Romain.33:23[music] [applause]33:25-Thank you, Sam.33:27Good morning.33:29Wow.33:30It's fantastic to see you all here.33:33It's been so inspiring to see so many of you infusing AI 33:37into your apps.33:38Today, we're launching new modalities in the API, but we are also very excited 33:43to improve the developer experience for you all to build 33:46assistive agents.33:48Let's dive right in.33:50Imagine I'm building $1,33:52travel app for global explorers, and this is the landing page.33:56I've actually used GPT-4 to come up with these destination ideas.33:59For those of you with a keen eye, these illustrations 34:02are generated programmatically using the new DALL-E 3 API available 34:06to all of you today.34:07It's pretty remarkable.34:11Let's enhance this app by adding a very simple assistant to it.34:15This is the screen.34:16We're going to come back to it in a second.34:17First, I'm going to switch over to the new assistant's playground.34:21Creating an assistant is easy, you just give it a name,34:24some initial instructions, a model.34:26In this case, I'll pick GPT-4 Turbo.34:29Here I'll also go ahead and select some tools.34:31I'll turn on Code Interpreter and retrieval and save.34:35That's it. Our assistant is ready to go.34:39Next, I can integrate with two new primitives 34:41of this Assistants API,34:43threads and messages.34:45Let's take a quick look at the code.34:48The process here is very simple.34:50For each new user, I will create a new thread.34:54As these users engage with their assistant,34:56I will add their messages to the threads.34:59Very simple.35:00Then I can simply run the assistant at any time to stream the responses 35:04back to the app.35:06We can return to the app and try that in action.35:10If I say, "Hey, let's go to Paris."35:15All right.35:16That's it. With just a few lines of code, users can now have 35:19a very specialized assistant right inside the app.35:24I'd like to highlight one of my favorite features here,35:26function calling.35:27If you have not used it yet, function calling is really powerful.35:31As Sam mentioned, we are taking it a step further today.35:34It now guarantees the JSON output with no added latency,35:38and you can invoke multiple functions at once for the first time.35:43Here, if I carry on and say, "Hey, what are the top 10 things to do?"35:49I'm going to have the assistant respond to that again.35:53Here, what's interesting is that the assistant knows about functions,35:56including those to annotate the map that you see on the right.36:00Now, all of these pins are dropping in real-time here.36:04Yes, it's pretty cool.36:06[applause]36:09-That integration allows our natural language interface 36:13to interact fluidly with components and features of our app.36:16It truly showcases now the harmony you can build between AI 36:21and UI where the assistant is actually taking action.36:25Let's talk about retrieval.36:27Retrieval is about giving our assistant more knowledge 36:30beyond these immediate user messages.36:33In fact, I got inspired and I already booked my tickets to Paris.36:37I'm just going to drag and drop here this PDF.36:40While it's uploading, I can just sneak peek at it.36:43Very typical United Flight ticket.36:46Behind the scene here, what's happening is that retrieval 36:49is reading these files,36:51and boom, the information about this PDF appeared on the screen.36:55[applause]36:57-This is, of course, a very tiny PDF, but Assistants 37:01can parse long-form documents from extensive text 37:04to intricate product specs depending on what you're building.37:07In fact, I also booked an Airbnb, so I'm just going to drag that 37:09over to the conversation as well.37:12By the way, we've heard from so many of you developers how hard 37:15that is to build yourself.37:17You typically need to compute your own biddings,37:19you need to set up chunking algorithm.37:21Now all of that is taken care of.37:24There's more than retrieval with every API call,37:27you usually need to resend the entire conversation history,37:31which means setting up a key-value store, that means handling the context windows,37:35serializing messages, and so forth.37:37That complexity now completely goes away with this new stateful API.37:43Just because OpenAI is managing this API, does not mean it's a black box.37:47In fact, you can see the steps that the tools are taking 37:49right inside your developer dashboard.37:52Here, if I go ahead and click on threads,37:56this is the thread I believe we're currently working on and see,37:59these are all the steps, including the functions 38:02being called with the right parameters, and the PDFs I've just uploaded.38:08Let's move on to a new capability that many of you have been requesting 38:11for a while.38:12Code Interpreter is now available today in the API as well,38:16that gives the AI the ability to write and execute code on the fly,38:20but even generate files.38:22Let's see that in action.38:24If I say here, "Hey, we'll be four friends staying 38:29at this Airbnb,38:33what's my share of it plus my flights?"38:40All right.38:42Now, here,38:44what's happening is that Code interpreter noticed that it should write some code 38:48to answer this query.38:49Now it's computing the number of days in Paris, number of friends.38:53It's also doing some exchange rate calculation behind 38:55the scene to get the sensor for us.38:58Not the most complex math, but you get the picture.39:01Imagine you're building a very complex finance app 39:04that's crunching countless numbers, plotting charts,39:07so really any task that you'd normally tackle with code,39:10then Code Interpreter will work great for you.39:13All right. I think my trip to Paris is solid.39:16To recap here, we've just seen how you can quickly create an assistant 39:20that manages state for your user conversations,39:22leverages external tools like knowledge and retrieval and Code Interpreter,39:26and finally invokes your own functions to make things happen 39:32but there's one more thing I wanted to show you to really open up 39:35the possibilities using function calling combined with our new modalities 39:39that we're launching today.39:41While working on DevDay, I built a small custom assistant 39:45that knows everything about this event,39:47but instead of having a chat interface 39:49while running around all day today,39:51I thought, why not use voice instead?39:54Let's bring my phone up on screen here so you can see it on the right.39:58Awesome.39:59On the right, you can see a very simple Swift app that takes 40:01microphone input.40:04On the left, I'm actually going to bring up my terminal log 40:06so you can see what's happening behind the scenes.40:09Let's give it a shot.40:12Hey there, I'm on the keynote stage right now.40:14Can you greet our attendees here at Dev Day?40:21-Hey everyone, welcome to DevDay.40:23It's awesome to have you all here.40:25Let's make it an incredible day.40:27[applause]40:32-Isn't that impressive?40:33You have six unique and rich voices to choose from in the API,40:37each speaking multiple languages,40:39so you can really find the perfect fit for your app.40:42On my laptop here on the left,40:44you can see the logs of what's happening 40:45behind the scenes, too.40:46I'm using Whisper to convert the voice inputs into text,40:50an assistant with GPT-4 Turbo, and finally,40:52the new TTS API to make it speak.40:56Thanks to function calling, things get even more interesting 40:59when the assistant can connect to the internet and take 41:02real actions for users.41:04Let's do something even more exciting here together.41:07How about this?41:10Hey, Assistant, can you randomly select five DevDay attendees here 41:15and give them $500 in OpenAI credits?41:18[laughter]41:20-Yes, checking the list of attendees.41:25[laughter]41:27-Done. I picked five DevDay attendees and added $500 of API credits 41:30to their account.41:32Congrats to Christine M,41:33Jonathan C, Steven G, Luis K, and Suraj S.41:38-All right, if you recognize yourself, awesome.41:40Congrats.41:43That's it.41:44A quick overview today of the new Assistants API 41:46combined with some of the new tools and modalities that we launched,41:50all starting with the simplicity of a rich text 41:52or voice conversation for you end users.41:56We really can't wait to see what you build,41:58and congrats to our lucky winners.42:00Actually,42:01you know what?42:02you're all part of this amazing OpenAI community here 42:04so I'm just going to talk to my assistant 42:06one last time before I step off the stage.42:10Hey Assistant, can you actually give everyone here in the audience $500 42:15in OpenAI credits?42:17-Sounds great.42:18Let me go through everyone.42:21[applause]42:26-All right,42:28that function will keep running,42:30but I've run out of time.42:32Thank you so much, everyone.42:33Have a great day. Back to you, Sam.42:44-Pretty cool, huh?42:46[audience cheers]42:49-All right, so that Assistants API goes into beta today,42:52and we are super excited to see what you all do with it,42:55anybody can enable it.42:57Over time,42:59GPTs and Assistants are precursors to agents 43:02are going to be able to do much much more.43:05They'll gradually be able to plan 43:07and to perform more complex actions on your behalf.43:11As I mentioned before,43:12we really believe in the importance of gradual iterative deployment.43:16We believe it's important for people to start building with and using 43:19these agents now to get a feel for what the world is going to be like,43:23as they become more capable.43:25As we've always done,43:26we'll continue to update our systems based off of your feedback.43:32We're super excited that we got to share all of this with you today.43:35We introduced GPTs,43:37custom versions of GPT that combine instructions, extended 43:42knowledge and actions.43:44We launched the Assistants API 43:45to make it easier to build assistive experiences with your own apps.43:49These are your first steps towards AI agents and we'll be increasing 43:53their capabilities over time.43:56We introduced a new GPT-4 Turbo model that delivers improved function calling,44:01knowledge, lowered pricing, new modalities, and more.44:05We're deepening our partnership with Microsoft.44:09In closing,44:10I wanted to take a minute to thank the team that creates all of this.44:13OpenAI has got remarkable talent density, but still, it takes 44:17a huge amount of hard work and coordination to make all this happen.44:21I truly believe that I've got the best colleagues in the world.44:23I feel incredibly grateful to get to work with them.44:27We do all of this because we believe that AI is going to be 44:30a technological and societal revolution.44:33It'll change the world in many ways 44:35and we're happy to get to work on something that will empower all of you 44:38to build so much for all of us.44:41We talked about earlier how if you give people better tools,44:44they can change the world.44:46We believe that AI will be about individual empowerment and agency 44:50at a scale that we've never seen before and that will elevate humanity 44:53to a scale that we've never seen before either.44:55We'll be able to do more, to create more,44:58and to have more.45:00As intelligence gets integrated everywhere,45:02we will all have superpowers on demand.45:05We're excited to see what you all will do with this technology 45:08and to discover the new future that we're all going 45:10to architect together.45:12We hope that you'll come back next year.45:14What we launched today is going to look very quaint relative 45:17to what we're busy creating for you know.45:19Thank you for all that you do.45:21Thank you for coming here today.45:22[applause]45:28[music]

7,046 words · 919 lines

Do more with this video

Preview

Unlock AI summaries, chat, and study tools — free with the JumpWatch extension, right on the YouTube page.

AI Summary

Key points & TL;DR

Chat with video

Ask anything

Key quotes

The best lines

Study guide

Learn it fast

Quiz

Test yourself

Flashcards

Memorize it

Mindmap

See the structure

Sentiment & topics

Tags & tone

Add to Chrome — Free

No sign-up · Works on any YouTube video

Transcribe another video

Code Interpreter

Retrieval

JSON Mode

TTS model

More transcripts

Games that Don't Fake the Space

Games that Don't Fake the Space

Why Do Horror Games Sound So Beautiful?

Why Do Horror Games Sound So Beautiful?

P.T. and MK Ultra Mind Control - Monsters of the Week

P.T. and MK Ultra Mind Control - Monsters of the Week

Solaris (And How it Influenced Silent Hill 2) | Monsters of the Week

Solaris (And How it Influenced Silent Hill 2) | Monsters of the Week

SILENT HILL f is Brilliant (And Here's Why)

SILENT HILL f is Brilliant (And Here's Why)

The Kojima-P.T. Conspiracy (Silent Hills|Konami) - Monsters of the Week Special (feat. Fungo)

The Kojima-P.T. Conspiracy (Silent Hills|Konami) - Monsters of the Week Special (feat. Fungo)

Stephen King in Silent Hill - Monsters of the Week - RagnarRox

Stephen King in Silent Hill - Monsters of the Week - RagnarRox

Silent Hill 4: The Room or (Horror ≠ Fun) ➣ RagnarRox

Silent Hill 4: The Room or (Horror ≠ Fun) ➣ RagnarRox