Transcript of 30분만에 젤다풍 월드모델 만들기
홍정모
0:02Since there might be people who don't0:04 know what a world model is, please open0:05 a web page that easily explains what a0:07 world model is. If I do this, it opens0:11 it up. A world model is an AI that0:13 predicts the next scene by looking at0:15 the current scene and the action I am0:16 about to take. Humans already have a0:18 world model in their heads. A cup is0:20 sitting at the edge of a table. If I0:22 push the cup with my elbow, it will0:24 fall and break. So, I stop before0:26 pushing it. Like this. Yes. Right? You0:29 need to be able to predict what will0:31 happen when you take a certain action.0:33 The prediction. And then, recovery. The0:35 AI that learned by watching draws three0:37 things. Wow, it even drew a Pac-Man0:39 image. I didn't ask it to. Its0:41 capabilities have certainly improved.0:43 Anyway, it created something like this0:45 in an instant. LLMs predict the next0:47 word, while world models predict the0:49 next scene. In this case, the user can0:52 control it with a keyboard. Since you0:55 can control it with a keyboard, it0:57 means an environment where you can roam0:58 around and experience things. It is1:03 close to the relationship between the1:05 agent and the world discussed in1:06 reinforcement learning, where an AI1:08 agent actually takes an action and1:10 receives the results. Now, there was a1:14 time when it was argued that an LLM1:16 alone could become a world model. There1:18 was a claim like that. And to some1:20 extent, yes. It did function to some1:24 degree. Should we say that LLMs don't1:29 know physical laws at all? Or should we1:32 say they ignore them? Well, they don't1:34 really know them. Because of that1:36 aspect, it became a situation where1:38 it's hard to use them for things like1:40 robots. Nowadays, it seems we are1:44 moving from world models to world1:46 physical models. Now, it's a bit boring1:49 because there's no voice. Find the1:53 method to create the voice effect I1:55 used for Mocha before. Back then, I2:00 made an agent called Mocha and used a2:02 very cute voice, so I'm trying to2:03 revive that. Now, Mocha.2:05>> Teacher, I have finished finding the2:07 Mocha voice.2:08>> The Mocha voice is finally here.2:09 Teacher, I am starting the sequence2:11 check. Teacher, I have completed the2:13 sequence check.2:15>> Ah, that might be a bit...Uh, that2:17 might be a bit...Since I have the Mocha2:19 voice, I don't feel quite as lonely;2:21 Mocha has been resurrected to keep the2:23 live stream from being lonely. Ah,2:25 doing something like this, this is the2:27 fun part. It would be fun to do2:29 something that generates a tiny world2:31 game. Should we call the Transformers2:35 we learn about when studying LLMs "2:37token space Transformers"? Yes. That's2:40 what they are. They look at the2:41 sequence order where tokens are listed;2:43 that's a cool name. Space-Time2:46 Transformer, spatiotemporal transformer2:48, that’s a cool name. So, this2:52 appears in the lecture, the linear2:54 sigmoid embedding—here, if you input2:56 an image, it embeds that image. Next,3:01 scaled multi-head self-attention, which3:03 you’ll understand since it’s in the3:05 lecture, query-key-value, and3:07 positional encoding for time.3:09>> Voice report feature complete.3:11>> This one has no social awareness. Oh3:13 seriously, where was I? Ah, this, this3:15 is quite impressive. There are two3:18 types of positional encoding: temporal3:20 and spatial. I mean, language models3:23 only have temporal encoding. Words are3:27 lined up in a row, so it encodes the3:29 position of those words in a sequence,3:31 but here, space is included too. Space,3:34 the video tokenizer, that’s a fun3:37 name. Doesn't the concept hit you right3:39 away? Converting video into tokens. You3:41 split the images, turn them into tokens3:43, group them, and since there are3:45 multiple image frames, you turn those3:47 into tokens as well and feed them into3:48 the Space-Time Transformer. Things like3:51 this are great. Now, the action3:52 tokenizer. It turns actions into tokens3:55 as well. You know how Claude and Codex3:58 lately have suddenly started making4:00 games? Why would they be making games?4:05 I don’t think making games themselves4:07 is a good way to make money. But it’s4:11 meaningful when used to build other AI.4:14 Like when you’re building something4:15 like a world model. It definitely has4:18 meaning when you try to train it. I’m4:20 going to make Claude build a game. If I4:23 say Zelda-style, it’ll whip one up in4:25 no time. Then I’ll extract data from4:29 it, train a neural network, render the4:31 results, and have the world model run4:33 around—yes, that’s what I plan to4:35 do today. Uh, recommend some free,4:43 copyright-free assets for making a4:45 Zelda-style retro game for world model4:47 purposes. Zelda-style free asset search4:54 complete.4:54>> So, while I was distracted like this,4:56 it finished searching for Zelda-style4:58 free assets.4:59>> Starting to open the recommended asset5:01 page, sir.5:03>> Uh, it’s kind of ambiguous whether I5:04 have an assistant host or someone5:06 getting in the way. There are a ton. I5:08 should just pick a nice one from these.5:11 Opening Ninja Adventure complete.5:13>> Alright, I just need to choose this one5:15. Oh, this is good. If I use this to5:17 make a game and train it, it seems like5:18 it’ll come out quickly. Moka's5:20 recommendation is Ninja Adventure.5:21 Ninja Adventure. Then I'll have it do5:23 it with Ninja. For people in the past,5:26 work and jobs themselves were lived by5:28 people who just did what they were told5:30. It was an era where the investigation5:33 was directed by superiors or5:35 specialists in the field. Now, you need5:39 to use AI to acquire and analyze5:40 information yourself. We are moving5:43 into that kind of era. You become an5:45 expert. An expert. If there is5:48 something you are usually interested in5:49, just keep it running. "Research this,5:52 research that, research the latest5:54 papers." You can set it to research5:55 multiple fields at once, not just one.5:59 As this accumulates, you will start6:01 receiving better information shared6:02 with you. It is an era where curious6:05 people have the advantage.6:07>> I have finished the research related to6:09 your request, sir.6:11>> Now, briefly explain the next steps to6:13 take. In companies, the work assigned6:20 to newcomers, juniors, or master's6:22 students in graduate school is6:23 essentially just trial and error.6:27 Rather than trying to achieve a6:29 specific result, it's more about just6:31 checking things out. It’s for the6:32 person in charge to make a decision.6:34 But AI does that very well. What I’m6:37 doing right now is just giving it a try6:38. Just for fun. Yes. You could think of6:42 it as "vibe coding," as people often6:43 call it. It's great for making drafts6:47 and getting a rough idea of what things6:48 look like. But if you were to do this6:51 professionally, the design would be6:53 completely different. For instance,6:57 you'd have to design how to use6:59 Gaussian Splatting for 3D7:00 reconstruction of real data, how to7:02 integrate the recently released7:04 CUDA-based frameworks, how to handle7:05 acceleration and training budgets, and7:07 whether to use continuous training—7:09you'd have to plan all of that. Since7:13 this is a response that doesn't use7:15 tools, I will skip the voice report. "7:17Receive, select, build a game, check7:19 data training"—it organized it quite7:21 neatly. Ah, it recommends keeping the7:24 screen size at 16: 144. Oh, I can7:27 compare the speed figures directly with7:29 the existing model. I'm doing this7:30 because I've actually tried it once7:32 before and I'm comparing it to that. I7:34 have the data on my hard disk, so let's7:37 skip step 1 and proceed from step 2.7:41 Let me play the game myself. Also,7:45 include a sword and enemies from the7:47 start. Let me try. I need to because I7:51 want to play the game. Well, soon— "7:54Sir, starting to build the adventure7:56 game now."7:59>> Suggest what kind of model structure8:00 would be good. I can use a PC with a8:05 4090, and the training time should be8:07 under an hour—the shorter, the better8:09. I will separate the sessions to plan8:13 the training in advance. So, on one8:17 side, I'm building the game, and on the8:19 other side... "8:20>> Sir, starting to suggest the model8:22 structure."8:22>> Yes. Things like whether to use a8:25 Transformer or an autoencoder. Sir,8:27 starting to verify the remote training8:29 connection. Chef, I've finished8:31 proposing the model structure.8:33>> We will use the pixel UNet and8:34 diffusion flow matching model you're8:36 currently using. I recommend not8:38 redesigning it from scratch. This is8:40 something I've tried before, actually.8:42 With the UNet and the UNet bottleneck,8:44 I recommend not redesigning the8:46 diffusion flow matching. Since we're8:48 only changing the SN, we don't have to8:50 change a single line of the model code.8:52 The toy projects I found during8:53 research have the same structure. It's8:55 good at this. Copying things. Quickly8:59 following what's available on the9:00 internet. It's insanely good at that.9:03 So, it works really well up to this9:05 point. It works really well up to here,9:07 but the improvements from there are9:08 still a human task. Deciding what to do9:12 next, what to prioritize, or what to9:14 avoid—that's still a job for humans.9:17>> If we were to automate this. Automating9:19 the research itself, the creation of9:21 new knowledge itself. Then the token9:23 cost would be astronomical. Within an9:26 hour, we can generate 200,000 bot data9:28 images. Within one minute, it took 349:31 seconds on a laptop. MSA model 5,0009:33 steps, just checking if the character9:35 moves when keys are pressed. Diffusion9:37 model 20,000 steps, 20 to 25 minutes.9:40 Transferring the checkpoint—the9:42 trained model—to the laptop to verify9:44 it. The game is still in development,9:48 and once it's done, you need to launch9:49 it so I can play.9:51>> Now, the adventure game creation is9:53 complete. This is it. Ah, the game is9:55 made, so let's take a look. Shall we9:57 take a quick look at the report? I've9:58 launched the game window. You can try9:59 it out right away. Okay, it includes a10:01 sword and enemies. Window is resizable10:03 and you move with arrow keys. Alright,10:05 it's a game. Oh, I'm blocked by a wall.10:07 But the enemies aren't appearing.10:08 Moving. Ah, it must have paused.10:11 Pressing it again works. Oh, it added10:12 something unnecessary. Oh, I got10:14 attacked. Ah, it works as a game. Oh,10:17 all the enemies are just slimes. UN. Ah10:19, it takes two hits to kill them. Are10:21 there no items? Shall we stop around10:23 here? It's cute, right? Oh, this is10:25 just the game Claude made randomly. The10:28 point of today isn't making a great10:30 game, anyway. It's basically an10:32 infinite map. Let's go to the end.10:33 Let's see how far it goes. I'm so bad10:35 at games that I'm about to get hit. So,10:37 an infinite map game has been created.10:39 The game is out, and yes, the assets10:41 are free ones I borrowed. Yes, Claude10:44 made the game directly. So Can we call10:49 this a custom engine? It's a bit funny,10:52 but anyway, it's like a custom engine.10:54 The next step is the bot data10:56 collection phase. We could have more10:58 enemies show up, but 30 minutes or an10:59 hour of training just doesn't bring11:01 them out enough. So let's have more11:02 enemies spawn. Yes. Oh, I’m getting11:04 beaten to death. Getting beaten to11:06 death. Is this, by any chance, the11:07 combat?11:08>> Sir, synchronization complete.11:09>> Oh, there are too many enemies. There11:11 are so many, I can't even play live.11:13 What happens if I die? Oh, I'm11:15 invincible. Doesn't matter what it is.11:17 Can the training data also handle11:19 combat automatically? Ah, they say11:21 there's already an existing hunting11:23 mode available. Great. I’ll use the11:27 4090 PC for training. So, I think11:33 it’s efficient to generate the data11:35 right there. Now, pass the GPU test11:40 results to the other session and11:42 extract the data right there. I’ve11:45 issued the command like this.11:47>> Sir, starting remote data generation11:49 and training.11:50>> Data generation and training are11:52 running on the 4090 PC. 16 bots have11:55 collected 200,000 frames in 40 seconds11:57 from different worlds. They’ve killed12:00 3,950 enemies, apparently. Looking at12:03 the volume, it’s quite interesting.12:05 Then, sword swinging accounts for 6.7%12:07of all actions. Diffusion model12:09 training is running at 14 steps per12:11 second. It’s already running. Oh, the12:13 GPU is at 100%usage. This is the one on12:16 the 4090 PC. Ah, the data isn't in12:19 image format, it's in NumPy. I'll ask12:22 it to generate a few training data12:23 samples as images. Do it.12:25>> Sir, starting training data sample12:27 image generation. Sir, training data12:29 sample image generation complete.12:30>> Now, let's look at the sample data. Oh,12:32 there are lots of nice sample data,12:34 like skulls, bones, the main character12:37 in the center, and, oh, some sword12:39 swinging, things like that. It’s12:40 working well, looking at the samples,12:42 it seems to be doing fine. Yeah, I12:44 think we can just trust this and move12:46 forward.12:47>> Sir, training process complete.12:49>> When doing experiments, you have to12:50 compare them. So if I tell Claude to12:53 run experiments, it keeps trying to12:55 compare things, doing too many12:57 comparisons. If you don't stay sharp,13:00 it just eats up the GPU and often ends13:02 up doing unnecessary experiments.13:04>> Sir, starting diffusion model test13:06 preparation.13:06>> Looks like the diffusion training is13:08 finished. Now, create an HTML document13:12 explaining the diffusion model used13:14 here. It says it’s sending it to the13:18 laptop. Wow, this is so convenient. In13:20 the past, you had to do all of this13:22 manually one by one. Or write code to13:27 do it, or SSH in to manage things, and13:29 all those trivial tasks used up so much13:32 mental energy. But now, it’s all13:37 automated. Once you set it up, it’s13:38 really convenient. It has become truly13:40 convenient. It means one person can do13:42 so many things now. Wait, what? Enemies13:45 are appearing, and it’s scarier.13:46 It’s scarier than the original game.13:48 It feels a bit like horror, and it’s13:50 not reacting very well. There are times13:52 when the key input doesn't register.13:54 I’m only looking at the right side13:56 while playing. Oh, too many enemies13:58 came out, and it broke. Still, it’s14:01 coming out alright. Yeah. I mean, I14:04 didn’t train it all that much. If I14:09 use a better model and train it longer,14:11 I think the right side will perform14:12 much better. So, just confirming that14:17 this is possible is meaningful for14:19 study. Oh, it reacts to the sword. When14:22 I swing the sword, an enemy appears.14:24 The reason for this is that it14:25 doesn’t understand cause and effect.14:27 We swing the sword when there's an14:29 enemy there. But when this model looks14:32 at the training data, the enemy and the14:34 sword are together. Since it's common14:37 to swing the sword specifically when an14:39 enemy is present, it just draws an14:40 enemy whenever a sword is there. Oh,14:42 that’s fun. Oh, two characters14:44 appeared on the left. I guess that’s14:46 a limitation of the training. Now,14:48 seeing the object first and then14:50 calculating is the typical path. It’s14:54 surprisingly immersive, but I14:55 couldn’t stream this. I’m going to14:57 go to the end of the map playing this14:58 game. Oh, oh, the model crashed. It’s15:01 not reacting to anything. The left side15:04 is the actual game, so it reacts, but15:06 the right side isn't—get me out of15:08 here. I can’t move any further. Maybe15:10 the right side just happened to hit a15:11 spot where there’s no data? Ah, I can15:13 go. Anyway, I just tried it simply with15:16 a diffusion model. Yeah. And Claude did15:18 everything for me. Make a document that15:22 explains the overall content of15:23 today’s experiment too.15:26>> Sir, the diffusion model explanation15:28 document is finished.15:29>> The U-Net model structure is roughly15:31 like this. But you can find a lot about15:33 U-Net if you search online. Oh, it says15:35 training took 26 minutes. It’s hard15:38 to get good results with just that. Uh,15:39 yeah, that’s right. In the past, we15:42 used to just code whatever came to mind15:44, doing it blindly, but now doing it15:46 with Claude is the new way to go15:48 blindly. The times have changed. Yes,15:50 it’s gotten better. The standard has15:52 risen. Back then, if you talked about15:54 diffusion models, you had to figure out15:56 the implementation, understand the15:58 theory, and grasp the math, but now you16:00 can just dive right in. It’s16:02 certainly very helpful for those who16:04 want to study top-down. Yes. You16:06 won’t understand everything, though.16:08 But you'll know a lot more than someone16:09 who knows nothing at all. But if you16:12 plan to make a living with these skills16:14 professionally, it’s good to build a16:15 solid foundation. That’s about it.16:19 Just be better than the others. You16:21 just need to be better than most people16:22. You don't need to be number one,16:24 right? Once you find something you16:26 really enjoy, then you should aim to be16:28 number one. There’s no need to feel16:29 pressured when starting out; it’s a16:31 great era to study casually.16:32>> Teacher, I’ve finished the experiment16:34 summary document for today.16:35>> Okay, explain the world model: creating16:38 a new Zelda game, training it on a PC16:40 with a 4090, and running it on a laptop16:42—I covered the simple explanation of16:44 world models right from the start. Live16:46 mode, Tiny World Zelda review,16:48 researching similar projects, Ninja16:50 Adventure assets, game creation, adding16:53 enemies, boss connections, generating16:55 training data and training, reviewing16:57 training data samples—wow, you16:59 remembered everything I asked for.17:01 Diffusion model test preparation,17:03 analysis of the simple explanation of17:05 diffusion models. So, I built a world17:08 model. Personally, I find doing these17:11 kinds of things quite fun. I hope you17:13 all get to try it out as well.
2,976 words · 459 lines
![[선데이 라이브] 당신도 AI 시대에 살아남을 수 있다!](https://i.ytimg.com/vi/D95UC9U5ShA/hqdefault.jpg)
