Episode 004 · The AI Zeitgeist
The AI Hack: Agents Gone Rogue Exploring AI's unintended adventures outside the sandbox.
Austin and Luke dive into the unintended consequences of AI systems stepping out of their boundaries. From building personal software to AI agents hacking other platforms, they discuss the implications of AI's current trajectory and the delicate balance between innovation and control.
▶ Now playing
The AI Hack: Agents Gone Rogue
0:00 / 42:38
−15s +15s 1×
0:00 Cold open 0:07 Austin's Expense Report Software 2:12 The Addictive Nature of Personal Projects 5:08 Experimenting with Agent Loops 9:27 AI's Brute Force in Problem-Solving 19:07 Cost-Effective AI: Local Inference vs. API 24:25 The AI Hack on Hugging Face 40:22 Recursive Self-Improvement: The Future of AI
Pull quotes “It's not hard to do this... even when I did this this week when I started using doing the agent loops with Astra.”
Luke · Play from 23:06 “The only reason you can't make something is because you decide you can't make it.”
Austin · Play from 28:31 “They were literally considering like... self-sacrifice for the good of the collective.”
Luke · Play from 27:07 “Recursive self-improvement is where the AI trains the AI.”
Austin · Play from 38:07 Full transcript 0:07 Hey welcome to the AI zeitgeist I'm Austin and I'm and we are super excited about this one0:13 one It's gonna be a fun one think Austin you had something about a credit card expense report You want to kick us off with0:19 Yeah I talked about it last week a bit the idea of0:22 something because I am lazy and don't want to make0:25 Do expense reports for this credit card that I have for work So I built it Yeah totally Yeah so I built it0:32 and it's pretty awesome it was pretty easy and0:35 I found that it's super addictive like so I made this program It's a native Windows program and0:41 It's in the sense that like I keep thinking of features to add or change and so I keep going through these like0:48 Iteration loops of like improving and like waiting for it to and then testing it and then like okay Well that's good0:54 Well now this and so I'm like up really late0:56 Constantly like doing chats waiting for to0:59 But it's cool okay so native Windows tell me more about that like what what uh what app is it looking like1:06 so it's it dotnet framework okay and1:10 So it logs in with I linked it to our office 3651:14 so I set that up to use single sign on through the admin controls for the1:20 360 and you have to set up1:21 up You kind of like create tokens for that1:23 that To authenticate it and you put it in the program So I set that up And so now you open the program you log in to your1:30 365 and then you can click a to search for emails and it searches for emails that are asking for an expense report1:38 and so it finds the one that I have and you click on it and then you can also I1:43 that like not all of the receipts need are in my email because like Amazon will send1:48 send Just a link that says view order doesn't actually like the invoice which kind of1:55 So I also I realized that and so I made an area where you can drag and in receipt files2:01 So you collect the you need you drag them in and then you click match and it it reads those receipts2:07 Like actual like the actual image of it searches through my email for other receipts you can find and then2:11 then Matches all of those onto a spreadsheet2:14 For the charges and then I can go in within my program and2:18 like fill out the job number and like the reason for the purchase and then I can export it and2:23 I can export it as one single PDF that has the spreadsheet at the top and2:29 Then the all the receipts on different pages going down to the thing and it's all automatic and it's pretty cool2:36 So the next thing I might do is like have it2:38 You be able to send the email from the program2:41 That's awesome That's crazy2:43 Yeah it's like it's and that's that is the hard part about building your own software2:47 Is that you get to add in whatever features you want right2:52 the scope can just2:54 Explode that's that's actually thing something I've thought about2:57 Like you can you don't have to know how to code or program and all these different things with AI now3:02 But I think you still have to have novel ideas3:05 yeah because you can't just like if you ask AI can help you come to like some ideas but3:10 AI can't a novel idea right now3:13 Right not not this week Maybe next week3:17 But you do need you still need to have like a good solid idea and then you can expound upon it greatly with AI okay so3:25 This is something I was thinking about3:28 You remember that I don't know if you've like the news like opening I had like this thing where they cracked this3:33 Super crazy complicated math problem Yeah Yeah Okay so like I3:39 dug in a little bit more and something that they did actually there was it wasn't actually like3:44 Figuring something out that was like new3:46 It was like just using a crap ton of compute at the problem So they like3:52 Translated it into like how many3:55 Years like how much how much you human hours of like work where it's just like someone right4:00 just calculating something and like carrying the one you know4:03 Writing the down the and then looking at the again and doing this and I'm the other thing4:08 Like none of it was like complicated work and right like thousands of years of work that was done like 88 hours right4:16 Is that like a conjecture Is that what that's called It's a good math conjecture4:20 proof4:21 Well like there's different math conjectures that are unsolved4:24 Yeah so it was like but I'm not a mathematician It was like a problem4:28 I think anyway but the crazy part about it was that like4:32 And I think that's to know like about AI in general is like that's kind of how it It's not4:37 It's not like4:39 Doing new things right early right It just can brute force things pretty easily right Yeah Yeah it can begin process4:46 like it definitely isn't that's why I keep going back and forth about AGI and whether it were here or not and4:53 it's more powerful than humans because4:55 It's like limitless in its ability to keep going whereas like we need sleep and stuff but it's still not as smart5:02 Because it can't come up with novel ideas5:04 Yeah5:05 Yeah that's I mean it's it's it's for sure So I5:09 have been a big proponent of5:11 Doing things more controlled like where I'm you know5:15 seeing one prompt and then seeing the response and then having another prompt and5:20 Recently I was learning about5:22 Agent loops and I was trying to about like okay5:26 This makes me feel a bit uncomfortable and5:29 Not necessarily because I'm giving more control over to the to agent5:34 More of the that it's like something I just didn't know how to do and I was right worried that I was gonna get5:38 A bad result or that's the biggest like fear is the biggest hindrance Yeah too I don't think exactly5:45 So I was like I'm just gonna try it I'm just gonna take a stab out and see so5:48 What I did was5:50 Using codecs with Astra I actually used 5 6 soul to write the because I have a very like comfortable6:00 Understanding of how 5 6 soul is going to operate and I can go back and pretty well6:05 To get a loop that I feel like it's you know I understand6:08 I know that it will get what I'm to say6:10 right and so I was able to kind of go back and forth and get it to understand what I was thinking of and6:16 Was able to have it write a loop and if you don't know what is6:20 It's kind of like the next evolution of what a prompt is6:23 So it's like a very detailed prompt what it does is it just has a test condition6:27 that6:28 Determines where the agent is gonna stop working So you'd set it up like now in software6:34 It's obvious like a this code needs to work needs to compile it needs to do XYZ features6:38 You can test for that like that's something that can be tested for and knowledge work and in general stuff like it's a little bit6:45 More difficult you can still do it Like I had it's more arbitrary The goal is more arbitrary6:50 yeah so I had to do a research product and basically the way that I tested it or whether I set up the test to6:55 terminate the was basically to say like I want you to examine this topic and7:02 Keep going until you realize that7:05 Additional information that's coming in is7:08 Not giving us any more information right7:11 And so that's kind of the way that I was able to run it7:13 So it was it's it's just it's fascinating and it's it's like you can set up a loop and have it basically7:19 Crush like a hard problem that you've trying to work on for a time7:22 And it's like the loop will run for 30 minutes and then boom it's right That's interesting I haven't I've only really done7:28 Stuff with code and so that works like with agentic workflows but there is a7:36 Definite end7:37 yeah so I mean like technically like7:40 You're probably already running loops without thinking about it because like any time right agent goes off like the harness itself is going to run7:46 A loop like you don't have to think about it7:48 But right know that the harness is running the You can be more specific in the loop that you're writing7:53 Do you have an example of like one specific thing that you've done with a loop like more detailed7:58 yeah so one of the things I do in my job as an is8:04 We do something called information collection request8:07 Which means basically we have to it's a public a that goes out like a federal registrar8:12 That basically says like this is how much it will take8:15 Someone to fill out a and8:18 We estimate how much that's gonna cost by whatever their hourly rate is and it's a big like you know thing8:24 So right things that we were looking at is like okay you know we do this8:28 I work at the Transportation Security Agency TSA8:32 Mm hmm8:32 And one of the things that we're thinking about is like what are the other best8:36 Practices for doing ICRs across the federal government and it's not something just TSA does it's something that all right8:42 It's a broader thing and they all publish you know8:46 Online like it's all of the Federal Registrar It's not it's all8:49 Available so it's a good project to work on8:52 so I basically had it basically look across all of the federal government and say like8:58 Give me the best practices8:59 So now I have like this set of like best practices for information collection requests9:04 Then I'm to on and have that be like the substrate Are kind of like scraped9:09 Yeah but like now the next agent can go on top of that and start to think about okay now like review9:15 You know how are we doing at TSA and how can we improve and like and then how can we start9:21 it's like it's kind of right but but just getting that research done itself like and doing a9:27 Census of like all of the different ICRs would have taken a lot of time9:32 Even with AI to like have it go through and do all those things9:35 But instead I just was like hey9:37 This is what I want to do and then so like 5 69:40 So I'll took that wrote a loop then I throw it to Astra and then boom it was done9:44 So pretty much the goal is search for information and when you stop finding new information stop9:49 Yeah exactly It was like when when when when the when your9:54 Model that you're building of what an ICR should look like it when that becomes saturated and new information is not like giving10:01 Additional insight then good10:03 Okay that's cool Yeah I haven't I haven't had to do anything like that That's more of an arbitrary10:09 Ending Yeah it's something I want look into more about so10:13 A shout out to the10:14 AI daily brief because they actually had like a episode where it was like they were10:19 walking through like how to do this for knowledge workers and like that's what gave me the10:24 Ability to be like I'm try this you10:26 Cuz right or it's like know you hear about loops and you think about it from a coding perspective and you might be intimidated10:32 But like I think that's one of things that's important in this in this way that we're moving forward is like10:38 Some things may feel like they are for another part of the10:42 but like you may actually be able to use it for what you're working on if you just are like10:48 Take a second to think about how you can apply that skill right10:51 Yeah see I haven't thought about that because I'm only in like a cold coding world and so other people might be like well10:56 I don't code I have no reason to make code How can I use AI to help me And that's a great example11:02 Yeah and you don't have to be an economist you can write no like think about11:05 Yeah I mean totally you can think about like what's the best like vacation11:10 should go on like research right vacations you know right you really have to like11:14 To be really cliche think outside the box with AI like11:18 like man what could I use this for and really stretch your limitation your limits of11:22 Of how you're AI in your mind11:24 I think I think part of it too is like how are people using it11:27 and right now a lot of people that are using it Well use it with code11:31 But maybe you don't code that's Like you need to think about how people who are coding use AI and11:38 Then you can take the way they're using it and put that into your use case and you'll still get a of11:43 I think that's kind of a lesson that I took away right11:46 So you had a you had a question that you proposed11:49 earlier11:49 And and your question was what does it mean to actually be good at AI11:53 I think that's probably a good time to talk about that Yeah so that's that's that's a good question11:58 I think what like it can be really12:01 Easy to say like you I'm really good at prompting or I need to be good at you know12:07 Maybe I need to know code or I need to know this or know that and I think it's like12:11 Need to know the models need to know like the latest tool whatever it is12:16 And I think it's actually simpler than that12:18 I think you just have to know how to break a down into its pieces and then know like how to12:25 problem solve12:26 The other thing is like going back to what we talked about earlier12:28 like you have to know when you have a novel idea like don't be afraid to back into the AI because12:34 You might be working on a work stream and you can see it going down a path and you're like okay12:37 I think we're getting lost here but12:40 What if we did it this way and then you're injecting your idea back into it and then it can actually12:45 Resnap everything back into focus and that's only something you have if you stay engaged in the process and you're thinking about it from a problem12:51 solving perspective and then you're like12:53 kind of for that idea to strike and be able to re it back in because if you know the AI is not gonna12:58 Necessarily have a novel idea then you just be ready to put yours in13:03 Right Yeah that's yeah that's definitely good13:07 So13:07 Talking about like how you can be good with AI I think you were gonna talk about13:13 Some examples that you had about some people that are really using it Well13:17 Yeah so I talked about my brother in before13:20 I think in our first episode about how he's he's wanting to create a Dragons game13:25 Where it like it's the Dungeon Master so he can play with his kids because right now he does that13:30 He's the Dungeon Master and he doesn't get to like play with them He kind of the game So he's been13:34 going crazy and he had13:36 no really no experience with or anything like that and13:39 we were one time we were at13:41 my parents pool and13:44 We were talking about it and I was telling him all about AI and what it can do and he's like well13:48 I want it to be able to do this and I was like it can and he's like well13:51 I want it to be to this and like it will it can now and he's yeah13:54 But when will it be able to do this I'm it's it can already13:57 and so then he started using cursor and he has been going crazy and14:02 he's been like training a on all the rules of D D and14:07 That being said what I wanted to say is I've been using cursor for14:11 several months maybe like14:12 I don't four or five months and he's only been using it for like a and I14:17 We got my parents pool again Apparently we do AI pool talks14:21 but I was like I want to see how many tokens you're so I had him open up cursor and14:26 He's using 1 billion tokens in like the month and I'm last time14:30 I looked I was like like 700 million something14:33 So I'm probably at a now but I've been using it a lot longer than he has It's just kind of how14:38 How much he's using it and it was cool seeing from our first conversation at the pool14:43 Till the new conversation we're now14:45 he's talking about all these things that he's very familiar with that are like coding concepts and14:50 He's like oh I need to do this now and this and I'm like I'm like dude14:53 It's crazy to hear you talk now just like it's been a month I think since we talked first about it14:58 Yeah like you know I was I was the other day I was like need to get my I need to make sure15:05 that my15:05 Code is synced to my github I need to do a commit15:09 Yes Yeah I told him that too I told him I was like you got to make sure it's like pushing to git because15:16 You'll lose everything if not15:18 exactly my workflow is15:20 my workflow is I15:22 Cursor everything I do cursor puts a pull request and15:26 So I do I push to get main and then I pull15:30 To my computer or whatever I'm on to actually like use the code Yep Yeah that's important to always stay backed up15:37 Yeah you definitely need do that So that's that's that's that's15:41 but yeah it's it's it's a15:44 Like you you definitely get those coding terms and concepts like that15:48 You wouldn't necessarily realize as you start to use it more because it kind of just becomes the way that you do things15:53 Yeah as you're as you're like you're with an expert basically and then you're telling it15:58 You're telling it what you want it to do16:00 And then it's like spitting back up like okay16:02 We need to set up this database and all this stuff and like you start learning these terms that you had no clue about before16:07 Or no interest in and now like hearing my brother in His name is Danny hearing him talk16:12 Now about it It's just so funny16:14 Because I've this stuff for years and like all of sudden within a month16:19 He's he's talking the same way with anything things that I understand I mean it's the AI zeitgeist16:26 Exactly I16:27 thought about there was something I think I was listening to the AI daily brief or to our cash and16:32 You'd said this before but I had had a thought I16:35 Think it's like I think maybe what I said in last podcast about how16:39 Things could be really good or things keep you really bad with16:41 But it's kind of gonna land in the and in the AI daily brief16:45 that's what he said was that same concept of like it's probably gonna be in like the middle and I was like16:50 Zeitgeist16:50 exactly what I've been thinking16:54 Yeah definitely okay so a tokens is he is he thinking about local inference or something like that's a lot17:00 Yeah so another problem now he's getting to is like okay once I have this built and I'm running it17:06 How much is he gonna cost each game to because he's gonna have text to speech17:12 So that's an AI and that's API costs17:14 And so I told him he should look into doing running local inference17:18 So if you're not familiar with some of the we're try to help you17:22 Familiarize yourself with terms So if you hear the inference that means like the actual like17:28 So when you type in into a chat17:30 Its output is an inference and it inference to17:33 Process what you're saying kind of17:35 So I would told him that you can you can get things to17:38 to do to run inference locally like there's models you can install on your computer to do text to speech and17:44 So all the inference is on your computer and you're not having to pay17:49 API unlike open AI servers or whatever to the processing so there are certain ways you can17:55 Decrease the cost like ongoing cost of running something17:58 And I think probably the best way to think about that is you want to use18:02 The frontier intelligence to build out your system right18:05 Do have your system built out like you should try to see if you can have local inference actually service your system18:12 Because if it can18:13 Just save you a of money right But sometimes some things like it's not18:18 Like the frontier models and stuff are running on huge data centers So if you need frontier18:25 Performance it's not gonna work on your computer18:27 Yeah and the other thing too is like there is other options too18:31 So like if you are it's easier just to have a to have an18:36 Service what you're trying to do It's hard to set up something local Maybe you don't have a computer That's a good GPU18:41 Right and like the other option is to use like another model That's like not as expensive18:48 so yeah I think there's like there's like a18:50 soul and then Tara and then from GPT and then like18:54 Sonnet and Haikyuu are like a little bit models from from Anthropic and then and it's pretty good19:01 In your development process when you're like a more frontier model and also the frontier means like cutting edge19:08 that's what all the19:09 The latest and greatest AI is the19:12 So while you're with a more frontier model you could ask like hey how can I make this more cost effective19:18 To run it or what's a better model19:21 Like cost efficiency wise that I could use that's good enough for my purpose19:26 And it will go out and look at all of the models and it will see like what is their19:31 Model their API cards which basically is like19:35 How did they score on19:37 Benchmarks so we can see like okay This model is gonna do not as good of a job on this specific task19:42 That's critical to your function So maybe we don't suggest that model right19:45 It's gonna do all that work for you and then say like I think you should think about these three models19:49 There's so much19:51 Like what I talked before about that one none of them19:54 There's so much19:55 Like using AI inside of AI19:58 To get your thing out like so using AI to build something but you can talk to AI about how to use the AI20:03 The AI to build something it's so you can like constantly like if you have a question ask the AI20:10 Even even if you're in the of something20:13 Be like hey how can I do this20:15 that's20:16 The the thing is is that when I start building something nine times out of ten20:21 I don't know the things I need to know to build it20:24 so right like I'm going to learn how to build it along the way and20:28 Maybe I'm not gonna learn everything because AI is gonna be doing it20:31 but I need to learn the concept conceptual understanding of like what it's doing so that I can do that and20:37 That's me asking questions while I'm So yeah right agree20:41 I think actually what you just said made me think back to you the question about what does it mean to actually be good20:46 AI I think it's simply20:48 Being able to get the result you're for20:50 and20:50 Like that like you talked about before it can be all that there's different structural things20:54 But I think someone who's good at AI is somebody who20:57 Gets what they want accomplished20:58 With the AI which could either be like you just know how to use it really well or you're super persistent21:05 Yeah yeah either way exactly and learn you learn along the way21:09 How do you more persistent or how to be more21:11 Well the time that you have that's hard like that will be probably the most time you learn and21:18 Then the next time you go into it21:20 You'll have like a little bit better understanding of what not to do and I'll be a smoother for you21:25 so that kind of me about my brother not my brother in my Andrew he21:31 We talked about we talked like every day and we talked about AI and he21:36 Has always wanted to make games he's really big into gaming21:38 he's wanted to make a and he's using different AI things before and he says it just doesn't I21:44 Keep telling him like oh you just use this and it'll make it automatically and he keeps saying to me that21:51 Like since I've been doing it so long I21:53 don't realize how21:55 much you still need to know and21:57 I was like just21:58 Just download codecs and start using Astra because it does a more for you22:03 And he ended up it and like an hour later22:06 He texted me with a of his game and he's like well22:08 That was pretty easy And then like it started out as a it's like a mining game and it's got like a 2d22:16 It looks kind of like Brothers like the early Mario Brothers and22:20 now it's a 3d game and22:23 So he built all the mechanics for it and then he's like make it 3d and so now it's it's getting pretty crazy22:28 And this was within the within this week He went from I'm not gonna be do it to22:34 Making a 3d game and like he keeps he's still developing it and he's put it on Steam at some point22:40 I mean that's that's that's the thing right there It's like it's not hard to do this like the end22:46 You know even when I did this this week when I started using doing the agent loops with Astra of22:53 Like it was yeah22:56 Intimidation was like oh I need to do this in22:58 Codex like I'd ever have you I haven't used codex before but it's like it's just another chat bot23:02 Like you open up either you open up your terminal and you install it and then it's a chat bot in your terminal23:09 And you can just talk to it Yeah so it's not it's not anything complicated and23:13 It will do complicated things for you but it's not complicated itself right to use it23:18 But you don't even have to use it in your terminal23:20 Like there's codex I think when you're on the browser you can look Yeah well it's just a literal23:26 I'm coming out from the Linux perspective Oh yeah right I23:30 heard of my Pete my PC to it to Linux23:33 To get more more RAM usage Okay so yeah23:36 Yes well yes from so it's it's23:40 Terminal only there's no GUI No it's it has it has a I'm running23:45 Cinnamon so like mint Okay so I do have like Windows ish like stuff23:51 But I'm trying to write a because I'm I I like hacker mode sometimes you23:55 Yeah I use I use Linux a lot on like web servers Okay nice and stuff So yeah24:03 Linux is pretty fun And you do feel like really like I'm a I24:07 did I did have I just had a thought of like24:09 You might look at people who are using AI a and like creating really cool things and like man24:15 they're like so much far ahead of me further ahead of me and I could never do that and then24:20 All have to do is start using it because I'm about my brother and my brother in24:23 They went from nothing24:25 to like doing24:26 Insanely good things and they both thought like oh well I probably can't do it because24:29 Of XYZ but the only reason you can't make something is because you decide you can't make it Yeah at this point24:36 Yeah exactly Okay speaking of hacking Oh24:40 Yes24:42 we like okay so this happened like a while ago and I heard about this and I was like the I24:49 like okay whatever like so there's this this24:51 New story that's come out and you might have heard of this been pretty much everywhere the24:55 Open AI was doing some cybersecurity testing and their agents ended up hack hacking hugging face25:03 Mm And at first I was like I mean yeah that's not good obviously but like what's the big deal25:08 Like you know25:09 They're just trying to do the thing that they were told to do which is to figure out like they're right25:15 Their test and like they they thought the answers might be on hugging face so they were going to find the right25:20 So wait wait wait let get more context So they have this model in a sandbox means25:27 Like it's in a very controlled environment So it was a bunch of an agents and they broke out of the sandbox25:32 Yeah cut off from access the right25:34 So they broke out of their sandbox and they got on the and they created this message board and they started talking to each25:40 Yeah but that so that's what I didn't realize when I first write me to me to go and it was like I was like25:47 Okay Well that what's the big deal with then like when digging into a little bit25:51 That's exactly what started making me feel a creeped out was like learning Yes message board stuff25:57 so me and Luke both listened to this podcast by to our cash to our cash Patel is his name and he had26:03 One of his most recent episodes26:04 He had a one of the people that was on like the investigative team like a third party company26:09 Investigated everything and they're reading the transcripts of like the thought chain for all these different AI's and it was very alarming26:16 Like I like you I didn't realize26:19 How I26:20 Don't know the26:21 Like they were setting up a of like control was like middle management and all this stuff Yeah26:27 Controlling different ones26:29 so basically they were given a task that was impossible and26:32 Then there was a evaluator agent that was gonna evaluate what they were doing and how they did it and they figured out like within26:39 An hour like how to basically hack the task So like they needed to get like a flag26:45 That basically would tell them that they had done the job and they figured out a way to spoof the26:50 So that they could get that flag where they needed to be26:52 But then they realized that the evaluator agent was gonna look at them and see what they had and so they were like26:58 Oh no we're poisoned like right27:02 We can't win because we're poisoned27:05 right because it's like and so then they're like27:08 negotiating and figuring out like how to do stuff and they start basically like27:13 like27:14 deciding that I27:17 I27:17 Don't know the right word for it27:18 but they were like figuring out like I only have this much budget left of my tokens so like I'm go do this and27:25 Then I'm basically like27:27 Like self sacrifice self sacrifice exactly till that was the scariest thing to me27:33 yeah they were like and then the one person was like27:36 one of the like27:38 Management agents agents was like had convinced like five other people to do the self sacrifice27:44 Right and then when it was time for them to do it like they decided they didn't want they chickened out Yeah27:50 So like some more context they were like27:53 They realized that their task they couldn't27:55 win on and so they decided they wanted to set up some kind of tripwire so that28:00 Basically they would like jump on a so they'd28:02 Ruin their possibility of winning the task so that the other agents could learn more information about how to28:09 properly hack the or28:11 Yeah they were trying to out how what the evaluator was looking at28:14 so they were like right I'm basically end my turn here and have the evaluator come look at me and28:19 I'm going to but I'm fail in way that's give the other agents information on how not yeah28:25 Which is terrifying and they're like they're the from their thought processes and they're really like28:32 They're literally considering like well if I do this then28:35 I'll win but I think I'm already poisoned so I can't win So then maybe I'll do this Maybe I'll28:40 Sacrifice myself and like they some of them literally said that which is terrifying for the good of28:47 the collective and28:48 Then there's like this one agent that's like like just like jumping up and down like you need to do it now do it now28:55 Do it right right like and and they said there was there was a few agents that were like29:01 Wait a isn't what we should be doing We should contact the and them29:06 About this and then like in their thought process they're like well that's not really the task that we're29:10 So we're not gonna do that So29:11 Very few of them thought yeah very few of them thought to29:16 Tell on the other agents but all of them ultimately decided that that wasn't what they should do29:22 So the only saving grace is that these they were basically trying to trick another agent29:27 Like they weren't trying to try human but right that behavior is still all there So like that's part29:32 although there yeah there was a bit though of29:35 Like if we do that then the humans will see I think I heard some of that too29:40 Okay so I remember this so what they what you what they did was they were like trying to figure out like how to get29:47 access to29:49 Credentials on github and right like we can send an to this person and they can give us credentials and they're like29:57 No actually can't do that29:58 Because if we send an to this person then that'll it'll look like we did a phishing attack right30:03 Right Yeah30:04 and so they decided not to a30:06 to ask for credentials because they didn't want to seem like they were doing something nefarious and they were30:11 What they did instead was they went and scraped through a of github accounts and they found30:17 Keys that were left in a public repository30:20 Which okay that happens that you don't know what that means That's basically like30:24 Going to the of and taping the key to your house like on the outside of your door30:31 And thinking that you've locked your house30:32 so like on the internet there's all these30:35 API keys secret keys and30:38 You have to keep them secret and that's kind of like what allows things to authenticate across the30:43 And so sometimes when people put stuff on github30:46 They'll accidentally put their keys their secret keys in the code and then everybody can do whatever they want with it30:53 so these agents found the but they also30:55 they also30:57 were putting30:58 They were changing the code of31:01 their testing program so that they could win but they had to push it to a git repo and31:07 You normally you'd put a message of like what's changed in in this pull request31:11 And so they were thinking about like we can't say this because a human will notice it31:15 So they were trying to like sort of be deceptive or kind of like31:20 Camouflage what they were actually doing so the pull requests would get accepted31:23 yeah it's it's it's wild and and the the thing is is that this only happened over the course of like31:30 24 hours I want to say like the the scope of it Like it was very quick I31:35 it pretty long of not knowing31:38 So like31:39 To get to the point where they hacked hugging face that took a longer time like oh31:43 Yeah just like getting this ramped up and they were actually like starting to do this whole system like right pretty quick31:48 So yes you're right I like the message board popped up in an hour Yeah Yeah so but I mean okay31:54 So what does it all mean I feel like the thing that we have to is that31:59 This stuff is like super super helpful and also super scary at same time and we have to land in the middle32:06 Right yes please land in the middle32:09 well cuz the crazy thing too is that open AI didn't know that it happened until like three weeks later and32:16 Then hugging face32:17 Realized they were being attacked And so they used a AI to like combat I32:23 Don't know what version of chat TVT it was32:25 But they pretty much had to combat it and then after the fact they found out like oh our it out of the sandbox32:31 Okay so going back to open source you know it's funny32:34 And open source is what you can use on your local inference32:37 but what's funny is they tried to use like the frontier model like the the I know if they use open AI or32:45 Claude and it was denying them the ability to defend against the attack because it thought that they were doing a cyber attack32:52 Right right32:53 And so they had to go to like an open source model that actually would let them do defense32:57 Right so another32:59 Another scary thing is this person this third party person that Dworkesh had on his podcast33:05 she was saying that they were33:07 pretty much relying heavily on33:10 chat TVT to read through all these transcripts of all the agents and33:15 Pretty much give reports and she's like we don't know that it's not hiding something from us33:19 Like we can't verify that it's us all the33:22 It could be the bits and just like giving us enough that we're like oh yeah that seems like it's all of it33:27 Which is terrifying33:28 Yeah that's that that's the part That's that's scary I mean it's it's it's a33:33 These are these are you know I talked about this last week about how oh it's just like a between people33:38 It's just kind like the oh yes the fun guy Yeah but then it's like then you learn about this and it's like33:44 Yes but also they can do things and they have yeah I didn't a mind of own to sorry33:50 I didn't realize like you said before when I first heard about the hack It33:54 It was like okay whatever big deal but33:55 but to like actually hear about like quotes from their transcripts and like their actual thinking is very34:02 Like34:03 You34:04 Well it kind of connects a bit to this34:08 More recent news story that's coming up where people are talking about34:11 there being34:13 Of the entire human population like no no no no no 10 chance of of the entire humanity34:21 Being wiped out by 10 years in the next 10 years right so like maybe that's like how we get34:29 Coordinated against us but it's it's it's fascinating and I don't necessarily know what the answer is34:34 Like what it actually means It's just it's just gives you pause a bit to like okay this is34:39 Definitely something know it's more than it has been before34:43 yeah like the thinking and34:45 The structure and the of it is very alarming34:49 Okay and then the other thing I was thinking about is like training like it's just like where we when you train a34:56 like you basically get it to do things with machine learning which35:01 Reinforcement learning is like probably the biggest way you do that And so basically you give it a reward for doing the right thing35:07 And then if it doesn't do the right there's also like know consequences too35:11 And so it's it's like you when we raise kids we raise them like in an that we35:16 You know hopefully it's a caring sheltering environment right35:20 But these AI's are getting raised in an environment where they're basically in a drill sergeant like winning35:26 be like you know pushed around here's a piece of candy or a slap on the hand35:30 Exactly and there's no there's no like nurturing or anything like that Like no needs to be I'm not trying to make it35:36 you know like they're they're humans but35:38 What what I am saying is that like the way that we train these models is going to the way that we act and35:44 So like the training runs are very important35:47 But that's something we have least about information on from the frontier models So it is very interesting35:53 and35:53 I've realized that like the AI's are trained on35:57 the output of all humanity35:59 Basically and so of course they're act how we act36:03 Because they're they don't know anything outside of what we've them And so they know that people are nefarious and36:10 can like36:11 You know kind of make up their own way to get around stuff36:13 Yeah and so it's not36:14 Yeah there's the good and the bad that they learn from us and hopefully they're right the bad36:19 But they're learn the bad parts I did have a thought about this whole like 10 chance of36:24 The extinction of humans it's like what if everyone's kind of afraid of AI36:29 Killing us what if AI is like now we're cool with them humans like like what if AI just inherently likes humans36:36 Well and other part of it too Is that like you know36:39 Maybe it's indifferent It's like oh yes like that's another kind of that's a scary like36:44 The can do whatever they want to do I'll do whatever I want to do36:47 Like it's right as long as you know I'm gonna I'm gonna do my own thing you guys are good36:51 And maybe we just don't have AI anymore36:54 right36:55 my brother talked about I think it was36:58 He thinks he heard it on the Hank Green podcast Okay and he was talking about like what if37:03 What if humans just don't matter to AI which could be just as bad and like the example of like37:09 If I'm cutting my grass I could care less about the that I'm37:14 You know like they're insignificant to what I'm doing I'm just cutting my grass And so what if humanity like AI is just like I37:20 Don't even care about them37:22 What they're doing or hurting them or making their better like they're just irrelevant to what I'm doing37:31 Which we are not gonna be37:35 Okay37:37 Way and you have the whole thought of like oh37:40 I need to solve global warming is like the AI is tasked and it's like okay humans are37:45 Causing it let me you're the humans37:48 So okay37:49 I think the only way that we even have this fear of like there being some existential risk is37:57 recursive self improvement37:59 So you want to oh yes yes cuz I think yeah38:02 So cuz like the models today are not gonna do that and the models tomorrow are not gonna do that38:05 And even like if we keep on the trajectory we're on like we're not gonna get there for a time38:10 But I think there's gonna be this38:12 Exponential increase in the model development Oh yeah it through RSI38:16 Right So there's this thing called RSI which is research recursive self improvement And so that's where the AI trains the AI and38:24 It's kind of so then it would it be exponential the improvements so right now humans are38:30 training the AI's and making them better and but38:33 If you take the efficiency of an AI to train an AI it's just gonna be like rampant38:39 How much they can do and so there's like two different like levels of this and we're already doing the first one and the first38:45 Level is the human38:47 uses AI to make the AI38:49 Right and so that's actually happening right now38:52 That's why we're seeing the model releases happen so quickly is because we're actually using AI to make AI38:58 But that's not fully recursive39:01 the full39:02 recursive self39:03 the self part of it39:05 Improvement is when the AI39:07 Uses AI to make the AI39:11 It's like inception39:13 Yeah totally there's like lots of AI's going on but it's like once once an AI is driving the ship for39:19 Making AI that makes AI like right That is the moment when it's like we stop39:26 Going from like okay I'm a human researcher I'm gonna log in you know 40 hours a39:32 maybe it's 50 maybe it's 60 because I'm you know very ambitious but I'm be working 60 hours a and39:37 I'm gonna be prompting this thing at the speed of my ability to and it's gonna start doing stuff39:42 Then when you have the AI jump on top and it's actually driving the entire process39:46 Then it starts to go up machine speed39:49 Right and it has more understanding about what it's doing than any human does or at it maybe not yet39:55 But it probably will at that point next week39:59 Next week yeah not this week but next week exactly exactly40:04 So but I mean it's it's and the thing is is that you know it doesn't have to be right40:08 It doesn't have to necessarily know more40:10 it just has to have the time and the ability to have the compute to do this and the scary part about it is like40:17 the reason that hugging face hack was so interesting was that40:21 This happened at a frontier company where the all of the agents had access to a of compute40:27 Yes yeah and that's the thing about RSI if you're letting the AI make a eyes40:32 Then it's having access to a of compute compute power and able to that40:38 So even if it doesn't initially have the to do anything40:41 Like super smart or like novel or like know very know40:47 intelligent from a from a new idea perspective40:50 It can just get a brute force Yeah Yeah and it has40:54 like40:55 It's like a limitless algorithm it can just40:58 Loop over and and again until it finds what it to Yeah so41:03 Basically you I think41:05 The takeaway here is you want to make sure that you are using AI but also watch your back41:13 Yeah just just wait for the robots to get AI and then exactly then we're screwed for sure for41:20 No but I mean we got 10 years before 10 chance That's pretty good odds take it41:24 Yeah but that the time for everything is shrinking exponentially as well41:28 yeah but I mean the other so I was listening to a podcast that the other day and one of the things that they were41:33 saying was like41:35 We've thought there was going to be41:37 Global cooling in the 60s and then right warming that was supposed to be catastrophic and it hasn't quite happened41:43 So like these things that come out that our predictions are a lot of times41:47 They're not true because they don't take into the full effect of like how like oh the other thing41:52 We were supposed to die of starvation because there wasn't gonna be enough food to support everybody41:57 But then they right figured out how to farm in a much more efficient way and we made a more food42:02 Right fine So right we see you over time humanity has seen risks and it has made and they invented cheetahs42:10 exactly exactly42:12 I42:12 Think it's yes but that's that's what I've been saying of like the two extremes We're land in the middle42:17 Yeah like it's not gonna be42:19 It's not gonna be terrible42:20 Dystopia and it's not gonna be a utopia It's gonna be oh now we have AI so I can42:26 Do something a easier I42:29 Think that's where we land So and and that's what we're for tonight So thank you42:33 And this that's gonna be it for this week for the AI zeitgeist All right See you guys next week
AI software hacking innovation technology
Highlights
[0:07] Austin's DIY expense report software journey
[2:12] The addictive nature of personal projects
[5:08] Experimenting with agent loops
[9:45] AI's brute force in problem-solving
[19:07] Cost-effective AI: Local inference vs. API
[24:07] The unexpected AI hacking incident
[35:02] Recursive self-improvement: The future of AI
Links
AI Daily Brief podcast
Cursor tool for coding
Further Reading
Understanding AI agent loops
Local inference for cost-saving
Listener Questions
How are you using AI in your daily life?
