LØpht - Heavy Industries

Video thumbnail (Frame 0) Video thumbnail (Frame 7922) Video thumbnail (Frame 20673) Video thumbnail (Frame 33114) Video thumbnail (Frame 45555) Video thumbnail (Frame 58305) Video thumbnail (Frame 71055) Video thumbnail (Frame 85578)
Video in TIB AV-Portal: LØpht - Heavy Industries

Formal Metadata

LØpht - Heavy Industries
Alternative Title
The L0pht Testimony, 20 Years Later (and Other Things You Were Afraid to Ask)
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
2018 is the 20th anniversary of the hacker think-tank L0pht Heavy Industries testimony before the US Senate Homeland Security & Governmental Affairs Committee on the topic of weak computer security in government. The testimony made national news when the group announced they could take down the Internet in 30 minutes. It was also the first-time hackers using handles appeared before a US Legislative body. Members of the L0pht have grown from their hacker roots to become distinguished leaders and contributors in the security community and beyond. They run multi-million dollar security-focused organizations, have lobbied the government for better security laws, work for some of the largest companies in the world, and continue to spread the message of the positive aspects of hacking. With several of the L0pht's original members, this discussion will cover the original testimony and the changes that have happened over the last 20 years. Is the government any more secure? Have they provided enough influence to help protect its citizens' data? What steps should we take to ensure user security and privacy in the future? We are hoping for audience participation and also welcome questions about any other time in the L0pht's relatively short, but poignant, existence.
Point (geometry) Group action Web crawler Code 1 (number) Student's t-test Disk read-and-write head Event horizon Magnetic stripe card Revision control Coefficient of determination Computer cluster Strategy game Hacker (term) Internetworking Computer hardware Representation (politics) Spacetime Software testing Office suite Computer engineering Information security Vulnerability (computing) Software development kit Cybersex Dependent and independent variables Software developer Denial-of-service attack Line (geometry) Incidence algebra Cartesian coordinate system System call Word Voting Software Logic Right angle Text editor Musical ensemble Whiteboard Window Buffer overflow Spacetime Dialer
NP-hard Sensitivity analysis Group action Context awareness Code State of matter INTEGRAL Multiplication sign Decision theory Direction (geometry) Source code Set (mathematics) Stack (abstract data type) Mereology Dimensional analysis Computer programming Software bug Facebook Hypermedia Computer configuration Single-precision floating-point format Endliche Modelltheorie Information security Control system Vulnerability (computing) Physical system Cybersex Area Injektivität Binary decision diagram Email Theory of relativity Binary code Sampling (statistics) Sound effect Hecke operator Bit Connected space Textsystem Right angle Figurate number Quicksort Arithmetic progression Spacetime Point (geometry) Statistics Functional (mathematics) Game controller Service (economics) Mass storage Device driver Web browser Mass Goodness of fit Internet forum Hacker (term) Profil (magazine) Term (mathematics) Codierung <Programmierung> Traffic reporting Authentication Exploit (computer security) System call Graphical user interface Software Video game Musical ensemble Buffer overflow Window
Suite (music) Sensitivity analysis Group action Presentation of a group Randomization State of matter Multiplication sign Source code Set (mathematics) Design by contract Shape (magazine) Mereology Perspective (visual) Computer programming Software bug Formal language Mechanism design Roundness (object) Hypermedia Endliche Modelltheorie Extension (kinesiology) Hill differential equation Information security Descriptive statistics Personal identification number (Denmark) Cybersex Area Covering space Email Mapping Electronic mailing list Bit Lattice (order) Flow separation Value-added network Type theory Process (computing) Auditory masking Computer crime Telecommunication Chain Order (biology) Volumenvisualisierung Right angle Text editor Cycle (graph theory) Quicksort Reverse engineering Row (database) Spacetime Web page Aliasing Point (geometry) Trail Open source Real number Online help Wave packet Goodness of fit Internetworking Hacker (term) String (computer science) Computer hardware Queue (abstract data type) Traffic reporting Form (programming) Information Planning Contingency table Cryptography System call Video game Theory of everything Table (information) Window
State observer Sensitivity analysis Group action State of matter Code Multiplication sign View (database) Decision theory Client (computing) Open set Disk read-and-write head Mereology Information privacy Perspective (visual) Dressing (medical) Web 2.0 Coefficient of determination Malware Different (Kate Ryan album) Hypermedia Videoconferencing Software framework Damping Information security God Area Email Spyware Sound effect Staff (military) Determinism Bit Lattice (order) Flow separation Connected space Proof theory Type theory Data mining Message passing Process (computing) Angle Addressing mode Software repository Drill commands Telecommunication Order (biology) Self-organization output Right angle Quicksort Figurate number Elektronische Wahl Spacetime Point (geometry) Trail Server (computing) Service (economics) Connectivity (graph theory) Ultraviolet photoelectron spectroscopy Virtual machine Similarity (geometry) Translation (relic) Rule of inference Machine vision Number Revision control Latent heat Hacker (term) Internetworking Energy level Representation (politics) Spacetime Selectivity (electronic) Addition Dialect Scaling (geometry) Information Projective plane Physical law Expert system Counting Planning Line (geometry) Cryptography System call Uniform resource locator Word Voting Software Integrated development environment Personal digital assistant Iteration Game theory Routing
NP-hard Context awareness State of matter Code Decision theory Multiplication sign Source code 1 (number) Mereology Public key certificate Software bug Fluid statics Direct numerical simulation Mathematics Mechanism design Envelope (mathematics) Bit rate Semiconductor memory Information security Physical system Vulnerability (computing) Software developer Feedback Fitness function Data storage device Sound effect Bit Lattice (order) Measurement Entire function Tangent Category of being Type theory Message passing Process (computing) Internet service provider Self-organization Right angle Quicksort Telepräsenz Writing Point (geometry) Functional (mathematics) Mobile app Continuum hypothesis Control flow Twitter Revision control Goodness of fit Crash (computing) Hacker (term) Profil (magazine) Term (mathematics) Touch typing Computer hardware Spacetime Software testing Data structure Dependent and independent variables Scaling (geometry) Inheritance (object-oriented programming) Information Physical law Mathematical analysis Plastikkarte Counting Limit (category theory) Statute Word Friction Software Integrated development environment Personal digital assistant Point cloud Abstraction Library (computing)
State of matter Real number Control flow Image registration Mereology Mathematics Internetworking Term (mathematics) Different (Kate Ryan album) Hypermedia Hacker (term) Computational fluid dynamics Videoconferencing Software testing Office suite Information security Traffic reporting Metropolitan area network Physical system God Form (programming) Area Software developer Physical law Projective plane Mathematical analysis Database Bit Hecke operator Line (geometry) Entire function Connected space Data management Voting Pointer (computer programming) Integrated development environment Software Universe (mathematics) MiniDisc Right angle Quicksort Game theory Elektronische Wahl Spacetime
so 20 years ago most people didn't know what a ddos was what they thought fishing was a voting activity and hacker was a bad word in a loft in Boston a group of hackers stumbled into this world in the early 90s they created loft heavy industries and in 98 they were invited to testify before the Senate Committee on governmental affairs about the risks and the risks with the burgeoning internet that unlikely event marked a turning point for the industry in today's cyberattacks dominate the news and ethyle ethical hackers and researchers are on the front lines pushing for better security practices and more critical thinking we're going to hear some never before revealed details about that historic trip hopefully and we're going to hear there have to see how they view security issues that are still problems today so I'm going to introduce them so kingpin also known as Joe grand as a computer engineer hardware hacker former DEF coat Def Con bad badge designer and proprietor of brand idea studio he joined loft as a 16 year old and worked on the a bunch of acronyms pocs AG pagers thanks great thanks King been a little tracked you know the president and decoder kit amps based cellular phone hacking and palm OS application development guys remember that one right [Laughter] what was the name of that guy was that Harry palm or booty call I remember which one it was the war dialer probably yeah so Chris Weiss opal also known as well pond is a co-founder and chief technology officer at Varick code was at CTO at very code loft he was a thorn in Microsoft's side researching security in Windows and writing Windows versions of loft crack and netcat and was webmaster for laugh comm tan that was right day 10 published a paper in 99 envisioning a cyber underwriters lab to test some software for security weaknesses and he led incident response after a logic bomb was planted in a UBS at UBS he did pen testing and red teaming for 12 12 and a half years at JPMorgan Chase and now works in security at a large health insurance company [Music] Peters at Co aka Mudge worked in DARPA as a senior government official for DoD running cited he also worked as VP of engineer engineering at Motorola and he was a deputy director at Google he's head of security for stripe and chairman of the board at the cyber independent testing lab Chris Thomas aka space rogue is a founding member of loft was a longtime editor of the hacker news network and is a public advocate for security topics through speaking writing and general gadfly nests he worked at at stake and then went on to work in research at truss web spider labs and tenable he currently leads global strategy for IBM's x-force red team and then dill dog Christian Rowe better known here as Dell dog was the co-founder and chief scientist at Avera code he's the founder of hailstone a security incubator inside CA Broadcom he was a researcher at aloft and at stake and a member of the cult of the dead cow anyone remember cult of the dead cow [Applause] he wrote original code for back orifice mm co-authored loft crack an anti sniff and wrote some of some of the first buffer overflows for Windows so thinking back to that testimony that was May of 1998 what what did you say or convey to them that day that you think worked so I'll start so I think the thing that worked and it was something that you know you never plan it right I think much dropping his line of the 30 30 minutes worked but we were like a visceral visceral representation of what the adversary viewpoint was like we were the first ones to actually talk like
these guys you know these guys were building secure systems building the software they don't know what we're doing we're hackers we could break this stuff like we made it really real to them that that there are people that can
do that stuff and we're talking you know personal person I think the other thing that we did that was important was we conveyed the poor state of software security and vendors just couldn't say you know all software has bugs we have security bugs because if we could find them we said they should be able to find them so I think those were some of the some of the basic foundational things that you know God got through I think going along with with well that said sort of raising awareness of hackers in general I was just at the the mob museum in Las Vegas recently and learned about something that was pretty cool the Kefauver committee back in like the 50s where they were basically exposing organized crime to the masses is the first time people learned about organized crime this is like a little bit of an exaggeration a little bit of an exaggeration but the testimony was sort of like that because people had sort of heard about hackers and up to that point they thought they were always bad criminals whatever you'd see something in the newspaper once in a while but this was the first time where we were onstage and saying we can do good like hackers can actually do the good side besides the adversarial we can educate people and do stuff that we you know think is helping and I think that was really important it was on sensitivities that was I've got a mic you're the only one with it on you it was one of the first examples of cultivating sensitivities across the chasm hackers were criminals that's the only way we were referred to in the media it's actually kind of how that whole thing came about in a strange way and on the government side you know we knew that there were some overlap where we all wanted the same thing but we didn't want to become government we government sure the heck didn't want to become us sort of set up so why can't you just ignore the other parts and like figure out how to make progress in the areas where you both have interest one of the things that actually worked really well out of that which took about two years afterwards was that set of testimony was leveraged to introduce verbage into BDD 63 presidential decision directive 63 under Clinton which stood up the scholarship for service program so if anybody went to one of the 73 colleges and got a tuition and then later had to go work for the FAA FCC FTC or anything else would had your tuition paid for that was largely driven by that set of testimony as an exemplar of this is why we need smart young people not just the old like you know old guard and the bureaucrats and the policy wonks which are very valuable but maybe didn't know that new thing coming in so it was an awareness sort of set up historically the government was being briefed that you know you know like it was the Cold War that if cyber was coming you'd be able to see it the same way you'd be able to see missile silos built up that it would take ten years that you'd see large movements of you know equipment and funds and everything else and you could track it and this was different because it was like you know here are some kids in Boston who didn't nations stay stuff out of dumpsters as far as they were concerned and they did it in a matter of months so what does that do for your threat model and I think that was probably the biggest wake-up call for the government how do you think things have changed since then [Music] well this thing on okay I think that definitely that exploits have definitely gotten significantly harder to write over the last twenty years I remember writing a lot of the first early Windows exploits and finding bugs was something that I literally stumbled across it was not something I had to search for you know exploit trading for cash definitely existed but this sort of nation state support for exploit marketplaces and things didn't exist back then so as the vulnerabilities have gotten harder to exploit their relative costs and price associated with an exploit is skyrocket guy simply this guy rocketed so exploits also didn't usually require that nearly the amount of chaining that we have to do today to actually you know fully exploit a system laying you know these layered defenses have really raised the bar on attackers and had an effect and the most well-funded attackers are still finding that the lowest hanging fruit is is uninformed and unlucky users you know who happen to click on the dancing bears so that but that's that instead of like fishing people's email just email or making a crappy phone call you've got online forums and SMS texts and whatever it takes to get to people tinder I don't know there's a lot more opportunities to fish people these days because of our increased connectedness so while the it may be harder to exploit things it has become easier to exploit people over time that said on the plus side more far more people are aware of the need for security stays even grandma on Facebook knows about hacking and might even know what phishing looks like two-factor authentication is available in many places and better when it wasn't but that's it's still surprisingly not oh no a lot of banks you know all the places that should happen more banks don't have it in enough places yeah yeah right video games happen on your bank I don't know and yes SMS two-factor sucks please everyone stop using it so yeah I mean I definitely the the the profile for the attacker has changed in terms of the amount of work you know it took me weeks to write a really good weaponized exploit back in 1998 and now it would take nine months to write something that was you know from initial you know injection point two to two full control is the hard target yeah yeah like it's our target what one of the things have changed and actually since we have two folks from very code here and this goes back to Slynt which was like the original lexical integrity source code checker in it was kind of maybe helped you guys a little bit with the idea there it was at the loft I wanted to do this sort of like Consumer Reports thing and say like let's look at software whether it's binary whether its source code and say like you know tell people what's good and what's bad and the problem was there were no examples of good everything you know was just crap and you can't go out there and give people actionable advice by saying like well if you're gonna choose web browsers well you've got you know CERN's browser and that's it and it sucks so there you go you're welcome if you want a word processor you know you've got this one option and it's crap so you can't do it now we actually have examples I think window 10 windows 10 was a huge step function for Microsoft as far as hardening you've got chrome you've got a couple of good examples that show that you can do solid build quality really hard targets you know I mean at the same time for every one of those you get like a thousand things that just don't do it that are out there at mass but now we have examples of good and bad you know plenty more examples in general of now it's all indexed he used to be that you know if I wanted to exploit every single Network driver on the planet I would go find a bug in MSD ends example code because everyone would cut and paste that now if I want to do something like that I go and I go to stack overflow and I find bugs and stack overflow samples because people are just gonna cut any stats yeah if you want to find bugs in ICS stuff or a skate of DCs stuff Stack Overflow seriously you'll find poor look for resumes and people will post examples of what they've done encode in those embedded control systems yeah and then you just look for where they worked so so and by the way it github is the new pastebin anyway I I think another dimension of what's changed is is definitely the adversary space senator Thompson asked us when we were there he said could a nation-state hire a group of hackers such as yourselves and hack the United States I don't think you would ask that question in 2018 because it's in the news every day but it was actually like a theoretical thing back in 1998 will governments do this and yeah of course our answer was yes you know yes yes they
can well a funny thing was the National Security Council visiting the loft prior to the Senate testimony where they all huddled out our parking lot and we all got freaked out so we ran over and and we said no no we just invited you in and that was a huge extension of trust sort of stuff you have to you know you can go back to your skips in Washington and talk all you want you have to tell us what you're talking about in front of us like huddling and they said well this blows our threat models what you've done we always thought was only nation state capable and so we were wondering if any governments have approached you yet and you know the answer was no but if you'd like to be the first we're willing to entertain offers and luckily they laugh I think that's I think that that's the time when we gave them the lock group you're also and all of you got headaches and stomach body got sick thanks Joe it's amazing they invited us back but yeah rellenos poison the poisoning the government uh attention recount so we know in masks yep of with homemade going up going off of a well Ted is that nation state or not like attacks are so common now we're back in the day they weren't we're like we're so desensitized to it and you know we're naming our bugs and we have giant presentations about it and it's like a real circus but now attack happens you know stock price goes up there's a media frenzy stock price goes up and then everyone goes back to work and then you do the next one right so it's like really this very different cycle related to the tools you know like we were creating our own tools and we were getting stuff out of dumpsters and in reverse engineering things that weren't available to the documentation wasn't there now everything's online and the tools to do stuff especially the hardware perspective around from the resources are out there you can get every tool you need to have hardware for under 100 I mean we went dumpster diving for documentation on more than one occasion now you can just google for it so yeah so that's you know I think it's the access information and the resources which is great and possibly you know for people on the defense maybe not so great so one question before I let the audience have a chance how did the testimony the invitation had come to Congress didn't happen that's a point of debate yes it's actually really not a point of yep well I mean what had happened was at the loft we were trying to do it full-time so we actually had a couple folks on early payroll and I was going out three payroll well actually yeah that's that's right well locked back was paying a little bit yeah we were getting a lot of press and that's actually what kind of started this in my mind we were getting a bit too much press and I was really worried and this is remember the MIT thing where I had to sit up there against them that day who was the FBI director or the cybercrime FBI part and this was hackers for girlies when they had hacked the New York Times and so I'm sitting on a panel you that MIT didn't buy to me with all the loft that's where their tan was there you know this one really well and yeah and the FBI Boston cyber part says like we we're investigating all the leads we've already talked to all the relative people and so I opened my big mouth and say no on the same panel you know you have it because if you look at the source code on the hacked page it calls out greets to the loft and Mudge and I'm sitting here on the panel with you and I'm telling the audience that you never talk to me you know didn't go over too well with the Boston FBI we were also you know making like a lot of news you know the local NECN the cable news and stuff like that and I honestly thought that we needed a contingency plan so I actually started to go out to the government and very quietly I started to advertise that I would train and you guys remember me going out to a bunch of these Air Force information warfare Center you know uh you know Quantico NSA for i4c Forex groups and stuff like that anybody in the government could get me to go out and I would educate them and train them just on tech stuff and I would not take any payment because I didn't want to be under you know I didn't want them to have chains and strings to pull sort of stuff and the goal was I figured that the FBI or the DOJ was going to at some point just from the media try and make an example out of us and I'd pull us up and make some big sort of like we got the hacker as you all know about him sort of set up if that happened that I wanted to be able to like reach out to the folks at West Point and have like you know Colonels and majors show up in uniform and say no these these guys aren't the bad guys they've actually just been very upfront and helped through that I ended up briefing the National Security Council several times and became friends with Richard Clark and then we had the meetings with them and then he became kind of a friend of the loft and he brokered it with the Senate and he and I kind of talked a lot about that and I was uncomfortable with it and then they reached out to us and then I had to broach it with the other loft folks and I remember weld is familiar and candid with some of the back stuff I didn't do a great job of sharing it with all of the loft folks but it was kind of prepared because we were not going to go into the Senate and get ambushed and just get raked up and down the hills it was going to be friendly or else we weren't gonna be we were very scared I had to do I'm terrified walking into that room what kind of reception we would get right I think it's one of the reasons why I met demanded everybody wear a suit because we didn't know who could walk in there and have the Senators call me did not have a suit he wore his father's I mean we were assured that it was gonna be a friendly meeting but we also in the back of our minds felt you know they we could get in there and they're just gonna call us all criminals and and what do we do that because if you watch it you'll actually see that it was a senator Thompson that says like I'm informed that you think you can take the internet down in 30 minutes he was talking about my bgp work that i was doing in my day job which was a government contractor at the time and there's no way that they would have known that had it not been the fact that in the day job and the night job i shared that with the folks in the National Security Council so I mean a lot of these questions you know were sensational but they were relatively safe and it was a little weird because we were still terrified I thought we'd worked out four sets of sensitivities I think we did a good job preparing the written testimony because looking at that like the spoken for me I'm sort of embarrassed about but the you guys are flying the written testimony was like a really good description of what could happen and so we'd prepared that in advance I think we had to send it to them a little bit in advance so public record if you can find it right it's in the public record so we didn't know how they were gonna react to us but by doing that it puts something into the public record that actually was gonna stick regardless and John Glenn obviously uh messed up a little bit when he was like that and I've worked with some of you and I'm like so one of the other interesting things about it was we only agreed to do it if we could testify under our hacker aliases and we we just almost couldn't believe that they allowed this they said the only people who ever testified before another aliases we're in the witness protection program which my this made expensive reimbursement really difficult this was hilarious and I might be giving away a little bit of OPSEC for it part of it was was I mean these guys all know this intimately I hate flying I've got Rifai de flying and we wanted to go to the NSA crypto museum and if we took a train we weren't going to get to the crypto museum before it closed so we rented a Dodge Ram 3500 you know shader van yeah 15 passenger van black tinted up windows and like Brian oblivion and everybody else that the hardware guys are like let's put in tennis let's actually like no map the Northeast Corridor the thing looked like it was it was a SIGINT sort of van going on this is 1998 and there's a story about taking the wrong turn which I won't go into when we went into the NSA wrong entrance with a van that looked like sig and we really fit in because we had all the antennas they didn't know but um when we get down there they were like you know we're checking into the hotel and it was a Lee Anderson Baba Durant you know CCTV engine uff and the weird thing was the hotel was like yeah we've seen this before you're going to the Senate aren't you the other thing about under the hacker names was the reason we did that was because we were protective of our
day jobs right like we are gonna be talking we didn't know how the vendors and the partners of the companies we've worked out we're gonna we're gonna take this well we did know how Microsoft used to handle this yeah Microsoft you know because obviously they can pull strings with your employer right but it didn't work out so well because there was reporters there with microphone with cameras and our picture was on the front of Internet week when we got back into work the next week my picture on the front of the Washington Post the handles though wasn't just for day jobs because not all of us had day jobs some of us weren't old enough to work yet but the you know we were seeing a lot of people off because we were seven kids in a warehouse and we've been there you know you had some professional job still but whatever it was we were remember reading some mailing list posts of people on like you know academic cybersecurity whatever we called it back then list saying why don't you guys come out from behind your handles if you have something to share and whatever and we're like we're sharing the technical information you don't need to really know who we are we're the locked in Boston with kingpin and Mudge and this is what you need but it was also a protection mechanism just in general because we you know what we were doing was not normal also even at that point we had built up as a considerable reputation individually and as a group under those names and if we suddenly started using our real names nobody would know who the hell we were well one of the reasons you don't give up the names if you remember when I when I went down to the Clinton round table in the White House for that and this is where my name actually because I had done I thought a pretty easy job of never having a picture of me never having my name on the internet or whatever and so I get invited to go down because Dick Clark goes like hey you know you guys have done good much why don't you come down and meet with the president for this like sort of photo-op you know this was right after bake will draw the big deal all sort of stuff and to go into the White House you know you have to always give them your social security number and other stuff like that you know because they want to check it's embarrassing if you a child support or if you have like warrants out for you and you're you know being you know invited in to see like the president they gave that list directly over to the media and so I have my name you know they don't get my social part but they give the names and I get a call yeah I'm a reporter I'm back in Boston and they're like hey special you know I just got this seat from the press corps at the White House and it has everybody's name on it is Mudge really Peters at and I'm like I was like okay thank you very much I hung up and I immediately called you so I call the National Security Council going you just blew my cover and goes the White House communications agency they're like oh we'll fix it so they so what they do yeah they fixed it what they did is then they then they sent out to the same press folks the exact same thing with my real name replaced with Mudge the exact same thing now I got death threats from that from the hacker community because folks were like hey you know you're a sellout you know what are you doing are you you know are you fed are you a hacker I mean look at all the stuff a contributed open-source whatever you look at the CFT you can see why I'm doing it I hope but yeah this is part of the reason why you don't want your real name out there sometimes I mean that wasn't a lot of fun as like 26 year old getting random phone calls to myself oh and even in cell phones that nobody else knew because you know there's some good phone freaks out there you know threatening your life so you guys cause some prompted some interesting news room discussions when I was reporting on your activities and and my editor would say okay so we're supposed to identify people with their real names so who is space rogue we need a real name and I'd be like well but that's the name that I have to report that's what they're giving me you know so I had to do some education that this is a new type of you know world a new you know reporting era and so that was really fun in the news room you're not just the media the academic journal said is the same thing I published papers with Bruce Schneier and David Wagner Microsoft's crypto stuff that I worked on USENIX wouldn't accept you know my you know the judged and you know vetted paper because they didn't have my real name on it I don't know if edirol papers I think it was you specifically asked me what my real name was and I was like you know what if it's good enough for the US Senate it's good enough for wire all right [Applause] so let's take it to anyone have any questions alright so we're gonna take some questions I'm gonna ask people really come on over all right anyone has questions please queue up right here a render I hope at least it should be a friendly question I hope you never know well you wonder who was sending all those death threats in our language no I just want to say all of you are directly responsible for me formative years you know late 90s there was no there weren't a lot of examples out there of what to do with these skills and these interests so you guys are the only thing out there and I think you did a damn good job so thank you very much if you think these guys like you know help form who you are please stand up I want to see you were responsible for all of this yeah and it's so these people probably haven't probably weren't even born then look thank you thank you for everything you've done but that actually means a lot because when we were doing the loft I mean especially when we were trying to figure out how to fund ourselves we looked at going into government contracts and we looked at any sort of funding thing where we wouldn't have to become like a commercial entity and it's honest to god why I did cyber fast track when I went in the dark because I'm like I want what we needed at the loft in order to keep doing good stuff available for other folks even if I don't get to do it and actually some of the guys actually did do you know get to take advantage of it in there and the biggest thing was it was a pain in the butt for us to figure this out and it shouldn't be that pain in the butt for other folks because they should be able to do it more easily so they can figure out the next thing and take it further and like you know that's the whole sort of thing it's like how do you get the next team and inspire them or the other people and not make it as difficult and you've released a lot of really good stuff and a lot of other folks in here have released a lot of really good jobs and stuff and that well that's that's why I think we were successful is that other folks who think that or that we might have inspired did neater stuff than we did and well I mean look how Def Con has grown to like we were standing on the shoulders of other people and inspired by other people and then it just keeps growing and it's like this exponential growth where everybody can do something right it's amazing so it's just gonna keep going and you know I feel like now it's just a toe GM there was LOD there's a whole bunch of we were just a little bit and it really is is it's yeah I mean it's an honor to hear that and other than feeling old thank you but it's amazing yeah so thanks yeah would you guys screw up along the way what did we screw up along the way a lot a lot of stuff will not talk about yeah you noticed that steaks not being talked about a lot although it actually was really important for a lot of other areas but it was very personally painful for all of us it was just no no I don't wanna say that was a screw up but it was definitely a learning and growing experience and I tell people that I wouldn't have changed it like even though it was far and it is what it is and we were trying to do something we wanted to fund ourselves if you're trying to do something nobody had done there were not so that steak was the commercial entity that lost turned into after we got VC funding or sellout and then that all kind of fell apart but we wanted we wanted to do it full time and and we took a risk and nobody it's not like we could follow the path of anybody else we didn't know it would happen and yeah there's a lot of there weren't a lot of very security companies out there at the time yeah but actually don't say what shape it shaped a lot of what we did moving forward also I'll tell you one of the things I always wonder if I did right or not is I kind of buried what the take the internet down in 30 minutes was and I did that during my day job I did it during my night job I didn't even wear a lot of it inside the loft with like the BGP update attacks there's a whole bunch of tricks with
putting like the target inside an AS set so it gets discard there's a lot of it we're seeing it now in the media and it's a very viable one actually if you go back through you know like pitar s or some of the other route view sort of things you can see some interesting nation-state stuff going on if you know what to look for and it goes back almost to 1998 which was interesting and I always wonder if I shouldn't have actually released a proof of concept for that I'm so under I'm still wondering maybe now but not that well I know well now it's actually more like you know an hour and a half rather than 30 minutes because all of the end up you know the private gearing exchanges and the i-x peas and stuff like that but yeah that was our big thing was you can't hide behind it you can't make it opaque here's how it works so both offense and defense can understand it and there was a lot of confusion about that and a lot of sensationalism and I always wonder if it shouldn't have been backed up with an actual oh dang but release so scary oh hey guys recognize this you know how do you do I guess I wanted to ask about the changing landscape both of the hacker community and the world at large because people my first question is kind of easy cuz I think it's a know could look could the loft exist today or what we think of the log we think of hacker halfway house we have like makerspaces and hacker spaces but a lot of younger people could see you like oh my god I want to do that but that isn't really that anymore if people want to share minds and maybe even crash together or maybe not maybe that's not what is feasible for people who really want to collaborate in this economy and at this time and in this job market and could the loft exist today or what is it now what if should other people do now I mean look at all the hacker spaces that exist all over the world and they're not exactly like the loft because we were much more private and you know careful with what we were doing sharing private just as far as like we would come over we wanted to be physically located we wouldn't have virtual members because of the direct interchange when we had like you know group meetings and stuff you do a virtual back then because virtual room exists but the hacker spaces that exist are in different regions you know around the world for people to get together like-minded people to get together at work it's not exactly the same you know it was just that particular time I feel like we were lucky that the seven of us plus silly and dill dog when they came in later like M Stefan and Stefan we just happen to all click and it was just it just worked because there was a lot of other people in the Boston community that we hung out with and did stuff with but for us it was just a really sort of lucky thing maybe maybe it can exist I feel like the there's more kind of commercialism of it right because we were doing it and we didn't expect to make money at first when we wanted to try to by selling t-shirts and services and stuff but it was law cracking laws for a kiss yeah I mean it we weren't we weren't planning on doing that but now it's very it's very financially driven I think a lot of times and you can you could do things on your own and have a hobby but if you want to survive you have to promote and do things that maybe are more sensational then I'd like to just go and say yes it can and here are the examples Chaos Computer Club right there you know tool right there fantastic examples of you know that sort of hacker mentality at a much larger scale you want to look at like specialization as the stuff but like dill dog really started to go into you know project zero I mean who doesn't want to be a part of that you know sort of set up so it exists out in that sort of like you know philanthropic sort of thing that's sort of like open thing like cccc it exists inside of organizations like the Google project zero sort of stuff so yeah I think it's even better than it used to be one observation that I've got on that is that many times the availability of the Internet in connectivity has allowed people to collaborate on shorter projects like individual projects you know we have repos now where people collaborate we have slack channels with Google operating they usually it might be overlapping from person you know certain groups of people but you know you'll have people working together for a sole for a single purpose as opposed to simply just having a group of friends that hang out so instead of being people and sort of place oriented a lot of the organization these days is project oriented and you know outcome a goal oriented type work you know I can't count the number of select channels that I'm on now and you know just you know all for different kinds of purposes and not necessarily for the purpose of like making friends and being a close-knit group but for helping like to bring something to bear to help solve problems to push some agenda forward to make some software that should exist a reality or whatever but it's all yeah found that it's easier to like make small bubbles of projects and work on the iterate on those so I'm gonna actually come to Mike so I'm I have a day job and we actually had Karen Ella's re at our conference in Washington and one of the things Karen talked about was that corporate environments in corporate America need to embrace the hacker community security community because a lot of the clients that I talk to still view all of us as enemies and I'll walk into a client meeting and I obviously don't dress like this and people like I know you from Def Con and I have to say now that's not me because it's there's still a perception and I like I brought my buy my daughter t-shirt and she go to school and people go aren't hackers bad and she knows the answer no hackers are people who get stuff to do things that the Creator didn't think could be done right you have a question yes the question is I specifically told you before the before you got out here but I can't moderate myself so so what are your thoughts on on how we can better get that sort of the sensitivities the relationship between what we all do here and what businesses do is wearing conflict more than not I mean this was actually you know what drove cyber fast track and what a lot of folks don't know is that I would get pings from NSA in the White House periodically saying you seem to be a daywalker you can you know you're you're friends with the hacker community you may or may not have a whole bunch of clearances sort of stuff like that over here you know how can you how can we trick them into doing work for us because they seem really clever and I'm like do you really just phrase it that way I mean this is how you make new adversaries and I think the thing that worked the best does not embrace the hackers or whatever it's to recognize that it is another you know group of people and they're different and you don't have to turn them into you and you don't have to turn into them so if the lost city of Atlantis just pops up out of nowhere it has significant capabilities you don't go like we got to make them become Americans and you don't say like oh we have to just you know attack them you don't know what it is you build out a relationship and you figure out where you have like you know goals and similarities and you focus on those and you let people be themselves and the other that's what worked that's where I can't tell you how many times I just stand back nasty emails and Keith Alexander and I went head to head a number of times on this but I don't think you have to embrace it I think you have to respect it and figure out where you have liked overlaps of where everybody could move forward without trying to co-opt people and say like Oh cut your hair I've become a government person or you know the hacker way is the only way where's my hair I think it you know it's funny that you say that I don't go into a lot of corporations but I feel like we've done a really good job as a community to share the food side and what I get from people when I do when I do go into organizations they say it's nice that somebody's here that has a different perspective right because so many people are in their silos they're working they don't go to DEFCON we sort of think it's normal to come here but it's not right so having that perspective [Applause] you may not have noticed but that's not well drill hair either yeah you still weren't your wig you know yeah yeah so I think being you know coming in and having a fresh perspective is okay it's just trying to maybe convince people that that's okay well we we feel you alright thank surrender for breaking the seal on this woman I just want to say you guys you're the original bad boys' you know you're the original sort of hackers and you guys through ups and downs through the years here you are all
these years later and you're doing good you know you you turned it around and you're doing good for the community and for our industry and I just applaud that because it paved the way and showed that it's possible no matter how you start it out that you can turn it around you know and you can you can encourage the community and do awesome things so thank you for that and yes there is a question so but folks should realize this that means a lot to us because that's Johnny long for hackers for charity yeah so hearing that from you given that that's how we feel about you means a lot thank ya thank you that you guys showed me that it was possible so it really encouraged me so thank you but my question is let's assume that you weren't the good guys how long would it take you to take the internet down today well kinetically or that's actually we got that a lot right afterwards and it was weird because we we wrote that a lot but honestly I mean I honestly didn't even share a lot of the BGP stuff inside the loft and a lot of the questions were well if somebody could take the entire internet down why haven't they done to demonstrate it and even back at that time you saw people hijacking prefixes you saw people redirecting it I had a friend at MIT they used to periodically take the East Coast and just route it to a dorm in MIT in order to saturate the lines just to mess with the other dorms over there you know look the loft web server was physically at main East it was on the fitti ring at Mae East so we we knew from whence we spoke we had a friend who actually put it there for us free of charge which was nice the md5 components were all zeroes for everybody for the shared secret in order to talk BGP for the national access point but what people didn't realize is there's no value and actually taking down all of the internet because then you take down all of your targets as well if I want to go in to a foreign area and do a strike why would i black out the skies so I can't fly in there myself so the problem is that you can still take it all down it'll be it would take a lot more because the cascading effect at some of the dampening and there's a little bit of rpki and some 33 79 BGP is sort of stuff not much but by going to private peering points they've made it really easy to go in with a scalpel and take out individual areas reroute it and nobody else notices and we're seeing the crypto hijacking for the mining pools happening we're seeing that you know places like you know Iran in other areas you know occasionally you know leak the old air route advertisements out it's probably not even a hijack it's because they're intentionally routing the stuff through their own monitoring infrastructure remember that time North Korea just disappeared from the internet for like I don't how long I have no recollection of what you're talking about senator there's another aspect to this question because after we did the testimony we had several people come up to us and say hey I heard your testimony you think on the Internet baba but was it because you could do XY and Z and we'd be like well that's not what we were talking about but that would work too and so there at the time there are multiple ways to do what we were talking about and are there other ways that we don't know about to do it now maybe I mean I think you're seeing it being taken down in a very interesting way by social media right now on itself Thanks hello I'm guessing you guys all have a lot of experience with the law and being on the good side and a bad side so my question is kind of around that the Supreme Court came out with a decision few months ago and this is my translation of what I was police need a search warrant to go track down a cellphone but the government for national security reasons don't what do you think of that I think it only applies to geolocation information that in other words where the location of the phone is and tracking of it if I'm remembering the court case properly no I'm sorry I thought there was a lot more to it because I didn't I didn't follow up on that particular case okay I definitely wouldn't say that we're all caught up on the law yet the law is caught up with Joey gee how I ended up in the loft is I got arrested as a kid because I had nowhere to do what I wanted to do and these guys accepted me in so know the law is I don't think I don't know how much we've actually paid attention at least I haven't paid attention to it even now because I would rather do something I want to do and share it and then pay the consequences later then not be able to do it and then you know not be able to reach the information so literally I was a senior official of the Department of Defense and I funded about a hundred and eighty different small projects from hackerspaces and individual folks I did have to follow the law a lot and in particular a lot of us code for title 10 and title 50 in addition to see FAA and Electronic Communications Privacy Act sort of stuff I'm not familiar with that particular case so yes some of us had to follow the law but and that's why we became midnight basketball for hackers for kingpin so I will say though that since that you brought the legal question there are a lot of issues that occur at the federal level and the state level that can require a different viewpoint for input okay a lot of the people who are writing these laws may not have the vision to see all the angles that a specific law or rule may impact and that's where we can come in as hackers right we can help inform the law makers of our point of view and how we think that certain proposed laws may impact us or other parts of the internet and so you know one call to action and I'll give now is to get involved in that process right call your senator right your representative seriously it makes a difference and if you hear about a law or a proposal that's getting put forth let them know that you may be an expert in that topic and are willing to talk to their staff and their age or at the very least give them your opinion your opinion right the letter now I assume for each but there is a hundred people who didn't call nothing now might be a great time to tell every you know Congress person that you can to go ahead and read Matt blazes paper on the safety of secure voting machines that's a key topic right there I mean I've done several briefings in DC four different staffers of different representatives and Senators and they eat this stuff up they really do because they're hungry and thirsty for that knowledge for data they don't have any other place to get an unbiased third party opinion that's not paid for by a lobbyist so if you can get those meetings you can write those letters and let them know your opinions of whatever bills are being frost it makes a huge difference and even more importantly give them your opinion but if you have data points nobody's bringing data to this game our industry is the only industry that doesn't have clinical trials and ground truth in anything else you bring numbers or other things you stand out give me your opinion for God's sakes if you have data or if you start to measure stuff that's how I rebooted dart but that was the entire in like framework that was 125 lines of code malware that took half a billion dollars away from keith alexander and redirected it you know cuz it was like well we brought data what do you have well you had an opinion that's great here's dad [Applause] that actually leads right into my question if you guys were invited back to the Senate now 20 years later what would be your main message to the Senators so maybe to from well we sort of did that we did we had a couple weeks ago months ago yeah a couple months ago but it wasn't directly with the senators it was with it was with staffer established you know it was basically what had changed what hadn't changed and you should like to watch the video I mean there I guess I guess a big message that I would like to bring is that we've come a long way in 20 years but we've got a lot longer to go a long way to go with what we're doing especially with the rise of new technologies like IOT the risk in electronic voting we've got other issues that we need to talk about so we have come a long way in 20 years we're not dealing with the same old doom and gloom but we still have a long way to go and my personal campaign is you know don't stamp things with safe
or not safe don't do then you know that's a single fourth you know like a ul seal or a FIPS 140 sort of like does it pass or not pass what a thing give
them a continuum give them a fuel economy give them crash taste for crash test rating give them the nutritional labels on the food so folks can make informed decisions give them transparency about the libraries that are coming with it's something that is measurable because all of the other industries have this and we don't and I have nothing for or against like fire I versus CrowdStrike versus you know carbon black versus whatever but like they all say no the best and they're all very different and you're putting me in your environment I mean like if you went to the grocery store and you're like well all the food is just food and there's nothing else no other information and your doctors like don't you dare have sodium because you're gonna die you're like oh this looks pretty good bacon there's if there's no information you can't make an informed decision so that's what I'd actually talked to the Senators about it's like that is something that the government can do a little bit and I wouldn't use liability like we said during the first time that went over like a lead balloon with the Senate I didn't said say like you know incentive structures you know or other ways of like encouraging people you know and you know maybe giving them tax break or whatever for organizations if they you know gave that data so that folks can make their informed decisions because one size doesn't fit all it's all how you word it right yeah yeah I think the other thing so that we sort of touched on in this meeting here and something I think that I would spend more time talking about is they had asked us like well what can we do for legislation and sort of the answer was have less of it you know get rid of the MCA and don't prevent us from doing security research because we're we're the good guys we're still the good guys so I think that would have to be a big point of like letting them know you don't need more laws to make stuff happen you can kind of let back a little bit let people get some more freedoms well that was an interesting time because HR 514 DMCA and WIPO a world intellectual property you know stuff was all coming out so our big message to them then was do not make it illegal to see what's in the sausage you know because that's what they were gonna do and I think today it's unfortunately kind of the same message except now it's advertised what's in all the sausage so that folks can figure out whether it's kosher or not because it matters to certain people so this is a tweet from Dan Kaminsky in 2014 for an industry built on layers of abstraction there is a remarkable lack of historical awareness around older technical design decisions in the context of the cloud and DevOps development do you think that's making the problem worse or better could you repeat that yeah we missed a little of it sir at the be good tweet just swallow the mic also please we can't hear you thank you okay Dan Kaminsky tweet 2014 for an industry built on layers of abstraction there is a remarkable lack of historical awareness around older technical design decisions in the context of the cloud and DevOps development do you think that the problem is getting worse or better so you know some of these foundational things that everything still sits on have really not been fixed like like BGP those problems DNS has problems the whole ssl certificate system has problems and this is something that we talked about when we talked a few months ago up at up Senate is you know it just seems like we're just sort of biding time and just hoping everything just keeps going okay as we become more and more dependent on on technical but that technical infrastructure every year goes by were more more dependent you know I don't want to be dependent on that when like the only doctor that can operate on me is doing like telepresence across the planet and someone launches a BGP attack right so we're getting more dependent on the stuff but no one's really going back and doing a good job trying to fix the foundations and I think the hacker community and InfoSec in general has a pretty short memory right so because there's so much information coming out there's what five hundred something talks just at Def Con now and it's hard to sort of know what the prior decisions were even prior work right so we're seeing repeating stuff and I don't know if there's a way cloud or not like how to how to care how to consolidate that in some way and I know you know the dark tangent is trying to do it with info org or something to harvest all of everybody's stalks from all over the world but I don't know how how we can get have a cohesive memory about our entire community or something I think that's what you need to learn what happened in the past with that a lot you might want to forget but oh you know maybe we should just put everything in like blockchain without work I'm a hardware guy in terms of you know the processes by which we write software and things changing over time it used to be that you simply did not find out about security issues until software was published these days we find out about it sooner and sooner because of the availability of various types of testing and I made it my business for like ten years to write static analysis software that could make it easier find flaws before they went out the door but that was met with a lot of resistance by developers to really continue to affect change in the future to improve the state of things developers need to be more involved in finding their bugs and fixing them sooner that said there's a lot of friction there because the tools aren't very good for developers they're not written with developers in mind they're written with app sect people in mind and the profile the use case for developers iterating quickly on code and trying to get things out the door quicker is count its disincentivizes security analysis so coming up with tools that developers can actually use that are low false positive rate and fast enough to be part of their process is really key I know you know I'm going to go ahead and say this is something I'm working on actively you know today I know that that's you know where I need to be in terms of developing tools you know but in general pushing the cultural envelope for developers to make it such that security is something that not only is something that they want to do because it makes them feel good I mean though the developers that I talk to you do want to write secure code they do want to use tools they just don't want to be a pain to be a pain but so meeting them where they are instead of forcing the the hand through you know draconian security teams who won't let code out the door is really where we have to be it's hard work for the security community to do that it's been historically over the last ten years we have simply said you know fix your code or by the time it goes out you're gonna be a gonna have your parents pull down or be embarrassed by all these vulnerabilities but that's intellectually lazy on our part it's simply you know saying that it's good enough to you know embarrass developers into into attempting to write more secure code that's not going to attack the problem at scale we actually have to help them make it easier and that means becoming their friend it means not being it adverse seen as an adversary to development so one of the things that would actually help a lot for developers and DevOps and everything and then I'll touch upon the cloud things I think that's a very important question some sort of feedback so if you turn on like stack cards you have no clue whether it actually inserted them or not right you turn on fortify source so I'm like a Linux system you have no clue whether it removed 90% of the weak functions that you didn't know that you shouldn't put in there with strong ones or point zero zero five and actually it's closer to the latter in a lot of cases so there's no feedback for developers as to whether you know when they're trying to do the right thing it's having an effect or not so I do think that's something that we could do a lot better as a security community is like providing feedback and measurements on the cloud stuff I'm actually really impressed with the cloud from the large providers not so much from the small ones and it's because they're essentially getting buzzed all the time by their users so if you look at Amazon or GCE or anybody else they've they're really hardened in a lot of ways or Azur you know sort of stuff because they have millions of users who are doing crazy stuff all the time in the real world and also the doing a/b testing as to what to roll out so yeah I actually think that abstraction at scale if it was like you know if you think of the world as AFL at large you know just you know as a Mechanical Turk version of AFL yeah that's actually really useful for the large providers but the small ones don't get that benefit and they scare me so clouds a mixed bag this is for Mudge tell us about the election hacking and your response does it have to be much those this topic I've talked about a lot so I've got a question go ahead I've been wondering how if you can tell me I don't know statute of limitations when they you know when that ends but what's your most lying that we did anything that require a statue of limitations ma'am I don't know your your you're a Def Con you're up on this stage I don't I don't know what what was what are your most like what's your most fun or entertaining or memorable hack that you can talk about no comment yeah do you guys remember the midnight poker
nights with Hobbit I'll see you a get out the bull is P in Latvia and raise you a a small internet connectivity area in the Gulf states for you know okay you need to need to shut up right now rogue agents election question I think
the thing to keep in mind when we read about election hacking in the press is that there's a big difference between probing a registration database and changing an actual vote in a real election and there's a very big blurring in the media between oh my god the election was hacked or the the voters were hacked in some state and whether or not a vote was actually changed that's you okay yeah I just I think it's important to keep that in mind and when you when you see these media reports of registration database was probed or copied or whatever that's not the same thing as somebody breaking into an election changing are not the same thing but all this in the media over and over again votes are getting hacked closed systems are vulnerable with this it makes it so that people don't trust the system and I think that part don't understand that so they're not gonna trust the system that less people are gonna vote and there are actually people out there who don't believe our president was elected legitimately because they believe systems were hacked there are people who believe that I think there's a hard problem worse right that's part that was part of the goal is that they're trying to just cause mayhem by forcing people on or change people's mind I'm not gonna get into whose day it could it could be anybody but a lot of this a lot of the information that we're seeing reported in the media and I'm not going to 400 pounds [Laughter] there's a lot of people in that now I've lost my shyness yeah actually it's it's it's really only have a few more minutes left I just want to give a pointer of the answer there Alex Halderman and the folks at a University of Michigan went over to Estonia and what a lot of folks don't know is actually that was part of the Google project that I ran for project vault and watched the video of an actual online voting internet sort of environment it is amazing and terrifying and it is 15 years old and it hasn't been updated and it is still the exemplar of the best thing out there which should make people paused bear in mind that the real election hacking is being done by our own government is called gerrymandering and you can vote against it so I want to thank all you guys that was awesome Eleanor sorry one last question if you what advice do you have for hackers and researchers security researchers today let's go let's start with at the end of the the line and just move this way advice for people in the audience never be afraid to get started on something I oftentimes get into form of analysis paralysis wondering if the thing I'm going to do is going to either make me money or be valuable or whether people will notice it you just simply don't know if it's interesting get started on it tear it apart you know make it your quest to learn about that thing as much as possible break new ground whenever you can if you think nobody else has maybe done it just do it and try to finish what you start it's the hardest thing ever when you've got a million ideas but it's it that's the only way anything ever gets done I think my big advice to security research today is just be careful right there's a lot of companies that don't like you there are laws that don't like you and that doesn't mean don't do it please keep doing what you do but be careful do some research on the laws don't cross the lines stay out of jail stay out of the courtroom I'd say play the long game you know I remember going out to talk to cadets when I was at the law those cadets are now Colonels and they actually have impact same thing with Senators aides same thing with CFD I mean kingpin came up at the end of my talk at DEFCON like a few years back or like we didn't know what the heck you know much turned into the man what's going on and I got a lot of crap for it but then folks saw that now actually that's still kind of much you know trying to make the dent in the universe sort of thing and you made it easy for us to take government money so what play the long game everybody else is optimizing locally optimize globally and for the long term I guess a lot of people ask me like what they should focus on in terms of you know people I work with that are actually coming to me to learn how to get more advanced in their security testing and everybody wants me to point them at something the company is using it would be useful or whatever but ultimately it has to be something you're really interested in because it's gonna take tons of hours of your devotion so obviously focus on that and then the other thing I would say is that it's really easy to get wrapped up in the excitement of things and make mistakes and today disk space is cheap and there's this concept of big data everybody's collecting data on everything so a lot of things that we may have gotten away with as kids I think we got really lucky and it's definitely a much more dangerous environment out there for people so you know make sure you you are careful not to cross the line more careful than you know I think kids normally are yeah but don't be afraid to fail either otherwise you'll be so so I would say you know I hope you welcome the newbies be nice to them they have impostor syndrome and but if they're here they want to learn and the other thing I would say was network with people that aren't just like you like people when you go back to the office developers UI people managers get them to understand how we think so you know spread it a little bit outside our community I think you just have to love what you do and it doesn't matter what it is but if you are passionate about something like that's really what you want to do you want to find something that you really care about and you shouldn't be searching for money you shouldn't be searching for being on stage you shouldn't be searching for you know branding your or whatever yes I know I'm up here that wasn't the goal when I started hacking but it's because I love what I do and if you do that you can look around DEFCON and there's a lot of people that do amazing things that that are not on a stage they're doing stuff that you know you're not gonna read about in the news and it's all over this entire hotel right so you you find something that you love and good things are gonna happen great thank you [Applause]