be here. So enjoy, and all of you guys. All right. So to kick things off, the title of the talk is Tales of a So Hopeful Journey. That's kind of a play off of So Hopelessly Broken, you know, given all of the devices that we've looked at and that we have in the contest back there are riddled with vulnerabilities. So to kick things off, I'm Jacob Holcomb.

This is my colleague, Rick Ramgotti. I'm a principal security analyst at ISE. I head up the technical team, coordinate all of the assessments, and I coordinate our newly founded research division, ISE Labs. I'm the creator of So Hopelessly Broken and an IoT Village organizer. Rick? Hi, I'm Rick Ramgotti. Oh, now it sounds loud.

Yeah. I'm Rick Ramgotti. I am a team lead at ISE. I'm also a security analyst there. I'm also the team lead of the research team. And for those of you that are playing the CTF, the new devices that you see there were devices that we identified during this iteration of So Hopelessly Broken, which is something we'll get to. Okay. So really quick about ISE, we're ethical hackers, computer scientists, really any, we're

just individuals that have curiosity on how systems work, and we attempt to break them. And we do it professionally. Our primary perspective is white box versus black box, wherein we prefer to have access to every piece of material we can, from source code to binaries to developer documentation,

you name it. Anything that will facilitate our understanding of a product, we're likely going to assess it or use it during our assessment. Okay. So the outline for today's talk, So Hopelessly Broken, as well as IoT Village and a few other topics, we're going to talk about the past, the present, and the future.

All right. So the past. Basically, I don't know, it was the end of 2012, shortly after I first started working at ISE. I was publishing CBEs in, I don't know, various services that run on network appliances from Cisco and a few other manufacturers.

And I was just kind of doing these as a one-off thing, you know, for fun. In my spare time, I'd come home from work, I'd be like, oh, there's a router. I had a Verizon Fios action tech router, I had access to Cisco Call Manager Express, as well as some other things. And I kind of would just poke around at them. I'd find vulnerabilities, I'd publish them online. And one day, my boss, Steve Bono, was like, you know, maybe we should do this in a more

organized fashion. So we kind of decided to, I don't know, look at network appliances, we focused specifically on SOHO routers, picked 14 of them, and started to conduct a security assessment of each of them. So the research originally started with 14 SOHO routers.

Our objective was to find vulnerabilities from a remote perspective that could be weaponized in such a way that it would allow a remote adversary to compromise a system and gain access, unfettered access, to the device and devices within the network. So we focused predominantly on network services that may be remote and local.

In some cases, the services only ran on a local interface, but we would leverage other vulnerabilities to expose those services to a remote perspective. And basically, once we concluded the research, it was time to present results. And we kind of came up with a talk name, SOHO Opusly Broken, and that's where things

kicked off. So it started out originally as a research project and a talk to kind of highlight the insecurities of SOHO routers, and it evolved into a contest. And we actually hosted the first SOHO Opusly Broken contest in 2014 in partnership with the EFF at DEF CON.

And it was a pretty small event. It's most of what you see today, but on a much smaller scale. We had only a subset of devices that you see back there today. And basically, it allowed our contestants to take some subset of the 56 CVEs that were published during the SOHO Opusly Broken research and use them to compromise these systems

in an attempt to find flags to redeem them for points. So if you're curious about the old research study, there's a link here, but you can just navigate to securityevaluators.com. There'll be a research tab, and you can kind of review all of our papers there. All right, so IoT Village origin story.

The first IoT Village was in 2015, and that was at DEF CON 23. And really, it was an expansion of SOHO Opusly Broken. So when we first kicked off SOHO Opusly Broken, as I stated, we were in partnership with the EFF, and we were kind of pushed into a small corner within the wireless village. So we didn't really have our own area. We were sharing a space. It wasn't very conducive for hosting an event of this magnitude.

But we were hungry because the event was successful. We wanted to do more. We wanted to bring more researchers together and like-minded individuals who would help kind of, I don't know, disseminate our ideology of building security into these devices instead of thinking of it as an afterthought.

So that's kind of how IoT Village started, and that will kind of bring us to where we are today. Okay, so here's some more statistics on IoT Village. Pretty much in the few years that we've been running this, we've had 30 plus devices in

our CTF at any given time. So those 30 devices span a couple of different networks, and contestants get to attack them. Challenges are of varying complexity. Some are extremely easy and some are very complex. And it's not really the type of vulnerability that determines complexity. It's the way in which the devices have to be exploited.

It's akin to challenges that would be encountered when performing network penetration testing. We've had 60 plus devices in our zero-day track. If you actually look in the very back room or stop back there, you can't really see it now. We have a very large table with 30-some devices on it this year alone. So that's pretty cool. I encourage you all to kind of go back there, grab a device, and try and find a vulnerability.

We have a handful of ISE analysts back there who would be willing to facilitate your efforts as well. So, yeah, go talk to them about zero-day devices. We've also hosted 38 talks. I believe there are some past presenters at IoT Village sitting in the audience here today.

So thank you guys for submitting your talks to our event. We're really appreciative of that. And finally, we've had 113 vulnerability disclosures. So through hosting our contest, researchers have submitted 113 vulnerabilities, and they've all been responsibly disclosed to the manufacturer.

So that's pretty cool, given that people could use them for nefarious purposes. They could sell them to a higher bidder than the reward that you would receive by submitting them through our contest. So that's a pretty neat statistic. All right. So I'm not going to read these, but here is just kind of a quick list of the different conferences where we bring IoT Village or some subset of the village, usually so

obviously broken. We're pretty much all over. We've been down in the Caribbean. We just got back from Hawaii. We come to DEF CON every year. So if you're curious and you want to check us out, spend more time with us, we're usually at all of these events.

So what we do here at the IoT Village I think is very important. And for those of you that are here today, not only for the pretty lights, I wanted to remind you as to why we actually do this. The IoT Village is formed out of, I guess, like three things here at DEF CON. One of them is the stage, which is what we're doing right now, where people have the ability to present on what vulnerabilities that they've identified or just their

methods for finding those vulnerabilities. What that means is that we, I guess a couple months or a couple weeks ahead of us getting here to DEF CON, we have a CFP and people can submit to that CFP so that they can have the opportunity to present at DEF CON in the IoT Village, but at DEF CON about what

they found and how they found it. We also have the CTF, which I feel is what most people are here to do today. And for those of you people that are staring at your screen right now, the reason why we have the CTF is that it's designed to both help you identify vulnerabilities and sharpen your skills and to see how it is that these things are exploited. And for newcomers, it's just an opportunity to learn on how to do it.

We've also had, we've also been very lucky that past contestants sometimes come back just to help people because it's a very joyous experience to finally pop a shell on an embedded device that you may actually have at your house. The other part, like Jake mentioned, is the zero-day contest. The zero-day contest is all the way in the back. And it's just a bunch of devices that some manufacturers donate to us.

Like, for example, Starbucks this year donated a latte machine, I think it is. And an espresso machine, very different. And there's also a washing machine that GE donated for you guys to hack as well. And there's a bunch of other devices there. I think Nest gave us a bunch of thermostats. I don't know. And other manufacturers as well.

So I really hope you guys enjoy it and make the most out of your time doing it. Like Jake mentioned, we're all here to help. So if you do see anyone with an ISE shirt like this, please come by. Please stop by, bother us, and we'll help you out. Okay, my shirt has it too. Mine's different. Oh yeah, yours is different. Yes.

Okay, so that brings us to ISE Labs. Really, ISE Labs is the research division of ISE. ISE has always been a proponent of security research, hence newly founded division, research projects hopelessly broken, 2012, 2013. But basically, this is a more organized approach to performing these types of projects.

We actually have two teams of researchers that focus on different technologies. And we're actively looking for security analysts to join ISE, but to also become a researcher on our team. So there are a few open slots. So if security research is something you're interested in and you like hacking products,

I encourage you to see Lisa Green in the back and talk to her about our open positions. But the idea here with ISE Labs is that we take a bunch of, we take a bunch of security researchers who are hungry to identify vulnerabilities, you know, to help make the world a better place, if you will.

And pretty much publish anything that we can find. So we choose a target, we figure out, you know, exactly what our mission is, we create a proposal, and then we'll execute, you know, a given research project. And that actually brings us into the first research project that Rick is heading up, Soaplessly Broken 2.0.

Cool. So a little bit about Soaplessly Broken 2.0. What we did this time around is it was very similar to what Jake had done in 2013 or 2012? 2013. And we also have 14 embedded devices that we're looking at right now. We're also looking at them from a remotely accessible perspective. What that means is that we're trying to hack these devices over their network interfaces

or any other application that they're hosting, or any other service that they're hosting. We're looking for weaponizable exploits. Weaponizable is not a word, I'm very aware of that, you don't need to remind me. But what we're looking for is that, we're looking for exploits where an attacker can remotely access a device over the network, or even if it is over the local network.

We're also looking for application security vulnerabilities. We do work for an application security consultancy, so this makes sense. And what we're looking for is how these devices work, because it is actually what we're interested in. Now let's get to the results. First off, we have, this is a subset of the results, Jake is actually very right about that.

Today I'm only going to cover three devices, and the reason why we decided on these three devices is because they're also in the CTF. So if you are playing in the CTF and you're wondering why you're not able to get to these, this is why. You'll be able to get to them once you find our blog post and all of our presentations. The first device is the Acer Store AS602T, which is a mouthful.

And we found out that it has an unauthenticated access to the SNMP API. This was actually found by Ian Sinderman, who is most likely hiding behind the table over there. So if you guys can please give Ian a hand, that'd be awesome. Ian, they can't see you.

Ian? Sam? Oh, never mind. Anyways, but we also found out that there was command injection in that SNMP API. So if you see the word command injection and unauthenticated, those things usually mean bad things together. They're bad independently. They're also bad independently. They're worse together. It's like me and Jake, worse together.

So we also have the file read directory traversal and a file upload directory traversal that was found by one of our other coworkers, Sean Marani, who is not here today. And the password was returned in plain text. So Ian found out that when you connect to the SNMP API, you had the ability to get the password for the device in plain text.

And there's also a couple of cross-site scripting that we identified as well. We also have the TerraMaster FS420. We added the 420 in green because I thought it'd be funny. But it doesn't seem like it went over that well. In the device, there is a bunch of vulnerabilities. One of them is an unauthenticated command injection.

Joshua Meyer from the research team identified this vulnerability and he had a blast with it because he started off at like 8 a.m. because he's an early bird and by the time he left at the end of the day, he had found many vulnerabilities. One of the funnest ones was the SQL injection, that's the second bullet point there. He found out that he was able to use that SQL injection vulnerability to get another command injection vulnerability

because it was shelling out whenever we tried to connect to the SQL database. For those of you that are developers, it probably doesn't make any sense because you're like, why would I ever want to do that? You should probably just talk to them and figure it out. And there was a file upload patch traversal in this one. We also found insufficient authorization verification. So a lot of the APIs that were on this device, specifically the web application, were accessible to anyone

once they knew the path of what they were trying to access. So it's kind of like a secret directory kind of situation or a secret path where it's more security through obscurity instead of actually saying, oh, they'll never find this and that's why the vulnerability is there. We also found a couple of cross-site scripting in there because cross-site scripting is still alive and well.

And cross-site request forgery as well because they were using tokens, but they weren't really worried about how the tokens were being sent. If you really are curious about how cross-site request forgery works, there are a couple of devices in the CTF that use cross-site request forgery to get the final exploit. And if you are curious about how that works,

one of us with these black shirts, either IoT Village or ISC can help you out. Then we have the Seagate STCR. This is actually another device that Ian worked on. And at ISC Labs, we're doing these things that every now and then we'll pick a device and the ones that were the most fun, and we go through the process of reverse engineering it

and exploiting it live on YouTube. So if you are curious to see what it is, what the frustration is it takes to hack one of these devices, I would recommend looking them up. It is our YouTube channel. I think it's just Security Evaluators? It's just Security Evaluators on YouTube and you'll be able to find it. It has the same color scheme and the logo at the bottom,

so you'll be able to find it quickly. Ian found out in this Seagate device that they were using a custom authorization mechanism where they would HMAC a version of the user's passwords and use that to HMAC each request, so you can modify the request. And he goes through the process of how we reverse engineered that HMAC process

to forge a request and then eventually find the file read-patch for us vulnerability that's listed there. And he uses that to read up all the source code that's running on the device because it's running a Python web server. And after he goes through the process of pulling all the source code, he finds more vulnerabilities and it's just a happy day at the end.

So what are some things that change between SOHO 1.0 and 2.0? This is only a subset. We're actually going to be publishing a white paper that covers the process of how we found these vulnerabilities, all the devices that we looked at and all that. Also for those of you that are wondering, as Jake mentioned during our first iteration of So Hopelessly Broken, we found 56 CVEs.

This year we doubled that amount and it's still growing because we're not done yet. So I really hope that you enjoy reading the white paper whenever it is that we publish it. If you're interested in seeing when we publish it, you can probably just follow us on Twitter. I think our handle is just security evaluators time. Our handle is ISE security evaluators.

Or you can just follow either Jake or me on Twitter as well and we're probably going to be retweeting it. So what have we found out different between SOHO 1 and SOHO 2.0? So the first thing that we found out is that there's still plenty of vulnerabilities out there. As I mentioned, we doubled the amount, although they did this in 2013 and it's now 2018. And we also have a different methodology. And we also have a different methodology now when we're performing these security assessments.

The other thing is that security through obscurity is still alive and well. Like I mentioned, both in the TerraMaster and the Seagate device, it was kind of more of a trying to hide things from the user instead of actually putting in a proper mechanism for protecting it.

And we go through the process of reverse engineering these to show how it is security through obscurity and how that doesn't really help anything. And for the watching the whole process of how this is done, you can just check out our live streams. When we get back from DEF CON, I think a week or two weeks in, we're going to have another one where Joshua Meyer is going to go through the process

of reverse engineering the TerraMaster. Some more things that have been different between SOHO 1.0 and SOHO 2.0 is that some manufacturers actually care. I'm really happy to say that we just bought embedded devices that I found on Amazon that were the highest rated. And some of the manufacturers are really interested in saying like,

oh, you identified a vulnerability. I'd love to work with you to try to resolve this. And they're actually pretty quick. A lot of the times when I responsibly expose, I responsibly expose the next day. So usually I get around nine and I report it. By the end of the day, most of the times, I have a response from the manufacturer. That's kind of good to hear because we often hear about how bad embedded device security is,

but it's probably a little bit comforting to hear that some manufacturers actually care. On the other hand, still some not so much. I've sent out a lot of emails, not always that they responded. Some manufacturers just kind of say like, hey, we'll get to this and they never do. It could just be that they're busy and I understand that. But from what it is, like a month or two in,

if we're not receiving anything, we go through the responsible disclosure process and we continue as necessary. Another good thing is that more companies have bug bounty programs. So we still haven't finished the responsible disclosure process. I'm not going to drop any names. But the good part is that some of these manufacturers have been responsive saying like, oh, this is awesome.

I'm going to send you a shirt that says our company name on it. Or we're just going to list your name on our bug bounty program. But knowing that they have a security team that's actually involved and cares about securing the device is a good thing to hear, I guess. On the other hand, some of them don't respond at all. And I really wish that this was something that was held more dear to them

because security issues is something that can definitely cripple a company, which is why there are so many people here at Def Con anyways. And that's something that more manufacturers need to be aware of and they should probably pay more attention to. Another thing is that some manufacturers are interested in working together. Like I mentioned, we have some manufacturers that are actually our sponsors here today.

And they are curious to see if people can identify vulnerabilities in their devices so that they can responsibly expose them and help them make them more secure. Repeating the second bullet point again, some manufacturers really don't seem to have a lot of time to respond to our messages. And that's something we hope to be changing with the IoT Village becoming more popular.

And now I'll pass it over to Jake. Okay, so that kind of brings us to the future. We talked about the past, we talked about the present, you know, where we are today. Let's really kind of talk about where we're going tomorrow. So ISC Labs, we're producing content as the result of various research projects.

We're doing live streams, blog posts, we're writing white papers, we're publishing exploits, most of which we haven't done yet as we're in the process of, you know, we're in the midst of working on these, a couple of research projects. But we, as Rick mentioned already, we put out a couple of live streams,

we're going to continue to do so. So more content is coming, it's going to be coming on the regular. So please stay tuned, you'll be able to see all of our sweet stuff. We'll have everything from vulnerability walk-throughs to full-on, like, tutorials. I know of how you would employ a methodology of, to perform an application security assessment on a device or some application.

So the future of Soaplessly Broken. Soaplessly Broken has, I don't know, it's gone through many iterations and we've improved the contest iteratively each time. However, there are still some challenges that we're encountering and we'd like to address to really enhance the experience for participants.

So in the future, we're going to be adding more devices. One of the challenges currently, we don't have internet access for attendees that are connected to our network, meaning you're going to have to leverage two network interface cards to connect to our network and some other public network that will give you internet.

We're going to change that. So you'll plug right into our network and you'll be given network internet connectivity. We're going to create a more robust system for reflashing and resetting systems. So if somebody corrupts the system in such a way that prevents other people from getting in after they're done with exploitation, that is, the devices will automatically reset.

So it'll be a little easier for you guys to play and a little less frustrating. And then also we'll be adding more devices, both physical devices and virtualized or emulated systems for scalability. So there's definitely some improvements coming to the event. So play it today, play it tomorrow, play it the rest of DEF CON.

Provide us with feedback and we'll go ahead and add to our list to continue improving it so every iteration is better and better. With regards to the IoT village, the things that we'll be working on making better is that we would like for more people to submit to our CFP and we're also going to be working, as we are also,

ISC Labs, we're also going to be working on creating more exploits and helping people identify vulnerabilities in embedded devices. One of the other things that we're going to do is, as Jake mentioned, we're going to have a lot more embedded devices and that's going to come as part of ISC Labs as well. Or just people identifying vulnerabilities and responsibly exposing them to the manufacturer.

Another thing is that we're actually working with more and more companies that want to come to DEF CON, bring their devices and have people hack them just so they can make the world a more secure place. And that's something we really hope to provide here as the IoT village. And also in future iterations of the village, we'll have some kind of like hands-on workshops

where you'll get to work with some of the researchers from ISC Labs and some form of exploitation exercise. I don't really know what that looks like today, but it is something that will come in the future. So we'll have more programming that's hands-on that you guys can interact with and interact with our staff. So yeah, let's wrap up.

Really, so obviously broken, hacking contests, IoT village, a community of researchers to identify vulnerabilities to make the world a better place from an embedded electronic perspective. We're really passionate about this event.

We're really passionate about the work that we do and trying to secure the industry and all the consumers of devices. So with that said, I mean, we're pretty much ready for questions. I would like every one of you who is here today, whether you have experience hacking a device or this is your first DEFCON, to please go to the back table

and talk with some of my colleagues. I'll be back there as well. We can kind of walk you through participating in the CTF, so it was broken. Or maybe if you're interested in looking at a zero-day device, we can at least sit down with you and kind of talk about a methodology you could use to possibly identify a vulnerability.

So yeah, any questions? Yes. So the methodology change was really just looking for additional vulnerabilities. So obviously broken 1.0, the objective was to find vulnerabilities that would lead to a compromise of the device.

And 2.0, we're still trying to achieve that same objective, but we've expanded our scope to include any type of misconfiguration or issue that could be considered a security vulnerability that when changed by the manufacturer would increase the security posture. So you could think like in a web server, certain security headers missing.

So that would be an example. Yes. You answered it. Yeah. So for the most part, I try to make it as warm and fuzzy as possible, and a lot of times you're just keeping up with them

and ensuring like, hey, we're trying to do this so that both of us can make this world a better place. And with regards to language, there's only been like three companies that haven't responded in a proper way or just haven't responded at all. And for those, it happens to be the companies that don't have a security email address or don't have a security division at all.

So the ones that we have been having a lot of success with are the ones that actually have a security department and they publish that.

So for that part, we actually found out with one of the manufacturers that they were saying like, we don't see this being a vulnerability. So what we ended up doing is that we recorded our screen while we were doing it, and then after seeing that they said like, oh, there's actually some real world applications with this vulnerability, we'll get it fixed.

Yeah. And additionally, we abide by U.S. CERT's policy pretty much after a 45-day window. We'll disclose the vulnerabilities unless we've had some formal agreement with the manufacturer. So any other questions? No? All right. Well, that's it, guys. We'll be in the back pretty much all weekend,

so come see us. Check us out. Thank you.