Dissecting the Teddy Ruxpin: Reverse Engineering the Smart Bear
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 322 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/39762 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
DEF CON 26217 / 322
18
27
28
40
130
134
164
173
177
178
184
190
192
202
203
218
219
224
231
233
234
235
237
249
252
255
268
274
287
289
290
295
297
298
299
302
306
309
312
315
316
00:00
Reverse engineeringPlastikkartePlastikkarteContent (media)CausalityInformation securityRoundness (object)Sound effectRight anglePresentation of a groupReverse engineeringFrequencyComputer animation
00:32
Information securityHacker (term)WebsiteComputer networkGraphical user interfaceSoftwareComputer hardwareContent (media)Function (mathematics)Presentation of a groupSoftwareInformation securityString (computer science)MereologyMultiplication signMusical ensembleReal-time operating systemHacker (term)BitIntegrated development environmentTwitterStreaming mediaTopological vector spaceSound effectComputer hardwarePresentation of a groupExploit (computer security)WebsiteError messageRoot
02:01
Compact CassetteComputer hardwareComputer fileCompact CassettePeripheralTouchscreenRevision controlComputer animation
02:46
Infinite conjugacy class propertyMobile WebData storage deviceMassMemory cardDynamic random-access memorySynchronizationFitness functionData storage deviceAuditory maskingWhiteboardMobile appMemory cardRevision controlLogicAreaProper mapMusical ensembleFlash memoryRight angleDevice driverComputer animation
04:29
LogicGreatest elementWhiteboardComputer animation
04:44
Module (mathematics)Memory cardDevice driverModule (mathematics)FirmwareSlide ruleComputer animation
05:04
Module (mathematics)DiagramDifferent (Kate Ryan album)SoftwarePersonal identification numberComputer animation
05:38
Utility softwareCore dumpFirmwareSoftware developerFirmwareVideo gameConfiguration spaceLink (knot theory)Projective planeComputer animation
06:05
Mobile appWeb page
06:29
Android (robot)Content (media)Wrapper (data mining)Wrapper (data mining)Content (media)Graphical user interfaceComputer fileSource codeXMLProgram flowchart
06:49
Flash memoryShape (magazine)Source codeXMLProgram flowchart
07:13
Asynchronous Transfer ModeWeb pageMobile appPurchasingInformationFirmwareSheaf (mathematics)Structural loadAddress spaceComputer fileExclusive orPresentation of a groupComputer fileUniform resource locatorFunctional (mathematics)File formatFirmwareSheaf (mathematics)Data storage deviceString (computer science)PlanningReading (process)Greatest elementMultiplication signRadical (chemistry)Slide rulePersonal identification numberTransmitterMathematical analysisPoint (geometry)Content (media)PurchasingCharacteristic polynomialMultilaterationError messageWebsiteSet (mathematics)Computer animation
09:33
Computer fileEmailRow (database)String (computer science)Table (information)Computer-generated imageryData storage deviceRaw image formatAudio file formatEmailTable (information)Row (database)String (computer science)File formatComputer fileMedical imagingComputer animation
10:00
VideoconferencingFrame problemComputer-generated imageryFile formatSynchronizationSingle-precision floating-point formatSet (mathematics)BitComputer fileTable (information)Medical imagingFile formatSlide ruleMultiplication signComputer animation
10:52
EmailSample (statistics)Bit rateAlgebraic structureComputer fileTable (information)Data structureError messageMathematicsString (computer science)File formatSoftware testingEmailFigurate numberSampling (statistics)Computer animation
11:44
EmailTable (information)Position operatorComputer fileAlgebraic structureTable (information)InternetworkingPosition operatorComputer fileSingle-precision floating-point formatStylus (computing)HexagonComputer animation
12:42
VideoconferencingElectronic visual displayComputer-generated imageryFrame problemClosed setTable (information)Open setTable (information)Position operatorMedical imagingFrame problemMultiplicationComputer fileNumberComputer animation
13:49
Dressing (medical)Computer fileTable (information)Position operatorSoftware testingComputer fileTable (information)Data compressionMultiplication signRandomizationInternetworkingComputer animation
14:24
Table (information)Bit rateSample (statistics)Sampling (statistics)Table (information)Bit rate2 (number)
14:44
outputFunction (mathematics)Message passingOrdinary differential equationData compressionoutputData structureContent (media)CodeMatrix (mathematics)Computer fileMass storageData storage devicePresentation of a groupReverse engineeringComputer animation
15:35
Content (media)Matrix (mathematics)Slide ruleSharewareVideoconferencingRule of inferenceRight angleMusical ensembleComputer animation
17:17
Computer hardwareStaff (military)Smith chartPresentation of a groupStaff (military)Computer hardwareServer (computing)Video gameFamilyExploit (computer security)Hydraulic jumpFreewareLecture/Conference
Transcript: English(auto-generated)
00:00
Let's give Xenofex a big round of applause. All right, hi, everyone. I guess we're gonna get started two minutes early, which is great because I packed in a lot of content into this period, so. My talk is Dissecting the Teddy Ruxpin Reverse Engineering the Smart Bear. It's essentially my experience taking my child's toy
00:23
and trying to see if it was gonna cause any security nightmares for me. But we'll get into some fun things here on this presentation, so. A little bit about me. First of all, my Twitter handle's at Xenofex. I go by Xenofex. I'm a security researcher at Stylance. I'm a founding member of a hardware hacking group
00:42
called The Exploiters, and I'm a contributing member to a local community that we have called Austin Hackers. A little bit about Exploiters. We have roughly 10 members, give or take a few, that don't officially associate with us, but they are pretty much part of the group. We have Agent HH, CJ, Cody, Gynophage, Maximus,
01:04
MBM, Sarek, TD Wang, and XOO String or Null String, and of course me. So our general goal is to just hack things, anything, anywhere. We originally started hacking Google TVs, and then Google killed off the Google TV, so then we just started hacking
01:22
anything we could get our hands on. We have a pretty decent community, and we're all very helpful, so check out our Exploiters site where we have like 60 plus embedded devices that have roots and other hacks. We also have an IRC network that I'll talk about at the end. So a little disclaimer, first of all. The data within this presentation
01:41
was all stuff that I reverse engineered. I didn't have official documentation or anything else, so a lot of my attempts were just essentially trial and error in reversing what I could, when I could. I literally have been working on this for a very long time, and you'll see why it's essentially an RTOS environment,
02:03
which can be a little more difficult because after you want to interface with the peripherals. So terminology. You Teddy-o-philes will, will not, or will already know this, but an Iliop is a brown bear-like creature with a kind disposition. You might think Teddy Ruxpin's a bear,
02:22
but he's actually an Iliop. So the OG Iliop. This was the 1980s Teddy Ruxpin, I think a lot of us are probably familiar with. It used cassette tapes on its back that you replaced, and you had physical books that read along with the cassette tapes.
02:41
His mouth moved and his eyes opened and closed, but it was a physical movement, not an LCD screen, like the newest revision. So the new Iliop. The new Iliop is this guy. Essentially, he has animated eyes, a moving mouth, speaker, Bluetooth low energy, USB mask storage
03:01
that is used via an internal micro SD card, and a companion mobile app. Getting inside Teddy, so I'm about to show y'all what this looks like. It's terrifying. So a little about Teddy. This particular revision comes with a mask
03:23
that you put on him so he doesn't scare your kids when he's off. It's needed, too, because like I said, it's crazy. But let's take his jacket off and get him out of here. This is essentially Teddy Ruxpin. This is Teddy's skin. So it's pretty enjoyable.
03:41
And here's Teddy on. No. No. I actually tried to wear his skin.
04:00
It doesn't fit on my face. So inside of Teddy, we have a logic board. And this is the top of the logic board. It's actually stored right in his eye area here. There's this chip called the Sonix MCU. I don't remember what the actual name of it was.
04:21
But there's a speaker driver, the Bluetooth low energy, there's an SPI flash, and then there's this SD card slot that actually ends up holding the storybooks that are stored on Teddy Ruxpin. This is the logic board's bottom. You can see there are two 128 by 128 LCDs
04:41
that are used for each individual eye. And then on that previous picture, we'll go two slides back, you can see that there's, on the, I guess, right side, there is a module that says Teddy Firmware 1.1. That module is an MYN822 BLE module, which is essentially just a module for the Nordic NRF 51822 chip.
05:05
So here is the diagram for interfacing with its SWD port, and also all the different GPIO pins that are used within Teddy. They only have roughly 11 GPIO pins in use
05:22
of the 22 or so that are available, or 26 that are available. And then, of course, they have SWDIO and SWClock hooked up, which is the debug pinout for software debugging Cortex-M0 chips. So with that particular pinout, we were able to dump
05:43
the firmware for Teddy, and also the RAM. You can dump it with OpenOCD, but I had a Seger JLink based on doing some badge development for the whole badge life project, and I just used that with an RFJ prog to dump the firmware and RAM.
06:01
Like I said, you can use OpenOCD, and if you ping me, I can give you a config to make that work. So this particular Teddy, instead of having physical books, he uses a mobile app. And so you can see in the mobile app picture, they have a little cartoon picture of Teddy and his best friend, Grubby. So the mobile app works essentially by using BLE
06:23
to communicate when each page is turned and when a next story should be read. So I took that, and I threw it. I took the APK, and I threw it into JADX GUI, and it was really nothing more than a wrapper for a bunch of Adobe Air content.
06:41
So within the APK was a SWF file, and I took that SWF file, and then I threw it into JPEGS. Essentially, this is a Flash decompiler, and all the BLE stuff was within this SWF. So if you are poking at your own Teddy,
07:06
you're better off not even looking at the APK. Just unzip it, grab the SWF, and throw it into JPEGS. So I went ahead, and I listed all the BLE info, the receive and transmit UUID characteristics,
07:21
and all the commands to jump between storybooks and to choose individual storybooks. I don't plan on reading you all that. You can look and reference the slides. At the bottom of the presentation, I didn't mention this at the beginning, but at the bottom of the presentation is a web address for our website. If you go to that website, here after the talk,
07:41
I'll have slides and some of my research content, but I've been having trouble updating it, and I really don't want to on the hotel wifi. So just check it when you get home. So you take the firmware that you dumped with SWD, and you can throw it into IDA, and it shows up as just binary data,
08:00
but then if you choose ARMv7 LE and enter in these settings, you can actually go and look at the disassembly and try to reverse some of it yourself, but realistically, most of my stuff was done through visual analysis of the storybook files and a ton of trial and error, so I got it in IDA a little too late
08:23
for me to spend too much time in this particular section. So the Teddy Ruxpin books. So I showed you all Teddy's face and body. On the back, there's a micro SD pin, pin out or header, and you can essentially connect into that, and it pops up this mass storage device
08:41
and has all the books on it, which are an intro file, an idle file, and then the 10 storybooks that they provide. I'm thinking the idea was that they would release books at a later point for purchase, and you just copy them over to the bear because they don't provide functionality to transfer it over through BLE. So within the storybooks,
09:02
the files that are the container format for the storybooks are these SNX ROM files. Since I don't have the documentation, they could be called something else, but the magic string at the top of each file is SNX ROM in wide character, so each one is individually null terminated.
09:20
The target exclusive contains two extra stories, but it was always a little more expensive than I wanted to pay, and I bought six of these, so I have way too many Teddies. So the SNX ROM files, I'd mentioned that they start with a magic string at the top, but then there's also the header format.
09:40
The header format starts with a record stop and a record end, and then the table itself ends with FFFF. The data, after you use that to extract the files within, you end up with the raw image data first and then the audio files. The audio files always start with AU, and the raw image files are all the rest of them.
10:01
If you take the image files and you throw them into GIMP, GIMP has a feature that you can import raw data, and then you can kind of play with the settings to see what the data is, which is on the left, you see the picture there. You can see that they're RGB 565, and then they're 128 by 128 sizing.
10:21
So then the audio files, this is where things get a little bit crazier, and this is where I spend a ton of my time. It's a proprietary file format used by Sonix, and it actually uses this thing called a mark table to store the triggers for the data
10:41
that shows on the eyes, and then also for the mouth movement. Essentially, and I'll get on that in the next slide, but we'll get there now. So the header structure for this format is essentially the AU string, two bytes,
11:00
an unknown constant value, since I was using a lot of trial and error and I didn't have the documentation or anything for this format. There was a bunch of values that didn't seem to impact my tests, and I couldn't figure out what they were actually used for. So the unknown constant value is the two bytes that seems like it's on every single file
11:21
and never changes. There's then the sample rate, which is two bytes, the channel, which is always one, but it's also two bytes, another unknown value, another unknown value, and then a zero or one to dictate if there is a mark table, a silence table,
11:40
another unknown value, and then the mark table data, silence table data, and audio data. The actual data structure, when I say audio data, mark table data, and silence table data, is the mark table is defined by two to four bytes that signify the position, two bytes that signify the value of the data at that position,
12:01
and then that particular table ends with 0xFFF. It also, if the position value is over 8,000 hex, it takes that and uses the next two bytes, adds them together, and uses that as the position value.
12:20
The silence table, it was 0x0 in every single Teddy Ruxpin file that I checked, and so I don't know too much about that particular table outside of just what I've done for some internet sleuthing, which I'll talk about soon, and then the audio data,
12:40
which is 16-bit signed Little Indian. The mark table, so when you're looking at the mark table, how this thing actually works is the mark table has the position value, and then the actual value. If the value is a zero, the mouth is closed. If the value is one, the mouth is half open.
13:02
If the value is two, the mouth is full open. Now, anything after that that you specify will correlate to image frames that are within the storybook, so if you wanna make, let's say, a special logo pop up or his eyes blink or something, you would essentially put the value of that image data,
13:24
and then you would set up multiples to essentially make it a moving image or whatever you're trying to do, but everything that you do, let's say your image file is number one, well, you're still gonna have to be offset by the mouth open, mouth closed,
13:41
and full mouth open values, so whatever it is, you have to offset it by three. So then let's look at the silence table. So I talked about it earlier, but I didn't actually mention what it's for. The only thing this is here for is compression. They don't use it. I guess the stories weren't big enough
14:00
or it wasn't needed, but it essentially just references silent data and marks the position in the table, and the only reason I knew that is because of just random internet searches on the subject. Unfortunately, Teddy doesn't use it. I just know that the files themselves have that field,
14:21
and in my tests, they completely broke any time I tried to enable it. So then we go to the audio data. It is 16-bit Little Endian signed data that's stored after the mark table and silence detection table. It only supports 16 kilohertz sample rates, and then it supports bit rates
14:40
from 16, 20, 24, 28, and 32 kilobits per second. So what I've done for people who wanna hack their own Teddy Ruxpin is I created this Teddy Ruxpone, and essentially, there's no ODE in this presentation. It's simply just reverse engineering stuff. So I threw together some Python code that essentially takes an input file.
15:02
It breaks it down into a folder structure that contains an i folder, an audio folder, and it throws all the i's and all the audio into those folders. You modify what you want, and then you use that folder as an input to recreate a new file. So if you take your Teddy, connect them to USB,
15:21
take one of the files, decompress it, or extract all the portions, modify it, rebuild it, then you can put that on the Bear's mass storage drive or device and be able to see the new content that you created. This is an example of said content. It's been the background for all the slides,
15:40
but I felt like for DEF CON, it was important to throw the DEF CON logo into the i's. So let me show you a little demo that I created, which is generally all the fun.
16:05
I hope you don't mind. I know there's like a no video photography rule or used to be, but I got this 3D camera that is awesome and I really want to use it. So everyone can just deal with me breaking that rule.
16:23
Okay, cool. So let's make sure that it's nice and zoomed in. Let's put this guy right there. Let me get this mic. Hello, okay, cool.
16:50
Here's a newly upgraded, here's my best friend.
17:24
And that's the outcome of months of work. So I hope you guys enjoyed that. Let me give thanks to the exploiters, one of my ex colleagues, Ryan Smith, the DEF CON staff for helping me on every presentation I've ever done.
17:40
My family, especially my kid and wife for tolerating me, destroying all of my kids' toys and then filling the kitchen with tons of hardware hacking gear. By the way, if you're leaving, I got free stuff to give out, so you may not want to. Hack all the things. We have an IRC server where people just jump in and they tell us what they're hacking on and if they have any problems and we help them.
18:01
So if you are hacking on something, you're new, you just want to chat with like-minded people, jump on a free node, channel exploiters. There shouldn't be a dot in that channel name, my bad. So just exploiters without the dot. And yeah, if you go to the last three, the back three doors, I got some of my exploiters' friends.
18:20
They're going to be handing out these SD breakouts that we created based on a previous talk, but you can grab one of those. We've got some stickers. We have some SAOs for your badges. They don't work, but you can just tape them on or something, who cares? And yeah, thank you everyone for coming out and braving the heat to get here.