Dissecting the Teddy Ruxpin: Reverse Engineering the Smart Bear

Video thumbnail (Frame 0) Video thumbnail (Frame 807) Video thumbnail (Frame 3032) Video thumbnail (Frame 4159) Video thumbnail (Frame 6729) Video thumbnail (Frame 7104) Video thumbnail (Frame 7607) Video thumbnail (Frame 8453) Video thumbnail (Frame 9137) Video thumbnail (Frame 9725) Video thumbnail (Frame 10221) Video thumbnail (Frame 10825) Video thumbnail (Frame 14337) Video thumbnail (Frame 15007) Video thumbnail (Frame 16299) Video thumbnail (Frame 17599) Video thumbnail (Frame 19054) Video thumbnail (Frame 20729) Video thumbnail (Frame 21605) Video thumbnail (Frame 22095) Video thumbnail (Frame 23382) Video thumbnail (Frame 25922)
Video in TIB AV-Portal: Dissecting the Teddy Ruxpin: Reverse Engineering the Smart Bear

Formal Metadata

Dissecting the Teddy Ruxpin: Reverse Engineering the Smart Bear
Title of Series
CC Attribution 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
The Teddy Ruxpin is an iconic toy from the 1980's featuring an animatronic teddy bear that reads stories from cassette tapes to children. In late 2017, a new model of the toy was released with improvements including Bluetooth connectivity, LCD eyes, and a companion mobile application. While the new bear features a number of improvements, the Teddy Ruxpin's original ability to add new stories by replacing the included cassettes is no longer applicable, and it requires users to supply files to the bear in a proprietary format. This presentation aims to show how the new Teddy Ruxpin was reverse engineered down to a very low level in order to create new content. I will reveal the inner workings of the hardware and software within the bear and document the process used to reverse engineer it. I will then examine the communication between the mobile application and Teddy Ruxpin as well as the custom structure of the digital books read by the bear. I will end the presentation by releasing a toolset that allows users to create their own stories followed by a demo showcasing the Teddy Ruxpin greeting the DEF CON audience.
Frequency Presentation of a group Roundness (object) Content (media) Plastikkarte Sound effect Right angle Plastikkarte Reverse engineering Information security Reverse engineering
Topological vector space Musical ensemble Presentation of a group Computer file Multiplication sign Real-time operating system Streaming media Content (media) Mereology Twitter Revision control Graphical user interface Peripheral Hacker (term) String (computer science) Software Computer hardware Compact Cassette Website Information security Error message Presentation of a group Touchscreen Sound effect Computer network Bit Exploit (computer security) Integrated development environment Software Computer hardware Function (mathematics) Compact Cassette Website Information security Hacker (term)
Area Musical ensemble Mobile app Memory card Mobile Web Flash memory Memory card Fitness function Device driver Mass Data storage device Dynamic random-access memory Proper map Revision control Logic Auditory masking Infinite conjugacy class property Synchronization Right angle Whiteboard
Module (mathematics) Slide rule Module (mathematics) Memory card Device driver Firmware
Personal identification number Module (mathematics) Link (knot theory) Software developer Projective plane Core dump Software Different (Kate Ryan album) Utility software Configuration space Video game Diagram Firmware Firmware
Web page Graphical user interface Mobile app Wrapper (data mining) Computer file Wrapper (data mining) Android (robot) Content (media) Content (media)
Point (geometry) Purchasing Slide rule Asynchronous Transfer Mode Presentation of a group Greatest element Functional (mathematics) Computer file Structural load Multiplication sign Flash memory Characteristic polynomial Sheaf (mathematics) Set (mathematics) Shape (magazine) String (computer science) Information Error message Firmware Personal identification number Mobile app Purchasing File format Web page Computer file Mathematical analysis Data storage device Content (media) Planning Multilateration Transmitter Exclusive or Uniform resource locator Sheaf (mathematics) Website Firmware Address space
Email Frame problem Slide rule Table (information) Computer file Multiplication sign Computer-generated imagery File format Set (mathematics) Data storage device Raw image format Medical imaging String (computer science) Row (database) Email File format Computer file Audio file format Bit Single-precision floating-point format String (computer science) Synchronization Videoconferencing Table (information) Row (database)
Email Stylus (computing) Email Table (information) Computer file File format Computer file Algebraic structure Bit rate Position operator Mathematics Hexagon Sample (statistics) Internetworking String (computer science) Single-precision floating-point format Software testing Figurate number Data structure Table (information) Error message Position operator
Frame problem Multiplication Randomization Table (information) Computer file Closed set Multiplication sign Computer-generated imagery Computer file Open set Frame problem Number Dressing (medical) Medical imaging Internetworking Software testing Videoconferencing Electronic visual display Table (information) Data compression Position operator
Presentation of a group Table (information) Computer file Code Content (media) Data storage device Bit rate Sample (statistics) Bit rate Function (mathematics) output Data structure output Table (information) Message passing Reverse engineering
Slide rule Musical ensemble Videoconferencing Rule of inference Shareware
Presentation of a group Server (computing) Computer hardware Computer hardware Video game Staff (military) Staff (military) Family Smith chart
let's give Xena effects a big round of applause all right hi everyone I guess we're gonna get started two minutes early which is great because I packed in a lot of content into this period so my talk is dissecting the Teddy Ruxpin reverse engineering the smart bear it's essentially my experience taking my child's toy and trying to see if it was going to cause any security nightmares for me but well we'll get into some fun things here on this presentation so a
little bit about me first of all my twitter handle is adding effects I go by seeing effects I'm a security researcher at silence I'm a founding member of a hardware hacking group called the exploit ears and I'm a contributing member to a local community that we have called Austin hackers a little bit about exploit ears we have roughly ten members give or take a few that uh don't officially associate with us but they are pretty much part of the group we have agent h h CJ cody gyne Fage maximus MBM Sarek TD wing and XO string or an ol stream and of course me so our general goal is to just hack things anything anywhere we originally started hacking Google TVs and then Google killed off the Google TV so then we just started hacking anything we could get our hands on we have a pretty decent community and we're all very helpful so check out our exploit ear site where we have like 60 plus embedded devices that have fruits and other hacks we also have an IRC network that I'll talk about at the end so a little disclaimer first of all the data within this presentation was all stuff that i reverse-engineered I didn't have official documentation or anything else so a lot of my attempts were just essentially trial and error in reversing what I could when I could I I literally have been working on this for a very long time and I you'll see why it's essentially an RTOS environment
which can be a little more difficult because after you want to interface with the peripherals so terminology you tell yo files will will will not already know this banana Li op is a brown bear light creature with a kind disposition you might think Teddy Ruxpin a a bear but he's actually an Illya so the og Illya this was the 1980s Teddy Ruxpin I think were a lot of us are probably familiar with it used cassette tapes on on its back that you replaced and you had physical books that read along with the cassette tapes his mouth moved and his eyes opened and closed but it was a physical movement not an LCD screen like the newest revision so the new Illya the
new layout is this guy oops essentially he has animated eyes a moving mouth speaker Bluetooth Low Energy USB mass-storage that is used to be an internal micro SD card and a companion mobile app getting inside Teddy so I'm about to show you all what what what this looks like it's terrifying so a little so a little about Teddy this particular revision comes with a mask that you put on him so he doesn't scare your kids when he's off it's needed to because he it like I said it's crazy uh but let's take his jacket off and get him out of here this is essentially Teddy Ruxpin this is Teddy skin so it's it's pretty enjoyable and here's Teddy on [Music] hi my name is Teddy Ruxpin can you and I be friends no no I actually tried to wear his skin it doesn't fit on my face so inside of Teddy we have a logic board and this is the the top of the logic board it's actually stored right in his eye area here there's this propriety or this chip called the Sonics it's the song's MCU I don't remember what the physical or the the actual name of it was but uh there's a speaker driver the Bluetooth Low Energy there's an SPI flash and then there's this SD card slot that actually ends up holding the story
books that are stored on Teddy Ruxpin this is the logic boards bottom you can see there are two 128 by 128 LCDs that are used for each individual eye and then on that previous picture will go to
slides back you can see that there's on the I guess right side there is a module this is Teddy firmware 1.1 that module is an my n8 to two ble module which is essentially just a module for the Nordic NRF five one eight to two chip so here
is the diagram for interfacing with its swd port and also all the different GPIO pins that are used within Teddy they only have roughly 11 GPIO pins and use of the 22 or so that are available or 26 that are available and then of course they have stdio and SW clock hooked up which is the debug pin out for software debugging cortex m0 chips so with that particular pin out we
were able to dump the firmware for Teddy and also the RAM you can dump it with open OCD but I had a sec urge a link based on doing some badge development for the whole badge life project and I just used that with NRF J Prague to dump the firmware in RAM like I said you can use open OCD and if you ping me I can give you a config to make that work so this particular teddy
instead of having physical books he uses a mobile app and so you can see in the mobile app picture they have a little cartoon picture of Teddy and his best friend grubby so the mobile app works essentially by using a ble to communicate when each page is turned and when a next story should be read so I
took that and I threw it I took the apk and I threw it into jad X GUI and it was really nothing more than a wrapper for a bunch of Adobe AIR content so within the the apk was a swift file and I took that Swift file and then I threw it into
shape X's essentially this is a flash d compiler and all the ble stuff was within this swift so if you are poking at your own teddy you can you're better off not even looking at the apk just unzip it and grab the swift and throw it into jf X's so I went ahead and I listed
all the ble info the receive and transmit UUID characteristics and all the commands to jump between storybooks and to choose individual storybooks I didn't I don't plan on reading you all that you can look and reference the slides at the bottom of the presentation of inventions at the beginning but at the bottom presentation is a web address for our website if you go to that website here after the talk I'll have slides and some of my research content but I've been having trouble updating it and I really don't want to on the hotel Wi-Fi so just check it when you get home so you take the firmware that you dump with swd and you can throw it into Ida and it shows up as a just binary data but then if you choose our mv7 le and enter in these settings you can actually go and look at the disassembly and try to reverse some of it yourself but realistically most of my stuff was done through visual analysis of the storybook files and a ton of trial and error so I got it and I - a little too late for me to spend too much time in that in this particular section so the Teddy Ruxpin books so I should go Teddy's face and body on the back there's a micro SD pin pin out or header and you can essentially connect into that and it pops up this mass storage device and has all the books on it which are an intro file an idle file and then the ten story books that they provide the I'm I'm thinking the idea was that they would release books at a later point for purchase and you just copy them over to the bear because they don't provide functionality to transfer it over through ble so within the storybooks the files that are the container format for the storybooks are these S&X rom files since I don't have the documentation they could be called something else but the magic string at the top of each file is s and X ROM and wide character so each one is individually null-terminated the target exclusive contains two extra stories but it was always a little more expensive than I wanted to pay and I bought six of these so I have way too many Teddy's so
the snx roam files I'd mentioned that they start with a magic string at the top but then there was also the header format the header format starts with a record stop and a record end and then the table itself ends with ffff the data after you use that to extract the files within you end up with the raw image data first and then the audio files the audio files always start with a u and the raw image files are all the rest of them if you take the image files and you
throw them into GIMP GIMP has a feature that you can import raw data and then you can kind of play with the settings to see what the data is which is on the left you see the picture there you can see that there are GB 565 and then there one by 128 sizing and yeah so then the audiophiles this is where things get a little bit crazier and this is where I spend a ton of my time it's a proprietary file format used by Sonic's and it actually uses it uses this thing called a mark table to store the triggers for the data that shows on the eyes and then also for the mouth movement essentially and I'll get on that in the next slide but we get there
now so the header structure for this format is essentially the au string two bytes an unknown constant value since I was using a lot of trial and error and I didn't have the documentation or anything for this format it there was a bunch of values that didn't seem to impact my tests and I couldn't figure out what they were actually used for so the unknown constant value is the two bytes that seems like it's on every single file and never changes there's that then the sample rate which is two bytes the channel which is always one but it's also two bytes another unknown value another unknown value and then some then zero or one to dictate if there is a marked table a silenced table another unknown value and then the marked table data sounds table data and audio data the actual data structure
when I say audio data mark table data and stylus table data is the mark table is defined by two to four bytes to signify the position two bytes that signify the value of the data at that position and then that particular table ends with Oh X F F F it also if the position value is over eight zero zero zero hex it takes that and uses the next two bytes adds them together and uses that as the position value the silence table it was Oh X zero and every single Teddy Ruxpin file that I checked and so it made and I I don't know too much about that particular table outside of just some what I've done for some internet sleuthing which is which I'll talk about soon and then the audio data which is 16-bit signed a little endian the marked
table so when when you're looking at the marked table how this thing actually works is if the marked table has the position of a position value and then the actual value if the value is a zero the mouth is closed and now use values one the mouth tap open the values to the mouth is full open now anything after that that you specify we'll sit will correlate to image frames that are within the storybooks so if you want to make let's say a special logo pop-up or his eyes blink or something you would essentially value put the value of that image data and then you would set up multiples to essentially make it a moving image or whatever you're trying to do but everything that you do let's say your image file is you know number one well you're still going to have to be offset by the mouth open mouth closed and full mouth open values so whatever it is you have to offset it by three so then let's look at the
silence table so I talked about it earlier but I didn't actually mention what's for the only thing this is here for is compression they don't use it I guess the stories weren't big enough or it wasn't needed but it essentially just references silent data and marks the position in the table and the only reason I knew that is because of just random internet searches on the subject unfortunately Teddy doesn't use it I just know that the files themself have that field and in my tests they completely broke any time I tried to enable it so then we go to the audio
data it is 16-bit little-endian stored signed data that's stored after the marked table and silence detection table it only supports 16 kilohertz sample rates and then it supports a bit rates from 16 20 24 28 and 32 kilobits per second so what
I've done for people who want to hack their own Teddy Ruxpin is I created this Teddy Ruxpin and essentially there's there's no O'Day in this presentation it's simply just reverse engineering stuff so I threw together some Python code that essentially takes an input file it breaks it down into a folder structure that contains an eye folder an audio folder and it throws all the eyes and all the audio into those folders you modify what you want and then you use that folder as an input to recreate a new file so if you take your Teddy connecting the USB take one of the files decompress it or extract all the portions modify it rebuild it then you can put that on the Bears mouth storage drive our device and be able to see the new content that you created this is an
example of said content it's been the background for all the slides but I felt like for DEFCON it was a important to throw the DEF CON logo into the eyes so
let me show you a little demo that I created which is generally all the fun [Music] I hope you don't mind I know there's like a no video photography rule or used to be but I got this 3d camera that is awesome and I really want to use it so everyone can just deal with me breaking that rule okay cool so let's make sure that it's nice and zoomed in let's play this guy right there and let me get this spike hello okay cool [Music] is the newly open ilium is my best friend [Applause]
and that's the outcome of months of work so I hope you guys enjoyed that
let me give thanks to the export ears uh one of my ex colleagues Ryan Smith the DEF CON staff for helping me on every presentation I've ever done my family especially my kid and wife for tolerating me destroying all of my kids toys and then filling the kitchen with tons of hardware hacking gear by the way if you're leaving I got free stuff to give out so you may not want to hack all the things we have an IRC server where people just jump in and they tell us what they're hacking on and if they have any problems and we help them so if you are hacking on something you're new you just want to chat with life behind people jump on freenode channel exploit ears there shouldn't be a dot in that channel name my bed such as to exploit ears without the dot and yeah if you go to the last three the back three doors I got some of my my exploit ears friends they're going to be handing out these SD breakouts that we created based on a previous talk but you can grab one of those we've got some stickers we have some si OS for your badges they don't work but you can just tape them on or something who cares and yeah thank you everyone for coming out and braving the heat to get here [Applause]