Modeling the App Store and iTunes with Ember Data
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 20 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/34996 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EmberConf 201412 / 20
3
6
13
15
17
18
20
00:00
Correlation and dependenceVideoconferencingFormal languageRight angleMobile appGoodness of fitData storage deviceNumberBusiness modelXMLComputer animation
00:31
Software development kitBoilerplate (text)Inclusion mapSkeleton (computer programming)Scripting languageElectric generatorCodeSoftware developerData structureMobile appPreprocessorBranch (computer science)Goodness of fitSoftware frameworkConnectivity (graph theory)PlastikkarteNumberBuildingCartesian coordinate systemWeb-DesignerMoving averageSkeleton (computer programming)TouchscreenBitSinc functionPrototypeDirectory serviceComputer configurationWeb 2.0Computer fileMultiplication signQuicksortInheritance (object-oriented programming)Plug-in (computing)XML
03:08
Data conversionMereologyRegular graphLibrary (computing)Mobile appLogicWeb page
03:43
Group actionMobile appData storage deviceSoftwareGroup actionKeyboard shortcutState of matterTouchscreenMultiplication signOpen setType theoryXMLUML
04:37
Observational studyResultantMultiplication signState of matterConnected spaceQuicksortInheritance (object-oriented programming)Keyboard shortcutData storage deviceMobile appVideoconferencingInternetworkingComputer animation
05:44
Taylor seriesGamma functionIRIS-TStructural loadSynchronizationCuboidMultiplication signLink (knot theory)ResultantMobile appVideo gameXMLUML
06:05
Uniqueness quantificationTouchscreenDirected graphWebsiteSearch engine (computing)Metropolitan area networkMultiplication signInternetworkingMobile appTwitterUniform resource locatorWrapper (data mining)Server (computing)Web applicationNumberSoftware testingWeb page1 (number)Food energyComputer animationLecture/Conference
07:52
Category of beingSearch engine (computing)Subject indexingMetropolitan area networkXMLUMLLecture/Conference
08:19
Data modelGroup actionReverse engineeringStructural loadData typeXMLUML
08:50
Type theoryNachlauf <Strömungsmechanik>Software developerSoftwareBusiness modelAerodynamicsSocial classComputer-generated imageryTouchscreenRevision controlTemplate (C++)Uniform resource locatorTerm (mathematics)Business modelType theoryMobile appIterationCodeRoutingWeb pageLink (knot theory)Game controllerMoment (mathematics)Physical systemScripting languageView (database)Shared memorySpherical capProcess (computing)Slide ruleExistential quantificationComplex (psychology)Functional (mathematics)Latent heatDifferent (Kate Ryan album)String (computer science)Template (C++)Single-precision floating-point formatSocial classData typeConnectivity (graph theory)Core dumpDebuggerData storage deviceMedical imagingInheritance (object-oriented programming)Context awarenessDynamical systemData modelOffice suiteCode refactoringMereologyAttribute grammarKey (cryptography)ParsingUniform resource locatorFlow separationCodeFitness functionDot product
12:54
CodeSerial portAdaptive behaviorBitDifferent (Kate Ryan album)Object (grammar)CASE <Informatik>Field (computer science)Point (geometry)XML
13:21
SoftwareHistologyData typeComputer wormIntrusion detection systemResultantTable (information)Web browserDifferent (Kate Ryan album)File formatRegular graphData typeReverse engineeringCodeBusiness modelIdentifiabilitySerial portElectronic mailing listLevel (video gaming)Category of beingQuery languageEncapsulation (object-oriented programming)Type theoryMobile appSpecial unitary groupWritingMereologyGroup actionUniqueness quantificationSocial classPolymorphism (materials science)Virtual machineLecture/Conference
15:32
MereologyInheritance (object-oriented programming)Content (media)XMLUML
15:56
Database normalizationHash functionCategory of beingLink (knot theory)Query languageFilter <Stochastik>Structural loadResultantTwitterType theoryWordHypermediaCrash (computing)String (computer science)Serial portFlagBusiness modelLink (knot theory)Parameter (computer programming)YouTubeSlide ruleHookingUniform resource locatorKey (cryptography)Implementation
17:25
Identity managementData analysisResultantState observerQuery languageLevel (video gaming)Game controllerWeb pageVideo gameMaxima and minimaSet (mathematics)Error messageParameter (computer programming)CASE <Informatik>Business modelIdentity managementSearch theoryGroup actionPopulation densityFigurate numberXMLUML
18:57
WebsiteUniformer RaumMetadataTemplate (C++)View (database)Web pageDynamical systemCodeBusiness modelSheaf (mathematics)XMLUMLLecture/Conference
19:37
Template (C++)Template (C++)CodeLecture/Conference
19:56
Translation (relic)HypermediaWeb pagePartial derivativeDescriptive statisticsYouTubeStructural loadMobile appLogicType theoryContent (media)Lecture/Conference
20:18
Game theoryUtility softwareInformationContent (media)Structural loadMultiplication signBusiness modelPhysical systemComputer iconMobile app
20:40
Server (computing)Structural loadProduct (business)Web browserRectangleMobile appAmsterdam Ordnance DatumData storage deviceResultantMultiplication signInformationWeb pageYouTubeUniform resource locatorWeb 2.0Computer animationLecture/ConferenceXMLUML
22:13
Abelian categoryPoint (geometry)Selectivity (electronic)Video gameINTEGRALSubsetXMLLecture/Conference
22:40
Mobile appWebsiteState of matterComplete metric spaceHydraulic jumpResultantWeb pageStructural loadDemo (music)Type theorySoftware developerTouchscreenFreezingComputer animation
23:33
RhombusOpen setNumberHydraulic jumpDifferent (Kate Ryan album)Disk read-and-write headMusical ensembleMathematicsXMLUML
Transcript: English(auto-generated)
00:16
Hey. Okay. I sound much more godlike now. Okay. That's good.
00:21
All right. So, yeah. The talk is Modeling the App Store with iTunes and, or, Modeling the App Store and iTunes with Ember Data. It's an incredibly long title for a talk and, yeah. So who I am. I was a Rails developer for many years, like probably most people who started in web development in the last five, six years, and I was quite productive with Ruby
00:43
on Rails, was able to make lots of good apps that were functional and, you know, did the whole CRUD thing very easily. And I was happy with that way of working, kind of dabbled a little bit in Sinatra, and that was how I built apps.
01:00
Later kind of became a backbone developer, as a lot of people did, when that became the technology that let me do things that I could not do with Rails, like structure my JavaScript code. I definitely remember Googling around looking for a way to structure JavaScript code and finding, you know, just write a jQuery plugin, and that's structuring JavaScript code. And I wanted something a little bit more, and thankfully, Backbone gave it to me, and
01:24
I built a large Backbone app, and that's why I really loved Ember when I first saw it, back in July 2012. I pulled this off the Wayback machine, and it's also the same documentation I've been referencing while I've been building the entire app.
01:41
So it's been great, and yeah, I really like the big download buttons with the rounded it, and that's Web 2.0. So I'm now an Ember developer, and that's all I do, is Ember development, if I can do it. I don't really pitch any other options to people, because that's where I'm most productive. I can churn out prototypes in hours, not days, and the code makes more sense.
02:05
You're writing more direct to the UI, as you all know, and it's a wonderful way to write applications. Over making, I don't know, in the last year, probably about the last year since today of 2013, I've probably written about 20 or 30 Ember applications, and you
02:23
can call an Ember app, you know, a single component on the screen that switches between two tabs, but I've written things that small and larger, and doing so, I ended up making this framework, Topus with Ember, and it's a skeleton for Brunch, and Brunch is a build tool for JavaScript and CSS preprocessors and all sorts of other things.
02:44
Brunch is a really good tool, because it's been around for a long time, has a lot of community support, and it's super fast, and so I use it for everything. I use it with my Vim workflow. I have a special plugin that lets me create files in the right directories, and starting so many brand new Ember apps, you kind of want to roll up these concepts into something,
03:03
and I've been using Topus with Ember while I've been building the thing I'm going to show you guys today. So when I hand out a card to people, it's that I create with Ember, and I like having that conversation with people about what it is Ember does for you, or why do I even care, because, you know, most people are like, well, why would you put a technology on a business card?
03:20
You're just going to scare people, but I think that conversation's a really good one to have, because I think people get it. You know, it's weird when you go to an app and you click on a tab that's obviously part of the sub-navigation, and the whole page refreshes. It's just, it feels broken. It's like there's something wrong with us, or why isn't the data live when I switch to things, and it's going to feel broken very soon to even regular users.
03:41
So I'm going to tell you a story. It starts in August 2013, and I was sitting around, and I had this brand new app that I really like called Launch Center Pro, and Launch Center Pro is an iPhone app that lets you make shortcuts to make everyday actions lightning quick, and that's definitely
04:00
the business speak way of saying you can just like pop it open, and it always works the way you expect it to. You just, you get to the main screen of actions, you choose one, you're off and running, and that's an awesome thing to have on a device that you pull out of your pocket hundreds of times a day. And so I wanted one specific thing that I do all the time, which is try other people's software by searching the app store.
04:20
So you pop open Launch Center, you start up with their new action for searching the app store on your phone, because you don't want to have to go in the app store app and like wait for it to load and try to find the search button and then clear out your previous search that's still there because everything saves state. So what you do is you start up Launch Center Pro, you click app store search, you type
04:41
in your search, which I always am searching for Angry Birds just to make sure nothing new has come out that I've missed out on. And this happens every single time. And you see it got my search in there, so it's really teasing me that something might have worked, like it certainly seems like iTunes or the app store app knows what's going on. I'm going to try it one more time just to be sure.
05:03
And this is, I recorded this video last night, so this isn't like it's something that is not still happening. So and just to prove that this is like actually a live internet connection, I'm not just like breaking this on purpose, let's go ahead and do the real search. Hey look, autocomplete, it certainly looks like it's going to do something this time.
05:21
And results. So that's kind of the experience if you want to shortcut search on the app store. And I'm like that is so crappy, like why can't I just pop open that search and make it work? So I started to think about it and I was like I might be able to solve this problem myself. Well I first started looking to see if there were other solutions and there weren't any.
05:40
And I started to think about what the experience is like to be shared something from the app store. And I feel like this is kind of the feeling you get. It's like oh I love that, you know. Or when you see the result in Google and you know exactly what's going to happen when you click that link is you're going to get to dismiss that nice dialogue every single time and what does that check box do and does that mean I can never
06:01
ever open an app again? Yeah. And then inevitably you get this guy who's very happy to hey, I'd love to sync your devices, do something like you know, anything. Like it's like sucks and it's terrible and it's you know obviously the thing that
06:21
Ember is great at. And so why don't we make something using those tools? And it turns out that there's like an API for iTunes and there's like a website for EmberJS. So I was like let's do this. Let's make something better. And so three hours later I got this thing and it is exactly what it looks like.
06:43
There's no like too light text you can't see. It literally just shows the app titles and you click on it and you get that and it's like okay I've got like a thing that wraps the iTunes API. And by the way this is ridiculously easy to do with Ember and Ember data. You just fire it up and you can make a wrapper around somebody else's API and run them out of business with their own API pretty much.
07:02
I mean there are a lot of apps out there right for the picking of they have APIs and they are not learning how they do things with JavaScript web application technology anytime soon. So I was like that's it. If I can put it up on GitHub pages, tweet about it and get my internet dollars and it's not going to get you know anything else and that's it. And I've done that plenty of times and it goes absolutely nowhere.
07:23
And you know it's like I use it and six other people do and I get 85 pull requests from one guy who wants to do one extra thing and that's about it. So I started to think about it and I was like nah this is like a lot bigger than this and the whole URL thing is kind of important. And I started to use my own app over the next week and I was like man this is really compelling
07:44
to have something that's better that you can use on every single device and not worry about it popping up your little friend down there on the task or on the doc. So I thought maybe it's time to make a search engine with no server. And it's the kind of crazy concept because you usually think like search engine you're
08:00
going to have this giant you know server that's going to go out and index all these things and do stemming and all this complex stuff. And so it's kind of like crazy. You can see where this is going because if it's madness then we got. Wait crap man. Yeah well oops. OK so no this is ember ember ember makes this easy like you know you can search
08:26
you can use other people's APIs do whatever you want so if somebody else has done this better then why don't we take what they've done better and use it ourselves. So that gets us to like the data model and it's more than just apps. iTunes actually has a crap load of stuff on it and I learned more and more about the
08:42
iTunes API over the last many many months than anybody should ever have to especially reverse engineering it not knowing how it all works. So there's a lot of data types and a lot of those data types have things in common and you want to be able to model them in their specific ways and use things like
09:00
or use common concepts between them so you're not constantly rewriting code and it's not just like components you actually need something more granular than that. So for me it all started with the type key. Type key is an attribute that every single ember data model gets automatically and it's the name of the model in a string and it's unified based on the way you
09:21
define the model's name so it's you do app dot iTunes or iPhone app in your iPhone underscore app will be your type key. So I define a whole bunch of things up front and so every single model knows a lot of things about itself like what its route name is and what its route path is and CSS class and all these other things which lets me do automatic things
09:43
to define all these complex models and make them work within my app. And oh just so you know everything in this talk is in CoffeeScript because it's like fits on the slide better that's why I'm using CoffeeScript. It's not because it's better than JavaScript for writing apps.
10:01
I did look at you at that moment I'm sorry. So you get to do things like this link to model name, route name and model and that's like working code and it will always work for every single one of my models it will link to the correct name of the thing and the route and I don't have to write like a custom template for every single thing and make it work or if I reuse it everywhere. Same thing I do it for route generation, iterate through all the models
10:23
and then create the routes dynamically. You might say well why didn't you just use dynamic segment for the first part of it? Well that's because then you have a dynamic segment at the front of all of your URL's that you would have to like parse everything through and I had some issues with it and I was like I'd rather have the routes be defined up front. You might be able to refactor into dynamic segment and have it work out okay but I just feel
10:41
like having a star at the front of your app is kind of weird. CSS class is super useful because then you can just chunk those on anything such as images there's different types of images for everything in the app store and I need to customize the way the CSS looks for those and by standardizing the names of the classes I can just write CSS and it will inherit from those new things if I add a new data type its new class gets added and I can go in
11:02
and start modifying it without thinking about it. So the core model is the iTunes item and it has a URL for its artwork, image suffix which is like PNG, TIFF, all these other things and price all this stuff that's common to every single thing on the app store and so everything inherits from this item
11:23
and that songs, apps, movies, all the different data types but there's still more than that there's still more specific complexity in different situations and it gets pretty deep into the rabbit hole. So mixins end up being a great tool for this and mixins let you encapsulate functionality into a little tiny piece
11:40
that you can include wherever you need it and so I made mixins for genre because some things have genres, a lot of things do actually and specifically the interesting mixins are app and review mixins. So the app mixin turns a thing into an app because Mac apps and iPhone apps and iPad apps kind of all share things but they aren't necessarily all descendants of an app class so that worked out well for me.
12:03
The review mixin is really interesting because that just means the thing has reviews and by including this mixin through my app then the item automatically gets added to its template the ability to view reviews, they will automatically be fetched from the separate API for reviews which we'll get into in a second here but it's a very powerful system where I can encapsulate it.
12:22
The same exact thing exists in Rails with concerns and it's a really great way to factor your codes so that you don't have to copy and paste or put all these things in one place or try to do it in the controller or something silly. So this is kind of the breakdown of the different mixins these things are using because it turns out you can't get reviews for a song because that's not even,
12:41
there's not even a page for a single song on iTunes. You actually see the song always in the context of an album but apps and movies have reviews and so you can kind of see how different things are using it. There's tons of these mixins throughout the app. There comes the really interesting stuff, the really Ember data stuff which is the adapters and serializers. So if you're not familiar with Ember data, adapters are the bit of code
13:02
that you make that's custom that lets you talk to your endpoint, whatever that is. In my case, it's a lot of different endpoints including iTunes API, the review API, some of the private APIs and the serializers are what turn that data into the stuff that Ember data is expecting so that it all comes in uniformly and you can use those objects cleanly.
13:21
So iTunes has very promiscuous data types. What I mean by that is when you make a request to an API, you don't know what kind of data you're getting back. Like you may actually get movies back when you're asking for songs if you don't do things right or if their API is having a bad day. It really does happen that sometimes they just return different data types for the same request. And so you have to be able to handle it.
13:41
You have to be able to filter them. And so what I ended up doing was making a serializer for all iTunes items into one thing. So I don't have a serializer per data type. I have one and it combines the notion of a classifier which I'll show you in just a second and then it's polymorphic so it gets back a list of things and we figure out the types and shove that in and say this is actually an app based on the criteria that I've looked at.
14:02
And it also supports side loading so that the things that it's loading polymorphically are already there. So when I ask for them, I don't have to make another request to the iTunes API, it's already been loaded. So the iTunes classifier looks kind of like this and it just has these different properties that I know about based on reverse engineering the iTunes API and figuring out what they call things internally.
14:22
They don't always use the same kind of identifiers to talk about data types so you end up having to like do different things like artist types, collection types and there's tons of other idiosyncrasies with their API. But this kind of encapsulates all of them, quick look up table and that makes everything really easily. So I also gather results from other APIs not just the main iTunes API and one
14:44
of them is the top charts and the top charts come in and they're in a completely whacky doodle different format than the regular iTunes API. But the one thing they do have is the ID, the unique identifier for that piece of data in iTunes. And so what I can do then is just load them all in as iTunes item results is what I call them
15:04
and then I just say get me my chart results and map by the actual result which will then go and make a fine mini query to the iTunes API which happily supports multiple IDs that will resolve all those IDs and will figure out what their data types are. So the end result is that I didn't have to write code that parsed the top charts
15:22
into a model that worked for me, I can just take the ID and let it be a second request. Turns out it doesn't take much longer in the browser to make that whole second request so it saved me a bunch of code. Another big part is relationships, I use tons of relationships inside the data. I map parent child relationships for everything, albums down to songs, back to artists,
15:43
even parent child relationships that iTunes themselves doesn't expose but that they have data for, it's really easy to do with Ember data. And then I also map relationships from external APIs that are interesting to have with your content. So I always load YouTube videos, reviews when they're mixed in and Rotten Tomatoes for movies
16:04
and what I do is I flag those relationships as always load and this is not a feature of Ember data, I've implemented this myself but it's a really simple implementation that just looks at this flag in the serializer, if it exists it's going to set the links attribute for that name to true and what that means is that Ember data will then try to fetch that no matter what and I'm going
16:24
to hook into it later to give it a dynamic URL based on the name. So the way I get YouTube video results is I need to make a query to the YouTube API. So I wait for the model to load from iTunes, I get its name using my custom logic, maybe I filter some stuff out of the name if iTunes does weird things with the name
16:41
and then eventually I'm going to make a query and the adapter is going to see that I had a URL that came in that wasn't a string as a remote URL and so I just say give me the relationship key URL name that I said in the previous slide. So you see how it says YouTube results URL, the thing was called YouTube results
17:03
and now just say the URL there, it's going to fetch it based on the custom parameters for that model and then I'll get YouTube results for that thing and it also lets me do stuff like jam in the media types so that you don't search for crash and get no results for the movie crash, it puts the word movie in there and I can customize it
17:20
and I've tweaked all the results for everything so that you get relevant results for this when you're searching. So pagination is another really hard thing to do with Ember data right now because it's not built in and I built my own model for it called the fetcher and it handles all these things for me automatically so it makes sure that I never go past the maximum result set because the different APIs I know how much,
17:43
like how many results they can possibly provide me and I don't want to make queries past it and be a bad citizen of their API and get banned or whatever, although it still might happen and does out of result detection so if I get back less results than I asked for I know that must be the end of the data set and my fetcher is very specific to the fact that I am using another person's API that I have no control over
18:03
so I have to be kind of robust and like handle error cases and things like that but I do do an identity map which is a concept that Ember data uses to keep a single copy of every single model and I do the same thing with a fetcher so that if you happen to back up to a previous search the fetcher will still be the same one as the search that took place before
18:22
so your results will be instantly loaded right there and as well since it has a unique ID I can do things like say oh you were looking at three pages of results so I'm going to show you three pages of results and that fourth page is still down there waiting for you preloaded in this fetcher. They're defined in the controller based on the query parameters that the person set up so they might have said show me movie results and that's where it belongs.
18:43
It took me a while to figure that out. I had it in the search model and I was doing observers to switch out the data manually and I realized no wait this belongs in the controller and once query param's new moved to the controller it made my life a lot easier and like I said it preloads results. So templates, this is kind of getting to the end of the technology that I used
19:02
to build the tool so you'll get to see it pretty soon. Dynamic partials, it's just what I call what I'm doing with the different views that I'm using for the iTunes items so each, the page is basically a template for templates and you can fill in the blanks however you want there and the idea is that the model can up front say I want a custom template
19:23
for my metadata one section or for my metadata two section that way I'm not writing a template for every single data type. I'm only customizing the pieces that I want because you want your site to feel kind of uniform when people are navigating it. So metadata one can be filled in with stuff and it's very easy to write this kind of code. You just basically define a method that's template with default and if it exists
19:44
in the ember dot templates then it's there and you use it. If not, you fall back to the base one that's just a generic template, might be empty even. And so for movies, I just define a few different templates and now I get this customized experience where you load up the movie and you see the Rotten Tomatoes score, you see the duration and you see the description
20:03
and you're seeing the YouTube videos that all came in from those three little partials and the page just loads up perfectly fine. The same thing happens with apps and I actually share the logic in the mix-ins so that all app types can share the same partials. And so you'll see that Angry Birds loads up and shows kind of the content there
20:22
and the different genre and all that kind of information. Same thing with a Mac app that shows up, fills in its blanks, puts the icon in, handles all that stuff. So what I built was this robust system for modeling their API and I've been working on it for a really long time and I've been using it every single day.
20:41
So what did we build? We built something that's better than this. That's for sure, that you don't have to wait for. It works on all devices and it's really fast and it's called Find. No vowels because it's cool and that's the URL for it.
21:02
It creates beautiful URLs with slugs in them that you can share with your friends and loads up faster than iTunes' server rendered pages ever could because I don't think they know the web very well. It works on all devices so you can access this on your Android device just as easily
21:23
and it's not going to pop open the app store there because there isn't one for there but if you send it to somebody on the iPhone, it will open up in the web browser and they can look at it and decide if they want to look at it. It's incredibly fast, it's faster than the native app store app. I've used it many a time and it's definitely faster than that. And the most important thing is that it's a product search versus like a document search.
21:44
So if you're looking for an app, you go to Google and you search and you're trying to find this one specific thing that you know about and you're looking for that rounded rectangle at the front there. And it's great for finding documents but it's not good for finding products. And this is the idea of Find is that you're finding the product first. You're getting there and then I'm going to give you the relevant information about that product.
22:01
Let's give you YouTube results, reviews and anything else I can pull in about the app so that you aren't looking through all the stuff sifting through these URLs to something that's meaningful to you. And this is the logo for it. And I didn't forget Launch Center Pro, which was the whole point of this.
22:20
So there's an integration for Launch Center Pro. And you just tap that button and it adds to Launch Center Pro and now you can search Find from there. And it is probably hosted on DigitalOcean, which I have not been paid for but will accept payment for. And that's it, Find.
22:42
And I can give you a demo of it now if you'd like to see it. Okay. So here it is. And so I'm going to search for angry birds. You're seeing I'm getting as you type completions there from some unknown API. And get back results pretty quickly. Those were already preloaded.
23:01
I might have had them in there. And browse through them, click on an app here. And you're seeing everything load in on conference Wi-Fi pretty fast. Go look at a screenshot, come back, look at the actual website for the person who created the app, very easy and quick. Jump to their developer page on Find and see all the apps that they've ever made.
23:24
And pray that conference Wi-Fi will eventually load these. Hey, it did. That's pretty awesome. And jump into another app. You're seeing loading states for everything. And it'll pop in all the data as it gets it from the different APIs it's using. But let's do one quick thing.
23:42
Jump to the top charts and you can see here that I've got the most important piece of an Ember app, which is a multiple select that changes. So top blues, songs, and music. And there they are. So that's fine.