Thank you for coming. Um, hope everybody enjoyed, uh, DHH's keynote, um, and are all ready to build

monolithic web apps. Um, we're gonna talk today about, um, API design, uh, and specifically API design, uh, that is focused on the people who actually have to write code against that API. Um, we are not going to be doing any coding at all today. Um, implementation is an implementation detail,

so, um, don't worry. We're just gonna design it really well and then implementing it is somebody else's problem. Uh, my name is Pete Holliday. I'm a lead engineer at MailChimp. Uh, MailChimp is an email service company and we will tell you, um, much more about it later.

Uh, I'm Kate Harlan. I'm an engineering intern at MailChimp and graduate from Georgia Tech next week. And after you graduate from Georgia Tech, what are you gonna do then? I'm going to come back to MailChimp and work there full time.

Um, Virgin 2.0, uh, of our API did a lot of things well. Um, about 250,000 users, uh, hit it every single day. Uh, billions of requests, uh, come, come and go. And it's fairly popular. Um, we did a survey not that long ago and we found 75% of people thought that it was

good or excellent. Uh, which, those are really good numbers, but, uh, as we were working with it ourselves, we discovered that there were some pretty big issues, uh, with the, with the design of it and, uh, with the things that people were trying to do with it. Um, and so this talk is sort of a, uh, came out of that process of us, uh, designing our new,

the newest version of our API, which is currently in beta. Um, it was supposed to be released a month or so ago, um, but software happens and so it's late. Um, we're also, this is gonna be a fairly interactive session, so, uh, during several portions, you're going to need to probably chat with your neighbors and so if you would, just take a moment, introduce yourself

to your neighbors, any of your neighbors you don't know, you're going to become good friends by the end of this, so just make yourselves friendly. Alright, I think you've got the next one, Zoe.

I want to know the gender ratio, I want to know the gender ratio of the whole conference. I'm just curious. Yeah. I think it's pretty good, but it's still kind of white-naily. Yeah, I mean, just looking around, there's like average-ish one, from what my perception is there's average-ish one woman a row,

like, or like half of seems about right. One in eight, maybe. Okay, okay, okay. Not that friendly. Okay, alright, let's bring it back. Let's bring it back.

Bring it back. I've lost the room already. Alright. Bring it back in. Awesome. Alright. So, so y'all probably at least kind of know what an API

is already, but it stands for application program interface. APIs are pretty much everywhere. If you've done any programming at all, you've used an API. Because that's just interfacing with that program. And APIs are just the way two pieces of software communicate across a boundary, and there are

all kinds of different boundaries everywhere you look. Ruby blocks are an example of a boundary. Language features typically are APIs of that language. Class methods, more generally object-oriented programming. All of the classes that you generate expose APIs. And then of course the thing that we're here today to talk about, web services. They could even

be microservices if you wanted them to be. Ah, yeah, yeah, I know. And the the principles that you use to design a web service API are, they're going to be very similar to the principles that you would use to design any other kind of

API, but we'll be using web services as sort of the thing we talk about today. I don't know, is it possible to dim the lights? Sure. I guess it's

hard to read in the light. Is that what I'm hearing? The screen is hard to read? He's working on it. It wouldn't be my first. Alright, so while we're working on that, I'll just keep on trucking I guess.

Developer-focused APIs they sort of mirror the movement we've seen over the past who knows how many years to bring user experience in web apps to get, for that to be a focus I guess. And I've noticed that that hasn't really

been mirrored in API design. APIs are still very much just sort of the wild west in terms of how they look. And so when we talk about developer- focused API design, we're talking about optimizing for the end user and not for your backend systems. Many APIs are designed

to make them easy to implement and maintain, but they're not necessarily easy for the end user to use. And when you focus on the end user, the person who's going to end up eventually using your API, that leads to great experiences in developing those. So what do

developers want when they are using your API? Well, if you think about any time you've used an external API, developers want basically one thing. They want to get in, they want to get done, and they want to get out. And they never want to touch your API again.

Ideally, that is the thing that they want the most is to just get their sprint done and be done. So, never

return is sort of an important part of that. Because once they write something using your API, you can't just change how it works because then you'll break everybody's everything and everyone will be sad. So

you have to version your APIs, which is really important, but also the versions that you put out have to be robust and functional. So who here has had a bad API experience? Raise your hand. Show of hands. Alright, so a couple of you. Good.

for a moment for, let's say, 90 seconds talk with your neighbors about what made those experiences bad. What was it about the API experiences you had that made them challenging? And when you're done, I'm going to ask you to actually contribute and

raise your hands and tell us what it was that you thought. I never go anywhere without a water bottle and I sort of have not been living in my apartment as much as usual and so I keep losing water bottles.

I don't know where they go. Really appreciate it. Thank you so much.

Doing his job at one of these would be awful. My dad does luncheons and theater and television, but this is like, just got to be so tedious. It's funny because Mara was like, well, maybe you

ought to do a light version of the slides in case it's hard to read. And I was like, yeah, I should totally do that and then did not do that at all. That's okay. Half of the room can see. That's all you need.

Alright, so this is where it gets interesting, I guess. Yeah, be avoiding naming specific companies if you can. No, no? Okay. You don't have to. That's fine too. Alright, so why don't you take over the rest of this slide, I guess. Do you want me to try and bring them back now?

Let's give them let's give them until an hour thirty. Okay. So until we get down to thirty seconds over here. And then we'll probably have them come back down by one hour.

Okay, let's try to bring it back. Alright, so raise your hands. We're going to try not

to name specific companies if we can avoid it. Please. But can you can you all raise your hands and tell me about some bad experiences you've had with APIs and why they were bad?

Okay. Sure, yeah. So we have in the back we had a get call used to create data.

We've got bad XML up front. Documentation, bad documentation. Quit stealing my future slides.

Okay, so yeah, that sounds like there are a lot of things there. So creating resources that don't give you the

resource back and then and then you have to make another call to get that resource. So lots of API calls to do one thing. Which yeah, sure. Let's get somebody from over there.

To repeat, just like a lot of baggage, it does a lot of extra stuff. Why would that be a problem? So payment system with no sandbox

mode. Let's do Michael in front and then... Okay. So schemaless APIs. I'm sure there are a thousand more ways. We could probably fill up the rest of the hour just talking about things

that are bad. But the short version I think of a lot of what you're talking about is easy stuff is hard. There are too many endpoints. Simple stuff is complicated. The docs are wrong or incomplete. Maybe no wrapper libraries. There are some other things folks said. But yeah. So that's

a bad experience. Raise your hand if you think you use more you have more good API experiences than bad experiences. Raise your hand. Okay. Alright. Well that's more people than I thought would raise their hands. But it's still not very many. So So today what we're doing, our hands

on design work is going to be using real MailChimp endpoints. We're going to start by learning a little bit about MailChimp. You can't design an API without knowing the product that the API is going to be used for. Probably really well. And you also can't design a developer focused API without

knowing what the developers are going to use the API for. Also probably really well. And like with a degree of nuance, not just like, well they could do this, but like more specifically what is more likely to happen. So that brings us to our advertisement portion of the talk.

MailChimp is an email service provider. Our mission is to help people send better emails. And one of the reasons that we have an API is that one of our co-founders a long long many years ago eight, nine years ago, he likes to tell the story where eight or nine years ago he was at a fork in the road and he had to choose do I hire a developer to build

an API for this new email service company or do I hire a salesperson. Fortunately for me, he decided to hire a developer and currently MailChimp has zero salespeople. So, seems like a good decision. So, what are our main features?

Lists, campaigns and reporting. Who are you sending to? What are you sending? And what are people doing with it? The other thing that we need to talk about is who uses the API. And for MailChimp, our particular use case, there are a couple of different options.

There are people who use the API directly. So your company wants to get your own data into MailChimp or vice versa. Our mobile apps are sort of an example of that. But then there's the case where you might have data in a third party web service that you want to communicate with MailChimp. Say, your CRM or your Shopify shop or

whatever else. And so, our API also enables that level of communication where the user is not, the user might not even know that an API call is being made but on the backside there are a ton of them. Usage patterns, that is roughly, that's about a week worth of API

calls to version 2.0 of our API and that's how they break down. The overwhelming majority is list management, getting people on the list, getting people off the list. Reporting is a close second and then campaigns which is probably the biggest thing that MailChimp does. If we didn't have campaigns, like what would we do?

Almost nobody uses that through the API. And so you have to kind of, I have to ask myself, is that because that's just not something people want or is it because our API is so bad that they can't? So, if we're going to look at API 2.0's

list management endpoints, what we saw is that is the most important, the most traveled part of our API, so we've got these endpoint lists that we're going to attempt to hand out. Right, so what we would, ideally what you'll do is in a minute we're going to go over the list management

like in a little bit more detail what list management means at MailChimp and while we're going through that, keep a hypothetical company in mind. It could be your own company, it could be one that you've just made up, like so for example, we'll give you some examples of an online store that sells like pens and pencils and such. And then once

you're done thinking about that, you'll have some time to chat with your neighbors and the reason we did a printout instead of a webpage is because last year the wifi was not sufficient, but this year I hear it's great, but there's still no webpage, sorry. So, so yeah, so we'll talk about list management and then we'll get to the meat of the, you know, showing you

showing you our dirty laundry here. So we have different parts within a list. You obviously have to be able to edit a list, so we're talking about adding people to lists, removing them. You also, we have merge fields. Merge fields are arbitrary data

that are set up at a list level. And then we also have interest groups, which are booleans, like favorite color was one example. Yeah, and you can set those up. So merge fields might be like a first name or the last time they bought

something from your store. An interest group is a little bit more of like they either are interested or not interested. Managing subscribers is another big one. Getting people onto the lists and off of the lists and keeping those fields that we just previously talked about, keeping those up to date. So did you change your name or did you just buy something in the store or have you decided you're

no longer interested in colored pencils? Segmentation, once you have them on the list and you have all this data about them, you then want to use that to send different emails. So you want to send if somebody's bought a lot of pencils, maybe you want to send them an email about a pencil sale that you're having, if that's a thing that happens.

And then once all that's done, your boss wants to know about statistics and reporting and who's opening and who's clicking and what links and all of that. So that is MailChimp's list management in a nutshell. One thing I'll say is that we do have don't go from this talk and then go try

to use our API because we're leaving a bunch of features out in the name of simplicity. So I guess then we have to talk about what's our intent. So, among yourselves, with your hypothetical companies in mind, let's talk about what sorts of things might you want to do with the API than as your hypothetical

MailChimp customer. So let's take two minutes and talk about what the big things are and then we'll come back and chat about them. So, go! ...to get this side dimmed at all? I don't think they've managed to do that yet.

Last one, they were doing a lot of really good deep dive

really good examples so hopefully that'll keep up. I think if you're here you probably are frustrated. You've got some reasons.

If anyone here or how many people here have actually worked with our API. That's a good question. I do a lot of hypothetical research studies in my mind where I just wonder about things like this. You should build an app. Yes.

Like to... Like, you would have it at the conference and it would send a push notification like, blah! Cause you're in this room or whatever. I guess maybe another minute they seem to be like...

There seems to be chatting. Those two who are chatting with their phones. That's okay. Maybe they're bad at personal interactions and just texting. That's fine, I do that sometimes. Text people I'm in the room with.

Long week and it's Tuesday. All downhill from here for me. Well, our senior design presentation looks like the app actually looks really good. As long as you only do the features that work,

they work really well. There's a couple of things that they look like they work, but they don't. Three steps down the line you're gonna not be able to get any data, but that's okay. How much of that is free range? Will your professors be like, oh, click this button? No, we just stand in front of the whole class

and we've got our preset demo that we're gonna do and we know that this building works all the floors, we know where the bathrooms are, we know that the dots should appear in the right place. We were doing edits this morning and one of my group members was like, y'all we are presenting very soon. I need you to finish. You're making me nervous.

Alright. Alright, let's finish it up now if we can. Sorry, I know I'm interrupting a lot of you mid-sentence. But if you could just...

Okay, alright, let's try to finish it up. Thank you. Alright, let's talk about it. Let's share with the group. What sorts of things might you want to do with MailChimp API? Who knows a thing? Nobody knows a thing. Okay, you might want to send an email.

That's a really good idea. What else might you want to do? Newsletter, okay. What about for like list management stuff? Yeah. Okay, great. That's a really common use case.

What else? Oh yeah, sorry. I'm really bad at that. When somebody at his company's store buys something, they then get registered on the MailChimp mailing list. Hopefully, only if they check a button that says I want to be on the mailing list. What else? What else might

you want to do with these merge fields or or any of that? Okay, yeah. Okay, what do you mean by that? Okay, sure. We typically recommend that you not have more than one list in the first

place. There are some use cases where you would want to, but we tend to, we try, that's what we have the interest groups for and the segmentation. So what you would typically do is have both of those lists on one list and then when you want to send an email to just part of it, you would segment it off for that one email.

Definitely a use case that some people have when they have multiple lists for sure. Yes, that's right. Okay, great. Being able to set somebody's interest groups as you

subscribe them. Anybody else? Yes, sir. Yeah, so the merge fields, being able to create those merge fields or create the interest groups through the API, that's especially useful, like I

think you were alluding to, if you create a new category for markers, maybe adding an interest group for that automatically as the category gets created in your store.

Sure, so like your internal, right, so adding your apps local internal ID to their Mailchimp profile so that you can link those up when the data comes back, say through a callback or whatnot. Okay, let's see. Let's see some of the other things.

I think we've covered almost all of these. And so now what we're going to do is we're going to pass out these endpoints. These are just list management and just version 2.0 and what we'll do, I'll come and cycle these around and just take a few minutes,

look over it and figure out what might be hard. What about these calls, either hard to learn or hard to use or anything like that. So give us just a second and we'll get these passed out.

So while this is all going on, just take a couple of minutes, look through those endpoints and see if you can find any problems. Don't be shy, we know this is bad. Sure, like what's bad about it?

Yeah, like what might be hard to work with or otherwise just use, understand, like what's confusing. Alright, so the handout took a little bit more time than I was hoping so let's go ahead and let's talk together about what what's wrong with this. And again, don't be shy, feel free to be as

brutally honest as you need. Everything is a post. It's restful though, right? What else, what else is wrong with it? Right, so lots of endpoints, so many endpoints. What else? Oh, and he said you could, the static segments and dynamic segments could really just be one

set of endpoints. What else? The API key is in the body of the request, that's bad-ish. Right, so there are a bunch of other endpoints in the API that are prepended with campaign or whatever, so

that is maybe more of an, more an artifact of of our right, so no context, so as documentation the handout is really awful and I'll give you a hint, the documentation on our website isn't much better. In the back? Right, so possibly the subscribe and batch subscribe could be the same endpoint that just takes

more than one thing, yes sir. Right, and aside from it just not being restful, what's hard about verbs in the URL? Right, okay great, so we'll talk a lot more about that in just a minute, what else? Right, so a lot of parameters for what should be a really simple call. Okay, let's do a couple more, what else do we have? Yes sir? Right, yeah,

how do you, what happens with errors? I still struggle with that to this day, okay. Interest group versus interest grouping, we're renaming that in 3.0 for exactly the reason you described, yes sir. Right, right. You want to repeat that one? Right, yeah, so there there's just some random hard limits in the activity

in the activity endpoint where it's like, oh it's 180 days, well why? Like why not 30 days, why not 70 days? Yes ma'am? Right, so the batch subscribe endpoint has a lot of like, you human figure out how to scale this yourself whereas we should probably do the scaling for you or just set a limit at a place where we know

we can handle the volume. So those are all really great and we could probably talk about what's wrong with this for hours, I know I have. We do. So, so many methods API key as a parameter, inconsistent naming how do you update merge fields, lots of, just lots of like it's just hard, I mean it's two pages to describe how to subscribe to somebody to a list.

So let's talk about the principles of developer driven API design. There are only in my opinion really two. You need to design for intent and you need to limit your API's mental footprint. How hard is it for somebody to load the API into their own brain? So what does

that stuff mean? So if you look at this, like if you're thinking about what Google has in Gmail, right, maybe you've got conversations and then you have the individual messages in a conversation, right. And you could set those up in a variety of different ways. You could make them

completely separate calls. You also, there are things you want to know to make this particular screen happen. So if you know that your intent is to make Gmail you know that you want to be able to get the subject but also the subject of the conversation but also a snippet of the most recent

message. And making both of those calls every time for every email could be really a pain and a lot of extra things to do. Right you end up with an N plus one querying situation where you have to get the list of conversations and then you have to make a query for every message. When you design for intent you put that message in the first list

that it's only one call to get the whole inbox. That makes everybody's life a little bit easier. So look at this for a minute and tell me if try and see if you can see what's weird about it. So there's a light there for the air conditioning and the light is on. But the

thing underneath the light says AC off. So the light is on when the air conditioning is off which doesn't really make any sense. You're going to look at that and say the air conditioning is on because the light is on because that's the way everything else works ever. But no

that's not how this one works. Right so if you live in Atlanta a light that tells you when your air conditioning is off in the summer is probably like how you might design it from the beginning but people who have gotten into your car have gotten into a million other cars and they're like well that's maybe an exaggeration. They've gotten into a lot of other cars and every other car when the AC is off the light is off.

And so in this way you don't want to be blazing new trails with your API most of the time. So follow don't lead. Use HT basic auth if you can. If you can't use OAuth and use whatever version of it works easiest for you. Accept and serve JSON.

Who can think of a reason why you might not want to use JSON? There are a couple. Yes sir in the back. Right so use cases you might need to serve XML you might also not want to use JSON if you've got a ton of binary data turning that into basic C4 encode. So there are reasons but start from the default of accepting and serving JSON. Make your API

as restful as possible and we'll talk about that in a little bit more detail shortly. Going along with the mental footprint of your API abide by the principle of least astonishment. Use your HTTP methods properly. Use your HTTP response codes properly and as a call back to what a gentleman over there mentioned earlier

please don't ever do that. Please. You've got a 200 OK response code and then a success equals true. Please just don't. That should be a 400 or maybe a 500. Who knows? Who knows what's wrong with it? It's just not successful. And then if you take nothing else away from this talk

about developer driven API design please remember this don't be clever. One of the biggest insults that somebody can give your API is that it's clever. It needs to be accessible. It needs to be easy. It needs to be simple. Clever save that

for I don't know when but please not for your API. Clever becomes confusing and you want to be able to not read every documentation that has ever existed.

So think about if you have a I'm trying to think about the cleverest thing I've seen in an API. A lot of it comes back to using the standards properly. You could be clever by choosing like a really like oh I'm going to use like 422

as my response code because like the description of the 422 sounds sort of like what's going on. I don't know what 422 is. You use like 48 What's that? Right. So that's maybe a bad example. That might be a good one. So there are some HTTP response

codes for example that mean something very specific and if a client is following the HTTP spec the client is then supposed to do something else. Right. Like oh well you a good example of the opposite of that is if you tell somebody the method isn't available it should return

what methods are available in the headers. So you can imagine situations where you just don't want to do something that somebody is then going to have to read documentation to figure out. And especially you don't want to do something that is clever that like somebody might think it does something else entirely

because of the context that they're in but really it does this other thing. I'm sure other people have examples of cleverness in APIs that is painful probably even in MailChimp's own API. But we can talk about that a little more offline after the fact if you and I can think of some better examples maybe.

So let's talk about REST. So REST is an architectural style. It's not a standard. It was coined in a PhD thesis. And the World Wide Web is living proof of the power of it. That's why it works. That's why we can do all the things that we do with it.

So if the World Wide Web worked like APIs do trying to figure out where to go for lunch after this talk would probably of course what you would do is you would pull out your phone book and you would turn the page to Yelp Incorporated and you would call Yelp Incorporated and you would say, hi Yelp

I really need to make a search on your website so could you give me a list of the data you have available and how to access it? And some nice person on the other end of the phone would say sure here are endpoints and here's how you get them and then you would say thank you you'd hang up the phone and then you'd open up Sublime Text or Adam or whatever you're using

these days. And then you would write a client, a web client for Yelp's website. That is how we write every API client ever. But REST helps the web make it so you open up Chrome and you go to any website anywhere and it works. And so when people ask oh why REST? That's why REST because

the network benefits of everybody doing the same thing are huge. And if it did work like that your desktop on your laptop would look like the desktop on your phone if you were to click yes to every time some website asks you to install their mobile app instead of going to the mobile site. So

XKCD to the rescue there. So HTTP at a glance. So HTTP there are just a few methods right? That's how you create things using post or put. You read using get. You can update using

patch or put depending on the implementation and then delete is pretty obvious. Use delete. The response codes are also pre-set for you. You've got your success, your redirect and your error response codes. And this is sort of what we were talking about earlier. You should

return the error code that goes with the error that's happening or the success code if it was successful. And not try to just try really hard not to mix those up because it's not helpful. Right. An example here of cleverness that just came to mind. There's actually a 400 error that says you are making too many calls too quickly.

But with 400 and 500 the general idea is that if you get a 400 error you shouldn't repeat that call. If you get a 500 error try it again later. So you might if somebody is overusing your API and blocked you might think oh it's clever I'll

send a 500 response code back because that will tell them you know try again later when you're unblocked. But 400 would be far more descriptive and would help them to understand oh I'm actually doing something wrong here it's not the service that's broken. So that's just one example that came to mind.

So REST is an architectural standard. It has six constraints. Many of these you get for free just by using HTTP properly. The client server model, the stateless server, cacheability and layered system. Those are all basically if you're doing HTTP properly you get all those. That's just wipe your hands you're done.

Code on demand is what enables JavaScript and Java applets and whatever it was that Microsoft tried to do. ActiveX components or whatever. That's the code on command part of on demand part of excuse me part of the spec. But the thing that really like the reason we're here is the uniform interface.

That is the part that makes that makes or breaks your API really. So the uniform interface constraint you can think about it in four different ways. Resource identifiers. Everything should live at its own address. And that is it's a

it works for developers because we are all well accustomed to file systems we're well accustomed to things to our files living in certain places and so it helps people understand your API better because it's something they don't have to learn all over again. If you look at MailChimp's list APIs you have to learn every single endpoint just to figure out what

anything does. Resource representations is the idea that a resource is not the same thing as its representation. So one example of different representations is you might serve XML you might serve JSON and it might be the user's choice as to which representation they request. You could have different language

translations as different representations. You could even have different representations that do have different fields in them if you really wanted to. So and one of the biggest parts one of the places where your API will fail to be RESTful if it

fails to be RESTful is in the self-describing message portion. Typically and what Royal Fielding recommends is that if you are going to have a self-describing message you should register your own media type with the IANA. And you should do that and that media type should describe how to consume your

API messages. We haven't done that for version 3.0 of Mailchimp's API yet in part because it's still in flux but also because if you've got to go read that media type I mean you might as well read anything else and so we use JSON schema and we pass back a link to the JSON schema that describes

the document that you received in version 3.0. API 2.0 fails because in order to understand the response that you get back you have to go read a giant page of documentation that tells you what every field is and how to process it. And then hypertext as the engine of application state this is what enables the World Wide Web

for you to just go to yelp.com and then eventually find a restaurant to go to lunch, at which to go to lunch. And in APIs this is sort of a controversial thing. The idea of Hadeas in an API is that when you you could find anything in the API

by hitting the root of that API it will give you back a list of links that you can navigate from place to place just like a website. PayPal's API does this actually. The problem that you have with Hadeas if you abide by it strictly is that you have it's a lot of calls then to trace

that API all the way back to the you know whatever sub-resource you're operating on every time you need to make that call. So in version 3.0 of our API we have Hadeas links, we do not expect you to trace them every time you have a deep call to make. We expect that you will bookmark a page inside of, or bookmark a page to use the metaphor, inside of our API. But the links are there for

discoverability. It means that you don't have to read a single page of documentation, you get your API key you call the version 3.0 root and you can then navigate and see everything there is to see. If you add in reading a couple of JSON schema documents you can actually figure out the whole API without hitting our docs page. And that

is what a RESTful API looks like. The Richardson maturity model describes this in a slightly different way. So if we start from the bottom you've got plain old XML and then the

resources and the verbs and it's not until you get to the verbs and the hypermedia controls that it's actually starts to be actually RESTful right? If you just have XML or you just have XML with resources you're not RESTful. You have to have the verbs and the controls at which point

you get to this magical REST thing. Right. And this is not to say that REST is the only way or the best way. It is probably the most commonly understood among web services today. And so having a RESTful API confers a lot of benefits. But maybe you're sitting there thinking this is

just not what I signed up for. I did not want to spend an hour listening to people blather on about REST. But it's important to understand if only so that when you release your API finally and some smug developer on Twitter goes well that's not really RESTful you can know how to respond. You at least know what

they mean when they say that and you can maybe say intelligently well we decided to deviate from REST in this way because of X, Y and Z use case if you don't ignore them which is what most people do. So let's get down to business. Let's fix the mess that we handed out to you earlier.

And just to reiterate the disclaimer you are not seeing the entirety of the MailChimp system. So don't if you're watching this on Confreaks later or you all don't immediately go then try to make API calls based on these docs or anything because it's just a subset.

So let's go back. In order to design this API we have to think about what the intent was. So we've got to remember our intent was to get customers onto our list keep their data up to date organize our list, send things and also have stats to make our boss happy so we know what's going on.

So is there a good reason not to use REST for this list management endpoint? Can anybody think of one? No. I think that's the right answer. In my opinion that's the right answer. There's no good reason not to use REST for this set of endpoints. So where would we start?

No need to raise your hand, just shout it out. Where would we start making this mess RESTful? Okay, what might the resources be? Okay, a subscription could be a resource. A list. List subscription. What are the resources? Anybody shout them out. List member would be a good one. You could subscribe then by creating a list member. So that's another option to having subscriptions as its own

resource. Interest group and interest grouping I guess if you're going to go that route. What else? What other resources might one have? A user. Yeah, in the larger API a user might be a resource. Okay, well we've got some other things. We've got static segments, right? That should be a resource. A report, right? Yeah, the stats

of like the different reports could each be a resource. Right. Yeah, so there would be an entire way to manage campaigns or emails as their own resource. You would also, we talk about merge fields. Those would be resources on the list, right?

But there would also be resources on each individual member then they have their data values of those things. So depending on how your use cases are, you might actually have you might actually have a sub-resource

on each member that shows their merge fields or shows their interest groupings. Interest groupings being the other. Alright, so take a minute and

talk amongst yourselves about what once you have those resources what I want to know is how if you were trying to subscribe 300 people or 3000 people or 300,000 people to your list what would you expect that to look like?

What calls would you expect to make? On our hypothetical brand new API that we've just designed in our minds, what would you expect that to look like? So take or if you know, does anybody know off the top of their head how they would want that to look right now? Okay, good. So what might be the reason

I know when I was talking earlier about having or when Kate was talking earlier about having all of the data available in one call, I've got some dirty looks from somebody over here on this side of the room. What might be a reason why you wouldn't want to do that? Okay, sure. So you might have some scalability issues with pulling all that data and sending it all. Are there other reasons you might not want to do that? Just send everything back? Sure.

Sure. That's definitely true. Right, yeah. That's great. So if you're doing batch processing you have to have a sane way to handle errors. If you send 100,000 subscribers and three of them have errors, well, how do you deal with that? Okay. Did somebody

up here have one? I thought I saw a hand. Yes. Right. And so why is duplicating functionality in your API potentially problematic? Right, sure. So you might it might just be confusing if you have if you go totally overboard on designing for intent you run the risk of creating too many endpoints, too many different resources that people have

to learn and that's where the sort of the tradeoff of this design happens. If you don't go overboard with designing for intent then you end up seeing the opposite problem which is to do one thing in your mobile app you have to make 14 queries and your mobile developers come to you and yell at you because your API is requiring it's just too slow. Yes, ma'am. Did you say?

Sure. Read-read conflicts, write-write conflicts. If you duplicate the functionality and then you've also got just the implementation detail of like if I've got three different pieces of code doing this you've got to make sure your architecture is such that you're not copying and pasting code all over your API. that two page, that two pager

that you've got in front of you if you follow RESTful principles and we talk about the resources we just talked about with lists and segmentation and that sort of thing it can be summed up like this. You can end up with what is that? Six, a half dozen resources

that you can operate on using very basic HTTP verbs and you end up with a far easier thing to understand. Now there's still the issue of what parameters are passed what data comes back and all of these so this isn't the full documentation obviously. But I hope that you'll all agree that if

you see this, this is far less daunting and far easier to understand and navigate than the two page ten point font mess that we handed you at the beginning. So that is the gist of it. I wanted to leave a lot

of time at the end of this for questions because I figured you would all have very specific things and so we're going to talk very briefly about how we worked with API version 3 both evaluating it and then some of our implementation details and then I'm going to leave a lot of questions for you to ask for your own things because we understand

that this worked for us but you might be having a problem how does that map onto your own implementation. this is how we discovered that 2.0 was a mess. so it doesn't look sketchy I'm just going to announce that you're leaving. Kate has to go. She's being given

an award by her university and she has to actually go receive it so thank you Kate. So we got direct feedback from our users. We looked through our support requests we had a ton of them. We looked through social media complaints

this sucks I'm having a hard time I don't understand MailChimp and sometimes those were user error and sometimes those were MailChimp error. We actually sent out surveys. We found people who were using our API especially the other web services that were integrating with MailChimp and we looked through we asked them what's hard about it and because

as we sort of saw earlier most of you have more bad experiences than good experiences with APIs excuse me the poor user experience on an API is the default so in my opinion if people don't love your API it's probably not very good. MailChimp's API

is not very good and it was relatively well received 75% of people thought it was great or excellent so when you're getting this feedback keep it in that context that if people aren't over the moon in love with it there's probably a long way for you to go to get ahead of the pack and the pack is not in a very good place right now so you definitely want to be ahead of it.

Hints from usage patterns really really important. Collect data, log everything about your API it sounds hard especially when you start doing volume but it's really important that you keep track of every call and as much data about that call as you can.

Excessive calls to certain endpoints relative to the customer size for example if we have a person with a 600 member list and they're making 600 calls a second there's probably something wrong and that thing wrong might be that they don't understand they've made a mistake, they don't understand something about the API. It could be that we're not

enabling a really obvious use case that they need. Underutilized endpoints for key features like our campaigns high error rates on certain endpoints or for certain users can sometimes indicate that there's something confusing about that and then really common but oddly specific queries so if

for example you see the vast majority of your queries to a certain endpoint include the same sort parameter maybe that ought to be the default. So think about that and above all when you're looking at usage patterns, collect as much data as you can and just be curious browse through it every once in a while, search it in different ways try to have some kind of easy

usable dashboard we pump all of our data into Elasticsearch and then have a nice dashboardy kind of thing on top of it so that really helps us to see error rates and sort of pair things down differently and then above all when you're looking at this data start from the assumption that

your users are not idiots some of them will be but assume that they're not to start with and that will get you a lot of information even people who are doing things in what you think is a stupid way it might be because that looks like the best way from your documentation or from your endpoints and then a couple of rules of

thumb we have for bad API design the best way for you to find these out for yourself is to code against your own API eat your own dog food as much as you can and if you don't have any internal need for using your own API somehow make some sample apps do the work do you have to make frequent references to your own

API documentation I do, this is what I do every single day is work with the Mailchimp API and when I write code against version 2 I have to go back to the docs a lot and that alone is enough for me to say that we've made a lot of mistakes if you find yourself copying and pasting from your examples

a lot, maybe the API, maybe the wrapper library you have is too verbose long argument lists are often a hallmark especially in wrapper libraries and then if you ever take something out of your API and immediately every time you get it, turn it into something else then that's probably you have a bad representation of that

data somewhere so for example in one previous version of our API we have a merge field called address that you can use it's just a data type, it has address 1, address 2, city, state, postal code you know the thing and at one point we were actually serving it as a two

space separated string all of those fields jammed together in that order and so what did nobody in the world needs the data in that format and so you would get it out of the system and then you would immediately explode it on those two spaces and then hooray, I've got the thing I actually wanted so that's one of those where

if you just start using it you'll start to pick up on those patterns pretty quickly and then some things that you can look at for your own APIs we use JSON Hyperschema to define the API define all the endpoints, define how links are served for Hadeos it's currently

making its way through the standards bodies, it's not a spec yet but hopefully one day we are looking into both Swagger and Rommel we haven't really made a decision yet but hopefully when we launch we'll launch with support for one of those two if you are using something like JSON Hyperschema you should be able to

you should be able to use that to generate wrapper libraries, to generate documentation and one really easy way to start finding problems with your with your documentation sorry, not with your documentation, with your API is how hard is it to write the auto generator for the documentation

if that becomes really hard then maybe think about why that is. If it's hard for you to write a program to understand your API how is anybody else going to do it and then make sure you have wrappers in sample code, in our version 2.0 our sample code are actually apps, like we have a rails app

I think it's still on version 2 3 or something crazy we have node app, we've got all we've got apps for all these different languages which first of all that's not what anybody wants to see right, like if you're trying to use the Mailchimp API you don't want to see a rails app, like you want to see the code that actually interfaces with the API and also that becomes a nightmare to maintain

and so try to keep your sample code easily copy and pastable, get rid of as much fluff as you can you don't need to give people examples as we did on how to build apps, just show them how to use your API and then make sure that you're thinking about catering to different kinds of learners some people want to read the docs so definitely have docs, some people

want to just play with your API so if there's time at the end I can show you the sandbox that we built that auto-updates based on JSON schema, but you can just browse version 3 of our API and then just sort of think about how different people are going to learn your API and then you know, architecture and scaling concerns

those are things that you just sort of have to learn in the hard way eventually somebody is going to use your API in a way that you hadn't anticipated and they're going to be calling a very expensive call frequently at a high rate and it's going to take something down and you just have to figure out a way around that

If you know already that you're giving up data that's expensive, find a way to cache it. Find a way to limit access to it. Find a way to keep it from becoming the default. And you can maybe try to avoid some of those outages. So we've got, if I am reading this correctly, we've got maybe 20 minutes left. So let's go ahead and take some questions.

And we have enough time that we can get pretty deep into some of these. So yeah. So the question is, let's talk about authentication. Signed requests, my experience and knowledge of signed requests is much more limited than a basic auth. But where I've seen signed requests most useful is in

single, like if you're using it, if you're using your API as a front end, if you're doing the thing DHH doesn't want you to do, which is use Rails as an API and then put a JavaScript to MVC on the front. My understanding is that signed requests really help with that aspect. But signed requests, if you think about the reason why I love basic auth for APIs, if it's possible,

is that the amount of time that it takes for your new user to go from, I want to figure out who's on my mail chimp list, to I now have a call that demonstrates that. You can do it in three lines of code. You pull up an HTTP party, or if you're using Python, you pull up in requests.

You hit one endpoint, boom, it's done. All of those libraries understand HTTP basic auth at a fundamental level. You can even do it in your browser with the username at API key, or username code, whatever it is, with the at sign before your URL. You can do that in the browser. So HTTP basic auth, if it works for you, is amazing

because it's supported so widely in so many different tools. There are times and places for other things. Definitely evaluate them all. But if you can support multiple kinds of auth, that's also good. Absolutely. That's a great question. So in version 3.0 of our API, we deviated from REST in one specific place, I think.

I'm not entirely sure if this is actually a deviation or if it's just a deviation from what people think of as REST. I haven't gone through the thesis to figure it out. But one of the things that people tell you about RESTful API is that you want to get verbs out of your URLs, get verbs out of your URLs. Well, that's great.

But there are some things that are really, really weird when you try to turn them into resources. So for example, we've got email campaigns. And maybe I'm just not as smart as some other people, but I couldn't figure out how to get sending a campaign into a resourceful architecture that made any sort of sense and wasn't overly complicated just for the sake

of being RESTful. So we follow Heroku's recommendation, which is if you have non-RESTful actions, basically, if you have actions that aren't like HTTP verbs, if you're doing something other than CRUD, essentially, to have an actions sub-resource and then the action after that. So in version 3.0 of our API, if you want to send

a campaign, you go to campaign slash the ID of that campaign slash actions slash send and you post to that URL. And that's one of the biggest places. We do that in several places where it just doesn't make sense. So you could have a sends resource that you then create one of, and we thought about that,

but that to me seems a little contrived. And maybe some of you disagree with that, but we thought it was contrived. So we're just like, ah, send, it's RPC, but we need it for this. So that's one example. Sure, yeah, and if we had, we don't have,

right now we don't have a tremendous level of introspection into our delivery side from the web app. So we don't know how many have sent in the web app. Delivery knows that, but we haven't pulled those APIs together yet. But yeah, that's certainly, I mean, I'm not saying there aren't ways to do it. But since you can only send a, and this is maybe

a feature of our product, you can only send that campaign one time, ever. If you want to send it again, you clone it and create a new one and send it again. So in that way, there would only ever be one send request, hopefully. And when you're at that point, it's like, well, okay. For us, we just thought it was better

to do the action send. I haven't, actually. Facebook API is a scary thing to me, and I don't have to touch it every day, so no. Is there a specific thing about it that you're curious about that I could maybe? It sounds awesome. It sounds really useful. It also sounds, I'm sorry, yeah.

So he was talking about how, apparently, in the new Facebook graph query language, you can compose different, and get different resources returned in one request. I think almost that that sort of thing might also be made very easy by something like web 2.0, or HTTP 2.0. Not the exact same thing, but it might enable it

a little bit more easily. Send off three or four requests at the same time, get them back, that sort of thing. Yeah, so there are, so when you're versioning your APIs, there are a couple of different schools of thought. One of them is, and many of these schools of thought are very, like, people feel very strongly about it. We version in our URLs.

To me, that's the easy, that's the most straightforward way. Another way is to version in your media type. So in your request, your users would send an accepts header, and the accepts header would include the version in it, saying hey, we're here, we're ready to accept version 3.0 of your API. To me, that's a little bit more dense.

It's a little bit less clear for people. It might be a little bit more semantically correct, but from my perspective, sometimes the semantically correct part makes things hard on users, and saying, oh, well, you're only, give you the example, what will inevitably happen if you only version in a header,

is somebody will write code that doesn't do versioning at all. They'll write code against version 2.0 of your API, you'll upgrade, and since they're not passing an accepts version header, you'll be like, oh, you must want the most recent version, and then you'll pass them 3.0, break those stuff, they'll complain on Twitter, and your boss will be like, why are people complaining on Twitter? There are obviously ways to avoid that.

If they don't send a version, you send an error back, but then we sort of get down that rabbit hole of like, well, why not just have a different endpoint entirely, which is what we decided to do. So Heroku's design guide is basically like, use REST, and the QR code on the top of your handouts there has a link to a whole bunch of different links

that I used in designing this presentation, so Heroku's guide is on there, some other API design talks. As far as when not to use REST, right, right, what to do when you can't avoid it, I think that Heroku's is pretty good, and in a certain sense, that's not really my area of expertise. I think that at least right now,

the benefits you get from REST in terms of HTTP caching, and different CDNs, like Akamai being really easy to work with if you're using a RESTful API, and not being quite so easy to work with if you're using XML, RPC, or SOAP. And I think that just like, with as many people as are using REST right now,

or at least trying to use REST right now, it's hard to blaze a new trail. Now if somebody did and it was amazing, then maybe five years from now, we'll be sitting here talking about why would you ever use REST anymore? Nobody uses that. And that'll probably happen at some point. But I think right now, I haven't seen a lot of good, and one of the reasons we went RESTful is because I couldn't find a lot of good reasons

to use XML, RPC, or SOAP, or any of those. Sort of. We have, in almost every case, we have, we use sub-resources. So you would go lists, ID, members,

email hash to like, find a member. We don't necessarily have a documentation of like, oh, well this member has been sent these campaigns. But we tend to try to make it as hierarchical as possible, so that when you are down in a member, you can get things about that member.

And so we try to keep it related that way, instead of trying to define a bunch of, oh, well this campaign has a sent to, so that's maybe a good example. If you look at, if you get the collection of members that have been sent to on a campaign, we tend to provide those member resources

inside of, not inside of the request, but if you go to campaign slash ID slash, or maybe it's reports slash campaign ID slash sent to, I'd have to look at the docs to be 100% sure, but that will provide you a collection of member resources that are the member resources

from the slash list slash members endpoint. And when those have their hadios links in them, they point back to the canonical representation of themselves back over on list slash ID slash whatever. So the answer to your question is not really, but sort of. That's a good idea. So documentation shows sort of like a top level

system architecture diagram of what your resources, how your resources relate to one another. Well, let me tell you, so okay, so I first have to tell you our dirty little secret, and that's that MailChimp is a PHP app. Sorry. But let me tell you how we,

so let me tell you how we do it in versions prior to 2.0, and that is when you create a new version, you open the directory of the previous version, you select all, I bet you know where this is going, don't you? You copy, and then you edit the code as appropriate.

I don't like that way of doing it, but that is one way. We found that to not be scalable for all the reasons you might think it's not scalable, in that, oh, we found a bug in version 2.0. Well, I guess we better make the change in 2.0, and 1.3, and 1.2. In 3.0, we've completely changed the architecture entirely to support REST at basically the top,

the bottom or top level of the API, whichever one is the most basic. And so what we'll do when we inevitably have version to 3.1 is we will have our router at the top level fall back, so if you request version 3.0 slash lists, and we haven't created a version 3.1 of lists,

the router will know to fall back to previous versions of the API. How we will implement that is yet to be determined, because we don't have a version 3.1, but eventually, that is our goal, that until a resource actually changes, there's no need to version it. And then once we do need to version, say, if lists does change in 3.1,

theoretically, much of it will stay the same, and so we'll be able to inherit from version 3.0 and just change what's different. So those are some thoughts that we have on moving forward and on what was bad about the way we did 2.0. Okay, so the question was how do we do our documentation? How do we make sure the documentation

stays up to date with the actual app? We'll do an old and new. In version 2.0, we actually generated the documentation from comments in the code. We also did this crazy thing where the API, as it was parsing the request,

would check against those doc blocks using reflection to make sure that your code was, your request was appropriate, so it was the right types and whatever, which led to a very interesting conversation I had with a coworker of mine in which one of us said, oh, man, we broke the API by making a bad comment there.

We're really gonna have to put some unit tests around those comments. If you ever find yourself saying you need unit tests around comments, you have made a wrong turn somewhere. So, but what we were doing is, on the documentation side, it's less crazy, but you still do end up with the problem

of having to, sorry, I lost my spot, of having to keep those comments up to date. We use JSON schema at the renderer level of our API. So the first thing you do in R version 3.0 when you're creating a new resource is create the schema, and the schema will have title and documentation,

title and description and data type and all of this information in it, and the renderer will use that to generate the response, essentially sort of at a deeper level. The controller, the resource returns some data, a data object, the renderer looks through the schema

and says, okay, let me pull this response, this request needs to respond with a title and a name and an email address and whatever else, and it goes and pulls that data out of the data object that was returned and constructs the, it serializes it, but it builds the serialization,

it starts with nothing and adds things from the other doc or from the other data object that comes back, and what that allows us to do is it means that unless something is really, really bad wrong, our documentation that is built off of JSON schema will always match the stuff that comes back

because the renderer is building that exact thing from the schema in the first place. And so we sort of fix that problem by forcing you to design the schema and keep the schema up to date because the schema is what gets returned. And so that's sort of our, we had that exact problem where it's like, well, we have to update these comments and all this, and so we just said no.

Like, design the schema first, design the code to implement that schema later, and then it all, so far it's been working out pretty well and then we use that JSON schema to generate documentation. Right, and I think Swagger actually sometimes uses JSON schema to define the resources, so. So the question was how does MailChimp determine

what the people are using the resources for? And for this, we don't have a very technical approach. Because most of the API traffic we get is from those integrations, those third parties that are making requests on behalf of one of our users, we ask them. And we try to open those lines of communication

so that they know that if something is hard or confusing that it shouldn't be and that they should complain to us and that from the volume of complaints we can sort of generate that. If you keep a lot of stats about your API, you can sort of start to intuit things. But I don't have a good technical way. I think that a lot of it is just keeping lines

of communication open with the people that are using your API and listening to them when they say they want something. I don't know if anybody else out there is like this, but I know my first reaction to, ah, we need this new features. You don't need that feature, this feature, this is perfect the way I wrote it the first time. And so I have to fight that internally. But it's really important that you do fight that because sometimes when they say I need this

and you're like, well, that breaks the purity of my API. It's like, well, yeah, but what is your API if it's not useful to people? So tools to maintain the documentation. Okay, we're five minutes over then. I don't want to cut into your break. Thank you all so much for coming. If you have further questions, please come see me.

Thank you.