All right folks, I'm going to get started here. Thanks for coming out. USB keys have two files you'll need, the vagrant file and the HTTP exploration box file.

If you don't have vagrant and virtual box, there are executables on there for Windows, Linux, couple of variants of Linux and Mac. You'll need to have those installed. I'm going to do a half hour lecture, hopefully a little shorter than that, and then we'll have about an hour to run through the exercises.

The exercises will walk you through starting a vagrant and everything you need to do. So I have Charlie Sanders helping me out. Raise your hand if you need a USB key to get the files from. So basically, you've got half an hour to get all that working.

If you don't, don't worry too much, you can pair up with a neighbor. I would appreciate feedback, negative or positive. You can tweet me at Craig Buchek. My email address is up there. My presentations are on GitHub. I haven't put the latest versions of this presentation yet, but it should be there tomorrow.

If you guys want to follow along, tinycchvexploration with an underscore will take you to this presentation. Actually, I think I need to update that. So I'll do that when we get to the exercises. The exercises will be in there too, so I'll update them when we start the exercises.

So the reason I started doing this is I actually have a previous life as a sysadmin and a network admin, so I've actually done a lot of troubleshooting of HTTP, going through networks, going through firewalls, and now I'm a Rails developer, so I've seen both sides.

So we're gonna talk about HTTP basics, requests, responses. We'll talk about proxies, some troubleshooting, and HTTP2, and then we'll get into the exercises that touch on all of those. So HTTP's been around since 1991. The first version didn't actually have a version number,

but retroactively we call it 0.9. It was standardized in 96. The version that we typically use now was standardized in 2007. That RFC is very handy if you have any questions about how HTTP works. I was just updated a few months ago. It was broken up into multiple pieces, basically in preparation for HTTP2.

HTTP2 has been ratified, standardized, but it hasn't been published yet, but that's a good URL to check out if you're interested. So HTTP is stateless, which means that each time you connect to the server, it doesn't really remember what you did

the last time you connected to the server, and the only way around that is through cookies, pretty much. It is text-based. That's kind of the whole point of the presentation. We're gonna look at the text that's going across the wire. And it's a request-response cycle. Your client, your browser, will make a request to the server. The server will respond.

So we're gonna talk real quick about URLs. So pretty much every piece of that URL on the screen will go from the client to the server except for the fragment. The fragment just tells the browser to go to this specific spot on the page.

The scheme is going to be HTTP. The host is obvious. You can specify a username and password in the URL. Recommend against that, but it's possible. And then the path is actually the piece that we'll be working with for the most part.

All right, so the HTTP request looks like this. That thing in green is called the method, so we'll talk about that in a bit. The next thing that's passed is the URL. Usually it's not the full URL. Usually it's just a slash. It starts with a slash, and it's relative to the top of the site.

And then you have the version of HTTP specified there. In blue are the headers. So the headers are sort of metadata about what you want to get or what you want to do. In this case, there's a host header and a content-length header. And the body, not all requests have a body. If you're just getting something, it won't have a body.

But if you're posting something or putting something, it will have a body. In this case, I tried to put something to Google's front page. Probably not gonna work. I actually tried that, and it gives a 405 error, which is hard to find. So that's the body text, basically the content that you want to upload.

So the methods I talked about, normally when you go to a website, you're going to do a get request. Say, get this page. You go to Google's front page, you do a get. A post is when you want to update something or when you submit a form.

Although some forms can actually do get, depending on if you're changing something or just doing a query. Like when you go to Google's front page, that form, when you type in the query, that's a get, because you're actually just asking for information you're at not asking to change anything. A put is actually, it's an update,

or actually it's more of an upsurge. You're saying, I'm giving you the whole thing. Replace whatever you got. If you have anything, otherwise create it. Rails is a little weird on that. It doesn't actually do the create part of that. So it uses put a little bit oddly. Delete does what it says. If you have permission, you actually can delete pages

or resources through the web. Head is basically a get request, but without the body. You're saying, show me the headers that I would get if I did the get request for this URL. There's a concept of safe methods.

So safe methods don't have any effect on information. When I talked about the post, it changes something. When you submit a form, it changes something on the server. A get is not that case. So the takeaway here is don't have your app change things when you do a get because when Google comes scraping your site, it's gonna do gets on your site.

If your site changes something on the back end, you're gonna have a bad day. So just keep safety in mind. There's also something called idempotency. That means that you can call the same thing multiple times. You can do a get to the Google front page multiple times. You'll get the same thing.

Same thing with the head. Same thing with if you delete a resource. The resource is gonna be gone if it was there or not. A put, you're actually saying replace something. So if it's there, replace it. If it's not there, we'll add it. So multiple times, doesn't matter. The one that's missing from this list, again, is post.

So if you post something multiple times, you'll end up with multiple copies of that thing. So request headers. We saw that we can provide those headers. These are some common headers. The host header is basically required. It says the name of the website we're accessing.

And the reason that that was added is so you could host multiple websites on a single server. The accept header says list the types that the browser would like to receive. So the browser may say that it likes to have HTTP.

It likes to have JSON, but not list XML. So if the server knows how to do HTML and JSON, it'll pick one of those, but it wouldn't pick XML unless it didn't know anything else. The weird thing about the accept header is you can actually specify relative qualities. Sometimes you'll see a queue number

if you look at the accept header. And that says I prefer these types of things over the other things. The default quality setting is 1, which means top priority. If you prefer JSON over XML, you could set its quality to 0.9 and XML's to 0.8. The content length on a request header

is the content of the body of the request that you're sending. And if you don't specify that, it actually doesn't let you specify anything. Once you've uploaded the content length number of bytes in the body, the server will close the connection. The content type tells what you're uploading. Am I uploading some HTML?

Am I uploading some XML? Am I uploading some JSON? The referrer is if you are a browser, it's the page you're on when you click the link. That is spelled incorrectly, but it's in the standard document, so that's what we use. The user agent is another name for a browser or web client.

It could be a web crawler. It could be various things. It's a string containing the information about the browser. And if you take a look at those, those get to be pretty crazy long, especially in Chrome and Firefox. They try to pretend. So back in the day, there was something

called browser sniffing. The server would look at that string and try to determine, oh, I'm Internet Explorer. I'm going to give you this copy of the page. And you're Firefox. I'm going to give you this copy of the page, which could be completely different. Google didn't like that, because Google wants there to be one canonical version of every page

that it indexes. It doesn't want you to fool it, so it sees one thing and then the user sees something else. So browser sniffing is not used too much anymore at all. But you end up with these really long strings that tell you sort of all the backwards compatibility that the browser tries to do.

And we'll take a look at those headers in the exercises. Yeah, yeah, yes, yes.

And then content type is only for a post or a put. And it's what type the body that you're sending up to the server is. So authorization is for, if you ever go to a website and you get the pop-up box, authorization

is basically that information you type into the username and password box. It's not encrypted. It is hashed and sent. And there's an exercise that we're actually going to find that going across the wire. Except encoding, you can actually gzip things going to the server,

can gzip the content coming back to you. This basically says that's OK. You can say content encoding gzip comma deplate. That tells the server, hey, I can gzip this and save some bandwidth across the wire. Connection usually is for keep alive. It is the client saying, hey, I've made this request.

But when I'm done, I'm going to make another request. So don't bother tearing down the TCP connection. I want to reuse it for the next request. So if I'm getting a web page and I know it's probably going to have some JavaScript, it's probably going to have some CSS, it lets me get those all in one without dropping the connection.

Cookie. So I talked about a cookie is the server has sent us this cookie. And we are supposed to return that cookie back to it. And that maintains sessions with the server

between the server and the client. And that's the only thing that maintains a session between the client and the server. And that header is limited to about 4K of information. So if we do our cookie-based sessions in Rails, we do have to worry about that limit. You don't want to store a whole lot in that cookie.

All right. So we've made the request and the server responds with the HTTP response. That first line is called the status line. It's got the HTTP version. Interesting thing, you can actually make a request for one version and get a different version back. I find that a little odd. The 201 there is our status code.

And then there's the description of the status code which is created in this case. And then like the request, the response actually also has headers. And then it has a body. Not every response has a body. There's a few that don't have to have a body. Created is actually one of those.

So if you are doing a put or a post, it could actually just say create it and then don't give you anything back. But you probably want to do a redirect. So you probably want some extra information in there maybe. So status codes. So the status codes have different meanings

and they are actually standardized. Although occasionally you'll see some non-standard ones. The 100s are informational. You'll rarely see those. The 200s are what you'll see most of the time. Usually for a get, you're gonna get a 200. For a post or a put, you're probably gonna see a 201.

Redirection, if you go to google.com, it'll actually redirect you to www.google.com. That's a redirect. I believe that would be a 301. If you send headers that are involved with caching, which we'll do in an exercise,

you'll get a sometimes you'll get a not modified. And that says, hey, you've told me you already have a copy of this in your cache. So I'm just gonna give you the metadata and the headers. I'm not gonna give you the body. You should use the body that I gave you last time. So then we've got the error response codes.

So 400s are client errors where the client made some sort of mistake. Unauthorized is if you have not sent to an authenticate header and the web server requires you to authenticate, the client is supposed to retry with the authenticate header.

And so before it does that, it pops the box up, has you type in your username and password, resends with that header. Forbidden means you've probably, either you can't authenticate or you've authenticated, but you still won't have permissions because maybe you're not an administrator on the box.

404, we've all seen that one before. Page not found. 407 is proxy authentication required. It's like a 401, except it's the non-transparent proxy that is asking for a username and password. That's pretty rare to see. We will do a little bit of proxies, but we're not gonna do any proxy authentication.

422 is unprocessable entity, which is a weird way of saying I don't understand what you're trying to ask me. That is recommended, that's what I would recommend, either a 422 or a 409 if you are making an API request and it doesn't have the right information

that it needs in the JSON or XML or whatever that you sent. Server errors. So if your server crashes really badly, you will see a 500 error. I believe you see that in Rails a lot of times when you're debugging. 502 and 504 are gateway errors.

So if you've got a reverse proxy sitting in front of your servers and your servers have gone down or take too long to respond, you'll see a 502 or a 504. There are plenty of others. I ran into a 405 earlier. There's one of the April 1st RFCs

called I'm a teapot response, a 418 code. Not sure when you'll need that. I'm sure someone has implemented it though. Response headers. So like a request, response has headers. The content length will almost always be there because you'll have a response that has content in the body.

The content type, it's the MIME type, so it's gonna be like text slash HTML, text slash plain, application slash JSON, or is it image slash JPEG, image slash PNG. That tells the client what type of file this is

and then it can handle that however it expects to. The content encoding is, I talked about accept encoding where you can gzip it. The content encoding says that the body has been gzipped. Notice that the body is gzipped, but the headers are actually still in plain text.

Content disposition is a little trick that you use when you want to, when the person clicks on the link, you want them to download the file instead of have the file display in the browser. In that case, you would use content disposition header and you can also provide a default file name that the browser will try to save it as.

Location, location is used for redirecting and we'll have an exercise on that. Usually you want to provide a response code that says redirect and then you provide the URL to redirect to in the location header. Set cookie, we talked about cookies

for maintaining state, maintaining sessions. Set cookie tells the browser to remember this token. It's just roughly a random string of characters and when it gets sent back by the browser, the server knows to associate it with a session. It looks it up in the database.

www.authenticate is basically telling the browser that to pop up the box or if they've already typed their username and password to provide that information to the server. All right, so we'll run into proxies. We'll have some exercises on proxies. So a proxy is something that acts in place of another.

In the case of a HTTP proxy, a web proxy, it intercepts our HTTP requests. So it can modify that request. So what it does is it intercepts the request, modifies it probably, does some caching perhaps and sends it onto the server, gets the response back from the server,

can modify it then again and then sends it back to the client. So it sits in between the client and the server and it can modify pretty much anything that it sees in there. So proxies are good for caching. You can add security. So I've actually, actually our exercises,

we will add some SSL to our Rails app. So you do this to simplify so you don't have to have the Rails app understand SSL. Also can save some CPU time. Also can be used for load balancing.

You can be used for authentication. I've had it where I had Apache in front of a application server and we added the pop-up authentication for it with Apache, acting as a reverse proxy which I'll talk about in a second. So there are transparent proxies. Basically you don't have to set anything up.

It's inputted itself into the stream of the network and there's non-transparent proxies which a lot of times if you're at work and at a big company, you'll have to configure a browser to point at the proxy. That's a non-transparent proxy and we've got an exercise on that. Reverse and forward proxies.

So the proxy you have at work that sits sort of right next to the firewalls would be a forward proxy. A reverse proxy sits next to the server. So right near the servers, anything that's, any proxy that's added on the server end is a reverse proxy and we will actually have exercises on both of those.

A CDN is a content delivery network. It's basically a paid service that does proxying for caching purposes. So you can cache all your static content. Technical things in here I won't get too much into. One of the nice things that can provide you,

protect you from distributed denial of services. If you've got a big site, you should probably be looking into these. Troubleshooting. So any network problem you have, you have to think about the OSI model and I wish I had a picture of that. I forgot to put that on there.

So you've basically got the physical layer, you've got the network layer, you've got the transport layer, which is TCP and then you've got your application sitting on that. So there's a lot of things that could go wrong. You could have a network cable pulled out. You could have your routers down or the server's not running. So all those different layers you have to think about when you have a problem. Troubleshooting is trying to figure out which layer it is

and then narrowing down on that. So one of the first tools is ping. Can I connect to the IP address that the server is on? Problem with that is sometimes the firewalls will prevent that either firewall in your company or the firewall on the other end.

Traceroute is similar but it shows you all the hops in between. So maybe the network is down between your internet provider and Google. Traceroute might be able to help you find that out. Telnet we're gonna use in our exercises. It's a good way to tell if the port is listening. If I've got connectivity to the IP address,

Telnet will tell me if the service is listening. And that's actually kind of where I start. I kind of start in the middle and then if Telnet doesn't work, I'll try the lower layers and if it does work, I'll try the upper layers. If you're on your server and you want to see if your service is listening, you can do a netstat on Linux.

That's dash P-L-A-N-T. It's easy mnemonic for me to remember. On Mac, it doesn't have all those options. You'd use dash NA and then grep for listen. And then so that's gonna list over on the left side. It's gonna list all the IP addresses which is usually gonna always be your IP address

and then colon in the port number. So if you're looking to see if Rails is running, look on the left side for something colon 3000, assuming you use the default port for Rails. Telnet doesn't work with HDVS because it's encrypted. So you have to use this tool that OpenSSL provides called S client and we've got an exercise on that.

TCP dump, if you really want to see all the details of what's going across the wire, it'll tell you everything. We've got a short exercise on that. There's so much information, we could probably spend a couple hours on that. Wireshark is a good way to visually see what's going on. You can actually take the output of TCP dump,

save it to a file, pull it into Wireshark. Wireshark has a nice feature. TCP dump shows you each individual packet and their disjoint. When you're having communication between a client and a server, your packets can only be so big. So the communication will be broken up into pieces. Wireshark has a nice feature

to put all those back together, which is kind of cool. All right, so as I said, HTTP 2 was recently approved and ratified. It came out of a project at Google called SPDY, S-P-D-Y. Apparently they wanted to make it speedy to say, to write the word speedy.

So as I said, we can compress, we can gzip the body, but we can't, in HTTP 1.1, we can't compress the headers. HPAC is a part of HTTP 2 that allows header compression. Another thing, the standard doesn't require it,

but every HTTP 2 implementation out there requires TLS. And that's probably because it's got, TLS or SSL has a protocol negotiation built into it. And that protocol negotiation can say, hey, do you have HTTP 2? I'd like to use that. Do you have SPDY 3?

I'd like to use that, but if not, fall back to HTTP 1. So I said that HTTP is all text. HTTP 2, that turns out not to be the case because we're compressing the headers. It has the same semantics. You won't be able to use Telnet, you won't be able to use TCP dump,

but you will be able to use some of the other tools that we'll be looking at today. I did get an example to work with HTTP 2 that we can do as an exercise. When you're making a lot of small requests, like let's say you've got a lot of icons on your page,

ideally you would just get each icon individually, but those icons are only 25K or something. At that point, the headers at a couple K are starting to become large overhead. And if we could compress those down to a couple dozen bytes, it would save a lot of bandwidth. And there's some other features in HTTP 2

we'll get to in a sec. Basically just to save bandwidth. The other thing about HTTP 2 is it multiplexes the connections. You can actually have multiple files coming across at the same time with a single TCP connection. Right now, your browser has to make

multiple TCP connections. I don't know what the browsers are up to. They started it like four at a time, they got up to eight at a time. But that means you've got, the TCP connection takes up resources and it takes time to set up. So if you could do just one TCP connection, that would save some time. That's one thing they've done. So you can grab your HTML file

and you can actually start processing it and see, hey, I've got some JavaScript, I've got some images, I've got some CSS, I need to grab all those too. So I can grab all those simultaneously. So that's gonna save a lot of time. Server push. The server can actually know, hey, he's getting this HTML file,

he needs a CSS file too. I'm gonna start giving it to him even if he didn't ask for it. And the client can say, well, I'll take that, hey, I finished with the HTML file, I didn't need that CSS file, you can stop giving that to me. Which is kind of weird with caching. I don't know how it knows what's been cached and what doesn't, but the possibility is there

to save some time. So when we do web design and web development, we've got the asset pipeline, right? And that combines all our JavaScript files into one big file. It combines the CSS, we probably combine that into one file. The images, we probably do something called spriting

where we put a bunch of images in one file and then the CSS has to go grab each piece to put each icon on there. If you look at the, I know I've looked at the homepage for Yahoo and it's got all the icons on the left, those are all in one file and the CSS gymnastics we have to do is a pain in the butt.

HTTP2 will hopefully allow us to stop doing that. We can just write it the way that it should have been written in the first place, not worry about performance. HTTP2 should do that for us. HTTP2 is kind of weird that it starts with a 1.1 connection

and then it upgrades. So the semantics of that are pretty crazy. You probably don't want to get involved in that. We'll see a little bit about that in the verbosity of the tools we use. We'll show you a little bit of that. So where is HTTP2 working?

Chrome 41 has it, Firefox 36, IE 11 but only in Windows 10. So nobody has that yet except in beta. Nginx says they're gonna support it by the end of 2015. They already support SPDY 31, which is pretty darn close.

I could not get cURL to work with Nginx using SPDY or HTTP2. So cURL 4, right now you have to specify the HTTP2 flag and for this I had to manually compile that feature in. Wireshark 2 will have it.

They're currently in the beta series 1.99. Patchy doesn't seem to have plans, which seems a little weird, but it does have mod speedy available. All right, so it's time for the exercises. I need to update that URL real quick.

But Charlie will be walking around helping people get started. The exercises are basically starting on page 27 of this. And so the first step is basically add the vagrant box with that command there, do a vagrant up and then a vagrant SSH, you'll be in the box

and then you can move on to slide 28. Will Rails support HTTP2? Probably not for a long time. So Rails really doesn't even support HTTP. Your Unicorn or your Puma are what supports that.

So when those support HTTP2, we will have it built into Rails. Rails itself sort of sits right behind that. Right now if you want it, you would put a proxy in front. You would put nginx is my recommendation. In fact, nginx said they have 95% of the, well it was speedy at the time, servers on the internet.

So you would use a reverse proxy. Well, Rails also doesn't support HTTPS. So you probably want HTTPS on your site, so you probably need that proxy anyway. Another question? Oh, the link?

You should see this version without the vagrant init. If you did the vagrant init, you'll need to get the vagrant file back. And I think you'll need to do vagrant box remove. But raise your hand if you did the init and we'll help you out.

So once you guys get into vagrant, with the vagrant SSH, just work through the slides. Raise your hands if you have any questions and we'll come around to help.

Oh.

That's like turn off.

Can you remember your vagrant file?

Oh, okay. Okay, we're just going to tip on it. Oh, sorry.

Hey, here's one. I'm just saying. Where are you at right now? I'm trying to tip right here. I was for a little bit, and we were friends of yours. We didn't have time today. I'm sure. You guys, we're fine.

Sure. Paul's going to do it.

Um, I think pause, but I'll pause for a minute. Yeah, whatever. And now the installer isn't working. All right, cool, thank you. Thank you. Next one.

It should be like the next slide. Yeah, there's no, nothing wrong with that one. I think they're working on it.

I think the next thing we're gonna do is start the event.

Yeah.

It's tall. It held 99 cents. Yeah.

Completely valid.

Yeah.

Yeah.

Yeah. Yeah. Hey folks, for the HTTP 1.0 and 1.1, make sure you enter the blank line when you're using telnet.

If you don't do the blank line, I'll just sit there waiting for you.

Well, oh, I think just, uh,

oh, yeah, Good. Uh,

yeah. Oh, yeah. Right. It's in the part. Oh,

well, I think this is like press enter now. no, no, no,

no.

Oh, no.

No.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

. .

.

.

.

.

So I run a company's website, and for so much that I just can't do it. Google. So there's only one idea.

One idea is when I'm doing a company's 48-minute process, it doesn't know if I own the future at that time, or if I own the company's 48-minute process.

Go for it.

It'd be really funny.

Okay.

We can get to the rail so far.

Yes.

Okay.

Okay. Thank you.

The sauce is like up there.

Thank you.

Thank you.

Thank you.

Hey, folks. We have 10 minutes left, and then lunch will be served.

You've got this virtual machine that you can start up, or you can leave it running. You can beg your necessity and do it later and complete the exercises at home. So my email address, my Twitter are in the presentation. You can ask me questions later. I can finish this at your leisure.