We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

gRPC, Protobufs and Go... OH MY!

00:00

Formal Metadata

Title
gRPC, Protobufs and Go... OH MY!
Subtitle
An introduction to building client/server systems with gRPC
Title of Series
Number of Parts
561
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
gRPC is an open source remote procedure call framework built on HTTP/2 that provides an easy way to create high performance, scalable APIs in a multitude of programming languages. In this talk we’ll describe what grPC is, how we it came to be, and compare it to some other alternatives. We’ll talk about the pros and cons and hopefully convince you to take a closer look at it and add it to your toolbox. Best of all we’ll walk through an example of how to build a gRPC based application using gRPC, Protobufs and Go! We’ll even take a look at how easy it is to add a client using another programming language.
10
58
80
111
137
Thumbnail
15:21
159
Thumbnail
18:51
168
Thumbnail
26:18
213
221
Thumbnail
15:22
234
Thumbnail
49:51
248
Thumbnail
23:06
256
268
283
Thumbnail
28:38
313
Thumbnail
1:00:10
318
Thumbnail
21:35
343
345
Thumbnail
36:13
353
Thumbnail
18:44
369
370
373
Thumbnail
44:37
396
Thumbnail
28:21
413
Thumbnail
16:24
439
455
Thumbnail
25:10
529
Thumbnail
15:36
535
Thumbnail
28:04
552
System programmingBuildingComputer configurationPrototypeTwitterInheritance (object-oriented programming)QuicksortDifferent (Kate Ryan album)Computer configurationGroup actionBlogVirtual machineClient (computing)Goodness of fitMultiplication signSpacetimeServer (computing)Computer animationLecture/Conference
Buffer solutionCommunications protocolStandard deviationStreaming mediaControl flowDataflowComputer programmingProxy serverBlogImplementationServer (computing)Client (computing)Interface (computing)Loop (music)SCSIInformationLoginData storage deviceSoftware testingStorage area networkBuildingService (economics)VolumeLogicElectronic mailing listString (computer science)Inclusion mapContext awarenessStreaming mediaCuboidServer (computing)Volume (thermodynamics)Parameter (computer programming)iSCSI2 (number)LogicService (economics)Inheritance (object-oriented programming)Storage area networkCommunications protocolReverse engineeringData storage deviceQuicksortRight angleComputer fileMiniDiscElectronic mailing listSpacetimeKeyboard shortcutInterface (computing)LaptopClient (computing)CodeProxy serverDevice driverDataflowProjective planeBuildingGame controllerProfil (magazine)ImplementationGoodness of fitCursor (computers)Computer configurationInformationComputer animation
Gastropod shellProgrammable read-only memoryKeyboard shortcutMultiplication signDirectory serviceCodeRight angleReal numberComputer clusterData structureVolume (thermodynamics)NumberCuboidCommunications protocolMessage passingParameter (computer programming)Client (computing)String (computer science)CompilerService (economics)Computer fileServer (computing)PasswordProfil (magazine)Virtual machineCASE <Informatik>Group actionDependent and independent variablesComputer animation
Normed vector spaceGastropod shellMessage passingVolume (thermodynamics)Context awarenessDemo (music)Software testingComputer fileCodeIP addressServer (computing)CuboidClient (computing)Service (economics)Partition (number theory)Physical systemLine (geometry)System callTerm (mathematics)LaptopKeyboard shortcutInheritance (object-oriented programming)Dependent and independent variablesMusical ensembleRight angleReal numberMultiplication signCompilerFunctional (mathematics)Computer animation
Computer animation
Transcript: English(auto-generated)
My talk is gRPC, Protobufs, and Go, oh my. So how many of you have already used gRPC and Protobufs? OK, so a bunch of you. This might be a review for you. This is kind of like an intro, a sort of getting started, and so we're going to go through.
I'm John Griffith. There's my Twitter, there's my GitHub. Reach out to me after this if you guys want to chat, if anybody has any questions, or wants to tell me I did something wrong, or whatever, or just wants to meet me and talk. I work at Red Hat, and I'm mostly around the Kubernetes space these days. So, RPC, basically we're just talking about machines,
talking to machines. The whole gRPC thing and everything, it's not necessarily a groundbreaking new idea. This technique and these kind of ideas have been around for a long time. I'm old enough to have worked with most of these, unfortunately. So, gRPC is kind of the newest take on this idea.
Simple idea is basically you have some clients that you want to be able to actually affect actions on a remote server. Usually that transport that you're going to use is TCP. One of the nice things about gRPC is that it's going to use HTTP2 specifically, which gives us a lot of flexibility and a lot of capabilities
and much better performance and efficiency. So, my talk really is actually just to tell you that gRPC is really the only option out of all the different things that are out there, and it's the only thing that you should use. So, that's really why I'm here. Okay, not really.
Actually, I'm the opposite. There's a lot of really good options out there. There's a lot of really good technologies, a lot of different things you can do. But gRPC is actually really cool. It's got a ton of super, super good features, things like that. So, what I always tell people is just please always investigate new things
and look at other options out there when you're doing your research. Don't just listen to what somebody said at FOSM or anywhere else, right, or what they wrote on their blog. Check it out for yourself and see what you think. So, some of the things that I think are really nice about gRPC is
you get pretty much a complete solution out of the box with gRPC. So, I lump protobufs into gRPC because they're hidden when you're actually trying to do an implementation or build this. So, these protobufs, it's got some really cool advanced features that we might get into today, hopefully, like streaming.
So, you can even do bi-directional streaming. So, you can have a client and server stream back and forth, open a channel. Blocking and unblocking options, flow control, all the sorts of things you would expect. And one of the really cool things is the ability to build in with protoc, a reverse proxy for rest.
So, if you're hung up on rest and you need a rest compliant API, you can actually still do that. You can actually use things like protoc to build that. So, that's really cool in my opinion. But the main thing, the reason I love gRPC and I started using it a while back is because it's super easy and I'm lazy. So, all right.
So, basically building a service with gRPC, it distills down to just three steps, right? So, the first thing we're going to do is we're going to create a interface definition. So, that's our protofile and that's going to just be saying what it is that we're going to serialize back and forth across the wire. The second step is to actually implement the server
and then the third step is to implement the client. Now, I did leave one important thing out of here, right? You have to have an API of something that you're building, right? To do something useful. So, but that's something you've probably already done. So, my argument is that it's so easy that we could actually
just build a gRPC service right here and now. And not just the usual ping pong, you know, anything like that. So, what we're going to do today is we're going to take this simple LVM package that I have, the simple LVM API and we're going to build a gRPC service for it right here on the fly in this session.
So, it's going to be really interesting because this is not my laptop and it has a UK keyboard. So, you guys should be entertained, if nothing else. All right, so what we're doing is, you know, you should always kind of have an idea of what you're trying to do.
We're going to basically build our own SAN device. So, I'm doing a lot of stuff with Kubernetes and containers and all these different things in the storage space and I don't want to go out and like buy a storage box from somebody or something like that because I left those startups and I don't do that anymore. So, what I want to do is I want to take LVM
and I want to put LVM on a server that I have with a bunch of disks and then use iSCSI and just do exports and just build my own SAN basically. I actually started to do this as a project for Kubernetes for iSCSI D driver that's in there and found that all of the code that's there is actually deprecated and obsolete so I'm rewriting it and this is what I'm doing.
Okay, so, I've got this package, it's called go-lvm and we're going to start off really simple. It just creates logical volumes, lists logical volumes and deletes them. So, everybody knows what LVM is, I'm assuming. All right, so, step one, I'm going to walk through the steps again real quick
and just give you a preview of what we're going to do. So, we're going to define our interface, right? So, that's going to be our proto file and then we're going to implement a server real quick and then we're going to implement an actual client and then we're going to deploy this and run it and check it out.
So, we'll start off and we'll just make this really simple. We'll just do like create volumes. So, I'll try not to talk too much while I'm... Well, that's not going to work. Who am I kidding? Yeah, okay. I can talk really loud. So, if I can find my mouse or my cursor.
All right, so, this is going to be good. Let's minimize this and get this over here. All right, so, I've got a couple of Fedora boxes running on VMs
that I'm running on this machine. So, what I'm going to do is I'm going to go on to those and let me actually mirror this because... Well, we'll just try it. Okay, so the first thing I need to do is I am in my go path. I'm in my GitHub directory.
So, if you look PWD, you can see there's Jay Griffith. Can you guys see that or should I make it bigger? All right, so we'll do that. So, I'm going to set up my directory structure real quick and the way I'm going to do that is I'm just going to say I'm going to create a directory for my proto files,
a directory for my server and a directory for my client. Okay, we're quiet. Oh, see, and there's the UK keyboard right there. Oh, no. All right, relax. Don't panic.
Here we go. We can recover. Okay, so, let's go into the proto file. The proto file is actually, this is really kind of the nuts and bolts, right? This is the main thing. And what we're going to do inside of here is we're going to create this file and we're going to call it lvm.proto. And what this is going to do is this is going to define
what we want our service to be, right? So, let me see here. So, we start off and we just say... Okay, so syntax.
We're going to use proto3 as our syntax. So, this just tells the proto compiler what protocol we're using in this definition, right? So, this is what we're looking for. The next thing we need to do is we need to give this a name, right? So, we use kind of a goish syntax here
and we say package and we're going to call it lvm. And if I'm typing things or spelling them wrong, don't holler at me because I can't see. So, the first thing we want to define is this thing called a message, right? So, a message is the actual thing that we're going to serialize and send over the wire, right? So, anything that you want to serialize and send over the wire,
it's basically called a message. So, we're going to create a message. We're going to call it a volume. And hit capsule lock instead of... So, we're going to call it volume and we're going to give it a string called name. The next thing you'll see is this one.
One is the parameter. So, what you do is you order the parameters that you have. So, it identifies that parameter inside of the message and that's what we have there. The next thing we want is our actual create volume. That's not even close.
Hey, I could get used to this keyboard. Okay. Create volume request. So, string, we're going to give it... To do a create, we're going to need a name.
And we're going to need... If you know anything about LVM, you're going to want a volume group. And in our case, we're going to use thin provisioning. So, we're going to want a pool. We're going to want to specify a pool. And then we're going to specify a size.
Num, num, num, num. All right. Okay. So, we've got that piece.
The next thing we need to do is we need to have a request, right? So, we do a message, create volume request. I'm sorry. Yeah.
Holy cow. Off the rails. All right. So, we're going to give this another minute and then if things don't get better and people keep leaving... Not cool, by the way.
Then what I'll do is I'll just go ahead and bring up my code. And we can just kind of walk through it instead. Okay. So, there we have a response. We're just going to call it. It's just a message, right? So, the last thing we need to do
is to find the actual service, right? So, we'll do like that.
And then we want to do the same thing. I almost got it. That's not terrible.
All right. So, here's what we're going to do because we're going to run out of time. What I'm going to do instead is I'm going to pull the code because I already wrote it. And we're going to look at it. It's so easy to do if you have a non-UK keyboard when you're a stupid American
who doesn't have his laptop, their laptop. Okay. So, I'll just use magic.
Voila, right?
Okay. So, if you look back, remember our proto file that we were building. Oh, stop that.
Oh. And he fell apart.
So, that's the proto file that we were trying to build. Those are the last two lines I was trying to enter there. So, the next step that we needed was we needed a server, right? So, if you look here, I have a pretty simple server that I put together. Just standard go. We have a main.go here. What we did is we went ahead
and we defined our function for create volume. So, you can see there we take a context. All of our GRPC stuff is going to pass context back and forth. And then there's that create volume request that we had. So, what we did with that proto file is we actually ran the proto compiler, the proto C compiler, and we compiled a pb.go file. And that's what I'm referencing there,
that pb.create volume request, right? So, that's where that's coming from. So, you can see we have the create volume request, we have the create volume response. And then I just go ahead and the server is actually where LVM is installed and everything. So, I'm just executing the LVM commands on that server when it gets this command. So, that's how it's working.
One of the things that I say that's super nice about GRPC and one of the reasons why I like it so much is in terms of setting up the server and get everything wired together, this is basically it. Unless you wanted to do some more advanced things, which you can, but this will get you a basic running server
and everything will work, right? So, it's pretty simple. All right, so we've got that. And then we would need a client. And the client is even easier. We just put basically the calls into our client.
We're going to just do this directly right now. We're going to go ahead and wire in our IP address for the server. And we're going to go ahead and just use this hard-coded create volume request, okay? So, if we look here, so if we do a pseudo-LVS, so right now you can see
we've got the system partitions and we've got that thin pool for LVM thin provisioning. And that's it. So, what I'm going to do right now is I'm going to kick off this server. Okay, so now the server, the gRPC service is up and running
and listening on this server, right? So, we can go over here. I have this other box called FETI2. And what we're going to do over there is we're going to run the client. And I'm already there, I think.
Okay, so this is the same client. It's all the same code and everything else. I just copied the whole thing over here. So, what we do is we do a client. We run that. Uh-oh.
Did I not pseudo that when I started it? So, very important. Okay, so we get our message says it was created successfully. So, we'll go over here and we'll do,
and you can see we have a new LV called test. And that's the demo that I was trying to code on the fly. I have a much greater appreciation for Just for Funk now.
I'm sorry, what? Oh, yes. How's that? Is that better? So, I'm about out of time, but real quick, anybody have any questions?
Alright, thank you.