FOSDEM 2023 - Go Lightning talks: Come speak!
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Number of Parts | 542 | |
Author | ||
Contributors | ||
License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
Identifiers | 10.5446/62056 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | ||
Genre | ||
Abstract |
|
00:00
Lie groupProjective planeModule (mathematics)Type theoryLine (geometry)Traffic reportingComputer programmingCodeSoftware bugBitMultilaterationOpen sourceError messageDiagramComputer animation
03:13
Maxima and minimaSimultaneous localization and mappingSlide ruleStaff (military)Element (mathematics)Server (computing)Software engineeringComputer animation
03:37
Server (computing)InfinityMaxima and minimaVertex (graph theory)Total S.A.Task (computing)Dependent and independent variablesExtension (kinesiology)Error messageContext awarenessMatrix (mathematics)Inheritance (object-oriented programming)Electronic mailing listWeb browserView (database)SpeicherbereinigungDampingRun time (program lifecycle phase)Complete metric spaceEvent horizonSynchronizationMaizeTablet computerFunction (mathematics)Query languageBerechnungskomplexitätBefehlsprozessorCodeQuery language2 (number)BitComputational complexity theoryOnline helpGreatest elementTracing (software)Cartesian coordinate systemRun time (program lifecycle phase)Point (geometry)Graph (mathematics)System callMultiplication signQuicksortSoftwareFunctional (mathematics)Profil (magazine)DialectDatabaseAlgorithmInformationCrash (computing)Different (Kate Ryan album)Context awarenessSpeicherbereinigungWordTask (computing)Line (geometry)Link (knot theory)Server (computing)BefehlsprozessorComputer animation
06:59
Matrix (mathematics)Formal languageStrutLevel (video gaming)Software repository2 (number)LaptopDiagramComputer animation
07:24
CodeGroup actionMathematicsStrutView (database)Codierung <Programmierung>Online helpNavigationString (computer science)Hacker (term)Surjective functionCAN busElement (mathematics)MathematicsVideo gameCodeRight angleDemo (music)String (computer science)ImplementationSheaf (mathematics)NumberProduct (business)Different (Kate Ryan album)Computer animation
12:07
Mail ServerEmailComputer virusMultiplication signServer (computing)Computer animation
12:51
Software maintenanceSource codeMail ServerEmailEncryptionTransport Layer SecurityState transition systemDigital filterContent (media)Physical systemComputer fileKey (cryptography)PasswordLetterpress printingCodeSoftware testingLine (geometry)ImplementationDisintegrationFeedbackCodeEmailSoftware testingBitInternet service providerElectronic mailing listDirect numerical simulationDomain nameLatent heatInformation securityConfiguration spaceServer (computing)Open sourceRow (database)Message passingPublic key certificateComputer fileAdditionLine (geometry)Arithmetic meanINTEGRALImplementationCodeMultiplication signStandard deviationConnectivity (graph theory)Address spaceFeedbackComputer animation
16:24
Maxima and minimaDevice driverAxiom of choiceComputer programMenu (computing)Web pageWeb pageSlide ruleBackupComputer animation
17:40
FluidPersonal digital assistantKeyboard shortcutUsabilityComputeroutputService (economics)MaizeSlide ruleObject (grammar)View (database)File formatComputer fileTouchscreenZoom lensState of matterFluidBitComputer animation
18:10
Projective planeSoftware maintenanceMultiplication signBranch (computer science)Computer animation
18:34
Open sourceComputer animation
18:57
Configuration spaceLoop (music)Multiplication signType theoryProcess (computing)Goodness of fitFeedbackMathematics
20:11
FluidCodeSoftware developerRule of inferenceExtension (kinesiology)Graphics tabletSource codeOpen sourceRule of inferenceComputer animation
20:34
Wechselseitige InformationExplosionComputer virusEmailLine (geometry)Multiplication signFunctional (mathematics)Order (biology)MathematicsCodeStructural loadRight angleAuthorizationTouch typingCognitionComputer architectureComputer animation
21:50
Error messageBoolean algebraBoolean algebraFunctional (mathematics)String (computer science)Error messageElectronic signatureProgrammschleifeLine (geometry)Sign (mathematics)Computer animation
22:41
VideoconferencingSpeicherbereinigungMultiplication signBinary fileWritingRun time (program lifecycle phase)Goodness of fitComputer animationLecture/Conference
24:34
Program flowchart
Transcript: English(auto-generated)
00:05
Hello everybody, I will talk about GoEvil which is a personal project which allows you to do one-liners in Go.
00:20
So you just type your Go code and call it with GoEvil and you can simply write a Nello world from the command line. So this is like magic. I will show you a bit under the hood how it works.
00:46
The whole project is about 300 lines of code, not more. Here is another example. You can call GoEvil and tell it to take the code from the stdin.
01:09
But here is how it works under the hood. From your Go code, from the command line, GoEvil generates a full Go program.
01:25
So the "-e", allow to print that Go program that has been generated. It is sometimes useful when you want to debug the syntax errors that you make. And then the Go imports equal allow to stop using Go imports.
01:50
Because here you see that in that code there is no import of the fmt package. But it is introduced by the Go imports which is called by GoEvil.
02:05
I am announcing today that GoEvil 1.0 has been released just a few hours ago. The new feature of GoEvil 1 is that Go modules are supported.
02:23
With Go modules you get locked versioned for your dependency code from GoEvil. So this allows you to submit, to share your one-liners with other people.
02:44
Because the previous code that I showed was depending on the dependency to be installed in GoPath. And so that's it. Try it, use it, report bugs.
03:00
And I'm available for questions. Later. Thank you. It's weirdly enough not the first open source project to be released when people are in the dev room. If this is your slide you can come up now.
03:29
Hello everyone, my name is Keegan, I'm a staff software engineer at Element. And I've been spending the past year debugging why Go servers are slow. So hands up, who's made a CRUD application before?
03:41
Create, read, update, delete. That's basically everyone in this room which is what I thought. Who's tried to speed up their server before? So this is a slow request, 3.6 seconds. Fewer people but still a fair number of people. Okay, cool. Who's used pprof before? So you know, flame graphs. Okay, it's great. Who's used runtime trace before?
04:03
Not that many people. Okay. Who struggled to figure out what was going on when you're using this? Right, okay, great. This talk is for you. So the first thing you need to do really is use spans to make these traces readable. Very easy, if you've ever used Jaeger spans before, they're basically the same sort of thing.
04:23
So you can create a new task and then you get a new context, you pass the context through to new functions. You can create regions from those and you end up getting something that looks a bit like the stuff on the bottom there. You can also add log lines for some contextual information. That'll appear on the UI which we'll get to in a moment.
04:42
And the crash course in using runtime trace is you make a trace in the same way that you'd make a CPU profile with pprof except you hit a different endpoint. But you also tell it how long you want to trace for and then you use go tool trace to open that trace. You don't use the go tool pprof, confusingly. And you'll get something like the bottom over here which is quite a lot of scary words and links and you have no idea which thing to click.
05:07
The only thing you care about is the user-defined tasks. If you click on that, you'll see something a bit like this. The only thing you care about is this go routine view and if you click on that, you can profile basically everything.
05:20
So for example, here's a bit of a request which is slow because of garbage collection. And if you click on any one of those Gs at the bottom which are highlighted with the red circle, you'll see stack traces that mention GC. Also the blue bar in the middle there says GC, so no spoiler. Other thing, if you have slow SQL queries, you can find that as well because if you click on any of these things, you'll see stack traces.
05:45
And those stack traces refer to any point where the go run time kind of yields away for kind of network IO or syscalls or things like that. So you can clearly see, oh, it's doing something with SQL and it's just doing the same thing for SQL for, you know, not particularly long here, only 20 mils.
06:02
But still, it takes a long time. You can do the same thing for profiling functions if functions are being slow. So you may, this is calling the same function over and over and over again, which it probably shouldn't be doing in this particular scenario. But again, it depends on your actual code as to whether or not this is the right thing for it to do.
06:24
Sometimes that is normal behavior. In this case, it's definitely not normal behavior. So the TLDR is you should probably use runtime trace next time and not CPU profiles. So for me, I've sped up requests that were taking 3.6 seconds to 96 milliseconds for the same request.
06:42
And there are bottlenecks from various different things. So from garbage collection to poor database queries and poor computational complexity on certain algorithms. And some of these things will only be visible if you use runtime trace. So, you know, flame graphs don't help you for debugging slow SQL queries. But runtime trace will do. Thank you very much.
07:08
Thank you. If this GitHub repo is yours, come to the stage. And you've got 10 seconds to switch laptops.
07:25
And it works, which is a miracle for Linux.
07:52
Hi. I actually didn't create the slide. And this will be the fastest lightning talk in my life.
08:01
Basically, I just wanted to talk about the JSON package and the issue what we faced with. And a lot of people faced with it. Basically, it's the have you ever used struct with omit empty?
08:22
Then basically this is where the issue come in. And there is an open issue here which trying to fix this. But it's basically abandoned and it's a pretty big issue. Because there is it's created in 2015.
08:42
And there is nearly 200 comments under that. And basically I just wanted to make an attention on this ticket. Because if someone fixing this ticket, that means that basically you can do something like what I show you in this code.
09:10
Sorry, it's really hard with one hand. Yeah, probably you use this package, the encoding JSON, yeah? I have a struct here which is here.
09:24
I'll become your mic stance. Oh, thank you. I'll use two hands. Thank you. So this is basically I introduced a new struct which is basically a new string or something like that. And here I added the omit empty. In this case, I implemented the zero method here which says if it's not valid, then it's basically a zero.
09:46
So I wanted to remove it from the JSON. But if I run the actual code, please run it. Live demos in a lightning talk, you're brave.
10:03
Yes, live coding. You see that it's basically here inside the JSON. However, I wanted to basically an empty JSON. And there is another implementation with exactly the same code.
10:22
But I just created a pumpkin seed JSON which is exactly the copy of the built-in JSON. The only difference here that the issue what I mentioned is basically suggesting an implementation that the omit empty section of the built-in JSON should check for the zero method,
10:43
whether it's existing in the struct or not. And if I run this one, it's basically doing what it should do.
11:01
And basically that's it. So this is something that I think should be implemented in Go. And this ticket with this number is basically showing actual implementations for that.
11:22
Right now, most of them are not declined but not processed. So I think if anyone has a good idea how to implement it in Go, then basically it would be nice to put into this ticket. There are also, this is the actual change request in the Go language,
11:44
what the guy made, and I just copied his code. Yeah. One disclaimer that the pumpkin seed JSON package you shouldn't use in production.
12:01
And that's it. Thank you. If this is your slide, come to the stage.
12:26
All right. Hello. My name is Michiel. I created Mox. I've been working on this for quite some time. I started using it two weeks ago, released it earlier this week. It's a mail server, so I'm curious. Is anyone here running their own mail server on domain?
12:43
One, two persons? Wow, okay. Three, room for improvement. So let's go right ahead. This is the tagline. It's a modern, full-featured, open-source secure mail server for low-maintenance, self-hosted email. So let's break it down. It's modern because it supports all the latest mail standards,
13:04
and there have been added quite a few over the years. It is full-featured in the sense that it aims to do everything at once, meaning all the relevant email standards. So you just need this one thing. You don't need a whole bunch of components to make a working system, just really to make it easier.
13:20
It's MIT licensed. It is secure, meaning it supports all the latest security things about email, like TLS, et cetera, and of course a bit of secure coding. And low-maintenance, so you actually started using it because I hear many people are moving all their email to the cloud, some big providers, because it's too hard, apparently, to run a mail server.
13:42
So it's for your self-hosted email. Email is one of the oldest decentralized messaging protocols, but we're making it more centralized by moving everything to the few big providers. So mocks is an attempt to make it so easy that you will all start using it.
14:02
So a bunch of features, a list of acronyms. IMAP, so you can access your mail. SMTP, so you can send mail. Nowadays, if you want to send mail, you need to configure SPF, DKIM, DMARC. Does anyone know what that means? Yeah, I see nods. That's good. Automatic TLS, so you don't have to worry about any certificate stuff, so it's like a caddy for email.
14:22
TLS reporting, MTA STS, that's one of the latest additions to secure email. There's a reputation-based junk filter in there, so if you receive messages from people and you don't like those messages and you mark them as junk, then next time those people send mail, it's rejected. So news senders don't have any reputation.
14:41
You can look at the content, so there's a content-based BSIM spam filter. It's all in there. Internationalized email, so you can have smileys in your domain names. That's what you'll want. And auto-configuration, so you get your Thunderbird. Setup is just instant. No need to worry about all the port numbers, et cetera. It just works. So getting started, of course, now you're all convinced you want to use this.
15:01
Luckily, there's a quick start. You just set up a Linux machine, probably, get your email address for your domain, and you get a configuration file that has this all configured. You just can start it right after. Not only does it make a configuration file, it also prints some commands and all the DNS records that you need to create. So you don't have to think. You can just copy-paste and be happy.
15:23
Then the code. 40k lines of implementation, 10k lines of tests, quite some test coverage. There's integration tests, fuzzing tests. It's all pure Go, no C-Go. It's just Go install, cross-compile, all the good stuff that you get from Go. The implementation is heavily cross-referenced with the RFCs.
15:43
So both ways, you can go from code to the RFC and back from the RFC to the places in the code where it's used. So this is supposed to help with maintenance. So it's implementing all these protocols, and it gets a bit overwhelming to understand all of that. So if you would code it once, you cannot go back to the specification and back to the implementation.
16:03
You don't know what it's going to do. So how you have to fix bugs, etc. Let's move on. So what's next? I just released it. I'm looking for feedback. Please use it and tell me if it works for you or why it does not work for you. So I aim to make it very simple. So if you find something that's not simple, let me know.
16:20
And of course if you find bugs, let me know. And this is where you can find it. All right. Thank you. If this is your slide deck, you can come to the stage. If this is nobody's slide deck, I'll just skip it.
16:45
Something with postgres. If this is your 404 page which you sent to me, please also come to talk to me. So yeah, also the speaker is not found. That's the thing with last minute talks. Then I had one backup speaker. You can come to the stage.
17:02
And the gophers are also falling down. They are tired. I understand. Me too. Can I connect? Yes. I have HDMI. I also have USB-C. USB-C.
17:42
Okay. So thank you. First of all. So I want to tell a goal of story that we and why we use Go to have to implement
18:01
this idea of fluid pull requests at ReviewPad. Before starting that, I need to talk a little bit about pull requests. So bear with me. So for that, I brought Robin and Kat with me. So Robin wants to contribute to a project that Kat is a maintainer. And what Rob does is what everyone does or at least they try to.
18:21
They open a branch. They create what they have to do. And then at the end, it comes a time where it needs to merge into main. And then when Kat comes in and says, wait a minute. We need to review those changes. So this kind of methodology is important for critical contributions from interested parties. And it's well known as open source project, especially with the name of pull requests.
18:44
We also use it inside our own companies. But it's well known at the open source community. And it's quite popular. As you can see, in 2021, we got a lot of pull requests being merged. And the process goes like you do whatever you want to do.
19:00
Then the CI triggers. You get the review. You get some feedback. And then you have to apply the feedback. And we enter a loop here until someone decides that it's good to go. And we get our approval. Then it goes to merge and that everyone is happy. And the problem here is that Robin goes through this process every, every, every, every time. Regardless of the type of change it is.
19:23
And we are unaware of the fact that Robin and Kat have been contributing and working with each other for some time. So, this idea that all pull requests are the same is, can we actually be improved? For instance, this scenario where Robin is just trying to do some configuration change.
19:40
Why do we need a pull request? Maybe we can just go directly to main without a review. Another scenario where Robin just go and duplicate an API with some documentation or some warnings. Let's imagine, why can it go to main? And then we can do a review afterwards. And then when it comes to critical changes, then when we want to stop the process and say, okay, this is critical.
20:04
We need to have a very good review here. And maybe instead of just asking one guy, we can ask two people for them to get their own approval. So, this idea of full pull requests is that all that I just said could be the final rules. And we can apply those rules into our own process and minimize the time.
20:23
That's where we came with the review pad, which is done on Go and it's full open source. And that's where we can define all these ideas of what are the rules for our team. So, here's how we could work with this terminology.
20:41
Behind this is Go, of course. Then it can, for instance, if my changes are all on markdown files, I want to merge my pull request right away. So, no review. If, for instance, my author actually is considered a new joiner, a new joiner could be someone that didn't do 10 PRs like Spotify does.
21:01
I want to assign a reviewer from my tech leads. And then, for instance, if I want to get some compliance, make sure that my pull request has an issue. I can confirm that and make sure that the user gets notified as soon as possible in order to iterate on that. And then we can do some more incredible things.
21:23
I want you to look at the line at the top where we have an annotation saying that it's critical, saying that every time someone changes that function, that function is critical. If the function is critical, if my code touches a function that has this annotation, then I want to trigger my pull request review that is for critical changes.
21:42
I want to assign a label, I want someone from the tech leads to review it, and I want to notify John, which is the tech architecture. We had a talk this morning about reducing cognitive load from Frederick, and I want to show how we could do that with this terminology. So here's how we could look into line of sign and make sure that if someone uses a lot of tabs,
22:06
so it means that we have a lot of loops between each other, or if and else, we can actually send a warning to the user. For instance, our error validation, making sure that they don't use string contains for errors or equals, but they use error is.
22:21
And last one, the mysterious Boolean, making sure that no more than one Boolean is used in the function signature. Yeah, that's pretty much it, how we could use to make our lives easier on pull requests. Thank you all. Thank you. The last lightning talk of the day is from me, again.
22:47
What do we want to talk about today? Well, two subjects. What is naming? No. I want to talk to you, first of all, a big thank you, again, to everyone. First of all, to all speakers who came here today to give an amazing talk, to stand here with a lot of stress, to say things.
23:04
I also want to thank Eva again for helping me out. I also want to thank the two FOSDEM engineers in the back who made our audio video work all day. I want to thank the people from FOSDEM who brought me food today.
23:25
I also want to thank everybody at FOSDEM. And I also want to thank all the volunteers, I think they all left right now, who helped us with... Oh, no, still one left. Still one left. Good. They're writing. Who helped us with doors, helped us with video, even what they couldn't solve today.
23:42
Thank you very much. Thank you all for coming, by the way. Thank you for staying so late. Thank you. And now my second subject, which is that Go is a garbage pocket language. And you know you can trigger the garbage collection by doing runtime.gc.
24:02
So when the time is 19 o'clock, I want you all to do runtime.gc and grab some waste you see around it and put it in any of our bins. But I think Eva wants to say something. You thanked everyone that has been here to help you, but without you this wasn't possible.
24:20
So a big thank you to Marja and thank you for coming. Thank you.