Is it safe to use unsafe?
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Untertitel |
| |
| Serientitel | ||
| Anzahl der Teile | 8 | |
| Autor | ||
| Lizenz | CC-Namensnennung 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/52206 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
Rusty Days 20205 / 8
2
5
6
00:00
Virtuelle RealitätMailing-ListeCodeInvarianteCompilerInterpretiererInformationsspeicherungGruppenoperationAnalysisAutomatische IndexierungProdukt <Mathematik>Projektive EbeneMultiplikationsoperatorNP-hartes ProblemSoftwareYouTubeTypentheorieVideokonferenzGüte der AnpassungObjekt <Kategorie>SoftwareentwicklerProgrammierungSoftware EngineeringProgrammiergerätStichprobenumfangTeilbarkeitWasserdampftafelWort <Informatik>Rechter WinkelOpen SourceDatenverwaltungProzess <Informatik>FlächentheorieArithmetisches MittelOrtsoperatorVektorraumVerschlingungCASE <Informatik>Streaming <Kommunikationstechnik>MereologieBitSystemzusammenbruchExogene Variablep-BlockTouchscreenSystemaufrufKonfiguration <Informatik>ZahlensystemPhysikalisches SystemBenutzerbeteiligungDifferenteFormale SpracheHMS <Fertigung>EreignishorizontComputeranimation
09:53
QuellcodeStichprobeCodeMailing-Listep-BlockMaßerweiterungWurzel <Mathematik>Physikalisches SystemAttributierte GrammatikSystemaufrufWrapper <Programmierung>BildschirmsymbolKryptologieFunktion <Mathematik>Hash-AlgorithmusÄhnlichkeitsgeometrieMini-DiscLokales MinimumBefehlsprozessorRuhmasseParallele SchnittstelleProjektive EbeneBrowserDateiverwaltungNetzbetriebssystemZeiger <Informatik>Hash-AlgorithmusCodeFahne <Mathematik>Wurzel <Mathematik>Konfiguration <Informatik>FunktionalPhysikalisches SystemTypentheorieZeichenketteProgrammiergerätMailing-ListeAttributierte GrammatikDokumentenserverRechter WinkelElement <Gruppentheorie>MereologieVektorraumGanze ZahlCompilerKryptologieBefehlsprozessorStrategisches SpielBitCliquenweiteMultiplikationsoperatorp-BlockAutorisierungKomplex <Algebra>Message-PassingSichtenkonzeptPunktFeuchteleitungFamilie <Mathematik>Dreiecksfreier GraphLokales MinimumBimodulSystemprogrammMaßerweiterungCASE <Informatik>ServerSystemaufrufGeradeElektronische PublikationVIC 20MomentenproblemComputeranimation
19:46
Mailing-ListeVirtuelle RealitätLambda-KalkülDienst <Informatik>Graphische BenutzeroberflächeCodeStandardabweichungDesintegration <Mathematik>Turing-TestSoftwaretestRechenwerkMittelwertInformationInterface <Schaltung>Formale SpracheAxonometrieLaufzeitfehlerDualitätstheorieLucas-ZahlenreiheROM <Informatik>Ebenep-BlockEin-AusgabeCodePhysikalisches SystemMusterspracheProjektive EbeneProgrammbibliothekFunktionalGebäude <Mathematik>BildschirmfensterTypentheorieDickeSystemaufrufBitSpeicherabzugLaufzeitfehlerProgrammiergerätPhysikalische TheorieVolumenvisualisierungFormale SpracheImplementierungYouTubeZahlenbereichMereologieElement <Gruppentheorie>VersionsverwaltungKonfiguration <Informatik>PunktZeiger <Informatik>StandardabweichungSchreib-Lese-KopfNichtlinearer OperatorProgrammfehlerSoftwarewartungCompilerMonade <Mathematik>Ganze ZahlMathematische LogikDifferenteElektronische PublikationInterface <Schaltung>Generator <Informatik>DatenverwaltungStrategisches SpielGraphische BenutzeroberflächeNetzbetriebssystemp-BlockKomponententestKernel <Informatik>Wrapper <Programmierung>VorhersagbarkeitTouchscreenOpen SourceMultiplikationsoperatorKontinuierliche IntegrationKlasse <Mathematik>HalbleiterspeicherKanalkapazitätFormation <Mathematik>TaskObjekt <Kategorie>GarbentheorieZweiMaschinencodeEntscheidungstheorieSelbst organisierendes SystemVorzeichen <Mathematik>BitrateHilfesystemMessage-PassingSprachsyntheseLambda-KalkülGüte der AnpassungVerschlingungQuick-SortComputeranimation
29:39
CodeMailing-ListeBetriebsmittelverwaltungROM <Informatik>Minkowski-MetrikZeiger <Informatik>DatenbankGraphDefinite-Clause-GrammarInterface <Schaltung>Physikalisches SystemInformationsspeicherungVariableDickeGanze ZahlFunktion <Mathematik>IndexberechnungZeichenketteGoogolKernel <Informatik>IdentitätsverwaltungSoftwareentwicklerInvariantep-BlockHilfesystemCASE <Informatik>Exogene VariableProgrammiergerätPunktProjektive EbeneEigentliche AbbildungRechter WinkelInformationsspeicherungWrapper <Programmierung>Konstruktor <Informatik>DickeKollaboration <Informatik>p-BlockFunktionalLesen <Datenverarbeitung>HalbleiterspeicherStrategisches SpielZeiger <Informatik>GarbentheorieProgrammierungAutomatische IndexierungFormale SpracheMultiplikationsoperatorRandwertGruppenoperationImplementierungSpeicherabzugStandardabweichungQuick-SortGoogolKernel <Informatik>SoftwareNetzbetriebssystemProgrammdokumentationGanze ZahlQuellcodeInvarianteMini-DiscSpeicherverwaltungBitrateVektorraumMinkowski-MetrikCodecBimodulCodeVIC 20NummernsystemPhysikalisches SystemSchreiben <Datenverarbeitung>Zusammenhängender GraphInterface <Schaltung>DatenbankGraphVideokonferenzGenerator <Informatik>Mandelbrot-MengeYouTubePrinzip der gleichmäßigen BeschränktheitInteraktives FernsehenBitRechenschieberCompilerSelbst organisierendes SystemWendepunktGebäude <Mathematik>InternetworkingVerschlingungFraktalgeometrieAutomat <Automatentheorie>ResultanteÄhnlichkeitsgeometrieEreignishorizontKondition <Mathematik>ExpertensystemWeb-SeiteSichtenkonzeptVerschiebungsoperatorLeistung <Physik>Computeranimation
39:32
InvarianteCodeMailing-ListeVirtuelle RealitätFunktion <Mathematik>BimodulAggregatzustandWrapper <Programmierung>InformationAliasingROM <Informatik>TypentheorieNotepad-ComputerDisjunktion <Logik>Modul <Datentyp>GruppenkeimGammafunktionHalbleiterspeicherYouTubeTypentheorieVerschlingungStreaming <Kommunikationstechnik>GruppenoperationRechenschieberMomentenproblemPatch <Software>ProgrammierspracheMereologieSpeicherbereinigungSelbst organisierendes SystemProjektive EbeneFormale SpracheFunktionalRenderingRechter WinkelFormation <Mathematik>ProgrammiergerätZweiCompilerDruckspannungZeiger <Informatik>Strategisches SpielGrundraumWort <Informatik>Zellularer AutomatGarbentheorieCodeAliasingMinimumCASE <Informatik>ExpertensystemTouchscreenDickeMultiplikationDokumentenserverAutorisierungLokales MinimumEinfach zusammenhängender RaumMusterspracheElektronischer Programmführerp-BlockWrapper <Programmierung>FlächeninhaltBimodulDeskriptive StatistikGoogolFunktion <Mathematik>BitSichtenkonzeptComputeranimation
49:25
Mailing-ListeVirtuelle RealitätCodeFlächeninhaltDomain <Netzwerk>FunktionalSummengleichungSoftwareentwicklerCMM <Software Engineering>Produkt <Mathematik>GeradePunktZeiger <Informatik>MultiplikationsoperatorMathematikHidden-Markov-ModellEinsStrategisches SpielLokales MinimumMAPProjektive EbeneDickeFormale SpracheSchreiben <Datenverarbeitung>DokumentenserverHalbleiterspeicherCASE <Informatik>Eigentliche AbbildungPhysikalisches SystemSoftwareProgrammierungMaßerweiterungProgrammiergerätProgrammbibliothekGüte der AnpassungProzess <Informatik>Ein-AusgabeSoftwareschwachstelleQuick-SortTabellePufferüberlaufCompilerYouTubeLineare RegressionVersionsverwaltungProgrammierspracheZweiPropagatorKonstanteHumanoider RoboterWissenschaftliches RechnenBAYESElektronische UnterschriftNeuroinformatikWrapper <Programmierung>StandardabweichungVerschlingungSkalarproduktRechter WinkelFormation <Mathematik>PerspektiveSpeicherbereichsnetzwerkComputeranimation
59:18
BefehlsprozessorMailing-ListeFormale SpracheSchnelltasteVirtuelle RealitätNeuronales NetzProgrammiergerätWeb-SeiteSichtenkonzeptPunktKondition <Mathematik>CompilerVerschiebungsoperatorZeiger <Informatik>VerschlingungGruppenoperationBitSelbst organisierendes SystemYouTubePrinzip der gleichmäßigen BeschränktheitCodeGanze ZahlSoftwarewartungMathematische LogikTypentheoriePhysikalische TheorieSchreib-Lese-KopfMultiplikationsoperatorPhysikalisches SystemProjektive EbeneOpen SourceRohdatenProgrammierspracheMereologieFormale SpracheSystemprogrammierungEreignishorizontZweiMaschinencodeEntscheidungstheorieComputeranimation
01:08:08
Virtuelle RealitätMailing-ListeEreignishorizontSchlussregelInternetworkingÄhnlichkeitsgeometrieZellularer AutomatSchlussregelYouTubeResultanteKonstruktor <Informatik>FraktalgeometrieVideokonferenzGenerator <Informatik>Mandelbrot-MengeComputeranimationProgramm/Quellcode
01:09:37
Rechter WinkelPrinzip der gleichmäßigen BeschränktheitComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:04
Hello, good morning. Good evening. Good afternoon. I'm speaking to you from Before dawn in New Zealand, and I am absolutely delighted to be part of rusty days I think it's fantastic that we've been able to take
00:20
although the Rust also team has been able to take what it has been quite a negative pandemic and turn it into a global event, which is which is perfect I I should begin so Just to reiterate I'm sure this will come through on the stream more than happy to take questions by any channel by a twitch or YouTube
00:45
or I think there's a third channel also and just ask questions online and I will try to Answer them as we go. The team is actually sitting behind me Monitoring all of those streams. So that's great
01:01
Okay, we should we go we'll start with Start with moving something Why is my screen not there we are introduction Who am I I tweet about rust on some clicks. I Spent a whole bunch of time wasting time
01:21
About rust on reddit. I do live coding on twitch. I make videos on YouTube. I write books So I've actually written this thing called rust in action one of the reasons why I do that is that I've kind of taken it upon myself to shorten everyone's learning journey by 100 hours
01:41
Now if you have just decided to come to this talk because you want to know how to apply unsafe to your own project I've decided to Just give you the answer We'll give you the answer straight away so you don't have to watch a whole hour of talking Firstly at the top of your crate add an annotation which is deny unsafe code
02:06
This will as will demonstrate in some other projects. This will prevent the compiler from allowing you to Use unsafe unless you have been very very explicit and toll opted in
02:21
The other one is that before or juror within an unsafe block You need to explain why it is that this code is safe. The compiler is no longer working for you And so you need to do the work of the compiler for any future rust program including yourself
02:42
Who might come along and wonder why on earth is this safe? and So one one way which I would recommend When you're during the code review is To ask them. Do you understand? Can you explain to me why that is safe? And if they cannot then either the code needs to change or the comment needs to change
03:08
our objective as rust programmers or One of the objectives let's say well the objective right now is safety And let's all remember while we're going through this process That other people make mistakes, right? We we never make mistakes, but other people do
03:25
So how do we prevent their mistakes from infecting our code? We need to create a System of software engineering that makes it extremely hard for stressed overworked and maybe Distracted individuals to do the wrong thing. We need to create the system as team leads as
03:47
Engineering managers as junior developers, we need to participate In a way that that gets the learns our objection so we want to learn about how the Projects are managing risk, but first I just I would really and I'd like to take the time
04:06
to talk about lemons and limes Now if you speak English as a second language, this may sound very strange but This is the story about how the British Navy or Britain in particular
04:23
Understood in the 18th century how to cure scurvy and by the 20th century They had completely forgot it In fact, the scientific advice at the start of the 20th century was so bad that it caused scurvy in the Antarctic and Expeditions quite famously and led to some
04:43
A horrendous tragedy and so there were many reasons why this occurred and one of the main reasons though, is that the thing that well a Contributing factor was that the English word lime? Included lemons also at the time in which the scientific and
05:02
At the time at which the cure was found which was used just drink lemon juice You know sprinkle some lemons into water and just drink that and so I Want to reiterate that if your code comments cannot be understood by your audience That is then they need to change and it isn't the the surface. It isn't the words themselves that are important
05:26
It's the meaning behind that people who are reading your code need to understand why it is that The code that you have written is safe You need to do the work of the compiler for it
05:44
Just a warning This allowing unsafe is actually insufficient to guarantee safety Unfortunately rust still has some problem. Well, no, it doesn't have problems. It just says something to be aware of It is actually possible to generate code that is guaranteed to crash your program
06:07
using only only safe code Now this is a ridiculous code example No one is ever going to wrap a vector of type T with another container But maybe you're doing something stupid
06:22
I say stupid Maybe you're doing something And you don't realize that you've created the situation where two of your primitive types may be inside a struct Actually are an invariant on the other one. So that is the position This position variable here
06:41
Relies on the fact like there's an intimate relationship between storage and position that The compiler cannot guarantee and so by mistake lives in this case is intentional but potential let's say by mistake I've written some code that breaks the link between two of the primitive types in
07:03
In this case I can see it position to something that is unreachable and then when the next court like if someone calls get this will Break and it will crash the entire program now, obviously I could have there are ways to to get around this I could have replaced
07:23
This index notation with the get method and that would return an option But this is completely safe code and it is 100% guaranteed crash So Just before we get to the projects I also thought I should explain a little bit around my methodology if there if there was any methodology. I
07:44
Firstly wanted to talk a little bit about the rationale about why did this I was really disappointed with the rust communities response to the ethics web unsafe Like basically driving if you've been around the rust community a little bit you'll be familiar with this, but if you're very new
08:02
One very famous example of unsafe or the use of unsafe code or Was this project called actics web where the developer happily used unsafe blocks? And then when people said look you can't do this it's you know, there's no there's no requirement
08:23
Eventually that person was driven out of the of the rust community Now because of this kind of cultural difference and And that made me think like well how like what is the right way to do this if that is the wrong way I think I think it was unfair to drive that person away
08:44
and The two main aims I think are to understand what it is that Professional companies people that are paid lots of money to write very good software and doing very hard things with rust Like what do they do? And I would also like to I just wanted to do some research to justify doing more research
09:05
Actually, I've got several ideas about how to extend this So, what did I do Sorry just to pause slightly doing qualitative research not quantitative what that means is I'm I'm not looking for a good. I haven't hacked the compiler to be able to do an analysis
09:27
This is quite I've used a lot of interpretation here. So it's it's it's it's not an analysis of every rust crate It's kind of looking very closely at team
09:40
And they were basically a sample of products or projects that I thought was interesting I tried to create a sample of open source From the GNOME project all the way through to like Amazon and Microsoft these big big companies as well and I was looking primarily at their documentation for new contributors and then I would go into look at their code snippets
10:06
via like Searching for unsafe inside their repository now, I intentionally did not communicate with any of these projects Because I That I was doing this. So none of these projects are aware that actually they have been part of this this project
10:26
So let's let's have a look server Servo is rust foundational project. It was kind of the reason why it was created to create a parallel web browser It's fascinating looking through the github because a lot of the documentation around
10:45
How to write code is written in about 2013 so it's quite old from us point of view One thing I found fascinating is that they include this annotation of their crates Which is also the annotation that he uses deny
11:01
but at the end the start of any Function that uses unsafe within it they or any module that requires unsafe They are required to allow it and opt-in so this I think is a really nice strategy for increasing the psychological barrier to including unsafe
11:22
Now just a provider of this is a again another convoluted example but this is a demonstration of what happens inside servo so you can imagine that they have modules and The module itself has this annotation that we deny unsafe The Deny
11:41
attribute allows programmers to later on Annotate internal things with allow so you can basically you're opting out of unsafe here But you can opt back in if you really need it the reason why I think this is quite a an interesting strategy is that I
12:00
Just feel like it's harder to and it it's harder to do mentally and it would there's no way it would pass code of view I think unsafe would It's unlikely to pass code of view, but this ugly annotation syntax. There's no way that would get through I was also curious as to what cargo Geiger does internally as well
12:23
so cargo Geiger is a cargo extension, which actually Inspects your own code and all of the code of your dependencies for usages of unsafe But how do they do it themselves like this for me was really really interesting They've gone further than deny unsafe
12:41
They've actually said forbid now the forbid keyword or the forbid attribute does not allow you to annotate internal methods as Allow unsafe so it just entails the entire the compiler that it's completely illegal
13:03
Future programmers in the project will only be able to include unsafe blocks if Somehow the project decides to like remove this annotation from the road of their crate So effectively the way this looks in code is We've we add forbid and then it's impossible to like the only way to compile this dangerous function is
13:28
By commenting it out like there we cannot we cannot opt in to allow it will not the compiler Will refuse to compile the code
13:40
Again kind of looking into some of Rust's Ecosystem well some of the long-standing utilities. I wanted to kind of get a sense as to whether or not The the cultural Wonder I wanted to know if the unsafe usage had changed so XA is a replacement for the LS command, which is a Unix utility and
14:07
It it's one of rust the rust community's oldest Command line utilities that is in public use So and XA talks to a file system, and it does that via system calls
14:23
It doesn't need many it uses it does not use much Unsafe at all, but for extended attributes inside it requires this The list extra attribute sickle family on Linux and Mac OS So the strategy that they have developed is to employ to only wrap the minimum of
14:48
What they need so basically every single function that they wish to call now I'll explain this syntax very quickly in just a moment but effectively this is the rust code and
15:01
all they are doing is Wrapping the C like wrapping the C function and So the strategy there is to put unsafe around the smallest element possible and the idea is I assume to make it very very understandable about what is the purpose of unsafe and
15:24
In this case we are The reason why we need unsafe is because the rust compiler cannot reason about what happens inside the operating system and So it just requires you it We just need to expect if the operating system is going to be well behaved now
15:44
going back to the To that comment around we need to understand why this is safe If you haven't used pointer syntax, this is probably confusing I want to
16:03
The first of all, we have a whole bunch of types that are not you really used in traditional rust code But if you're using libc, if you're using any FFI, you've probably seen that before And if you've programmed in C, this probably makes some sense We take a path
16:22
so we did the A Pointer to effectively and and rust syntax. This would be a VIC of u8 We've got a reference to C string a C string is like a VIC of u8 with a null byte at the end and Then we're creating a null pointer which in the convention of C programmers is to spend in null pointer
16:44
Kind of how a rust programmer would use an option Zero, I'm not sure what that does. So that is I think size And and we have a an integer being used as flags and
17:02
The way that that works is that in C a convention is that each bit represents an on off switch and And so that's what that's going to be used. So bear in mind. It's important to think about whether or not your team is Familiar with this kind of code if it is maybe it doesn't need comments
17:22
But if it is like if you have team members or you might have new team members that are less familiar with this type of syntax be verbose So that this is where it came from Another example that I think is quite interesting is like three so blank three is this new cryptographic hash function
17:42
That's supposed to be Really fast and also very good. Oh I think I've just received a oh, I haven't received a comment Oh If you're watching this live Do ask questions More than happy to receive them as we're going through you have stopped using MD 5 just just just just
18:09
Asking that question, right? It's no longer It's no longer best practice. So why is unsafe needed in this project? Well blank three wants to make use of very high performance
18:21
functionality within the CPU and that requires and Intrinsics access to intrinsics we have Victor instructions we can operate on more than one Element at a time so they have used their minimal breath of strategy as well Now this is a
18:41
Kind of a like us like a Victor why she was closer to an array That has a width of 256 bits of integers and we call this kind of crazy thing in the middle This is what the unsafe. This is the function that's provided by And if you've used making no sense
19:03
This is the function that is provided by the compiler when you opt into intrinsics Your And from the Rust code
19:21
We Only see it a lot of the complexity is hidden from us and I think this is a really a really interesting strategy Now one thing that the authors of the crate have decided to spend extra attention on is that when we use pointers in
19:42
Unsafe blocks that's especially dangerous and we need to be especially careful and so They've made extra effort to Annotate those sections with code comments now
20:02
Get into a project from Amazon and this is the foundation for the AWS lambda and AWS faget projects and It's a so this fist will start with a question like why do they need unsafe? Well, they're interacting with a hypervisor They're basically building an operating system manager
20:23
and They also use a lot of Chrome OS inside their own project. And so they've actually got a lot of code generation And so this strategy and I don't mean to blame them. It was just I thought I'm in it, you know They've just got like it's automatic code that includes unsafe
20:42
And so there's just this comment there saying this is automatically generated. And so that's their strategy Or a lot of for a lot of their uses just just a comment saying I Don't know why this is safe. So well The I wanted to call it out there contributing markdown file. Sorry. It hasn't rendered very nicely on the screen
21:08
The point I want to make here is that they have a big document Which explains why you should or how to contribute but it doesn't address unsafe at all It talks about code comments and and and so forth and pull requests and unit tests and and that's all great
21:25
But this is an operating system project and they don't mention unsafe in any of their guidelines Windows oh, sorry Microsoft is developing a language prediction Otherwise known as a rust interface for the windows runtime, which I think is ridiculously exciting like it's amazing to see that rusts
21:48
Usage across operating systems is first class. So thanks to Microsoft Obviously they speak to Windows API's whether or not that is Like kernel 32 DLL or whichever interface that they use
22:03
They need to trust the operating system and so they have to use unsafe so their strategy again are these minimal wrappers Done slightly differently so previous so this is a bit of confusing method to see What we're dealing with here is an array of type T
22:24
Which happens to be generated by the windows runtime? sorry It allows you to create so what we're trying to do in this simple block is Create methods that allow you to create Objects that the windows runtime understands
22:45
Now The there are several other methods as well but I want to call out with Lynn because I thought it was this is where we have unsafe and so The
23:01
So this is a create we creating a new object here So this is kind of like the width capacity only it's a bit with capacity But here we're creating an array with the with a length of guaranteed to be let's say 1024 or something We want to make sure first there's a session which is good practice and then
23:23
We're saying that this is this code task name a lock is the call that we need to use which is fine But I am personally I Mean if you are a Windows like a Microsoft programmer
23:41
You probably understand what this means like innately But it feels like to me as someone who's looking at the code fresh that this unsafe block is doing quite a lot of work Inside it and So what we're doing is we are calling this part of the
24:01
Windows API And we've got our link So that's the number of elements in our array and then we multiply that by the size of memory of our type and so, you know, you can kind of see why that's one function and then this returns something and then we coerce it to a
24:20
Pointer to T. So maybe if you are familiar with systems programming this comes very naturally to you but I Think that if something were to happen inside one of my projects I would have expected that I would have provide some explanation about what is happening But again, maybe if you are developing systems programming
24:40
Like if you're developing operating systems, this stuff is so natural that you don't need to I'm not sure I Do know as well that they provide annotations as to describe why it is that The Operation is safe and so we're actually writing to the pointer
25:02
to Starting at zero I assume so this is a point data is a as a pointer So we just created up there. We I assume it's we start at zero and we write length I'm wondering and I don't know enough about About this but I assume the difference interestingly we've got length multiplied by system mean like this the but here I'm using Ling
25:25
And I I wonder whether or not that's a bug. I'm not sure I'd be interested to hear from anyone Can I enlarge the code? Yes, it's okay Hi read me mark down. So what I really like is
25:45
The windows The windows of sort of the Microsoft team Believes that they want to create this runtime within 100% pure safe rust But they make this call out that sometimes that they have to go and talk to there to the to the API's and they are implemented in C++ and
26:03
And They make safety as like a first-class citizen in their readme, which I think is a really positive sign now a Project from the gnome project sub from a project within going on is there the rewriting Libby's be I think live a
26:28
C library in rust so this is a CSV renderer and the question becomes why do they need unsafe? Well This crate lib rust RSV G speech
26:41
talks to g-lib g-lib is a kind of a core of Of gnome in some sense and it is like another C library So again, you can see this pattern of requiring we're building out new functionality in rust and we need to rely on
27:04
Languages And also rust itself. Sorry, the rust code itself exposes the same C API as the library that preceded it One thing I think is really positive in this project is that the entire culture is Focused on questioning whether or not unsafe is a valid thing to do
27:26
Now this is a code comment from One of their code reviews inside a merge request which is get labs some version of pull requests and you know the project lead there makes this
27:41
Really good point, which is why are you using an unsafe construct because I think that we could use a safer option instead Now I found this inside their code after like looking through their commits and I think it's really positive to see that people have got in their heads this idea that
28:03
If we when we can avoid unsafe we should do so Because the compiler is the compiler does not get tired The compiler does not get distracted and the compiler Can have bugs. I'm sure that there are compiler bugs
28:21
but Rusts safe rust is as good rust We've seen a minimal rapid strategy before so I'll just pass on that one. I also wanted to look at the rust standard library They have provided explicit
28:40
advice for code reviews They require comments around Each use of unsafe and they have some tooling a lent and their continuous integration builds That checks that there is a comment
29:01
Obviously the lint probably can't read Code comments to check that they relate to the code block, but at least they're there And there are humans checking these things as well. So I Like the last sentence. We're unsafe code actually needs to be okay Like it don't put unsafe and code in the air that is actually unsafe
29:24
Like that is not a good thing to put inside the standard library So Another thing that they call out inside their their guidelines is that It's okay to ask for help so if you
29:41
So we we mentioned it we see a mention here of the unsafe code guidelines working group and I'll mention what they What that is very very soon properly But did there are experts with inside the rust Language that know a lot about rust and so if you are unsure
30:04
It's okay to ask them for help. In fact, they've made this point that Everyone loves Debating whether or not there is an unsound problem here like it whether or not so, you know Don't be worried, you know, you're actually making someone's day by
30:24
Being able to get some some reasoning here and some expertise Now if you are a tiny project, this obviously does not apply as equally to you. So you know, we don't have a large team of
30:41
Collaborators and colleagues to call upon and so My advice there would be to Be cautious and to kind of build yourself up into things that you know rather than Things that you don't and don't try to use constructs that you don't understand as long as I'm saying there
31:05
One thing I think is really really like is that The public documentation explains why things may panic and if you read the code comments of standard pointer read
31:22
You'll see that there is invariants that are described which is that the source of where you're reading from must be valid for reads and so and by the way, you must initialize their value before you actually try and read from a pointer and
31:40
so without those They the method is unsafe and So Even if size T has size zero the point pointer must be non now, which is interesting, but So I think that including if you have an unsafe method or unsafe function including a safety
32:04
Section as a very sound strategy a Further project which is less less well known as this thing called toolshed toolshed is a memory allocator It uses an arena strategy or provides an arena for you which is basically to the operating system
32:25
It looks like you've just asked for a large chunk of memory And inside your program you can divide that up internally. However, you want It's typically faster, but can be slightly less efficient. You might get some wasted space
32:41
Now why do they need unsafe Well, if you're dealing with memory blocks you probably need to deal with pointers and that means that you need unsafe and So this strategy has been to push every usage of unsafe into one specific module Which happens to be arena dot RS?
33:01
Even though the API is completely safe all of the unsafety is is Isolated with inside one module and so people know if they're touching that module like it's dangerous and Again, like we're trying to
33:20
Create a situation where as a team we are building safe software and Mentally, we're switched on when we go and interact with that module so again, here is a Another project which is far less well known which I think is really interesting as a new database that one new graph database actually being developed out of Ireland and
33:44
Inside it it's the storage engine is written rust but the reasoning engine is written in prologue Now they need unsafe because they're interfacing with this prologue implementation and Now have they done it they have gone further than isolating at the module boundary in some sense
34:04
They've actually created a third crate. So they have which provides the sort of a wrapper or an interface as a Completely outside of their project or their core storage project so the storage engine the written and rust that deals with
34:22
keeping data and persisting it to disk has no and Has no knowledge of the prologue implementation It's completely independent and so if there was a problem with inside that
34:40
Kind of rapid rate It wouldn't in fact this the storage engine They also have a very strong straight like a very strong commenting practice live inside the team and so almost You know seen on the slide. It's a bit difficult. So
35:04
Allow me to explain what's happening here, so Inside so inside their code When they store data to disk or Unsigned integers to disk they compress it using a variable length encoding scheme
35:24
There's a public method in codec that Allocate some memory. So we create a vector of size of size length and And this and then we call an internal method really which is encode unchecked
35:44
Even at this wrapper Like even at the This isn't really unsafe. We're just calling an unsafe function or an unsafe method There's already a safety block saying we know that we have created our Vic
36:02
with the required length and so therefore it is safe and inside the The unchecked method the encode unchecked there's even more commenting. So this is the internal method
36:22
Basically everything that happens inside this method is well indexed Possibly, you know, we don't necessarily need to say that about incrementing but what I think that this project gets right is that
36:40
They want to assume that people are looking at this code with very very blurry tired eyes and They're even specifying why we're using an integer sorry and an index integer, which I think I've never seen before and
37:02
This makes it very very difficult to Not be it makes it very difficult It makes it very easy to understand what the code is doing and very hard to to get it wrong in my opinion Okay, we're coming up to one of the final projects that I want to touch on and this is fuchsia from Google
37:30
So the fuchsia kernel is actually written It's not written rust rust is using as used and kind of systems components But the kernel I believe is written in C++, but might be pure C their team
37:49
their belief they have got code documentation that Makes it very clear that if you're writing adding unsafe to the code you need to
38:00
You need to Ensure that it's safe. It's your responsibility and It's essential that you identify any assumptions that are required by every unsafe block You need to ensure that those assumptions are actually meet not just that you've identified them but actually with it in this case that they've been met and
38:24
over time It's possible for those assumptions to continue to be met now. This is a really interesting and Really interesting bullet point to add because it means that the programmer today Is responsible for
38:41
Thinking about how people might use this in the future and This becomes even more explicit soon a further. Oh, right. I just added the class. Oh Well, I
39:01
Just Yeah, we'll carry on. So one of the things that is Very clear by now is that? Projects that do this. Well use unsafe properly. I said probably and comments to
39:22
the unsafe blocks to explain why it is safe and And if you're inside the future project, you also require a comment explaining, you know, what assumptions are being made like previously we saw that we made an assumption that the length being provided to a pointer read or pointer write was of sufficient length and so
39:44
That seems relatively clear but This documentation is actually available in public for every contributor to the project Oh, sorry about the rendering of this I'm sorry
40:07
We're possible so This is a very good explanation for the minimal rappers strategy. So We're possible it package any unsafety into a single functional module and
40:23
Then document what it is that needs to be the case before it becomes safe how things fail and How things what happens if everything succeeds Now this Seems like a very very sound strategy to me the Google this the future always project
40:44
Has the most robust guidelines in my opinion for writing unsafe blocks of code safely partially for this reason I
41:02
You know, I apologize for how this is rendered on the screen so there's a couple of sections here so There are three particular types That the documentation calls out as being particularly dangerous Starconced star not or unsafe cell. So these are pointer types there. I are does they are
41:31
Specifically called out as things that need to be very heavily documented and the comment on the Bottom is talking about memory aliasing and that
41:46
You Can either in rust you can you can either alias so that means you can have to shed two references It's got a read-only references. So this is the ampersand syntax you can have multiple of those you can have multiple readers, but you can only have a single writer and
42:06
You need to explain that if you have been able to if you've used one of these unsafe types That you have uphold that guarantee that is provided by the rust compiler Obviously, so
42:22
Now these resources you can't click on the links But you can definitely look them up and I will ensure that links are provided for anyone that would like them and Just thinking about the way to get them there
42:42
But Like to call out the future OS team Brian Anderson Ralph Young has provided two fantastic articles in particular was the person who created That example of having two types that are both safe and of themselves But when they relate to each other and you break that connection you can create unsafety inside safe rust
43:07
There's a really nice guide of rust patterns with inside the rust unofficial repository that talks about Condensing or containing or isolating unsafety within small modules or
43:21
and lastly the unsafe working groups unsafe code guidelines reference Which I want to Which I would like to See if I can bring up now So Where are we
43:43
inside the rustling org is a Unsafe code guidelines repository and you may notice this This author Ralph Young is the same person who wrote those two fantastic articles about unsafety and rust
44:03
Now This is a very good description of The things that you need to be aware of if you are writing Unsafe code and you want to be able to it isn't quite as mature as I expected it to be
44:24
But I think it is developing and if this is an area that you are interested in I believe that you should continue to I think you should participate So it's the main output of the group is a
44:40
Is a reference document And I'm sure that this is I'm sure that there are many patches welcome from anyone inside the community and so That is I think that is the talk actually well, I'm more than happy to
45:02
Answer any questions that people have but just a moment again to say thank you very much for the organizers of the conference I am Very privileged to be part of it even from the other side of the world and so I am more than happy to stay online and
45:22
Again, answer any questions that come through I'll try to get a I'll get a link try. I'll try to post that in the YouTube Comments to where I have the slides and so forth and And
45:41
Yeah, ask away ask away in any of the in any of the channels that you're more they have if you if you're watching the stream Ask the in the comments and I'm those comments are being being monitored. I'm just seeming that organizes a link and
46:10
Of the slides that I have as well as the research that I've done on all the projects because there's a little bit of More material there that you might be interested in so we've received our first question which I think is a really fascinated one
46:23
and it's from YouTube and I believe it's I'm going to get my English pronunciation is going to be terrible of this name It's a kind of view Sicaris what would be your advice to new rust programmers who are too happy to use unsafe?
46:44
so if I if I Re So the question there is you have someone who is new to rust maybe they have a lot of experience or they've just graduated from university and they look at unsafe and they say oh
47:02
Using unsafe just means I can apply the same idioms that I am used to in C++ Or in some other language and My advice would be that The reason is to ask them to step back and say well, why are you learning rust?
47:23
You know, you can write you can use pointers Manually in other languages and in fact in a way that is much more ergonomic rust makes it very Fussy it's it's it's not that nice to Use unsafe like use pointers in rust. I think it's done intentionally
47:44
it's intentionally clumsy in some scenes and If you have come to rust because you want to write in a safe programming language without a garbage collector then Use the compiler to your advantage
48:00
so There is a second question from from YouTube again, I apologize for my My English pronunciation especially the stress on the wrong part of the word but Michelle Lazowski great lecture. Thank you. Which strategy is the best in your opinion?
48:23
Yeah I am not an expert in this I would defer to The unsafe working group, but If you were going to ask me For my advice it would be to add a comment to each
48:42
Use of unsafe and That will force you to mentally check It will just slow you down and cause you to pause to double-check that What you are doing is sensible That would be my one
49:01
takeaway from twitch Has she done you've covered calling unsafe code inside functions But what about creating functions which are them themselves safe? Do I have any thoughts on that? I do and that is we I don't I could try and potentially try and scroll all the way through
49:21
So where was I I was over here. Oh But I think it was exa has created a function like a rust function around a C function I See, it's a safe function around an unsafe function. You see that wasn't quite what I want to do. I
49:45
Believe oh, no, that's not right either So my strategy for this is to write minimal rappers around Around and create functions that are very very easy to understand. Okay another question from twitch
50:09
Doc Chris, please. Thanks for the talk. You mentioned some lengths that we can use to assist developers in maintaining code with unsafe Are there any changes to the language or any? Unexisting ones that you would like to see introduced to make this easier and more lot more liable
50:26
Hmm. I would say to that use clippy, right? So I Think you see we can find it rust clipping
50:45
Not in that repository. Come on get up. Let's let's do this properly. Okay, there we go. There's rust lippy now Rust clippy is a community repository of good practices hundreds and hundreds of these things And I love this quote that it's just wrong do not use this in your own thing so
51:05
now You I wouldn't I don't I Expect that the lent that this quote comment linting that is applied by I think it was the One of the projects anyway, I think it was the standard library. I'm I assume that they have upstreamed
51:25
The lint into clippy this is where I would go and if they haven't well, you know, there's a job for this weekend Now oh, this is great again from Twitch
51:41
MRO WQA, why do you love so much? I love it. I Laugh because I'm nervous partially I I laugh because I Tried to remember that programming is primarily creative endeavor and
52:03
It's okay to learn it's okay to make mistakes it's okay to It's okay to grow and That means That we need to do the we need we need to learn in a way that is supportive
52:21
and so there's no sense and I I don't want to Start telling people dictate exactly how people should code in there and I don't want people to feel as if they are inadequate anyway One of the things that one of the reasons that I wrote the program and rust
52:41
Or and learned and took the time to learn rust and one of the reason why I teach rust is That it empowers everyone to write strong safe software even me Like I am an I'm a mediocre programmer in many senses Yes, I've and you know, I developed in Python for a long long time and
53:01
Probably over a decade and spent a long time in data science everyone every every piece of documentation that I wrote that I read around writing C extensions to make Python go faster was the first paragraph almost said only expert should do this or
53:21
you know, this is dangerous, you know or What was something some sort of language like that and I was always intimidated rust was the very that was the first systems programming language community That made me feel welcome And I want other people to feel welcome as well It doesn't matter where you come from doesn't matter whether or not you're from the San Francisco
53:42
You know from the Bay Area or whether you are from South Asia you should feel like a participant and and and You are you have a place at this table and I want everyone to feel welcome Okay, the question from YouTube from Andrew Bose if my unsafe code is unsound but works
54:06
For example, it might be faster should I change it for a slower but something that is guaranteed is a slower version that is guaranteed to be sound we have an example of this in the standard library or not so much so if
54:25
It probably is the case that if you have something that is fast, but dangerous that it is It works for no one good cases Like if you set everything up correctly, it will go fine. But if things are set up badly
54:41
Then they might blow up and explode on you. I Do not recommend anything that is like that takes user input or anything you're gonna install in someone else's computer. I Do not recommend that you expose them to security vulnerabilities caused by buffer overruns But if you're the kind of person that likes to play dangerous
55:04
Here's my advice to you create two methods The first one and the one that you should use primarily is safe code. It is pure rust. That is pure safe rust secondly have another method that has the same signature, but has underscore unchecked or underscore unsafe and
55:23
And Describe why it is that when you call this But like what are what you've done to ensure that it's safe when you call it if you know that certain edge cases Will cause it to explode Call the safe method like if you might be in a situation where they might occur
55:41
But if you know if you have initialized or if you're doing second scientific community scientific computing, for example, you know all of your data Pretty intimately, you know exactly how it's laid out in memory and so forth You can probably guarantee that the edge cases will never occur I'm not smart enough for that. So I I would urge you to
56:02
I would urge you to stick with safe, but if you want to you know like Yeah, then that that's over to you Okay another question from YouTube Zoran Lazarev kitsch have you ever had situations where your unsafe code has broken?
56:24
broke the safety and rust and Can't and if so, can you give an example of how that has manifest? No, I have not I haven't broken the rust compiler. I know that some other people have For example, I believe there was a regression in one four five
56:40
relating to The use of There was a strange edge case relating to Propagation of constants like and like toy code with references and
57:01
I haven't experienced that myself Okay, so YouTube michiel Lozovsky, is it possible to have in your project too many comments? That's like with terminus DB had quite a lot in my opinion. I agree. I mean
57:21
It's very very difficult to know where the balances Perhaps the balance exists It depends on the maturity of the project I can imagine that if you know that there are very competent developers and people who are very familiar with the domain
57:40
that you don't need many comments, but if Unsafe is a warning to you. Like if it there are some people they see unsafe. They're like, well, hey Like this is this is my play area. I love this stuff. I love pointers. I love and so it's all fine to me then
58:02
Maybe they don't need comments but if you're the kind of project that uses unsafe and less than 1% of their code base like probably Most projects and whenever someone sees unsafe they kind of tense up slightly Because they're worried that they're going to crash their own system Or potentially introduce security vulnerabilities for their users and more comments
58:27
I don't know if every single line needs to be commented. I thought it was interesting to find a Code in production at least it provides more robustness
58:42
Oh, okay from twitch Solace raffle, is that like somehow reflective back on me the username is Solace ruffle. I really enjoyed to talk Thanks, but I have one question What would be some good starting points for developers who are new to lower level programming language who want to learn how to safely?
59:01
Write unsafe rust Do you know of any resources that teach these skills from a rust perspective? Solace waffle do I have a book for you? I Will I feel very uh, I I don't do this because it's an ad I apologize if this means anybody
59:26
But I do think that my book It's designed to do that exactly. And so I think it's appropriate here the rust in action book Teaches you both rust and systems programming at the same time
59:43
It does that it will introduce you to unsafe it doesn't go through all of those explanations, but it does make you familiar with with some of the reasons why you might use unsafe and so Hopefully hopefully that's true not too much of an advert, but I
01:00:00
recommend in taking a look and checking the reviews and checking whether or not this seems like something that might be sensible for you. YouTube from Andrew Bogus. By the way, ask me if you want a Lint and Clippy. I think I want a Lint and Clippy. I think the community would love more robustness around
01:00:21
unsafe code and so I spent a little bit of time and probably too much time should have been preparing for the talk inspecting Clippy and I originally wanted to run linters over every public crate in crates.io and maybe some and try and find out whether or not
01:00:45
I wanted to go and inspect the uses of unsafe to say like what are people doing are they manipulating pointers are they doing some weird things like transmutation like in interpreting the bit patterns of like various you know like an array as integers and so forth
01:01:09
but no oh we've actually got quite a few oh two more comments I would and so yeah sorry answer that question yes add the Lint. I'm sure that logic the maintainer of that project will
01:01:25
be happy to accept it. Twitch. Lewis Co. Does Rust have a way of marking unsafe code and working smoothly with it like Haskell's IO monad? Maybe an unsafe trademarker. That is the unsafe
01:01:40
sorry that's the unsafe keyword itself I would say. You're not going I there are people who probably are type theorists and who understand Rust's type system very innately and I but I'm going to I'm just going to admit some ignorance there. I think that there are you could possibly have market traits you could
01:02:03
definitely create something was like you know you could create a market trait which implies unsafe I believe yes and I I'm just thinking off the top of my head I'm I've never done it but I think that there are unsafe traits and so as soon as you implement that you mark your
01:02:24
own type as unsafe but I yeah I'm going to express a little bit of ignorance there and just and just allow the type theorists and the back channel ask on Reddit ask on users.rustlang.org
01:02:46
and find out because again participating in the Rust community is exactly why Rust is great. These technical reasons are fine and perfect. You can see that other language
01:03:03
communities are adopting a lot of the you know D is investigating very is investigating lifetimes and C++ have lots of have changed their practices to be able to use smart pointers rather than raw pointers for example and so the technical parts of the
01:03:25
Rust community I think will be adopted by other languages. What we have that's special is the participation in the community aspect and so if there's something that's unfamiliar to you I would strongly encourage you to ask. Now Michael Ward on YouTube asks a question
01:03:45
why do you feel like bad about promoting my book? I'm eight chapters in and it's great. Okay I feel bad about promoting my book because people have to pay money for my book.
01:04:01
I've been contributing open source code for like 15 years or something like this and I just find it very difficult to ask for money from people and I think especially from people who are learning and who don't know what is good and what is not. I don't like you know about 90% of people that read my book think it's excellent.
01:04:25
No about 80% think it's excellent about 10% to 15% think it's very good and about five five percent want to throw it in the beer and like throw it in the fire. Now I don't want to recommend that you should buy it if you're going to throw it in the fire
01:04:42
and so yeah I don't know. It's just a personal thing. I should get better at asking for money.
01:05:03
I'm going to give like an extra 10 to 15 seconds for me to make a decision on the most interesting question. The most interesting question actually receives a I think we I think Manning my publisher has provided some codes to the organizers. If you are selected so if you
01:05:22
think of anything ask a question away like ask a question I'll try and include it but I'll try and pick a winner. If you are the winner please stay on whichever channel you have asked your question on such that the organizers can contact you to send you the code.
01:05:42
If you are interested in buying the book if you go to rustinaction.com there's a 40% discount code on that page. Now I actually really liked the first question. What would be my advice to Rust programmers who are too happy to use unsafe?
01:06:03
I think that this is very interesting from like a psychological point of view because it talks to the culture of people that are new to Rust and try and you know there's a lot to this idea that bringing people to a new apparently like it we there's a there are multiple
01:06:24
mental shifts that happen when you learn the Rust programming language and one of them is this idea that you need to trust the compiler you know the compiler is not your enemy your compiler is actually your ally. The Rust compiler actually is on your team and so
01:06:48
yeah I find that you know it's not so I definitely think that that one that for me is the most interesting or kind of the most compelling question I think it's the one that if I were to have a beer with you all after the conference because you know we're
01:07:04
going to the pub afterwards right. That's the question I think I would spend a lot of time trying to flesh out and explain and understand so that is the winning question for me. Again if I would just like to thank the organizers based in Poland for organizing this
01:07:27
worldwide event I think it was fantastic and I have been you know delighted with well it's been fantastic presenting to you all so thank you so much. But first I've just got one more announcement and that is I think the that is that there is
01:08:01
a hackathon if I go to rustydays.org and I look for a hackathon link sorry my internet has decided that it wants to go very slowly here we are the topic for this
01:08:22
year's hackathon is emergent phenomena or perhaps if you prefer can you amaze us with simple rules let's create an amazing result with a simple rule set if you've ever heard about cellular automata fractals or similar constructs this is what we're kind of talking about so allow me to do one more plug actually if you would like to learn about
01:08:49
writing a fractal in rust go to my youtube channel I have a video which is describes how to generate the mandelbrot set in rust and that can be possibly a good and I've got
01:09:05
some other generative art tutorials in there as well and so in particular this one is about creating some generative art in rust and we kind of work through a tutorial so I
01:09:24
I would encourage you again yeah to look at my youtube channel so that is youtube.com slash timflix and uh great I encourage you to participate in the hackathon it's going to be fantastic and um and have a lovely week oh a lovely weekend and a lovely evening and I'll
01:09:46
see you at the pub because it's the last talk of the conference right I'll see everyone later take care