Why everyone needs to know some coding: last-mile sandboxing
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 287 | |
| Author | ||
| Contributors | ||
| License | CC Attribution 2.0 Belgium: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/56904 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
FOSDEM 2022109 / 287
2
4
6
8
12
17
21
23
31
35
37
41
44
45
46
47
50
62
65
66
67
68
71
73
81
84
85
86
90
92
94
100
102
105
111
114
115
116
117
118
121
122
124
127
131
133
135
137
139
140
141
142
145
149
150
156
164
165
167
169
170
171
172
174
176
178
180
183
184
189
190
192
194
198
205
206
207
208
210
218
220
224
225
229
230
232
235
236
238
239
240
242
243
244
245
246
249
250
253
260
262
264
267
273
274
277
282
283
287
00:00
Boom (sailing)Line (geometry)Dijkstra's algorithmCodeGateway (telecommunications)ComputerPower (physics)Demo (music)Group actionBridging (networking)NeuroinformatikLine (geometry)Software bugSoftwareCollaborationismProjective planeDistanceFreewareDialectState observerComputerCodeSound effectElectric generatorGame controllerCASE <Informatik>Information securityMachine codeMathematicsCollisionView (database)Lattice (order)Decision tree learningCore dumpDiagramJSONXML
02:35
Gastropod shellTemplate (C++)Operations researchComputer networkAsynchronous Transfer ModeFunction (mathematics)Computer fileMomentumIntegerCursor (computers)Menu (computing)Lemma (mathematics)Price indexFrame problemMIDIWindowLetterpress printingBootingVolumenvisualisierungInformation securityOperator (mathematics)Student's t-testFunctional (mathematics)Mobile appNeuroinformatikOrder (biology)WindowComputer simulationSoftwareComputer programmingGreedy algorithmWeb browserComputer configurationPhysical systemTouchscreenSoftware frameworkInstance (computer science)Position operatorMoment (mathematics)Exception handlingSpeech synthesisSystem callError messageWhiteboardView (database)CASE <Informatik>Data structureMathematicsPhysical lawInternetworkingRow (database)Event horizonMachine codeFile systemGreatest elementRight angleWritingKeyboard shortcutAreaDifferent (Kate Ryan album)Type theoryLoop (music)Menu (computing)Price indexCodePropositional formulaSingle-precision floating-point formatHeegaard splittingMultiplicationAsynchronous Transfer ModeText editorComputer fileMiniDiscBasis <Mathematik>CountingTwitterDefault (computer science)Information privacyPower (physics)Pie chartSound effectComputer animation
08:59
VolumenvisualisierungCursor (computers)Function (mathematics)Event horizonWindowJunction (traffic)System callOperations researchComputer networkComputer fileAsynchronous Transfer ModeComputerCodeMereologyInformation securityMobile appOperator (mathematics)Position operatorMathematicsComputer filePoint (geometry)QuicksortSound effectFunctional (mathematics)Condition numberCodeSpacetimeInsertion lossLattice (order)Open setSource codeElectronic mailing listBit rateAdditionGoodness of fitOrder (biology)WritingTrailEvent horizonCommunications protocolData storage devicePerspective (visual)TouchscreenSoftware testingWebsiteFlagConstraint (mathematics)Line (geometry)Multiplication signSoftwareDifferent (Kate Ryan album)Internet forumGame theoryBoss CorporationTable (information)Transport Layer SecurityComplex (psychology)State of matterFunction (mathematics)outputDefault (computer science)Demo (music)BootingSet (mathematics)System callNP-hard2 (number)Online helpOptical disc driveCASE <Informatik>Power (physics)Flow separationWeightReading (process)Vulnerability (computing)Java appletJSONXMLUML
15:24
Element (mathematics)Computer animationMeeting/Interview
15:44
Connected spaceInformation securitySoftwareComputer clusterGroup actionInformation privacyTouchscreenDecision theoryWebsiteRight angleCASE <Informatik>State observerOperator (mathematics)Execution unitLevel (video gaming)State of matterFunctional (mathematics)File systemProgrammer (hardware)Exploit (computer security)Mobile appVector spaceFeedbackConfidence intervalDifferent (Kate Ryan album)BitAbsolute valueComputer programmingMultiplication signDenial-of-service attackComputer animationMeeting/Interview
18:02
PlanningFile systemComputer fileComputerMobile appHoaxFunctional (mathematics)Computer programmingVideo gameMachine visionHypermediaMeeting/Interview
18:54
Point cloudComputer animation
Transcript: English(auto-generated)
00:06
Hi everyone, I'm Kartik Agram and welcome to my talk, Why Everyone Needs to Know Some Coding. I don't want this to be clickbait though, so the answer is right there in the title as well, Last Mile Sandboxing.
00:22
In short, I want to argue that we should encourage people to take more direct control of their computers and that teaching basic security is a good way to do that. My talk will consist of a short preamble and then a few demos. The purpose is to solicit collaborators for my project to live up.
00:46
30 years ago, Tijkstra reminded us that lines of code is something to be minimized. This was obvious to the best software thinkers then, and it's gotten increasingly obvious even to non-tech folks today. Software is a liability.
01:01
And free software hasn't alleviated the ill effects of software as much as we had hoped. We'd hoped that with lots of eyeballs, all bugs would be shallow. But we haven't actually had the eyeballs that we were promised. My action to this observation is to stop trying to hide the code that runs on our computers.
01:25
It doesn't work reliably anyway for most people. And what's more, it's self-limiting to hide the code. If you do it well, you just make it more difficult to recruit the next generation of contributors who can improve free software. To everyone who says,
01:41
why can't my computer be more like a car or a bridge or a house? I say, it can't. It just can't. Software is more power than a car. Small changes to it can have action at a distance in our world. Either learn to harness this power or avoid software and dial.
02:00
The downsides can be huge when things go wrong. If we assume that exposing more people to code is a good thing, what would nudge them on a path to care more about it? My answer is security. People today are acutely aware that their computers are not accountable to them.
02:21
It might be a compelling case to say, do this small amount of coding yourself, don't delegate it to any third party and your computer will be more directly under your control. To see how that might work, let's switch to Tuluwan. Let's say you downloaded this app, TootToot.TLB, from somewhere of dubious
02:45
repute on the internet. The value proposition is a simple text editor that counts characters as you type. What's more, there's a single delimiter for splitting your toots up into multiple toots for Mastodon or tweets.
03:05
Each toot or tweet gets its own character count and you can split up or merge the toots or tweets as you figure out the best way to chunk your toots. All this looks mildly interesting enough for you to try out the app.
03:21
However, when you try to save your toot using this hotkey in the menu below, you get an error. Unlike most programming languages, Tuluwan doesn't assume you wrote a program just because you're running it. And as the reassuring green on the bottom right shows,
03:43
Tuluwan doesn't permit apps to make any file operations or network operations by default. In this case, you really do intend for the app to perform a file operation here. So you follow the instructions, enter the permissions area using this hotkey.
04:01
By default, everything in the permissions area is green and safe. An app can't do much, for better or worse, but you can adjust things. For example, to run this app in a browser like Sandbox, you can enable network access while continuing to disable access to local disk. Tuluwan now provides a concise summary of the
04:22
security model. This app can communicate with other computers but can't access private data. However, you can also disable network access and enable file system access by making an edit. Now the summary shows that Tuluwan is operating this app on an offline basis.
04:46
Already we have more flexibility than a web browser gives us. However, there's still a warning here. This app is now allowed to perform all file operations. That's a lot of power. If we were to enable network access now, Tuluwan goes red. It can't protect us anymore.
05:05
The app can read anything, exfiltrate it anywhere. We're relying on the owner system. Might as well go use a more mainstream framework, which will at least provide a more pleasing, less brutalist UI. So let's turn network access back off.
05:24
And what's more, we'll even make the file system access more restrictive. For starters, we could restrict writes. Oops, we got a syntax error. We're writing
05:41
Lua code here and the syntax for not equals is a little different. Now the syntax is correct, but we still see this warning that we're likely allowing all file operations. There's a typo in our mode. So let's go back there and fix the typo. So now we see we're still allowing almost all file operations. And the reason is that there
06:05
are more than two modes. So let's instead say what we allow. Now the pie chart here shows us that there is some increase in safety. An alternative way to do this is to restrict
06:25
just the file names that we care about. Now Tuluwan thinks we might be rejecting all file operations, and that's close enough to what we have. We're rejecting every possible
06:42
file operation except one. This is some decent protection. Let's go back to the app. Oops, we lost all that we typed in before. Oh well, we'll type it in again. And we'll now try to save, and the save succeeds without any error.
07:05
The indicator on the bottom right now shows that we aren't quite as safe as we used to be. But the app does what we want. Days and weeks pass. Now we have an idea for a change we'd like to make. We press the standard hotkey shared by all the apps Tuluwan runs. We want to edit
07:28
the app. When we edit an app, we first see a big picture view that contains the major data structures up top that the app uses. Second, it also contains the functions that
07:44
the app has implemented, defined in top-down order. All apps start at function main. Under function main, the next row shows the functions that are called by name. The row below that shows the functions called by those functions, and so on. Let's go into
08:04
main. You can see I'm typing below here. The code is in Lua, as we saw before. We see some initialization. There's an event loop which renders the window and then waits for a keystroke.
08:21
If we go back to the big picture view, we see that those function names are exactly what are being called by main. So this is expected. Let's now jump to render. We see some code that
08:40
executes to render the screen. We also see that this function is called from main, which is what we expect. So things are starting to make sense. Now we'll type in whatever our idea is for a cool new feature. We run the app, and we can immediately see the effects of our code.
09:06
Tuluwan transparently does what it needs to to get the app to run. In addition to controlling permissions and editing app sources, Tuluwan also gives us the ability to monitor what an app is
09:20
doing. One facet of the big picture of an app is the recent events view. This is a list of the side effects it has caused. Recent events. And now we see that Tuluwan did in fact open this file boot for writing. This provides us with an audit trail. It won't directly protect us, but it provides
09:46
some social accountability. It's also useful aid for browsing the app's source code. It gives us a different perspective on the functions that are important. I can hit enter here to jump to the update function. Let's go back to the permission screen. One challenge with the permission screen
10:10
is that there's a lot of power here. Code that runs here gets unrestricted permissions, and there's no safety net. To encourage safe behaviors, I'm imposing a couple of UI constraints.
10:23
The first, you've already seen my little summaries at the top. If you make the code complex, Tuluwan reminds you that it can't be sure how well protected you are. But as you learn more about the app's internals, Tuluwan gets out of your way. The second way that Tuluwan encourages
10:44
good behaviors is by only displaying a few lines of code at a time. This will hopefully remind us all to keep policies concise. When we keep college policies concise, we also reduce the odds of
11:01
a mistake or a typo. I've spent some time thinking about possible policies, and I haven't found any good reason for the code to even exceed a couple of lines. Perhaps if we start allowing different parts of an app to have different permissions, perhaps you've studied one function more closely and feel comfortable giving it greater permissions that won't be
11:22
available to some tiny function secreted in a dark corner of the app that does something insidious one day in a year. Anyways, that's future work. This is the end of my demo. Hopefully it gives you a flavor for the sorts of things I imagine being possible on Tuluwan.
11:40
There's still lots of open questions, some of which I alluded to during the demo. The first open question is how do you avoid data loss on that initial run? More broadly, generalizing away from this one app, what are some default policies that are easy for people to understand and also unlikely to put people at risk? I've spent some time thinking about this,
12:04
but it still seems hard. For example, it's not easy for a policy to stay aware of what gets written to a file once an app opens it. Or if you allow apps access to a private space to write persistent data, they could secrete sensitive data there and potentially exfiltrate it once they get broader permissions. Another concern is what do flexible permissions for the network look
12:28
like? Right now, as you saw, it's just a simple toggle. It doesn't make sense to separate permissions for read and write like we do for file operations, because networking protocols require both. It also has been challenging to think through scenarios for network use.
12:44
Any user-visible network operation involves multiple syscalls. Open, bind, connect, listen, accept. I start getting nervous keeping track of the state in trusted code. If I want to track that the output of one syscall is not permitted to go into the input of another, and so on, that simultaneously seems brittle for innocent apps, and it's also harder to
13:05
convince myself that it'll actually protect people from arbitrarily subtle, malicious apps. One thing I consider to be very important is that there should be no way for an app to test for permissions. An insidious app that can get a sense for what permissions it's operating under
13:23
can also alter its own behavior to avoid showing any red flags about what it's trying to do. In general, apps can be arbitrarily insidious. That suggests that Dilua needs the ability to specify where in an app we can perform file operations on network access. That way,
13:44
it's safe for people to grant permissions without having audited all of an app's code. They could gain lots of safety just by auditing one or two functions. Past these sorts of UX questions, there are some challenges in educating people.
14:03
How can we gradually teach people to put themselves in the shoes of a malicious app and think about the risks they need to protect against? For example, even if people start using Dilua, it would be a failure if we saw questions like this on public forums, because they lead to answers like these.
14:21
Oh, just toggle this setting on permission screen, or just paste in this code, which leads to people's tendency to do things without thinking about them. Another challenge is educating people on the need to keep trusted code on the permission screen concise, or to avoid side effects in it. If we see people reading policies from file somewhere
14:44
and then sharing large files of allow lists, that feels like failure. Hopefully it's clear by this point that I don't really know what I'm doing. I don't have a background in security, and I'm accurately conscious of my ability to introduce vulnerabilities with every line of code I change in the Lua code base.
15:04
Hopefully I've also kindled some interest in this problem space. There is value here for people. I hope you will come help me, whether it's by asking questions and helping me think through use case for security, or by helping to set the curriculum we want to educate people about in the course of using
15:21
the Java. Thank you.
15:44
And we are live now. You can start answering questions. Hi. The first question is, shouldn't you also restrict how much an app can write to your disk? And that's absolutely true. In general, IO,
16:04
what I've shown is a very bare minimum sense of what is possible. There's a lot that makes to be done. In particular, I didn't really think hard about denial of service vectors for attacks. I spent all this time thinking more about privacy and exploitation
16:21
and protecting against that. So I think, yeah, there's more work to be done on the denial of service and everywhere else. The next question is, it should be more possible to be more specific with network connections. Absolutely. And that, again, is future work. One thing that makes networking harder than file system operations
16:43
is that when you want to open a file, it's just once it's called. And so all the state is right there. For networking, you end up going through connect and bind and accept and listen. And so the states are getting strung through, which makes it harder for me to have confidence in the policies I craft. I've been thinking about potentially having a different API for
17:04
network operations that's more stateful and easier to control, but then I'm diverging from Lua. And so there's a trade-off there, and I'm still struggling to plot the right course. The next question, how does the function evaluate the safety level of the function that
17:22
checks permissions? I think the latter half of the talk as described, that permission screen is completely trusted. There's nothing preventing you from doing anything there. And so that's why I try very hard to provide a lot of feedback on what the consequences are of your decisions and also to nudge people to keep things really concise because you want to
17:42
really be able to see everything at a glance. And also, I don't want people to turn into programmers trying to control their security footprint. I just want them to have confidence that they are protected. They just need just enough programming to do that.
18:01
Scroll down a bit to read the rest of the questions or comments. Yeah, so the rest is just comments. Maybe you could define this as a special grant for writing the file. Oh, I see. So do the follow up on the quota question. And I think I don't fully
18:21
internalize the answer, but I'm definitely going to go think about it afterwards. Thank you. Can't a program work out whether it has five functions by trying to read, write a file and read it back? Yeah, so this is absolutely planned future work is to be able to lie to apps. And so you,
18:41
you as the policy crafter, the person who owns the computer should be able to say, give this app a truth, a fake file system that has these files in it, and I'll allow it to do whatever it wants. That is absolutely on the plan.