presentation of better software no matter what. So if you have not been showing up for the earlier sessions, we're talking about how to improve the quality of software and the basic topic we're talking about now is the use of static analysis and in particular we're talking now about the use

of static analysis tools such as Lint. I mentioned before the break that there's this notion that programmers don't write code unless they think it will do something. So for example, if you were to say x gets x, that just seems like an odd thing to write as a source statement.

And so some researchers say well all right, if we make the assumption that programmers don't usually write statements that have no effect, if we notice things that have no effect, then that suggests that there might be an error there. And so there's a paper which I'll be citing at the end by Z and Engler and what they point out is they were looking at the Unix kernel

and what they found was somebody had written da.snode gets sa.snode followed by da.snet gets da.snet. Now this is x gets x. That is a self-assignment and almost certainly what they were trying to do is they put sa up here and they probably should have put sa down there.

Now I don't want to say that maybe they did some copying and pasting but maybe they did some copying and pasting and then they failed to edit this afterwards. Now I understand that this would never happen to anyone in this room but think about the people who are attending different talks right now.

So that is an example of the kind of thing that static analysis can find. And human beings can read this 100 times and unless you know what you're looking for it's going to be really difficult to notice that kind of mistake. Now this is from an article by, I think it's Andrei Karpov

came out recently. Notice that what this condition says if it's not true that axResolution is greater than zero and axResolution is greater than zero. Do we really want to test that twice? I mean does that seem like what we want to do? Now it's not clear what the code should look like

but it's almost certainly true that this is not what they intended. So static analysis tools can identify things that where the code just looks funny and it's easier for human beings to overlook. So both of these papers are by people who actually are in the business of doing static analysis.

They have kind of a vested interest but what I think is interesting is the idea that by looking for code that just doesn't seem to make any sense without breaking anything you can find interesting problems. My experience is that once people are used to lint-like programs most of them never want to go back. Many people will complain about getting used to the volume of warnings that they get from lint-like tools

however most people say, you know, now I can rely on tools to find problems that otherwise would slip through. Static analysis tools are so useful that they have been developed for a whole ton of languages. I did some googling around one time. Certainly you can find them for C, C++, Java, VB, C Sharp, Perl, Pruthi, Rayban, you name it,

they've pretty much got them. What I think is kind of interesting is even languages that are dynamic like Python and Ruby are included. So the notion of static analysis tools is not something which is limited to old-fashioned languages like C and C++. The fact of the matter is there just are suspicious constructs

in pretty much every programming language and if you can have a machine find them why would you want to look for them yourself? So far I've been talking about source code static analysis. It turns out there's some interesting stuff

you can do with object code analysis too. So for example, you can analyze Java or .NET byte code which can lead to some really interesting results. So this is from an article by Anderson and Repps and they point out that, so here is some function they wrote and you have an array for password, you get the password, you do some secret operation passing in the password,

and now in order to make sure that you don't leave the password in clear text sitting on the stack what you do is you do a memset on password for the size of the password to scrub a copy on stack which seems like a perfectly reasonable thing to do. And what they discovered when they did byte code analysis was they found out that the compiler had said,

huh, I see you're doing a write here to the memory for password but you never read it ever again so clearly that write is unnecessary. Let me optimize that out of there for you. You clearly don't need that. So static analysis of object code can also find certain kinds of things that otherwise would go overlooked, let us say.

So now we've talked about compiler warnings, we've talked about static analysis tools such as Lint. The next kind of thing you might consider for static analysis are custom parsers and custom analyzers for source code. You might write your own parser or acquire a parser

and then perform your own custom analysis. Now I gave a talk one time at ETH in Zurich where I was asked to give a talk by Bertrand Meyer who's the man behind Eiffel and is probably one of the world's most rabid C++ haters.

So I thought it was very nice that he actually asked me to give a talk there. So I gave a talk actually in two pieces, one of them about how wonderful C++ was and the second one about how horrible it was. He liked part two much better than he liked part one. But one of the things I talked about was how you can do template metaprogramming in C++ and about how you can therefore generate

all the necessary types to do complete static analysis of scientific and engineering units, systems, which was kind of a nice result. And what he pointed out was that researchers at ETH had developed a dimensional analysis utility for Eiffel.net. Now they were able to write a static analysis tool to effectively have the same result as what was achieved with template metaprogramming,

but the important thing is they wrote a custom parser and a custom analyzer to be able to identify certain kinds of mistakes. Something else you can do is very simple. You can search for interesting comments in source code, things like to do or fix me or question marks or kludge or hack. This might be a place you might want to look

a little more closely, not a sophisticated static analysis tool, nevertheless a static analysis tool. By writing your own tools, you can also deal with the problem that in some cases you have incomplete or invalid code. So you can write partial parsers or relaxed parsers.

So for example, if all you need to do is come up with a class browser and you don't need to actually parse all the bodies of the functions, that's something which you can put together even if the functions themselves won't compile in many cases. There is an interesting implication to this. Easy to parse languages foster the development

of static analysis tools. And one of the reasons that we know that this is really important is because C++ got this wrong pretty much from beginning to end. C++ even now has a horrible dearth of static analysis tools, simply because it is such a difficult language

to write parsers for. And this is important for you in general. What this means is if you are designing a language or if you are designing a data format, you want to design it such that it can be parsed in a pretty reasonable, straightforward manner, because that permits people to write tools for it in a pretty reasonable, straightforward manner.

If you come up with some kind of a data format or some kind of a syntax for a language that is going to be difficult to parse, it's going to be hard to write tools for, and you're going to really regret that from that point forward. So you want to follow the lead of C++ and then go in the exact opposite direction as far as parsability goes.

The final kind of static analysis I want to talk about are code reviews. Now, code reviews is when human beings sit down and read code. They're not executing the code, they're just reading it. Therefore, this is also static analysis. Humans are actually really good at static analysis, and they can catch problems that tools typically don't find,

like domain and application-specific errors, where something looks okay in general, but it's completely wrong for your application, context-dependent mistakes, refactoring opportunities, concurrency errors. Humans are still, in many cases, better at that than tools are. Furthermore, if you have multiple human reviewers,

the coverage tends to be pretty broad. I mentioned this morning that if you don't tell programmers what to do, different programmers tend to do different things. Somebody will program for performance, somebody else will program for maintainability, somebody else will program for portability. Similarly, if you ask programmers to look at a section of code,

they tend to focus on different things. Some people focus on performance, some people focus on maintainability, some people focus on portability. So if you choose your reviewers carefully, you can get pretty broad coverage of the different aspects of the code that's being reviewed. When we talk about code reviews,

there's actually a scale of formality that we can look at. So the scale I'm going to talk about is taken from Carl Wieger's book, Peer Reviews in Software, and he divides the possibilities of code reviews into these categories. There's the ad hoc review, which is where somebody performs a quick review

of your code, they kind of go by, they look at the code, the ad, that looks okay to me. Now these days, over your shoulder could be a virtual over your shoulder using some kind of a collaboration tool, but it's an informal possibility. Then there's the desk check, which is where you review a printed copy of your own source code. What's interesting about this is the word printed.

Now my experience is that I find many more mistakes when I am looking at something which has been printed out and I have talked to many other people who say that they also find more mistakes if they're looking at something which is printed as opposed to looking on a screen. Having said that, I have looked for research results which actually demonstrate empirically

that you find more problems when looking at a printed version as opposed to an electronic version and I have not been able to come up with any evidence one way or the other. So it could just be that old people like me like looking at paper, but Carl's also old and he thinks that it works as well. But it's something to take into account where find out whether you get better results if it's printed as opposed to just looking on the screen.

Variations of that are the peer desk check, which is where somebody else reviews a copy of your code. Then there's the pass around where people perform independent desk checks. These are all pretty informal. Peer programming. Peer programming is two people programming at the same computer. Guess what, one person's typing and one person is doing a concurrent

instantaneous code review. So that also counts as code reviewing. Then there's the walkthrough which is where you walk through and explain your code and I suspect we have all had the experience of you sit down to explain how something works to somebody else and you go, oh, and you realize that there's a problem there which you had not seen earlier.

These are informal code reviews and the characteristic of informal reviews is generally speaking that data on the process isn't collected. So usually with these kinds of informal reviews you're not keeping track of how many defects did we find, what kind are they, what's the location, what is the underlying cause? So that makes it informal.

More formal approaches where data typically is collected, there's the team review and again I'm following Uyghur's categorization here. This is a planned event. It is somewhat structured. There's multiple official roles, possibly including post-review follow-up activities.

Generally speaking, the reviewers get the code to examine in advance. They often get then checklists or perspectives for specific things to look for. Now I said before, if you choose your reviewers carefully, it is often the case that different people look for different things so you get pretty good code coverage. In a more formal inspection, you would like to eliminate the possibility

that you'll be unlucky. As a result, instead of saying let me know what you find, you will often go to people and say, okay, I'm going to give you this checklist of things to look for. For example, you might be looking only for efficiency related concerns. So you're looking for unnecessary operations, you're looking for operations which could be performed more efficiently, you're looking for a loop fusion opportunities

and that's what that person is then going to focus on. Or you might give them a perspective where their perspective is you're the efficiency person or you're the portability person or you're the readability person. This eliminates the possibility that you're not going to get a broad analysis of the code that is being examined. And then traditionally,

there is the full-blown inspection of code that was originally brought forth at IBM. It's fully planned, it's completely structured, there's official roles, it's followed by very specific follow-up activities and this would include verification that defects found in the review have actually been addressed. Most people will probably never experience this rigorous and detailed a review

unless you are working on safety critical software. If you're working on safety critical software where lives are at stake or if you are working on software where ginormous amounts of money are at stake, then you might have something that is going to be subject to that level of scrutiny.

So the question is how effective at finding problems is human static analysis? The detection rates for behavioral, which is external quality defects, generally speaking, increases with formality. Steve McConnell in CodeComplete second edition,

he says that informal code reviews tend to find 20 to 35% of the total bugs in the system from an external quality point of view. Formal code inspections, they find between 45 and 70%, says Steve McConnell. Carl Wieger says, yeah, really good teams have hit 90%. Now, 90% to me looks really impressive,

that you can find 90% of the mistakes without ever even running the system. That's pretty good. Although frankly, what I find myself often wondering is how do you know what 90% is if you don't know how many total bugs there are in the system? I mean, there's got to be some way of calculating, I mean, 90% of what, right? It turns out that 100% generally speaking

is the total number of post-development bugs that are found either within six months or before the next major release. I mean, they actually do define what 100% means. So it's not 100% of the ones that are there, it's 100% of the ones that we eventually find. And we're simply going to assume the ones that we never find don't exist.

Programmers are incurable optimists. There are now online code review tools which have been developed, which works especially well with distributed teams. So the notion of everybody sitting around a conference table and going around and looking at the code still occurs in some places,

but online code reviews are increasingly common. They tend to be GUI-based. Use a GUI to view the code and the diffs. You can add and discuss and make comments on the code, things like that. They're basically collaboration tools. They support asynchronous reviews and distributed teams. No need for an actual physical meeting, works great for distributed groups.

Typically integrated with the version control system, which means that you can actually identify exactly which code got changed the last time it was checked in, so you have a better idea of what should be focused on. And often integrated with IDEs, for example Eclipse, and I think that these days there's some plugins for Visual Studio as well. As far as I have been able to determine,

there is no objective data yet on how well these reviews do at identifying defects compared to the more formal versions where there is some data on it. There is some data that is out there, it's just that it happens to be published by the companies that are pushing their collaboration tools. So I don't consider that terribly objective data.

Now, if you think about reviews of any kind, and we're talking specifically about human-based code reviews, there's a number of benefits. The first one is less sloppy coding. It is a fact of human nature that people behave better if they know that someone is watching.

So if somebody is going to be writing code and they know that somebody else is going to look at the code before it gets checked in or get approved, they are likely to do a better job. This is one of the reasons why pair programming can pay off. It's not like you can type in something without somebody noticing. Somebody clearly is going to be observing what's going on. Same thing for open source,

where in theory, the whole world is looking at your source code. Another benefit of doing these kinds of human reviews is knowledge transfer between programmers. So typically, after a while at a company, you've got some older, more experienced people, and they know a lot of stuff about the system, they have a lot of experience about things in the wild,

and that's great, that's very valuable information. On the other hand, you can have some new people coming in, either from other companies where they've had a different experience, or possibly right out of college where they just have finished some academic courses. Those people tend to be more up-to-date on the latest technologies, on the latest thinking, or they just have different background, and now you get knowledge transfer across the two people.

So this is, by the way, another way that pair programming can pay off, because you've got people who are now talking to one another who can share experience. A big advantage is earlier defect detection. Now, reviews during coding can find defects in external quality that otherwise would be determined later during testing.

The sooner you catch a problem, the less expensive it is to fix it. So if you don't have to wait until testing finds the problem and reports it back to development, it's going to be less expensive and less invasive to fix that problem. Furthermore, they can find defects in internal quality that otherwise wouldn't be detected later until maintenance.

And remember what I said this morning, that if you want to have quality software, it needs to exhibit external quality, it behaves properly, as well as internal quality. It has high quality code that can be maintained. So you're looking for both of those things. Now, obviously, it takes time

to perform these kinds of code reviews, to have people dedicate time for doing code reviews. So the question is, is it worth spending the time and the money for doing these kinds of code reviews? Now, the data that we have is generally based on the more formal approaches and is generally only concerned with external quality.

Even so, this is what the data shows. First, reviews and inspections tend to find more behavioral defects than testing, which is actually kind of an interesting result. You can find more problems by human beings staring at the code than you can by testing the code in a dynamic fashion, so that's kind of interesting. And at the same time, you can find defects in internal quality, which testing rarely exposes.

And reviews tend to be less expensive per defect than testing, and one of the reasons for this is kind of interesting. So let's suppose you write a test, you run the test, the code fails. You have two problems. Problem number one is you have to find the location

in the code that led to the failure, and then you have to fix it. But before you can fix it, you have to find out where in the code does the problem occur. But when people are reading source code, they go, oh, here's the problem right here. They've already identified where the problem is. See, all you have to do now is fix it. So code reviews have that advantage over testing.

And it turns out, according to the data, that code reviews tend to find defects faster than testing. So Carl Uyger's ultimate conclusion is that reviews don't slow projects down, defects slow projects down. So what you want to do is make sure you don't have the defects.

I would hope that it goes without saying you don't need to review every single line of code. In particular, you can reserve it for the highest risk parts, where the defects are either most likely to be present or where, if they are present, they'll have the most significant consequences. This is typically what people already do with system testing anyway.

You don't try to test the entire system completely and totally uniformly. You try to focus your testing in the areas where it's most important. There's nothing wrong with doing that with static analysis and with code reviews as well. Code reviews are complementary to testing,

but they cannot replace testing. It turns out that in practice, testing identifies different defects from code reviews, and code reviews identifies different defects from testing. So you still need to do testing if you're doing code reviews. However, we also know that every combination approach to defect detection does a better job

than any of their components alone. So if I have technique A for finding problems, technique B for finding problems, I'm going to get better results if I do both of them than if I do either one of them alone. And this could explain part of the success for Agile. We have pair programming plus test-driven development.

Effectively, that's the same as well. Pair programming is kind of like designing code reviews going on on the fly, and test-driven development has a lot of testing going on at the same time, so you've got reviews going on at the same time as you've got testing. Now, these kinds of reviews, reviews within a pair of pair programming,

they do have the disadvantage that, generally speaking, both people were responsible for producing the code at the same time, which means they probably share the interpretation of the spec, and they don't have the outsider's perspective you might get from an outsider's review. So it's not necessarily the same as an outside code review, but it's certainly better than nothing. What we do know is this.

Reviews may reduce the time and the cost of testing and debugging, and what that means is you might be able to spend less time and money downstream on testing compared to what you did before you introduced reviews. The general observation of software engineering is that well-spent time upstream virtually always saves time downstream.

So you may shift your resources a little bit to the upstream development process more into code analysis, but then that allows you to spend less time on testing later. Now, I've talked so far about code reviews because, as I said, I really focus on developers, developers are the people I feel most strongly about,

pretty much every argument for code reviews extends to all the other work products. So certainly you want to review requirement specifications, you want to review use cases, you want to review user stories, design specs, user interface specs, API specs, test cases, documentation, schedules. I used to think it was a joke, but there's now several companies now

where they actually review performance reviews. So it turns out that manual static analysis is just a technique that works, and because it is so effective, it is one that I encourage that you adopt. So the guideline is to embrace static analysis,

and the four things we talked about were compiler warnings, lint-like tools, custom parsers and analyses, and manual code reviews. Any questions about anything to do with static analysis?

Next thing I want to do is talk about the keyhole problem. The keyhole problem arose a number of years ago

when I had the revelation that a whole lot of things about software that really bugged me could all be traced to a single underlying problem, that there were a whole bunch of things that I found annoying and irritating

that could be made to magically go away if we could just solve one little problem. So here's the deal. Yes, this is the deal. It's a big, wide, wonderful world out there,

and way too often, I have to manipulate it and deal with it through this tiny little restricted viewpoint. I feel like I'm trying to deal with the world as if I'm going through the keyhole of a door, which is why I call this the keyhole problem. So this is what I'm going to be talking about

as regards the keyhole problem. The most important thing is I'm going to talk about what are keyholes. It is not a standard term. Therefore, before we can go any further, I need to try to get across to you what is this problem that I am trying to describe to you. Then I want to focus on why are they important? Is there more at stake here

than just Scott being irritated when he uses the software? Hint, there is. I would like to spend a lot more time on how we can eliminate them or how we can avoid them. That turns out to be a huge problem in its own right, so I'm going to spend less time on that, but I can at least introduce to you what are keyholes and why are they important,

and we can talk a little bit about how to get rid of them. What I want to point out is that almost everything that follows is something that I experienced personally. In fact, I am going to be showing you things that we all experience every day and most of you probably don't even notice it any longer. We've just become accustomed to these kinds of things. I don't seek out key holes.

It turns out they're completely unavoidable. This is not a gooey presentation. This is not a gooey presentation. This is not a gooey presentation.

I am going to start by showing you what I call visible key holes because they are visible on the screen because I think they get across the idea of this keyhole metaphor that I have been talking about. But the notion of keyholes is more pervasive than gooeys, as I will demonstrate afterwards.

So don't be fooled by all the screenshots into thinking, oh, it's a gooey talk. It's not a gooey talk. This is just example manifestations. So what I've tried to do here is name the various kinds of keyholes that we have. So this is what I call the vertical list box, combo box keyhole.

Now, this is a screenshot from my screen. I hope you admire it. But what I want you to notice here, so here I'm using Skype, and this is a dropdown for choosing, as I recall, various kinds of noises that it can make when various events occur. But what I want you to notice is the scroll bar here. So you can see that if it went down about that much further, it would show me all of the options that I have available. But they decided, no, no,

we're not going to show this to you. We're going to limit it. So on these kinds of dropdown boxes, if you're anything like me, what you do is you look at this and you go, hmm. None of those look like what I'm looking for. So then you go to the scroll bar, you scroll them down, and you go, hmm, nope, none of those are that either. And then you scroll back and you choose one of the ones that you saw before. So the question is, why didn't it just show me the whole thing?

Why artificially limit it to this many when they could have gone a little bit further down the screen? It's not like we're running out of space here. So this is the vertical list box, combo box keyhole, and this is still not a gooey talk. So here's a different manifestation of the same problem. This is from OpenOffice.

So what I like about this is, you probably can't read it very well, which is unfortunate. The red thing, this says more. So it already tells you, hi, I'm not showing you everything, click here for more. But it's not even showing me everything of the not showing me everything, because you can see the scroll bar here. So why didn't they just drop it down further and show me everything it was going to show me?

All it does is slow me down, make me do a little more scrolling when I don't need to. Any time you have two different arbitrary decisions being made, you run into the possibility that there's going to be inconsistent results

as a result of the two arbitrary decisions. Now, this happens to be from an older operating system. This is Windows 2000. But nevertheless, it's a good example. So here I have a situation where it's not showing me all of the drives here, and it's not showing me all of the drives here, but you'll notice that it's two different visual manifestations of not all of the drives.

So it makes it difficult for me to recognize that I'm in essentially the same kind of selection in the operating system. Fundamentally, arbitrary constraints, like how many options I'm going to show you, inherently lead to inconsistency.

Any time you have someone arbitrarily choosing how many elements are going to be displayed, you have the possibility that the number that they choose will be exactly one smaller than the total number necessary. Now, I think we do have to admire

the geniuses at United Airlines who decided that the number of things to show on a month drop-down box was 11. I want to remind you, everything I'm showing you is a professional released software artifact. Someone did this for a living. And this one is from Microsoft.

So I have a theory as to why this one is off by one. My theory is that the person who wrote it forgot to count select your operating system as one of the options. That's my theory. So I want to point out, it doesn't have to be this way. So this happens to be from Mozilla Firefox, and you'll notice that, so here's my window,

which doesn't quite go to the top of my screen, but you'll notice that when I try to see, I think this is my bookmarks, it actually fills the entire screen. It fills the entire space available. I was so happy, I took a picture. But my point is, it's not like it's difficult programming. It's just a matter of somebody has to think about

why should we restrict what people can see? So this is the horizontal list box combo box field. This is just the horizontal version of the same thing. This is from an older version of Windows Media Player, but nevertheless. So I'm reading through the list here.

Now, the thing about this is, you don't have a horizontal scroll bar, so you can't actually see what it says until you click on it. Now, maybe I'm a little bit overly sensitive because I've written some books, but when I see burn your own awe, I'm a little concerned that says burn your own author.

So, but the only way to find out what these things do, oh, and you'll notice, by the way, it's not showing me all the options either, because going down to here would be clearly incorrect. The next kind of keyhole is what I call the fixed size window keyhole, which is what it sounds like. It is a window that is fixed in size.

Doesn't let you resize it for some reason. Very common for program installation, so this is from, let's see, this one is from CC Cleaner. This one over here is from Greenshot. This, I think, is a shot from last week when I installed the most recent version of Greenshot. Why it is that you can't make this window bigger so you can see the whole dropdown, I don't know,

but somebody decided that that's the perfect size window, that you can't improve on it. Very common for about boxes, so this is for, let's see, xView, I think, is the name of this program, so you get a tiny little box here. I can't make the window any bigger, so if I want to read about all the wonderful things they've done, that's it.

Every once in a while, somebody gets it right. This is from Google Earth, so there's not a lot to display, but you can resize this window. I know, I tried. This is also a fixed size window. This is an older one. This is from Ameritrade. Now, Ameritrade is a company,

it's a financial company, that wants me to trust all of my money to them. That's their business. Trust us with all of your money. And so when I was looking at the account funding, it says account funding, it wants to know the account type, so I clicked on the question mark for the account type,

and up it popped this window, which could not be resized. And I said, you know, if you can't make your pop-ups resizable, I'm not gonna trust you with all of my money. I'm not kidding, that actually was the basis for my decision.

These are also fixed size windows. I'm not making this up. I can't make this stuff up. I'm not making this up. This is also from, these are also fixed size windows. Now, what I like about this is, you'll notice that they're almost the right size. They're not quite the right, they're almost the right size. Now, think about this for a minute.

I'm running Microsoft Windows, I'm installing Microsoft Office, I'm going to the Microsoft product update website. You'd think they could get it right, and they would, except, anybody know what the sin is that I committed? I configured my system to use large fonts.

What the hell was I thinking? What kills me about this is, if you have a webpage and you want it to be resizable, this is how much additional work it is. I'm not kidding. That's how much additional work it is.

It is funny, but the reason I mention this is because in many cases, people say, well, you know, we can't fix it, it's much too hard. It's not too hard. You simply have to be aware of the problem and decide that you want to address it. Some problems are harder than others. This is not a hard problem.

This is an easy problem. Now, related is the fixed size dialogue. I mean, in some sense, dialogues are almost, by definition, fixed size. This is one from ACDC Systems, so it's a fixed size dialogue, I can't change it, and, you know, really, that is unbelievably useful information.

I mean, I can clearly decide what it is that I want to select out of there. So, yeah. I can't make this stuff up, I'm not kidding. Another fixed size dialogue, but you know, some people say, well, you know, commercial software, what do they know?

Open source. This is from Mozilla, okay? Same kind of problem with Mozilla. So I can't resize this dialogue, and it runs off the edge here, it runs off the edge here, and I've got a scroll bar here. So this is not improving my open source lifestyle. And I like Firefox, I use Firefox all the time. It's just that I don't like it when it does this to me.

Okay, we have now the limited tree expansion keyhole. Now, before I go further, I want to remind you, every example that I am going to show you is caused by the same underlying problem, which is somebody imposed an arbitrary restriction.

Lots of different screenshots, lots of different manifestations, but the problem is the same. It's an underlying constraint that someone decided to impose. So this is from an older version of Visual Studio. So you bring up the options dialogue in the older version of Visual Studio here. And it turns out that if you choose

one of these choices here, it will snap closed whichever one is currently open. You can only have one level of the tree expanded at a time. And I've shown this dialogue on top of the larger window

because what I want you to see is the tree control immediately next to it, which has multiple sub-branches open. But I lied. I told you that you could only have one sub-tree open here at a time, which is true if you use the mouse.

But if you happen to know the obscure keyboard commands for running around the tree control, you actually can have multiple sub-trees open simultaneously because, general lesson here, any time you have somebody imposing

an arbitrary restriction, that will almost inherently lead to inconsistency. It's the arbitrary decision because what that means is somebody else might make a different arbitrary decision. That's what we're trying to combat. Here's the modal dialogue keyhole. So modal dialogue locks up your screen,

or at least locks up your application. That's the problem. It says you can't do anything else. So this is from FrameMaker 6, the one where it is correct behavior for auto numbers to go to zero when you freeze pagination. But I'm not bitter, really, I'm not. So here's my question. While I'm printing my book

with all of its auto numbers going to zero, which at the time took a while, my question is why can't I close the spell check window and clean up my desktop a little bit? I mean, does it really seem unreasonable that I want to close my spell check window while I'm printing? But with modal dialogues, you can't do it.

This is from Windows 7. So anybody besides me ever have the thing where you open a dialogue and you click on a sub-dialogue and you click on a third sub-dialogue and then you forgot what you were doing so you want to go back and look at the first one? And you can't because if you want to do that, you must be punished.

They probably had a focus group, you know? I mean, really. Every once in a while we get lucky. And we get lucky in the sense that we have two applications which behave more or less the same but have made different kinds of design decisions.

So it turns out that if I want to do bookmark management, in this case, so if I'm using Internet Explorer, this is a modal dialogue. So if I'm organizing my bookmarks, I cannot use the browser window. However, under Firefox over here, it turns out I can use my browser and I can organize my bookmarks simultaneously.

I bring this up to point out it's not like it's inherently technologically prohibitive to implement this. But application developers make these decisions so frequently we just become accustomed to seeing them and we don't think about them any longer. Then there is the fixed-width edit control.

So here's an edit control. Now, notice how the dialogue is this wide. And they probably hired some artist who said, you know, really, it looks better. I think the right amount of gray, I mean, really, what I really liked about this

was that the default setting doesn't fit. It's not like they never saw this problem. Somebody said, you know, it looks good. I really like that. So, okay, that's the fixed-width edit control. Here's the fixed-width edit control. This happens to be from onesweet.com's website. This is account setup. So you get to say, okay, what is your favorite, secret question, what is your favorite, whatever.

Oops, this doesn't get any bigger. So I can make this page as wide as I like, but that edit control doesn't get any larger. Unless, unless you go to the accountant's maintenance page, at which point it does get wider. Why? Because anytime you have somebody making an arbitrary decision

that will inherently lead to inconsistency. Arbitrary decisions beget inconsistency. How do you fix this problem? Pay attention, it's really complicated. Style equals width colon 100%. This says, make the edit control as wide as the window is.

I don't think that's an unreasonable amount of work to do to avoid these kinds of problems. This is the one that I really love. This is the lying fixed-width edit control. Now, the lying fixed-width edit control has become so pervasive, even my wife refers to it.

So I go to Marriott, I want to book a hotel room, so I type some information in here, and then I decide, what is your Marriott Rewards number? So I whip out my Marriott's card, I take a look at my Marriott's card. For anybody who is thinking about copying down

my Marriott Rewards number card and staying in a hotel to give me benefit, don't do it. I have digitally altered the number. But if you do want to give me credit, contact me privately. But what I want to point out is, so, this is my Marriott Rewards card. This is a picture of it. So, I go to the Marriott Rewards number,

look at all that space I have to type. I type, seven, seven, seven, space, five, five, five, space, four. This is a freaking lying edit control.

This is what kills me. You know, it lines up perfectly up here. It looks really pretty when it's empty. That's what I want to optimize for is the empty look. What's that? It's a glass, yeah, all right, it's a glass door. All right, so, this is from WinFacts Pro.

This is an old program, but I really do like this. This is something no one needs to know, but I'm going to tell you anyway. The smallest state in the United States has the longest official name. It's officially known as Rhode Island and Providence Plantations.

But don't try typing that here, because, I'm sorry, that's as much as you get. The rest of this white space, from what I can tell, that's just for show. You can't type it in there, sorry. At this point, I hope you're noticing that I'm mixing web applications and native applications

because this is not a web thing. This is not a native application thing. This is not a Windows thing. This is not a Mac thing. This is a general software development thing. The next one is the fixed width web page.

So, here's my screen. That's how much information they want to show me. Notice the scroll bars. They have more information to tell me. I said, look, I am willing to give you, I am willing to give you my most precious resource. There is nothing more precious than pixels. I have lots of memory.

I don't have very many pixels. I give you my pixels, and this web designer says, I turn my nose up at your pixels. I don't want your pixels. This web designer said, I like your pixels even less. This is the same problem with these two websites down here.

What I like about the ones down here, they employ what I call the children's reduced vegetable density strategy. This is where your children smear the vegetables around on your plate, hoping that the density will drop below parental detection threshold,

like I'm not going to notice that there's still a lot of white space on my screen. It doesn't have to be this way. Go to Wikipedia. Make the pages white as you like. Wikipedia just automatically adjusts. It's not like Wikipedia is a little-known website.

The technology is not terribly complicated. All right, so at this point, I hope I've gotten across the basic idea of a keyhole. It is an arbitrary constraint on what you can see or what you can express.

It manifests itself in lots and lots of different ways, but the underlying, there's only one underlying concept. Somebody arbitrarily shows a restriction when there was no good technical reason for that restriction. So now that we've seen the pictures, let's talk about invisible keyholes,

which are non-GUI components. When I say it's invisible, that doesn't mean it's undetectable. Just means it's not a GUI keyhole. This is one of my favorites. This is the too-few-bits keyhole. So I installed some software.

It says, if you have EZCD Creator 4 and have more than eight gigabytes of free space, stop. Let's be clear what the problem here is. The problem is I have too much free space, okay? That's the problem. Too much free space. Please create temp directories and fill up space

in these less than eight gigabytes of free space remains. And while I appreciated the suggestion on how to fill up my hard drive, what really got me, and this is such a sad sign,

I looked at this and I went, I went, eight gigabytes? That's 33 bits. That's just a weird number. I mean, four gigabytes I can sort of understand. That's 32 bits. 33 bits? Very strange. I have a theory as to how they got there, but it's, I have a theory, but it doesn't matter.

It doesn't let them off the hook. I mean, really. So a number of years ago, I was using a program. I used to really like this. It's called Naviscope, and what Naviscope used to do was it was an early ad blocker. And of course, it was one of the first companies, because it was an ad blocker, I mean, once you got ad blockers, it becomes a spectator sport, right?

So you start blocking ads, right? And you go, okay, I've got 100 ads blocked, I've got 1,000 ads blocked, I've got 10,000 ads blocked, I've got 20,000, 30,000? 32,767? And a little while later, minus 4,116 ads? I mean, really?

This is a good thing. Now, I used to complain about them using a signed value here. I mean, really? Did you expect to have negative ads, but it's just too few bits? You've never heard of Naviscope. I mean, come on, little company, who cares? Anybody heard of Microsoft? So you can't read it very well. Trust me, the first character there is a minus sign.

So apparently, Microsoft has a little trouble, too. Too few bits. Okay, the next one is what I call the restricted field size keyhole. I'm going to give you a very specific example of something that happened to me, because I want to get across

these arbitrary restrictions end up causing problems in ways which people don't typically foresee. So one of the books that I wrote, I have the following phrase, prefer iterator to const iterator, reverse iterator, and const reverse iterator. It doesn't matter what it means. That's what the phrase is.

And in order to enter this into the index, I want to do what is known as rotations, where you rotate the various words so you can make multiple index entries. So ultimately, this is all the index entries that I wanted to have. And because FrameMaker is

an interesting program, it is a what you see is what you get editor, except when it comes to entering things in the dialog boxes, in which case they actually have their own markup language, which really is what you're looking for in a WYSIWYG editor, is its own sub markup language. But that's a whole separate issue. So this is ultimately what I typed. And I typed it into this box.

Well, I'm lying. What I did was I typed it into Emacs like a real person, and I copied it into that box. Of course I didn't type it into that box. So I got a big keyhole here. But what it said was, marker text is too long, marker text is limited to 255 characters. Now, we know what's going on here. What's going on here is the person who implemented this said,

no one will ever want an index entry of longer than 255 characters. That's what happened. This is a word processor. Trust me when I tell you, it wallows in virtual memory, okay? And so they chose a fixed size array here. They didn't want a dynamically allocated array. Unbelievable.

But the reason I pointed it out, and you can tell I'm still mad about it, is because this was me trying to get my work done. I wasn't trying to break the system. I wasn't trying to do anything unusual. I just wanted to type in my index entry for a book that I was working on. So when somebody said,

no one will ever need more than 255 characters, they were simply showing a lack of imagination. And the point is, if you are smart enough to work on a program that effectively competes with Microsoft Word, you are smart enough to dynamically allocate an array. Speaking of Microsoft,

anybody here unfortunate enough to know what warning C4786 is? I'm so sorry for you. It turns out Microsoft said nobody would ever write a program

with a name in the program that was longer than 255 characters. No one would ever write an identifier of longer than 255 characters. And then some stuff happened in the standard library for C++, and Microsoft's compiler itself was generating names

with more than 255 characters, which could not be debugged by the Microsoft debugger because no one would ever have an identifier with more than 255 characters. So the end result was Microsoft managed to block itself into a situation where it could not debug the code it generated,

which I thought was really kind of a sweet move. But of course it didn't really punish Microsoft so much as it punished all of its users. Why? Because somebody said, well, no one's ever gonna have an identifier of longer than 255 characters. So why should we dynamically allocate memory? I've talked to the compiler team since, and apparently they fixed the problem.

The array is now larger. So for Microsoft Visual Studio 2010, read me, setup may fail if any path and file name combination exceeds 260 characters. I don't really understand this business.

Acrobat 9, Acrobat 9, not that old. If you wanna make an annotation, it's limited to 5,000 characters. You can actually type more than 5,000 characters into the annotation box, or, sorry, you can paste more than 5,000 characters into the annotation box, at which point the rest are silently discarded. Apparently they had a focus group meeting at Adobe,

and they said the maximum number of characters any reasonable annotation will be is 5,000. 5,001 characters is too many. 4,999, that's okay. But 5,000, that's the absolute limit. It's an array thing. Fundamentally, you may notice a theme here, arrays equals bad. Or to be more precise, fixed size arrays equal bad.

Other things from the past. For a long time in the United States, we thought that phone numbers would never have more than seven digits. Oops, then we went the 10-digit dialing and all kinds of things broke. Two digits will always suffice to identify the year. Oops, new millennium. 32 bits will cover all IP addresses.

Oops. There seems to be this recurring thing where people assume, we'll never need more than that. If you can dynamically size things without killing yourself, dynamically size them. It's just so much more graceful when the world changes, which it always does.

The next one is what I call the restricted domain keyhole. The restricted domain keyhole is where you have a domain from which people can select things, but you arbitrarily restrict what they can select. Usually this is my fancy way of saying you can't enter certain characters for usernames or passwords. So Ameritrade, you remember them,

the company that wanted all of my money? I have to have a pin. A pin, the lock that ensures that my online security is going to be upheld. The only thing between nasty people and my money, my pin, it must be a four-digit number. Four digits.

It cannot begin with a zero. It cannot include letters or symbols. Well, I feel so secure. Now, I know what happened here. What happened here was originally they offered access through telephony. You could use a touch-tone phone

and they wanted to do things with touch-tone, which is fine, I have no problem with that. So why would you carry this over to a platform where people have keyboards? Now, if I want to have something which will work both on my phone and on my keyboard, then I can choose a four-digit number that doesn't begin with a zero, which presumably calls the operator in the middle of entering my pin.

But why restrict that? Now, it turns out that a competing company, Charles Schwab, which offers essentially the same services, it allows digits and letters for both telephone and for keyboard input. I point that out, that's a demonstration that working around the technological problem is not difficult.

So for a while there, I was using a program, or at least trying to, using a program called Smart FTP. It's an FTP client for Windows. Now, think about what an FTP client does. An FTP client takes information and gives it to a remote FTP server.

That's basically what it does. And in particular, at what point it's going to say, enter your name and enter your password. So I enter my name and I enter my password, and the FTP client says, password must not contain spaces. Why are you looking at my password? All I want you to do is give it to the FTP server, which takes spaces.

I will yield some information about myself. I feel like I know you well enough. Every password I have has spaces in it, as long as I can get it through the filtering system. So basically, everything has spaces in it. But this actually prevents the program from working.

And what I like about this is this is more work for them. They actually have to put a check into the code to detect a password that they don't approve of, even though they're not even supposed to be looking at it. Oddly enough, I uninstalled that. Well, all right.

If one keyhole is good, then multiple keyholes must be better. Here's my path. I'm a programmer, I'm sorry. Here's my path. This is what I get to edit in. Path, edit.

Except that you go to environment variables and then I've got a keyhole up here and I've got another keyhole down here. This is not resizable, of course. That would be way too easy. That gives me this. This is not resizable. That would be way too easy. I'm just trying to get my work done. I mean, all I want to do is edit my path. I'm a programmer, sue me. This is from a number of years ago.

I decided to order some memory to go in my laptop. I wanted to make sure that it would fit in my laptop. And so I went to the Crucial Technology website and I wanted to make sure I was going to order the right thing. So I said, I'm going to chat with them. Here was their expert online. And it was really convenient that the website, the webpage was only this wide

because that let me scoot it over so that I had room for the expert online chat box, which was not resizable. So I couldn't actually see what was going on. So you can see my question here, which I typed into Emacs, of course, and then pasted in here. And it said, question should not exceed 255 characters.

I'm trying to buy some memory from you. All right, I mean, at this point, maybe you're thinking, that Scott, look at the hair. He's so excitable. Sometimes it's more serious.

I got some email from a guy who was working on software to be used in sawmills. And this is just a picture I found on the internet. So this does not show specifically the thing there. And it was a 16-bit machine. And you have these logs that go into the sawmill, and then they have to be pushed into place

so that they will then approach the saw blade directly. These are obviously very heavy trees. And the software was configured in such a way that basically, as I recall, they represented like tenths of an inch or something. I don't remember the exact details.

What I do know is that because it was a 16-bit system, when the log's width exceeded 65.535 inches, a number which I suspect we are somewhat familiar with, you would get overflow. And when you got overflow, it would push the log much, much closer than it should have.

And the way they found out about this was that when they started putting some old growth logs in there, it was the first time it had ever exceeded this width limit. The thing overflowed, that was not detected, and they started pushing the log, and the log started going into the operator's cabin where the person is sitting,

and I'm not making this up. Now, when you have logs that are in the wrong place, you can talk about, you can have blades on the saws that will.