04.10.08
Posted in Programming and People at 1:48 pm by Brooks
So, earlier today, Jeff Atwood wrote a rather interesting post about people who aren’t willing to pay money for software. And in there, he made this comment:
It’s tempting to ascribe this to the “cult of no-pay”, programmers and users who simply won’t pay for software no matter how good it is, or how inexpensive it may be. These people used to be called pirates. Now they’re open source enthusiasts.
And there’s been the predictable outcry of blog posts reading that as saying that open source enthusiasts are pirates, and saying things like this:
Jeff Atwood’s blog Coding Horror is one of my favorites. Until yesterday, I’d been recommending it unreservedly. Jeff’s made a big stumble, and I hope he corrects it soon….
Right. First off, anyone who’s read Jeff’s blog and is halfway observant knows a couple of things about him. He’s intelligent, and he’s outspoken about his opinions. If he wanted to say that open-source enthusiasts were the same as pirates, he would come right out and say it, and give you several paragraphs of reasons why he thought that. He wouldn’t just imply it in a throwaway sentence like that. So it ought to be obvious to any reasonable reader that that’s not what he intended, regardless of how he worded it.
More than that, though, I find the reaction of “I can no longer unreservedly recommend this blog” to be quite interesting. Suppose that Jeff did intend to say that open-source enthusiasts were pirates? I don’t agree with that, obviously, but so what? I don’t think I’ve ever read a blog that I completely agree with; that’s what makes conversation worthwhile, and if he has a good argument to back it up, it’s probably worth reading even if I think it’s erroneous. To me, what makes a blog worth recommending is when it presents intelligent arguments based on reasonable facts, in a way that I find interesting and thought-provoking and on subjects that I think are worth reading about. Making a “stumble” doesn’t change any of that.
Permalink
05.23.06
Posted in Programming and People at 1:25 pm by Brooks
Jeff Atwood makes the point that although experts generally don’t follow recipes (such as “best practices”) literally, the recipes are still valuable:
The idea that a recipe (eg, a best practice, a methodology, or a maturity model) is completely worthless is just as wrongheaded as the idea that everything should be based on a strict recipe. In other words, even an expert chef may occasionally find it helpful to refer to a recipe card.
There’s no reason these two models can’t coexist. You should always start with the common denominator recipe, of course, but you may want to provide some alternative guidelines and ideas for those cooks who have outgrown traditional recipes, too.
I strongly agree with his general point, but I think that “guidelines and ideas” are the wrong sort of thing to include.
To take the metaphor literally, for some things, I’m a level 4 cook. (Not necessarily an especially good one — it’s worth noting that these levels are not always parallel to the quality of the output!) A particular example would be some coleslaw I made a couple of weeks ago, when I found myself in an ill-stocked kitchen with a cabbage, an empty stomach, and the Joy of Cooking.
The Joy of Cooking is a pretty good cookbook for this sort of thing, as it turns out. It has a half-page discussion of what coleslaw is, without any recipes at all. It turns out that pretty much the only common factor in slaw is that it has cabbage in it. Everything else is variable. (This was an important point; we had no mayonaisse, and the person I was cooking with insisted that coleslaw has to have mayo.) It went on to describe, in loose terms, some of the common varieties. I skimmed through those, and then went on to look at the actual recipes. The standard recipe format is well-optimized for this sort of thing; there’s a list of ingredients at the top, separate from the instructions, making it very easy to skim to pull out specific bits of data such as the ratio of sugar to vinegar in a sugar-vinegar slaw.
After looking through the cookbook, I’d gleaned the following information: sugar and vinegar are a perfectly sufficient base for a slaw dressing, and it’s common to have them in X ratio, Y is a typical amount of salt, and Z, W, Q, and a dozen other things are some ideas for spices to put it.
Those, you may object, look exactly like “guidelines and ideas”. So, why am I objecting to including guidelines and ideas for experts in the recipe? The same reason that specific instructions are wrong for experts: They are processed information, and the definition of “expert” in this context is someone who does their own information processing. In my case, when I read the recipes, I was mining them for bits of information. Any processing that complicates that information-mining is a detriment, not a benefit.
And so, when writing recipes to be useful beyond basic instructions, the thing to include is facts. Not a guideline of “this squash should be cooked for 20 minutes”, but the fact of “if you cook this squash for more than 25 minutes it will be mushy and begin to lose flavor.” Not the idea of, “You could use peaches in this,” but the fact of “Peaches are also common in this.” The guidelines and ideas are certainly useful because they contain facts, but the parts that make them “guidelines” and “ideas” rather than bare facts rarely are. Often the process of distilling the fact into a guideline or idea loses useful information, too; suppose I wanted mushy squash, or wanted an uncommon alteration?
Permalink
05.12.06
Posted in Programming and People at 10:30 pm by Brooks
I was reading Kristin Abkemeier’s post on what it feels like to be a “geek girl”, and she linked to Matt McIrvin’s post about Nerd Bravado, which is the claim (originally made by an anonymous poster on yet a third blog post that I haven’t read) that in order to have any cred as an engineer, one has to at least once solve a problem in the manner of sitting down and working on it for 20 or 30 hours of continuous independent work, because there are some problems that can’t be solved any other way. (To be clear, that’s a claim that Matt was describing in order to disagree with it.)
I’m going to claim some authority in talking about this, because I’ve done that — and it wasn’t just 20 or 30 hours, either. A year ago, I thought that there was some chance that I could finish my dissertation by the end of the academic year if I got a certain set of simulations done, which required taking a computer program that was currently held together by Duck-brand Fortran Tape, and turning the structurally unsound mass of the core algorithms into a well-oiled robust, solid, machine that would do what I told it. I had two weeks in which to either accomplish it or give up.
So, I sat down on Monday to start working on it, and for all practical purposes, I worked on it straight through until Wednesday of the following week.
This is the first flaw of the Nerd Bravado claim. Hyperfocus doesn’t care if you sleep and eat and take a shower. You can think while you’re eating. Your brain keeps working on things when you sleep, and if you think about a problem when you go to bed and when you get up in the morning, you’ll often find that some tricky bit of it has become a lot clearer in the interim. You don’t have to sit for 25 hours straight to get the benefits.
That’s where the benefit of this style of working is, insofar as there is one: Hyperfocus. After a dozen or so hours into this project, I was so focused on it that I couldn’t really think about anything else — even if I tried, the problems in the code would keep distracting me back to the task. And, because I’d pretty much emptied everything else from short-term memory, quite a lot of stuff about the code could fit in there — enough to contain all the complicated interweavings of how the code all fit together. I could look at the results and see where the problems were coming from and what bits of subroutines wove together to create the particular effect that was happening, and I knew, without having to look, where to prod the program to change it, and when I prodded it, it changed. It’s a heady feeling.
I think that’s what the appeal of Nerd Bravado is, really. It’s like spice, from Dune. Get a bit of it into your system, and the universe opens up before you and whirls at your beck and call, and you have tremendous power at your fingertips to poke exactly the right thing in exactly the right place. But it’s also got downsides; do it a few too many times, and your eyeballs turn funny colors and it completely eats your life.
Now, I said I had two weeks to fix this program, and I stopped a week and a half in. Do you think that means I finished early? Hardly. A week in, and I’d gotten all the easy bugs fixed and most of the complicated ones, and I’d started tangling with the algorithms at the core of it. And they had a problem: The math that I was working with involved solving an equation where significant parts of both sides are multiplied by a factor that goes to zero. Divide that out to solve it, and very small errors in one term become very big errors in the solution, and it stops being well-behaved. There are ways around this, but they have to be designed into the algorithm from the start, not bolted on later. And by about Wednesday, it became quite clear that I was dealing with something that needed to be fixed from the ground up, and there was no way I could come close to that in the three days I had left.
This is the second flaw of the Nerd Bravado claim. It doesn’t actually solve the real big problems. Hyperfocus is a state of working on instincts, of having enough information actively in your brain that you don’t have to work through logic and deductive reasoning to know what the Right Thing To Do Now is. And that’s great, if what you need to accomplish is something that’s a chain of consecutive steps, like debugging a convoluted program or working through a complicated mathematical derivation or even writing a new program that fits a general specification that you’ve already determined.
Real big problems aren’t like that. Flash-of-insight problems aren’t like that — Einstein didn’t have his flash of insight about time dilation on the trolley because he’d been hyperfocusing on the problem for the past few days; he had it because he’d been rolling the problem around in his head for months. (Or years; I don’t actually know that bit of his biography well enough to say for sure.) And hard-work problems aren’t like that, either — writing a fixed version of my program isn’t going to work like that either, because it needs to be designed right from the ground up, and designing something right is a process that requires awareness of the outside world, and of a lot more than what fits in the hyperfocus. Yes, there are the legends of the great designers who sit down and in one day lay out the basic lines of an airplane or a car engine that are more right than their less-great colleagues create with a committee and a month, and that comes from having tremendous amounts of knowledge in their heads, but that’s the sort of knowledge that gets there over years, not hours.
This brings me to the third flaw of the Nerd Bravado claim: It’s not, overall, worth doing. When I’d finished my week-and-a-half of debugging, I was mentally exhausted. I was a week behind on all of the minutiae of daily life, I was emotionally starved for interpersonal interactions, I’d barely talked to my wife about anything that mattered to her, and my brain felt thoroughly fried. (My wife’s ironic laugh when I asked her if she remembered when I did this probably tells you all you need to know about what it was like from the outside.) It took somewhere around a month before my work productivity was anywhere near normal again. All totaled, I’d probably have accomplished as much if I’d simply worked at a reasonable pace for the month and two weeks, and I’m pretty sure the resulting solution would have been worth a bit more, as I’d have had the time to actually work out at least a first cut at a ground-up solution.
And, finally, the fourth flaw of the Nerd Bravado claim: the claim that “you must do it at least once to be a successful engineer … and to gain the respect of your peers” simply doesn’t hold true. The result of this particular experience, for me, was that I had a nicely debugged program that solved the equations the wrong way and didn’t produce useful answers. And all I learned from it was this: That style of working isn’t sustainable, and doesn’t lead to an enjoyable life, and I don’t want to do it again except in very small doses. As for the respect of my peers — well, does this cause any of you to respect me any more? No? I didn’t think so.
Permalink
04.24.06
Posted in Programming and People at 6:39 pm by Brooks
I’ve been thinking quite a lot about free software licenses lately, and how important it is to get them right. A couple of recent events will serve to illustrate some of the important points:
- The “listings” package for LaTeX is invaluable for including computer source code in a document. It’s something like syntax highlighting in a source code editor, but for typeset output rather than on-screen, and very easily user-customizable. Carsten Heinz spent almost a decade writing it and perfecting it, and it’s a marvel of intricate TeX programming — 218 pages of typeset user documentation and source code. Then, about a year ago, Carsten disappeared. As far as I know, nobody in the TeX community knows what happened to him; he simply stopped replying to email, and letters to his last known physical address just disappear. As a result, the official version of the listings package has effectively been frozen for the last year, despite a couple of important bugfixes that have been informally circulating. Luckily, this isn’t a permanent situation; it’s licensed under the LaTeX Project Public License, which provides a process by which the maintainership of the “official version” of a package can be transferred to someone else in such a case.
- In a blog post I was reading this morning (I’m leaving this vague so as not to point fingers excessively at only one example of a common problem), the author mentioned a general-purpose data structure that he had included in his open-source simulation code, and provided a link to the source file for the implementation. Like many such implementations, it’s sufficiently general that it would be useful in many programs that I’m writing, and it’s a better implementation than I could come up with (and test!) in a day’s work. However, there’s a problem: The license for this software package is not quite the same as any common software license, and it requires that its particular set of conditions and disclaimer be included in any redistribution. Further, the license has the relatively common clause that any scientific publications which contain data resulting from use of the program must cite it.
I’ll start with the last item. It’s completely reasonable that the author of a piece of scientific software would want to be cited when their work is used, so what’s wrong with putting that in the license? The answer is a lesson that should have been learned years ago. It only makes sense for cases where what’s being redistributed is pretty much the same as the original work. Beyond the well-known problems described at that link (short form: someone else builds on it enough to merit citation and adds their paper, and after a few repeats of that it quickly becomes unweildy), there’s the problem of partial reuse: if I include this little data structure implementation in my program, according to the license I then have to cite their paper in all of mine, even though my work is in a quite unrelated field. And so does anyone who uses my program. While that might arguably be appropriate for an advertising clause, it’s not appropriate for scientific citations.
So, functionally, I pretty much can’t reuse their data structure in my own software, if I wish to do so under its published license. That’s completely contrary to the spirit of free software; even though it’s open source and I can read it, the license prohibits me from simply copying it and reusing it unless I’m willing to agree to inappropriate terms. This almost certainly isn’t the intent of the program’s author; he simply didn’t consider the possibility of someone reusing a small piece of his program when he decided how to license it.
In this case, there’s probably a simple solution. If I want to use this code, I can write to him and ask for permission to use it under a different license than the one it’s published under — specifically, one which doesn’t contain this citation clause. Probably he’ll agree, and that will be that.
However, this is where the first example is relevant. What if it was something in Carsten’s code that I wanted to reuse, and he had a similar sort of clause in his license? Plain and simple, I’d be stuck. Even if I were willing to ignore the license on the assumption that his heirs wouldn’t find out or wouldn’t care enough to sue me, I certainly couldn’t relicense the code with a clear conscience.
That’s why license compatibility is important, and why it’s a bad idea to add private clauses like the “cite my paper” clause. Without it, free software isn’t a commons — it’s a balkanized set of little gardens fenced off from each other and unable to cross-fertilize. We’ve had this problem with railroad tracks, with fire hoses, with all sorts of things early in the industrial revolution; as engineers, you’d think we’d know better by now. Four feet eight and a half inches probably isn’t the optimal solution for any given railroad in isolation, but it’s certainly better in practice in the real world for nearly all of them, because it’s what everyone else uses, and in the long run it’s just not worth the trouble of being different.
“But,” you say, “I still want people to cite my paper when they use my software!”
Consider this: Most researchers and users of your program are, by and large, honorable and honest people. They understand the value of citing sources, and they appreciate that you’ve written this software and made it available for them to use. For the vast majority of them, all that you need to do to get them to cite your paper about the software is to ask.
License terms aren’t “asking”. They’re a demand, one with legal teeth behind it. They’re what you put in there for the people who you don’t trust to do right, and for the corporations that are too big to have morals at all. They ought to be the final line in the sand for the things that are vitally important: you don’t claim you wrote this, you don’t claim I’ll support it, you don’t put restrictions on it that I disagree with. If it’s not that critical, and if you wouldn’t sue someone for ignoring it, then setting it in that sort of legal inflexibility does more harm than good.
Thus, I propose the idea of adding a “Requests” or “Moral Obligations” section to software licenses, for this sort of thing. When I release a program that’s large enough to merit a paper citation if someone uses it, somewhere near the license clause in the documentation will be a section that says something like this: “The requirements in this section are not legally binding; however, they represent things that the author would appreciate: If you write a paper based on work that uses this software, please include a citation to my paper, and send me a copy of your paper. If you create a derivative work based on a this software, tell me about it and let me know how I can get a copy, and include this request if it’s appropriate.”
In virtually all cases, that will work just as well as adding those as terms to the software license. (How do I know it will work? Well, it already works quite well for the papers themselves, though there the request is implicit rather than stated.) And, importantly, it will work equally well in the cases where something I asked for turns out to be terribly inconvenient for some reason that I didn’t think of when I wrote it, even if I’m not around to change the license terms.
Permalink
04.06.06
Posted in Programming and People at 1:45 pm by Brooks
I’ve been thinking a lot about the idea of practicing programming lately, after reading Steve Yegge’s “rant” on the subject. His claim is that programming is a lot like playing guitar – just doing a lot of it doesn’t make you any better; you have to specifically do things that will improve them.
Dave Thomas suggests in his Code Kata post that one way of practicing programming is by solving small-but-complex algorithmic problems, and he has a list of suggestions.
In the shower this morning, I was thinking about a problem that would fit nicely in that list. I’m writing a complicated bit of code that takes the triangles in a triangular mesh and loads them into a data structure. It would be nice to test it by taking a given mesh and loading the triangles in various random orders, and that immediately leads to the classic card-shuffling problem: given a list of objects, how does one efficiently sort them into a random order?
Read the rest of this entry »
Permalink
« Previous entries