It’s tempting to ascribe this to the “cult of no-pay”, programmers and users who simply won’t pay for software no matter how good it is, or how inexpensive it may be. These people used to be called pirates. Now they’re open source enthusiasts.

And there’s been the predictable outcry of blog posts reading that as saying that open source enthusiasts are pirates, and saying things like this:

Jeff Atwood’s blog Coding Horror is one of my favorites. Until yesterday, I’d been recommending it unreservedly. Jeff’s made a big stumble, and I hope he corrects it soon….

Right. First off, anyone who’s read Jeff’s blog and is halfway observant knows a couple of things about him. He’s intelligent, and he’s outspoken about his opinions. If he wanted to say that open-source enthusiasts were the same as pirates, he would come right out and say it, and give you several paragraphs of reasons why he thought that. He wouldn’t just imply it in a throwaway sentence like that. So it ought to be obvious to any reasonable reader that that’s not what he intended, regardless of how he worded it.

More than that, though, I find the reaction of “I can no longer unreservedly recommend this blog” to be quite interesting. Suppose that Jeff did intend to say that open-source enthusiasts were pirates? I don’t agree with that, obviously, but so what? I don’t think I’ve ever read a blog that I completely agree with; that’s what makes conversation worthwhile, and if he has a good argument to back it up, it’s probably worth reading even if I think it’s erroneous. To me, what makes a blog worth recommending is when it presents intelligent arguments based on reasonable facts, in a way that I find interesting and thought-provoking and on subjects that I think are worth reading about. Making a “stumble” doesn’t change any of that.

There are, however, solutions. First, there’s the semi-official Microsoft solution, which involves some really hairy poking around in the system registry. (And, by “really hairy”, I mean that it looks signficantly more so than poking around in the system registry usually is.)

There’s also a handy little freeware utility that does those changes for you automatically, created by the fellow who runs IntelliAdmin.com. His blog post describing it is here, and the actual download is here. I tried it out, and as far as I can tell it worked perfectly (aside from not actually closing the window when done). He suggests setting your clock ahead to November to watch the time flip over to see if it worked, but it’s easier just to go into the “Time” control panel that you get from right-clicking on the clock and selecting “Adjust Date/Time”, and looking at the “Time Zone” panel to make sure it says “Pacific Daylight Time” (or whatever) rather than “Pacific Standard Time” for the time zone.

For those of you still running Windows 98, he also wrote a fix for that, even though there isn’t an official Microsoft solution at all. The blog post about that one is here, and the download is here.

It’s an interesting observation on advertising, too. It probably took him a day or so to code up this program and test it, and he’s now got a first-page Google hit for people looking for a solution to one of the bigger problems of the month — and people like me are passing around links to his blog, too. You just can’t buy advertising that good.

(The rest of his blog looks fairly interesting, too, if you’re doing Windows administration. This little trick for automatically putting the current date in a filename is particularly neat, for instance.)

One particularly nice feature is that, since all of the GNU/Linux stuff is operating in a single process on the Windows side, it’s easy to set the whole thing to a priority of BelowNormal, and thus even when I’ve got a compilation running and taking up as much CPU as it can hog, it doesn’t slow down anything else I’m doing at all. And it turns out that there’s enough slack in the system that this doesn’t give it any noticable lag in the terminals, even.

The one problem with that is that it completely confuses the software clock on the operating system in the virtual machine, though. When I’ve been using the CPU heavily on the Windows side of things, I’ve seen the GNU/Linux time lose 10 minutes out of 15! VMWare does a nice job of keeping the hardware clock up-to-date, but Debian is set up so that everything depends on the software clock, which only gets synchronized to the hardware clock on bootup. It’s not a critical problem, really, but it’s somewhat annoying to have to remember that a file date of three days ago (or whatever) means “a minute ago”.

The right solution to this is not ntpdate or some other NTP solution. The Windows clock is already synchronized over the network, and the VMWare virtual hardware clock is synchronized to that (I presume), and so there’s no need for adding extra traffic to the network. (Especially since it’s dialup, and disconnected from the internet and any NTP servers half the time.) Nor is the solution something like adjtimex, which tweaks the speed of the software clock; the time skew is a function of Windows-side computer load, and thus quite random, and a static speed tweak is not much of an improvement.

Instead, a better solution is hwclock --hctosys, which synchronizes the software clock to the hardware clock. Because the skew is quite large and needs correction quite often (and because cron is using the skewed software clock for timing, and thus loses time), I added the following to my root crontab:

* * * * * /sbin/hwclock --hctosys


So far, it seems to be working beautifully.

This marks the first significant bugfix release in nearly three years; I took over the official maintenance of this package early last year, after the previous maintainer simply disappeared. It feels good to have this back in the realm of actively-maintained packages again, even if I am a little slow about replying to bug reports sometimes.

Also, although this doesn’t show in the release, I was for a few hours this afternoon completely caught up on listings-related email. (I got another bug report this evening, though.) One of the perks of having one’s email address at the top of the documentation is that one gets a lot of questions as well as things that are truly bugs in the package; cases where someone wants to do something like highlighting parentheses and brackets and numbers, which the package doesn’t directly support, and so I spend 15 minutes or so hacking up a workaround for them. Sometimes these questions result in new functionality, too — the old versions assumed that if \chapter was defined, the user wanted the listing-number counter to reset at each chapter, and the only way I could find to change that on the user side was to temporarily undefine \chapter around the \usepackage{listings} macro. Now, there’s a numberbychapter key that can be set to “true” or “false” in the preamble.

This release also has the usual selection of small fixes to handle newly-discovered and newly-created conflicts with other packages, and I particularly want to thank Markus Kohm for his bug reports, which always contain an insightful analysis of whatever is conflicting with his KOMAscript document classes this year, and a small piece of code that I can copy in to make things work right again.

There’s also a new \lstMakeShortInline macro, which defines a character as an equivalent to \lstinline with the appropriate user-specified options. This is a pleasant case of unintended consequences; a couple of years ago, I was corresponding with Frank Mittelbach (of the LaTeX3 project) about some suggestions I had for making the LaTeX Project Public License — which is essentially the TeX-world equivalent of the GPL, used by nearly everything on CTAN — more appropriate for cases where one’s borrowing a bit of code from one project to use in another rather than distributing a slightly-modified version of the original project, and I believe a few of those ideas made it into updated versions of the license. (At the least, Frank clarified that the intent of the original license allowed that sort of copying, even if the wording was arguably unclear.) And it turned out today that implementing this bit of functionality was almost trivial to do, because I could copy an equivalent implementation for inline verbatim environments from the LaTeX documentation classes.

This post also wouldn’t be complete without thanking Carsten Heinz, the original author of the package. Although the code is deeply arcane and complicated in places, in the way that only a general-purpose parser implemented in TeX could be — which is not to be confused with the way that a TeX output routine to handle three levels of cross-referential footnotes in scholarly editions of originally-footnoted books can be, and I am glad David Kastrup is writing that piece of code and I am not! — it has quite a significant amount of documentation, including several chapters on the high-level design and implementation which are almost completely useless to anyone who’s not working on the internals of the package. There’s a line at the beginning of them, saying, “First I must apologize for this developer’s guide since some parts are not explained as well as possible. But note that you are in a pretty good shape: this developer’s guide exists!” And every time I read it, I smile, because he’s right: I am, and it makes a tremendous difference.

One of the things that I’ve noticed is how different websites handle feedback from their customers—and, in particular, bug reports (and feature requests).

For example, the American Institute of Physics has direct links to an online “submit query” form directly off their help page, and also has a prominently-featured email address—which was quite useful, since I got a server error when I tried to use the online form. We’ll see how quickly they reply (if they do; bug reports don’t really need replies), but in my opinion, that page is pretty much how things should be. It would be nice if they had an explicit mention of “bug reports go here”, but “Technical Support Query Form” is close enough.

Contrast this with the Wiley “Contact Us” page, which I’ve been dealing with for the past fifteen minutes or so. Why have I been dealing with it for fifteen minutes, for what started out as a trivial feature request? Well, first off, at the bottom of the How To Contact Customer Support page (yes, that’s a different page, which you get to by clicking a link on the “Contact Us” page), it says, “We regret that Wiley InterScience Customer Support does not accept queries by email.” That, of course, means that the only option to actually say anything to them is through their web system, regardless of whether or not it happens to be broken.

So, consider the web form. That “Contact Us” page has a couple of links to customer support: one labeled “Online Customer Support” promising FAQs and so forth, and one labeled “Contact Customer Support” and noting that one should use it if “unable to log in”. Despite the appearance of being different things, they both go to the same page, which is a login form, with a place to register an account in the fine print at the bottom. Now, I don’t have a personal account here, since I’m using a university site license for the content, so it looks like they’re asking me to go through the whole process of creating an account on the site just to file a simple bug report.

Eventually I noticed, down at the fine print on the bottom of that page, the link for “if you don’t want to register, click here”. Good. No, I don’t want to register; I want to file a bug report and be done with that.

That gets me to the main customer service web page, which then has a link to “Ask A Question”. It does not have a link to “File a Bug Report”, or to “Make a Feature Request”, or to anything at all which would be an appropriate phrasing for any problem that would require them to do anything whatsoever about it, but that sort of implicit assumption that it’s the customer’s problem seems to be the style these days, so I dutifully clicked on the “Ask A Question” link, and got presented with a web form.

This web form has a fair bit of clever Javascript to make my life “easier”, so that depending on which options I click for my problem, different subcategory boxes appear. Once I’ve got them all filled in, a box pops up for me to select my regional location and enter my institution. (Apparently it’s a barter system; I have to trade demographic information to them before they’ll answer my “question”.) Except that this doesn’t actually work in the version of Opera that I use, so the “Location” box never appears, which means that the “Error: You didn’t enter a location” message that I got when I tried to submit the form was really quite cryptic. And rather hard to work around, too, until I decoded it to “Error: We don’t support your browser,” and tried Mozilla, in which the Javascript works a little better.

Once I entered the question and clicked submit, I still wasn’t done. First, there was the little matter that since I hadn’t registered before, they wanted me to do so now, and were presenting me with the registration form to fill out. I gave in and gave them a password to associate with my email address and the other information they wanted, and again clicked submit.

And, again, I still wasn’t done, because that gave me a page along the lines of “We’ve selected five answers from our FAQ that our pigeons think may have something to do with your question. Do any of them answer it? If not, please scroll down a screen and click the ‘No, these don’t answer my question’ link at the bottom of the page.” Of course the FAQ answers don’t answer my question; it’s a bug report, and it needs someone on your end to do something about it. (This is why I am annoyed with customer service websites that phrase things as “ask a question” rather than “contact customer support”; it seems that it’s a short jump from there to setting up the whole process as if the only things the site has to deal with are questions.)

At this point, it finally accepted the question, gave me a tracking number, and told me that I should expect a reply in 24 hours. We shall see.

So, let’s recap on why this matters (beyond the obvious bug report). A good-sized chunk of my dissertation is going to get reworked into a journal paper or two, and both AIP and Wiley publish journals that would be good places to submit it. One publisher’s customer-service site is a clear, simple, single-page website with a no-javascript online form and a prominently-placed email address. The other publisher’s customer service site comes with its own 650kb(!) user manual in PDF form and makes it a 15-minute complicated web maze to file a trivial bug report. So, whose paper-submission process would I be likely to want to deal with?

Edited to add: I got an emailed reply to all three messages to Wiley, within four hours of submitting the questions. It was polite and clearly from a human being, and made it clear that my initial feature request was noted and relates to things they are currently working on. It was slightly off-putting that the response to my bug report about the web form was essentially “We’ve tested this on all current browser versions, and recommend upgrading your browser” (rather than something like “Sorry about that; I’ll let our web team know, though it may be low-priority for them since it’s not an up-to-date browser”), but other than that, the message was responsive and prompt, and seemed to involve at least as much thought and consideration as I’d put into the messages I’d sent. There’s more to customer service than just the website!

I’ll post some more about these results once I get home. I’ve been working pretty hard on this for the last few months, and haven’t had time to post much about it here, but things should be a bit less hectic soon. My code is finally producing results (luckily just in time for the conference!), and the results are considerably more dramatic than I was expecting — a pleasant surprise!

]]> http://notes.dpdx.net/2006/11/21/slides-from-the-2006-aps-fluid-dynamics-meeting/feed/ 0 http://notes.dpdx.net/2006/10/15/fortrans-50th-birthday/ http://notes.dpdx.net/2006/10/15/fortrans-50th-birthday/#comments Sun, 15 Oct 2006 18:37:53 +0000 Brooks http://notes.dpdx.net/2006/10/15/fortrans-50th-birthday/ IBM’s very first Fortran manual, Programmer’s Reference Manual: Fortran Automatic Coding System for the IBM 704, was dated October 15, 1956 — 50 years ago today. There are a couple of different scanned copies of it available online; bitsavers.org has one, and fortran.com has an autographed one.

There’s an old joke that, in a hundred years, programmers will be using a language that’s completely unrecognizable to modern users — and it will be called “Fortran”. There’s a lot of truth to that; in the first 50 years since this manual, Fortran has changed a lot. Among the signficant changes: subroutines, functions, explicit variable typing, character strings, block IF and DO loops, whole-array expressions, array segments, pointers, dynamic arrays, user-defined data types, pure functions, polymorphic data types, function pointers, generic functions, object-oriented capabilites, and (in the upcoming 2008 standard) arrays shared across multiple threads. Oh, and for those of you who remember the old version: lower-case letters and removal of the “statements must be in columns 7-72 of the line” restriction!

(Update: Regrettably, the article has been withdrawn due to a “serious flaw”. The piece described below is still interesting, however.)

The question that she’s solving is this: Given an infinite three-dimensional domain with arbitrary initial conditions, does a smooth solution to the Navier-Stokes equations (which are the equations describing the flow of water, air, or other fluids) exist on that domain for all future time? This problem was considered one of the most important open questions in mathematics by the Clay Institute, who are offering a one million dollar prize for a solution. It’s of some physical relevance directly, since if it’s not true then either there are cases where the Navier-Stokes equations don’t describe physical reality or else physical reality can behave in surprising ways. The greater significance, though, is that it and the methods that are used to prove it will be useful in further mathematical understanding of this particularly difficult problem.

Even though this is an existence proof and not a constructive one — which means that it proves that a solution exists, but does not provide an analytical method for finding one — there are some interesting construtive portions to the proof. Specifically, she shows that the solutions to the Navier-Stokes equations can be expressed as the limit of a set of hyperbolic equations. There is some interesting physical and computational relevance to how that works, which I thought would be interesting to more mathematically-inclined readers.

To start with, the Navier-Stokes equations are differential equations in four variables — pressure (p) and three directional components of velocity (u_i, for i=1, 2, or 3). They look like this:

There are some interesting features of this set of equations. First, there are four equations and four unknown variables, so it’s a closed set. However, if we start matching up variables with equations on a one-to-one basis, we find that the first three equations match up neatly with the three components of velocity, leaving the last for the pressure — even though it doesn’t actually have pressure in it (much less a time-derivative of pressure)!

What happens, then, is that at any point in time, the first three equations describe the time derivative of the velocity, as a function only of the local field at the current time (that is, they’re hyperbolic). This means that in these equations information travels across the domain at a finite rate, which is a very nice feature computationally; one can compute a solution at each grid point without needing to know the solution everywhere else. It’s also handy mathematically.

The fourth equation, though, messes all of this up. What it says is that the pressure needs to be chosen such that this velocity condition is satisfied everywhere. Physically speaking, if the velocities would be trying to compress a certain spot, the pressure there gets increased to counteract that, so it doesn’t compress. This isn’t a local constraint, because it’s the derivative of the pressure that affects the velocities, and so a change in pressure in one place causes immediate changes throughout the whole domain — essentially, the speed of information transfer is infinite. In a computational simulation, the vast bulk of the time is usually spent taking care of the pressure.

So, how does Penny Smith’s analysis approximate this by a set of hyperbolic equations? There are two parts to that. The first part is that she needs a set of first-order equations for her analysis, and this contains a second order term. Fixing that is straightforward, we simply introduce more variables, and a mathematical identity. The (nine) new variables are the components of a “viscous stress tensor”, defined as:

This suggests a rather simple computational method: solve the problem with a small but non-zero value of ?, and thereby obtain a solution that’s close to the desired one. This used to be a fairly common method, because hyperbolic problems are relatively easy to solve, and it’s called “artificial compressiblity”. It works reasonably well in some cases, although as ? gets closer to zero, the rates of information transfer around the domain get faster, and so the equations take more work to solve computationally. And, of course, if you use a larger ?, the rates of information transfer get slower, and so the solutions aren’t as close to the real solution. But it’s reasonably good for some problems.

Now, in Penny Smith’s case with everything in first-order equations, there are some additional equations that aren’t in hyperbolic form, so she does exactly the same thing to those equations, adding a time-derivative term to those as well:

(Note: Equations in this article were typeset with MimeTeX.)

I’m sure of this, even though I have no idea how this engine is supposed to work. Why? It’s a matter of applied thermodynamics.

First, imagine that the engine is in a black box. We can’t see into the black box, so it could be any possible type of engine that has been or will be invented. All we can measure is what goes in and out.

What goes in is basically just the “fuel”: room-temperature water. There might be some air with it, too. There are a couple of things that come out: The exhaust from the engine, and the power from the engine — maybe in the form of a shaft coming out of the engine that it’s turning, or something.

According to the First and Second Laws of Thermodynamics, available energy cannot be created, though it can be destroyed by turning available energy into unavailable energy. If we apply that principle to the box with the engine in it, it’s pretty clear that the sum of the available energy that goes into the box and what is in it when we start running it has to be at least as much as the sum of what comes out and what is in it when we finish.

Now, this is an “engine”, not a storage system; it’s not allowed to be running off some sort of hidden internal energy supply that’s also contained in the box. So, if the engine isn’t cheating, the amount of available energy in the box when we start will be the same as the amount when we finish. Thus, we conclude: what goes in must be at least as much as what goes out.

So, look at the available energy that’s in that exhaust. Whatever the exhaust is, it has to contain the same hydrogen and oxygen atoms that came in with the water, along with the atoms that came in with the air. It could hot steam — which has higher available energy than the water that came in. It could be some other compounds of hydrogen and oxygen and stuff in the air — and all of those have higher available energy than water and air do. It could be a cool wet fog, which has slightly higher energy than the water. It could even be ice, but even that has higher available energy than room-temperature water. The best possible case is that it could be room temperature water and air just like what came in, and have the same available energy.

That means that the available energy that comes out in the exhaust is at least as much than the available energy that goes in with the fuel and air. And there’s supposedly even more energy coming out via the power from the engine — thus, the output is claimed to be higher than the input.

This is obviously a contradiction. We just found that, by the laws of thermodynamics, we can’t have more energy coming out in the exhaust and power shaft than what came in with the fuel.

Thus, the engine is impossible.

The idea that a recipe (eg, a best practice, a methodology, or a maturity model) is completely worthless is just as wrongheaded as the idea that everything should be based on a strict recipe. In other words, even an expert chef may occasionally find it helpful to refer to a recipe card.

There’s no reason these two models can’t coexist. You should always start with the common denominator recipe, of course, but you may want to provide some alternative guidelines and ideas for those cooks who have outgrown traditional recipes, too.

I strongly agree with his general point, but I think that “guidelines and ideas” are the wrong sort of thing to include.

To take the metaphor literally, for some things, I’m a level 4 cook. (Not necessarily an especially good one — it’s worth noting that these levels are not always parallel to the quality of the output!) A particular example would be some coleslaw I made a couple of weeks ago, when I found myself in an ill-stocked kitchen with a cabbage, an empty stomach, and the Joy of Cooking.

The Joy of Cooking is a pretty good cookbook for this sort of thing, as it turns out. It has a half-page discussion of what coleslaw is, without any recipes at all. It turns out that pretty much the only common factor in slaw is that it has cabbage in it. Everything else is variable. (This was an important point; we had no mayonaisse, and the person I was cooking with insisted that coleslaw has to have mayo.) It went on to describe, in loose terms, some of the common varieties. I skimmed through those, and then went on to look at the actual recipes. The standard recipe format is well-optimized for this sort of thing; there’s a list of ingredients at the top, separate from the instructions, making it very easy to skim to pull out specific bits of data such as the ratio of sugar to vinegar in a sugar-vinegar slaw.

After looking through the cookbook, I’d gleaned the following information: sugar and vinegar are a perfectly sufficient base for a slaw dressing, and it’s common to have them in X ratio, Y is a typical amount of salt, and Z, W, Q, and a dozen other things are some ideas for spices to put it.

Those, you may object, look exactly like “guidelines and ideas”. So, why am I objecting to including guidelines and ideas for experts in the recipe? The same reason that specific instructions are wrong for experts: They are processed information, and the definition of “expert” in this context is someone who does their own information processing. In my case, when I read the recipes, I was mining them for bits of information. Any processing that complicates that information-mining is a detriment, not a benefit.

And so, when writing recipes to be useful beyond basic instructions, the thing to include is facts. Not a guideline of “this squash should be cooked for 20 minutes”, but the fact of “if you cook this squash for more than 25 minutes it will be mushy and begin to lose flavor.” Not the idea of, “You could use peaches in this,” but the fact of “Peaches are also common in this.” The guidelines and ideas are certainly useful because they contain facts, but the parts that make them “guidelines” and “ideas” rather than bare facts rarely are. Often the process of distilling the fact into a guideline or idea loses useful information, too; suppose I wanted mushy squash, or wanted an uncommon alteration?

I’m going to claim some authority in talking about this, because I’ve done that — and it wasn’t just 20 or 30 hours, either. A year ago, I thought that there was some chance that I could finish my dissertation by the end of the academic year if I got a certain set of simulations done, which required taking a computer program that was currently held together by Duck-brand Fortran Tape, and turning the structurally unsound mass of the core algorithms into a well-oiled robust, solid, machine that would do what I told it. I had two weeks in which to either accomplish it or give up.

So, I sat down on Monday to start working on it, and for all practical purposes, I worked on it straight through until Wednesday of the following week.

This is the first flaw of the Nerd Bravado claim. Hyperfocus doesn’t care if you sleep and eat and take a shower. You can think while you’re eating. Your brain keeps working on things when you sleep, and if you think about a problem when you go to bed and when you get up in the morning, you’ll often find that some tricky bit of it has become a lot clearer in the interim. You don’t have to sit for 25 hours straight to get the benefits.

That’s where the benefit of this style of working is, insofar as there is one: Hyperfocus. After a dozen or so hours into this project, I was so focused on it that I couldn’t really think about anything else — even if I tried, the problems in the code would keep distracting me back to the task. And, because I’d pretty much emptied everything else from short-term memory, quite a lot of stuff about the code could fit in there — enough to contain all the complicated interweavings of how the code all fit together. I could look at the results and see where the problems were coming from and what bits of subroutines wove together to create the particular effect that was happening, and I knew, without having to look, where to prod the program to change it, and when I prodded it, it changed. It’s a heady feeling.

I think that’s what the appeal of Nerd Bravado is, really. It’s like spice, from Dune. Get a bit of it into your system, and the universe opens up before you and whirls at your beck and call, and you have tremendous power at your fingertips to poke exactly the right thing in exactly the right place. But it’s also got downsides; do it a few too many times, and your eyeballs turn funny colors and it completely eats your life.

Now, I said I had two weeks to fix this program, and I stopped a week and a half in. Do you think that means I finished early? Hardly. A week in, and I’d gotten all the easy bugs fixed and most of the complicated ones, and I’d started tangling with the algorithms at the core of it. And they had a problem: The math that I was working with involved solving an equation where significant parts of both sides are multiplied by a factor that goes to zero. Divide that out to solve it, and very small errors in one term become very big errors in the solution, and it stops being well-behaved. There are ways around this, but they have to be designed into the algorithm from the start, not bolted on later. And by about Wednesday, it became quite clear that I was dealing with something that needed to be fixed from the ground up, and there was no way I could come close to that in the three days I had left.

This is the second flaw of the Nerd Bravado claim. It doesn’t actually solve the real big problems. Hyperfocus is a state of working on instincts, of having enough information actively in your brain that you don’t have to work through logic and deductive reasoning to know what the Right Thing To Do Now is. And that’s great, if what you need to accomplish is something that’s a chain of consecutive steps, like debugging a convoluted program or working through a complicated mathematical derivation or even writing a new program that fits a general specification that you’ve already determined.

Real big problems aren’t like that. Flash-of-insight problems aren’t like that — Einstein didn’t have his flash of insight about time dilation on the trolley because he’d been hyperfocusing on the problem for the past few days; he had it because he’d been rolling the problem around in his head for months. (Or years; I don’t actually know that bit of his biography well enough to say for sure.) And hard-work problems aren’t like that, either — writing a fixed version of my program isn’t going to work like that either, because it needs to be designed right from the ground up, and designing something right is a process that requires awareness of the outside world, and of a lot more than what fits in the hyperfocus. Yes, there are the legends of the great designers who sit down and in one day lay out the basic lines of an airplane or a car engine that are more right than their less-great colleagues create with a committee and a month, and that comes from having tremendous amounts of knowledge in their heads, but that’s the sort of knowledge that gets there over years, not hours.

This brings me to the third flaw of the Nerd Bravado claim: It’s not, overall, worth doing. When I’d finished my week-and-a-half of debugging, I was mentally exhausted. I was a week behind on all of the minutiae of daily life, I was emotionally starved for interpersonal interactions, I’d barely talked to my wife about anything that mattered to her, and my brain felt thoroughly fried. (My wife’s ironic laugh when I asked her if she remembered when I did this probably tells you all you need to know about what it was like from the outside.) It took somewhere around a month before my work productivity was anywhere near normal again. All totaled, I’d probably have accomplished as much if I’d simply worked at a reasonable pace for the month and two weeks, and I’m pretty sure the resulting solution would have been worth a bit more, as I’d have had the time to actually work out at least a first cut at a ground-up solution.

And, finally, the fourth flaw of the Nerd Bravado claim: the claim that “you must do it at least once to be a successful engineer … and to gain the respect of your peers” simply doesn’t hold true. The result of this particular experience, for me, was that I had a nicely debugged program that solved the equations the wrong way and didn’t produce useful answers. And all I learned from it was this: That style of working isn’t sustainable, and doesn’t lead to an enjoyable life, and I don’t want to do it again except in very small doses. As for the respect of my peers — well, does this cause any of you to respect me any more? No? I didn’t think so.

This has interesting ramifications for language design. A language is not defined by all of the syntax and semantics that are in its programs. It is, instead, defined by its primitives and by the structure upon which the programmer can add to the language.

For example, here’s a short Fortran program:

program sample
  use vectorModule
  real :: b
  type(vector) :: V
  call loadFromFile(V, 'myfile.dat')
  b = abs(V)
  write(*,*) b
end program

Fortran defines the words that are highlighted in red, and the punctuation; everything else is something that I’ve defined either in this program or in the matrixModule module that gets included by the second line. Some of these definitions are so common that we don’t even think of them as “language extensions” any more — for instance, the function and subroutine definitions. Beyond those, there’s also the user-defined “vector” type, and the extension of the absolute-value intrinsic to apply to it.

The invariants of the language are more interesting, though. Variables are declared with a “::” syntax, and must be declared before any executable statement. User-defined variables must be declared as “type(vector) ::” rather than just “vector ::”. Subroutines cannot be used as statements alone; they are prefixed with a “call” keyword. Expressions, likewise, cannot be used as statements — and an assignment is a statement, not an expression. These requirements are at the core of what can and what cannot be a Fortran program.

One interesting question is: What happens as we reduce the number of syntax invariants of the language?

Consider the TeX typesetting language. What if we had a programming language that worked like TeX?

TeX is, to a rough approximation, a two-level language. The first layer is a macro expansion language; the user’s code (along with some system libraries) is processed through a macro system that converts it into a stream of TeX primitive commands. These primitive commands are then processed through a second layer, which converts them into the typesetting equivalent of machine code. It would be easy enough to imagine a language like this where the second layer produces compiled programs rather than typeset pages.

This has some interesting results in how the TeX is used. Knuth expected that what he had written was largely a proof-of-concept language, and that people would rewrite the basic language processor to add extensions to it. However, the macro expansion language he wrote is sufficiently powerful that nearly any desired language extension can be written as a macro. (There are a few exceptions, surrounding things like file input-output and bits of math typesetting that are implemented largely in the second layer, but they are rare.) And, given the choice of writing a macro that works with the existing system, or creating a new system, people have generally chosen to write macros. Even the LaTeX typesetting langauge, which is sufficiently different from TeX as to be nearly a distinct language, is merely a large set of TeX macros that are treated as a system library.

Another benefit is that it makes experimentation easy, and makes it portable. To write a new bit of language syntax, I simply write a few macro commands and put them at the top of my file — and, so long as those commands go with my file, it will run on anyone else’s TeX system.

These would be good traits to have in a programming language.

Thus, the question is: What is an appropriate set of primitives for a programming language? And what is a good way to structure a TeX-like macro language that will produce them?

Presentations at APS meetings are only 10 minutes long, instead of the 25 at most other conferences, so these slides are much tighter and more concise than the ILASS slides. I think there are definite advantages to the format; most of the presentations still seemed to contain all of the critical aspects of the research, and most of what seemed to get cut was excessive belaboring of points, and recitations of the same justifications and basic background that we’ve all heard dozens of times before. And, of course, it means that there’s time to see more than twice as many presentations in a day; when the conference packs over a thousand talks into a three-day block, that’s quite important.

One unfortunate thing about the APS DFD meeting is that the talks don’t come accompanied by papers that one can look up and read afterwards. For my presentation, though, my paper from the ILASS meeting covers most of the same results.

Oh, and at some point I’ll work out an appropriate Creative Commons license for this stuff. In the interim, if you want to borrow one of the slides or figures for something, just send me an email.

A 1997 Geo Metro is one of the cars with an integrated headlight-and-signal-light unit that’s accessed from the front corner of the engine compartment, so I popped the hood and went to work. The lens is held on by a couple of obvious clips, so I removed those. No luck; obviously there was something else that needed to be unfastened on the bottom, too. The next obvious thing to undo were a couple of large screws holding the entire light unit on. I removed those, and still no luck — there was something holding it on the outside bottom corner, which I couldn’t figure out how to get to.

This, I concluded, was the appropriate time to read the documentation, in the form of the owner’s manual. The owners manual explained that one didn’t actually remove the lens or the housing at all, but replaced the bulbs by reaching in behind the light housing, unfastening the socket, and pulling it out. It went on to say — and this is only a mild paraphrase — “Reaching the socket to unfasten it is actually physically impossible unless you are a leprechaun. You will want to take your car to the dealer and let them do it.” Back to plan A.

Some more investigation found the third screw holding the headlight unit in place, hidden in a cavity behind the bumper that I could either look into or reach into at any given time, but not both. So I removed that while contemplating the pipes running across the ceiling, and pulled out the headlight unit, and pulled the socket out of the now-accessible back of it, and discovered a small light bulb entirely unlike the one that the parts store had sold me to replace it.

A small light bulb which, to all appearances, seemed perfectly intact and functional.

Some further testing of the corresponding lights on the other side of the car informed me that what I had just gone to all of that effort to access was not, in fact, the turn signal light at all, but merely the parking light, which was just as functional as it appeared.

So, thus enlightened, I reassembled the headlight unit with only slightly less effort than had been required to disassemble it, and then replaced the actual turn signal bulb, which is in the middle of the front bumper behind a lens that’s held in place by two quite obvious and quite visible screws that are, as it turns out, entirely trivial to remove and replace.

• The “listings” package for LaTeX is invaluable for including computer source code in a document. It’s something like syntax highlighting in a source code editor, but for typeset output rather than on-screen, and very easily user-customizable. Carsten Heinz spent almost a decade writing it and perfecting it, and it’s a marvel of intricate TeX programming — 218 pages of typeset user documentation and source code. Then, about a year ago, Carsten disappeared. As far as I know, nobody in the TeX community knows what happened to him; he simply stopped replying to email, and letters to his last known physical address just disappear. As a result, the official version of the listings package has effectively been frozen for the last year, despite a couple of important bugfixes that have been informally circulating. Luckily, this isn’t a permanent situation; it’s licensed under the LaTeX Project Public License, which provides a process by which the maintainership of the “official version” of a package can be transferred to someone else in such a case.
• In a blog post I was reading this morning (I’m leaving this vague so as not to point fingers excessively at only one example of a common problem), the author mentioned a general-purpose data structure that he had included in his open-source simulation code, and provided a link to the source file for the implementation. Like many such implementations, it’s sufficiently general that it would be useful in many programs that I’m writing, and it’s a better implementation than I could come up with (and test!) in a day’s work. However, there’s a problem: The license for this software package is not quite the same as any common software license, and it requires that its particular set of conditions and disclaimer be included in any redistribution. Further, the license has the relatively common clause that any scientific publications which contain data resulting from use of the program must cite it.

I’ll start with the last item. It’s completely reasonable that the author of a piece of scientific software would want to be cited when their work is used, so what’s wrong with putting that in the license? The answer is a lesson that should have been learned years ago. It only makes sense for cases where what’s being redistributed is pretty much the same as the original work. Beyond the well-known problems described at that link (short form: someone else builds on it enough to merit citation and adds their paper, and after a few repeats of that it quickly becomes unweildy), there’s the problem of partial reuse: if I include this little data structure implementation in my program, according to the license I then have to cite their paper in all of mine, even though my work is in a quite unrelated field. And so does anyone who uses my program. While that might arguably be appropriate for an advertising clause, it’s not appropriate for scientific citations.

So, functionally, I pretty much can’t reuse their data structure in my own software, if I wish to do so under its published license. That’s completely contrary to the spirit of free software; even though it’s open source and I can read it, the license prohibits me from simply copying it and reusing it unless I’m willing to agree to inappropriate terms. This almost certainly isn’t the intent of the program’s author; he simply didn’t consider the possibility of someone reusing a small piece of his program when he decided how to license it.

In this case, there’s probably a simple solution. If I want to use this code, I can write to him and ask for permission to use it under a different license than the one it’s published under — specifically, one which doesn’t contain this citation clause. Probably he’ll agree, and that will be that.

However, this is where the first example is relevant. What if it was something in Carsten’s code that I wanted to reuse, and he had a similar sort of clause in his license? Plain and simple, I’d be stuck. Even if I were willing to ignore the license on the assumption that his heirs wouldn’t find out or wouldn’t care enough to sue me, I certainly couldn’t relicense the code with a clear conscience.

That’s why license compatibility is important, and why it’s a bad idea to add private clauses like the “cite my paper” clause. Without it, free software isn’t a commons — it’s a balkanized set of little gardens fenced off from each other and unable to cross-fertilize. We’ve had this problem with railroad tracks, with fire hoses, with all sorts of things early in the industrial revolution; as engineers, you’d think we’d know better by now. Four feet eight and a half inches probably isn’t the optimal solution for any given railroad in isolation, but it’s certainly better in practice in the real world for nearly all of them, because it’s what everyone else uses, and in the long run it’s just not worth the trouble of being different.

“But,” you say, “I still want people to cite my paper when they use my software!”

Consider this: Most researchers and users of your program are, by and large, honorable and honest people. They understand the value of citing sources, and they appreciate that you’ve written this software and made it available for them to use. For the vast majority of them, all that you need to do to get them to cite your paper about the software is to ask.

License terms aren’t “asking”. They’re a demand, one with legal teeth behind it. They’re what you put in there for the people who you don’t trust to do right, and for the corporations that are too big to have morals at all. They ought to be the final line in the sand for the things that are vitally important: you don’t claim you wrote this, you don’t claim I’ll support it, you don’t put restrictions on it that I disagree with. If it’s not that critical, and if you wouldn’t sue someone for ignoring it, then setting it in that sort of legal inflexibility does more harm than good.

Thus, I propose the idea of adding a “Requests” or “Moral Obligations” section to software licenses, for this sort of thing. When I release a program that’s large enough to merit a paper citation if someone uses it, somewhere near the license clause in the documentation will be a section that says something like this: “The requirements in this section are not legally binding; however, they represent things that the author would appreciate: If you write a paper based on work that uses this software, please include a citation to my paper, and send me a copy of your paper. If you create a derivative work based on a this software, tell me about it and let me know how I can get a copy, and include this request if it’s appropriate.”

In virtually all cases, that will work just as well as adding those as terms to the software license. (How do I know it will work? Well, it already works quite well for the papers themselves, though there the request is implicit rather than stated.) And, importantly, it will work equally well in the cases where something I asked for turns out to be terribly inconvenient for some reason that I didn’t think of when I wrote it, even if I’m not around to change the license terms.

The basics of the port were pretty much the same as last time. There are a couple of key problems to work around to get OpenFOAM to compile on Cygwin. First, it’s written for a case-sensitive filesystem, and has files named “Vector.H” and “vector.H” in the same directory, and suchlike. This clashes when one tries to unpack the source distribution on a case-insensitive filesystem, and then when one goes to compile it, the #include lines in the source files pull in the system’s “vector.h” file instead of either of them. Fixing this is a tedious but trivial matter of renaming all the relevant files, and fixing all the #include lines. Someday I’ll write a script for it, but this time I did it by hand with the disk-search-and-replace in my text editor.

The more interesting problem involves how dynamically-linked libraries are compiled and linked on Windows versus Unix and Linux. OpenFOAM is, functionally, a big set of libraries that get linked into application-specific (and often user-written) wrapper programs. Most of the core functionality is in a libOpenFOAM library, although the parallel-processing parts are off in a libPstream library. This allows for different parallel implementations to be used depending on which of various libPstream libraries is actually used at runtime, while having only one version of the libOpenFOAM library. Unfortunately, this is written with a bit of a circular dependence; libPstream uses a lot of data types defined in libOpenFOAM, while libOpenFOAM needs to link to libPstream.

On Unix and Linux systems, this doesn’t actually present any difficulties; one simply includes the relevant header files when compiling the two libraries, and then all of the class methods from libOpenFOAM and the parallel functions from libPstream get resolved when the final wrapper code is compiled and the two libraries are both linked in.

It seems that this fundamentally doesn’t work on Windows, however, because (as I understand it), a dynamically-linked library on Windows cannot have unresolved symbols. All of those functions need to be resolved at compile-time, even if they get resolved differently at runtime. And so we can’t compile libPstream first because it needs to link to libOpenFOAM at compile-time, and we can’t compile libOpenFOAM first because it needs to link to libPstream at compile-time. This is a bit sticky.

The solution to this is a bit of modification to the OpenFOAM make scripts that Petr Vita created and posted to the OpenFOAM forums earlier this year. I don’t quite understand the details yet, but he’s written something that allows the make script for one library to “import” the source files from another library and include them, and thus the compilation of libOpenFOAM can reference the relevant libPstream functions that way, before libPstream is compiled. Then, once that’s taken care of, we can compile libPstream in the usual way by linking to the libOpenFOAM library, and all is well.

This time around, that seemed to work a lot better than it did in OpenFOAM-1.2 – or perhaps I simply had better luck with the compiler. In any case, in OpenFOAM-1.2, it didn’t work to import the (only three) libPstream files into libOpenFOAM, and instead Petr had to do things the other way around, which involved dozens of libOpenFOAM files getting imported. And then the libOpenFOAM files had to be imported into another library as well, which should have compiled fine without them, but apparently didn’t. This time around that one worked just as it should with just a link to the libOpenFOAM library, so now I’m wondering if perhaps something was just mixed up on Petr’s computer that day.

The rest of the port involved a few other things that shook out because of things that I’m guessing were the result of this Windows pickiness. There were a couple of cases where an additional #include was needed in a source file to get the resulting library to link, including one case where the relevant function prototypes weren’t actually in a header file at all. And then, there were a vast pile of cases where additional libraries needed to be given to the linker explicitly – presumably as a result of the fact that on Linux the link process didn’t complain if a needed library was omitted, and there weren’t any runtime failures if the library got referenced by the relevant applications somehow – so there was a bit of trial and error fixing those. Someday I should look up the relevant gcc options to tell the linker not to give me ten thousand (literally; Windows gives me a maximum 9999-line scrollback buffer, and it got overflowed) lines of errors, but instead stop reporting after the first five or so.

But, in any case, that’s all done, and I’ve uploaded the files and posted an announcement post on the discussion board. Now to see if it works for anyone else!

Dave Thomas suggests in his Code Kata post that one way of practicing programming is by solving small-but-complex algorithmic problems, and he has a list of suggestions.

In the shower this morning, I was thinking about a problem that would fit nicely in that list. I’m writing a complicated bit of code that takes the triangles in a triangular mesh and loads them into a data structure. It would be nice to test it by taking a given mesh and loading the triangles in various random orders, and that immediately leads to the classic card-shuffling problem: given a list of objects, how does one efficiently sort them into a random order?

With cards, this is relatively easy at first glance: Put the cards in a stack (i.e., a linked list). Pick a random card from the current stack, and use it to start a new stack. Pick another random card, and put it on the stack. Repeat until the original stack is empty.

In practice, this doesn’t work so well: people aren’t very good at picking really random cards. Or anything else random; this is why random play doesn’t win rock-scissors-paper contests. But that’s a digression; we’ll assume that we have a “good enough” random number generator to use, just like a computer would.

Having a random number generator doesn’t solve the problem, though. It picks numbers, not cards. There’s still the problem of mapping the number to the card.

The trivial solution is to number the cards in the stack from 1 to 52 (thereby making it an indexed array), get a random number from 1 to 52, and pull out the corresponding card. That works nicely for the first card, but then we’ve got a stack of cards that are numbered from 1 to 52, but one of them’s missing. Now what?

We could renumber the cards after the one we pulled out, reducing each one’s number by one so that the deck is numbered from 1 to 51, but that’s still a lot of work – for the whole deck, it’s O(n²).

(Now that I’m writing this out, I see an obvious solution here that I didn’t see earlier. I’ll get to that later.)

Alternately, we could give up on renumbering the cards, and just leave them in a linked list. Then, when the random number generator gives us a number, we count down that many cards from the top of the deck. Again, for the whole deck, we end up counting O(n²) cards.

So, consider an alternate approach: Take the top card off the stack, and put it in a random place in the new stack. This looks like the same problem – how is locating a random place easier than locating a random card – but there’s a twist. Because we’re doing this in a computer, we can keep the cards in two orders at once. We can store them in one order that makes them easy to find, and simultaneously in another order that’s randomized.

First, consider the new deck as a linked list; we pick a card randomly, and insert the new card after it. On a pointer level, this means that the new card points to what the old card had pointed to, and the old card now points at the new card. The trick is that, when picking a random card, we don’t have any reason to care what order they’re in in the linked list. So, if we’re actually storing the cards in an indexed array, and putting each new card on the end of the array, then all we need to do is pick a random index and go straight to that card. Once all the cards are in the new deck, then we can traverse the linked list and pull them out in the shuffled order.

There’s a bit of nice optimization, too; we could pull cards off the bottom of the old deck just as easily as off the top, and since we’re putting them on the top of the new deck (insofar as actual storage location), there’s no need to actually move the cards at all; everything below the current index is the new deck, and above it is the old deck. If the cards are just numbers, there’s no need to even store the cards at all; we just store the pointers.

So, how does that actually get implemented? Start at the beginning of an indexed array of pointers (which are really just numbers). Increment the index. Pick a random bin below the current index. Take the number from the random bin and put it in the bin corresponding to the current index. Put the current index as a pointer in the random bin. Repeat until done.

…

So, back to that parenthetical note from earlier. There’s really no need to renumber all the cards, because they don’t have to stay in the same order. We could just take last card on the deck, and renumber it to fit in the gap, and be done. So, we take a random card out of the deck, put it at the first spot of the new deck, and then move the last card from the deck into the hole that got left. Or, to do this in the same space, we could just pick a card, set it aside, take the first card from the deck and set it in the hole, and put the picked card in the now-empty first spot. Now, the “new deck” is the first card, and the “old deck” is the rest, and so forth as we go along.

How does that get implemented?

Start at the beginning of an indexed array of card numbers (which are essentially pointers). Increment the index. Pick a random bin above the current index. Take the number from the random bin and put it in the bin corresponding to the current index (i.e., the end of the new deck). Take the current index (i.e., the card from the end of the old deck) and put it in the random bin.

An interesting similarity, isn’t it?

The bad news: I know this, of course, because they’ve posted a bug report to say it doesn’t work for them.

Turns out that a large part of the problem was fairly obvious in hindsight. The original OpenFOAM package contains environment setup files for both bash and tcsh, and I had modified them both for use under Cygwin. However, since I use tcsh, I had completely neglected to test the bash scripts, and there were several typos in one of them. The other problem was that, since the OpenFOAM port uses a local copy of GCC 4.0.2, rather than the Cygwin system copy of GCC, this user had decided to uninstall his system GCC to avoid any possible confusion – and thereby flushed out a bug where the make system was hardwired to use the C preprocessor installed as /bin/cpp, rather than the one in the local copy of GCC. This one dates back to the original Linux version of the software, so I’ve in turn filed a bug report on that.

Aside from that, there were a number of complications that amounted to reminders that, whenever the build process goes awry, it’s important to clean up any incomplete results so that they won’t perpetuate the problem after the actual bug is fixed.

So, it looks like everything is fixed now, but I’m waiting for a final confirmation that it all works before I push all the bugfixes into a new version and post it.

Some time ago, I posted a comment on This Is Broken, noting my frustration with motherboard-mounted AC power jacks on laptops, and noting that I had fixed a couple of them by following some online directions. Several people asked me to explain further, and I decided it would be useful to write up a more detailed explanation of how I did it.

First, the keywords: My experience is with a Compaq Presario 1210 laptop (which is very similar to the Compaq 1600 series), and a pair of HP Pavilion xf235 laptops, but this should apply to most similar laptops; the advice is mostly generic.

The problem, in short, is that most laptops have their AC power jacks hard-mounted to the motherboard just like any other soldered-on component, and for many of them this is the only structural connection. Unlike most other components, though, power jacks regularly get yanked on when someone steps on the power cord or such, and the solder joints (or the mounting tabs, which after all are pretty thin metal) start to fatigue, and eventually they crack. At this point, the laptop will only charge if the plug is twiddled “just so” (if at all), and doesn’t work reliably.

The official solution, of course, is to replace the motherboard — which fixes the symptom, but not the overall design problem. On a laptop with a street value of $300, this is also not an economical solution. But it does mean that you don’t really have anything to lose if you toast the motherboard trying to fix it yourself.

So. Enough preamble. Here’s how I fixed the problem.

DISCLAIMER: THIS IS A DESCRIPTION OF HOW I DID THIS REPAIR. FOLLOW THESE INSTRUCTIONS AT YOUR OWN RISK, AND USE YOUR OWN JUDGEMENT; I MAKE NO WARRANTY THAT YOU WILL HAVE A WORKING LAPTOP WHEN YOU FINISH. SOME MINOR STEPS HAVE BEEN OMITTED.

On the Compaq, I used the solution that I found on http://www.delanoscientific.com/cpq1675.html. Essentially, you take a three-inch strand of some form of heavy-but-flexible two-conductor wire — that site recommends a good grade of speaker wire, but I happened to have a dead iBook power supply that I salvaged some wire from — and solder a socket on the end of it that matches the plug on the power cord, feed it through the original power jack hole in the case, and solder the other end to the motherboard where the original power jack used to be. One trick is to tie a knot in this new pigtail just inside the case, so that pulling on it only pulls on the knot, rather than pulling on the solder connections.

On the two HP Pavilion computers, I found a better solution. There is a hole in the case, right next to where the power jack goes, that’s intended for a PS/2 keyboard or mouse plug on other versions of the laptop. However, it’s unused and blanked off, which makes it a great place to put a case-mounted power jack — and it turns out that a standard 1/2″-diameter panel-mount jack fits perfectly. I used a Philmore No. 248 “DC Power Jack”, which has 2.5mm x 5.5mm pin and socket diameters, thereby matching the original plug. While you’re at the electronics shop, get a decent soldering iron and a power supply for it that lets you control the temperature. If you haven’t done much soldering before, find some sort of discarded computer-board junk that you can practice on.

Here’s what the new case-mounted jack looks like when it’s installed; the original power jack hole is just to the left of it:

Actually doing the replacement took me about two hours, on the last one I did. The time-consuming part is that you need to completely disassemble the laptop to remove the motherboard. This starts (on the HP Pavilion computers) with gently popping out the panel above the keyboard that has the CD playback and volume controls on it, and continuing by removing every screw you come across. A digital camera is a very useful timesaver here; photograph the screwdriver pointing to every screw you remove, put the screw next to a number on a numbered piece of paper, and photograph it there — then, go through the photos backwards and you have a detailed set of reassembly instructions.

When you have the motherboard all the way out, the next step is to remove the remnants of the old power jack, and solder on some one-inch-long wires (or the pigtail, if you’re not using a case-mounted jack; remember to feed it through the case first!) in its place.

Here, it’s worth noting that the original power jack has three pins, and acts somewhat like a switch. One pin goes to the ground, and becomes connected to the power-adaptor ground when the adaptor is plugged in to the laptop. The second pin is the +19.5V (or whatever the power voltage is) line; when the adaptor is not plugged in, it becomes connected to the ground pin. The third pin is a logic pin for determining when the adaptor is connected; it is open-circuit when the adaptor is not plugged in, and connected to the ground pin when it is. (Unfortunately, I didn’t keep a record of which pin goes with which connection to post here; sorry.)

This could theoretically be problematic with the pigtail-mounted socket, as that socket only has two connections — the logic line for sensing whether the adaptor is connected does not exist. I simply ignored this problem on the Compaq laptop, and the computer seemed to work fine; it may or may not work properly on other computers. On the case-mounted jack, there are three lines, so this is not an issue.

Meanwhile, speaking of the case-mounted jack for the HP Pavilion laptops, there’s the issue of attaching it in the case. Remove the blanking plate from the case by appropriate means (I did this by grinding off the heat-swaged mounting pins), and install the jack in the hole that this leaves. You’ll need to remove a bit of material from the inside top of the jack to prevent interference with the top half of the case later; now is a good time to do that. Also, if you want to be fancy, you can cut a small rectangle out of the blanking plate and superglue it in place to cover the old power-jack hole.

Anyhow, you then reinstall the motherboard (with its new wiring) in the case, and — if using the case-mounted jack — connect the wires to the jack. On the HP Pavilions, make sure the wires aren’t over the LEDS immediately behind it on the motherboard; there are clear-plastic “light pipes” that have to go on top of those, and the wires can easily get in the way. Then reassamble the laptop, and that’s “all there is to it”.

In comments on the original version, someone mentioned that http://www.laptopjacks.com sells exact replacements for the original laptop power jacks, if you want to go that route, though they’re rather more expensive than the generic parts. Other people mentioned repairing the following laptops (which I’ll list for the sake of Google) with this advice: HP Pavillion 1150, Compaq Presario 1685, HP ze1115, HP xt412, HP Pavilion ZE43xx series, Compaq 2195US, and a Compaq 2100. I’m particularly entertained by the fellow from Montana with the Compaq 2100; he couldn’t find the right replacement fitting, and so ended up using a trailer-hitch plug set from the local truck stop.

In Part 1 of this series, I talked about why it’s difficult to do an accurate computer simulation of a single drop dripping from a faucet. That’s a very simple example of a free-surface flow, however, and many of the flows that are relevant to other scientific or engineering questions are far more complicated.

One general class of more complicated free-surface flows are “sprays”, also referred to as “spray flows” or “atomization” – a liquid is forced at relatively high pressure through a small nozzle (or set of nozzles), and when it comes out it breaks up into tiny droplets. Sprays are nearly ubiquitous in any handling of liquids; for instance, in my day-to-day life, I might take a shower in the morning (under a spray of water from the showerhead), wash the shower walls with a cleaner in a spray-bottle, use an aerosol spray can of cooking oil to oil the pan to cook my eggs in, and wash the pan using the spray attachment on my sink. My car has a fuel-injected engine, which means that there are nozzles in it that spray the fuel into the air intake. If I were to paint the (sadly rusting) hood on it, I’d use a spray can. The train that I take to work has a diesel engine; the term “diesel” means that in it the fuel is sprayed directly into the engine’s cylinder, where it burns immediately as it’s being sprayed in.

In every one of those cases, it’s important to get the spray “right”. The shower-cleaner, cooking-oil, and paint sprays need to provide an even coating on the walls without spraying any off to the sides. The shower head needs to produce rather large drops that act like rain rather than fog. The fuel injectors in my car need to produce tiny droplets that will evaporate quickly, while the ones on the diesel locomotive need to produce droplets of the right size to get to the middle of the cylinder before they burn up. Thus, it would be very useful to be able to simulate the spray that comes out of a new nozzle design, so that it can be tested and improved without needing to do costly experiments.

As an example to talk about, here’s a fairly typical spray that’s easy to photograph – water being squirted through the nozzle from a Windex® bottle. I should credit my wife for patiently helping me take the picture; it took quite a few tries before I managed to get the flash timed correctly with the spray!

(click on the photo for a larger version)

Dilute Parts of a Spray: Lots of Little Droplets

One of the most obvious complications in simulating a spray like this is the number of drops involved. There are something on the order of 10,000 shown in this picture, and this is a fairly “tame” spray; a diesel-fuel-injector spray will produce millions at a time. If calculating the behavior of a single drop dripping from a faucet is compilated, this is obviously far, far worse.

This isn’t quite as complicated as it might seem from that estimation, however. For at least the left two thirds of the picture, the individual drops aren’t really doing anything interesting. Very few of them are breaking apart, and they’re mostly far enough apart that they’re not interacting with each other (except indirectly because the overall spray pulls along the air it’s moving through), and, though it’s not visible in the picture, they’ve pretty much all settled down to being nearly spherical. This part of the spray, where the drops are widely spaced apart so that they aren’t interacting with each other, is known as the “dilute region”.

That means that, for most of the dilute region, we don’t need very much information about a drop to know what it’s doing; if we know the size of the droplet and its velocity, it’s a reasonable approximation to say that it’s doing exactly the same thing as any other droplet of that size with the same airspeed. And that’s much easier to calculate than trying to track all of the individual details of where each point on the surface of each drop is.

Even when the drops are breaking apart – and sometimes they do, if they’re large enough and are going fast enough – they break apart in ways that don’t differ very much from drop to drop, and so it’s possible to approximate that by applying a statistical distribution that converts some fraction of the drops (depending on their velocity and size) into multiple smaller drops.

So, for the 10,000 drops of this spray, that reduces the calculation of the dilute region of the spray to something that could run on a desktop computer in a day or so. But even that’s a fairly long time, particularly if we want to simulate something with millions of drops. For those, there’s yet another simplification that we can make – with that many drops, we can start grouping them into “packets”; near a given spot in the middle of the spray, there are perhaps a hundred tiny drops that all have nearly the same diameter and nearly the same airspeed, and so there’s no reason to do a calculation for each one of them individually; we could just do the calculation once and say that they all do approximately the same thing.

There is, of course, the question of how good these approximations actually are in practice. This obviously depends on the details of how one does it and of the spray that one’s simulating, but it turns out that it’s possible to get quite good results for simulations of the dilute region of most sprays with this sort of method.

The Nozzle: The Other (Relatively) Easy Bit

The other part of the spray that’s relatively easy to simulate is the flow inside the nozzle and just outside it for the first half-millimeter or so. In this part, the surface is not doing anything especially interesting, and so it’s entirely possible to track all of the details of its motion in a simulation that could be run on a desktop computer in a few hours.

Dense Parts of a Spray: What About the Rest of It?

That, then, takes care of the spray in the left-hand two thirds of the image, and the first tiny bit coming out of the nozzle. The alert reader will have noticed that this leaves quite a significant portion of the spray undiscussed! This is where things get tricky. In this region of the spray, known as the “dense region” because the drops are sufficiently close together to affect each other directly, there are still too many drops and too much going on to simulate all of the details of the surface motion, but the process of the water breaking apart into droplets is far too complicated to accurately approximate as the motion of independent spheres.

This region is one of the biggest reasons why simulating free-surface flows that are of interest in engineering problems is difficult. At present, it’s an unsolved problem.

One common way to deal with the dense region is to simply take the independent-spherical-drops approximation from the dilute region, and continue it all the way up to the nozzle. The drops in the simulation are created at the nozzle in such a way that, when they get downstream to the edge of the dilute region, they match some experimental data for the distribution of real drops there. How good is this approximation? It’s not really clear; it works fairly well for cases where the dilute region is the only part that’s important (though it’s not completely clear how true that is in most interesting sprays), but of course it’s only applicable in cases where there’s good experimental data to start with.

Meanwhile, there are ways to do approximations working from the upstream end. If you look at the spray as it comes out of the nozzle, it’s (roughly) a conical sheet. To a rough approximation, this conical sheet first breaks up into rings, and the rings then break up into drops. (That’s a very rough approximation for this spray; it’s a bit clearer in some with more precise nozzles.) By assuming that these processes can be separated and calculated independently, it’s possible to get mathematical models for how quickly the sheet breaks apart and what size drops are produced, and these tend to be reasonable as a very rough guess for the actual drop distribution.

So, at present, there are two big remaining problems. One of the problems is that the existing sets of approximations don’t really meet in the middle; there is a part of the dense region after the calculations of the initial exit from the nozzle have done what they can accurately do, and before the independent-drop approximations of the dilute region are accurate. The second problem, even if we solve the first problem, is that the computational methods corresponding to these two sets of approximations are very different, and it’s not clear how to combine them in the same simulation in a way that’s mathematically consistent.

The research that I’m doing for my dissertation is something to address a tiny piece of both of those two problems. It won’t solve them – not by a long shot – but it’s a small step up the hill.