Silicon Valley Redneck

Heisenbug!

Grrr...

Spent a few hours bashing my head against a weird bug in a new feature I'd just added to the FPGA logic for Project Egbert.  (Would've been less time, but there's compilation delay, then download delay, and it makes for slow testing.)

It was a definite Heisenbug.  With Firesign Bilocation (i.e., it was in two places at once when it wasn't anywhere at all).

I could make it come and go by adding or removing certain lines of Verilog code.  In a synchronous section.  Dependent entirely on synchronous signals.

And, when it manifested itself, it didn't just misguidedly assert the status bit the new code was meant to affect.  Oh, no.  It also goofed up some other, unrelated logic.  Oh, and it manifested itself at impossible times, when the necessary trigger condition wasn't present.

This morning, in among surfing the Net and before going back to bed, I decided to route the internal state of the buggy feature to a register I could read back, just to see what was going on.  Naturally, this made it start working.

So I removed the debug route.  Now it works.

I'm developing a great mistrust toward Verilog for Xilinx.  But maybe a newer version of ISE would fix the problem.  And, maybe, introduce new ones.  (Update from 11.1 to 11.5?  But the updater says my license doesn't cover some features of the new version.  And doesn't tell which those might be.  Hmph.  Ah, but if I tell it to go ahead, and it tells me the license is OK for everything except DSP tools, which I don't know how to use anyway.  And 12.1 is scheduled for release month-after-next.  Treadmill!)

Beyond the Curate's Egg

We're assured by our betters that portions of the monster dogpile just passed by Congress poll well.

Yes, if you're very selective, you can pick out a dozen or so sentences, of out the two-thousand-and-however-many pages, that sound good when taken in isolation.

"I assure you that parts of it are excellent!"

Only... with laws, you're not allowed to pick out the good bits, nor to cut out the worms as with a questionable apple.  It's like being presented with a mug of the very finest coffee, mixed with sewage.

Supposedly, the presence of any trace of coffee makes it a mug of coffee.

In reality, the presence of sewage makes the whole thing a steaming mug of sewage.

Yes, the current health-care system is unsustainable, and the recession has pushed it into a death spiral as healthy people drop policies they can't afford, and companies jack up their rates to make up for the shrinking (and increasingly less healthy) customer base.

This new "health care 'reform'" bill is set up to lock everyone into the broken system, while making it even more unsustainable with guaranteed issue and community rating combined with a weak individual mandate - so that, for healthy people in any but the highest tax bracket, medical insurance becomes something most affordably available from the IRS by way of just paying the no-coverage penalty until major illness strikes, at which point one can sign up for a comprehensive medical plan with, effectively, retroactive coverage.  Until, of course, all the "insurance" companies go bankrupt.

A recession should be a time of innovation, as people and companies scramble to do more with less, i.e., become more efficient.  Prepaid third-party payment systems are inherently inefficient; adding layers of government bureaucracy make them even more so.  Locking in an inefficient system while simultaneously kneecapping it and forbidding innovation is a recipe for national disaster.

And for those who insist that health care should be entirely run by the government, consider this: if you get screwed over by your doctor or your insurance company, what do you do?  Well, you can complain to the appropriate state regulators, or you can sue.  If you get screwed over by a government agency... well, you're pretty much SOL there, as the government agency and the regulators are the same, and the government has sovereign immunity against lawsuits.

But don't worry.  Once the government has finished smashing the old system to bits, we'll end up with single payer.  And the government will get it right the first time and every time, just like it always does.  Because otherwise its customers would... uh....

Don't worry, be happy.  Because parts of it are excellent.

OK, that was weird.

A bit over half an hour ago, Firefox 3.6 crashed.

Well, that's not exactly news.  Browser crashes happen.  At least on Linux they don't corrupt the OS.

But... it didn't restart when I clicked the "Restart Firefox" button on the crash window.

I clicked the Firefox button again, and it started back up, restoring the tabs I'd had open.

Finishing what I'd been doing, I exited Firefox, with the intention of doing a clean restart.

Clicking the button got launch feedback, but no browser.  Huh?

OK, off to the command line: /opt/firefox/firefox gets... nothing.  Exits silently.  Editing the script to enable debugging gets the information that firefox-bin is exiting with status 1.

Oh, and on one occasion, early in the game, the browser actually launched.  Not after that, though.

Re-install Firefox 3.6 from a fresh download.  Still no soap.

Awright, I know this game. Into ~/.mozilla/firefox, and rename profile.eric, then try again.  I get a popup box informing me that Firefox is already running.  Eh?  Nary a Firefox process to be seen on ps, and I deleted the lock files.  Put profile.eric back, and the popup box goes away, but Firefox goes back to exiting silently.

Grrr... try moving all the sqlite files out of the way.  No change.  Move them back.

I wonder if Iceweasel will let me recover my saved data...?  Run Iceweasel (version 3.0).  Starts up OK.  My bookmarks are there, including one I remember adding recently.

Exit Iceweasel, restart Firefox.  Now it works.

Like I said, that was weird.

On writing printf()

A few days ago, I followed some links from Slashdot and ended up at this post by the Reinvigorated Programmer.

One thing kinda jumped out at me:

2. But you wouldn’t want to have to write printf()!

Well... commenting as one who has written printf()... there's something to be said for it.

I've got my own implementation of printf, which I still use for embedded applications.

It started out as a compact implementation, back in the 80s, when the version on offer was causing programs to get huge.  I ended up with something that did everything I needed, and almost everything the standard version did, and was vastly smaller.

It also turned out to run a lot faster.  The version that was making the rounds, copied from C runtime to C runtime with cheerful disregard for licensing terms, was obscenely slow.  As in, running on a typical 16-bit minicomputer of the day, if stdout wasn't buffered, you'd notice the delay between characters.  The library had many layers of function calls, bizarrely interrelated.  My version was simple, small, and fast.  And, I might note, used far less stack space.

Over the years (from the minicomputer to the microcomputer era), I've continued to use my printf for one reason or another.  In the early days, this was often to get around bugs in vendor-supplied implementations, or because I wanted an extra format (such as the ever-useful %r, which I saw described in an ancient document probably written by D. M. Ritchie but which never made it into the official standards).

I originally wrote it on the assumption that arguments were pushed on the stack, using conditional compilation to distinguish between stack-grows-down and stack-grows-up architectures.  As RISC architectures and ANSI C started cropping up, I adapted it to use vararg, and then stdarg.

I'm still using it, e.g., in AGROS, which sports an integer-only version with a few extra formats I've found useful over the years, such as %i (integer, with the precision field determining the radix), %q (string with special characters escaped), and the new-this-month %p (fixed-point, for scaled integers, with precision defining the number of digits that belong after the decimal point).  And %r is still there, though now it takes a va_list instead of a pointer to a format string.  Gone, though, is %R (Roman numerals).  The source code for that should still be around somewhere, but it just wasn't all that useful somehow.

Anyway: sometimes the library you're given just doesn't get the job done, and you need to reinvent things.  (This is bringing back repressed memories of the time, back when official DEC C for VMS was new, that I ended up replacing large chunks of the supplied stdio library to get around the weird behavior of a UN*X library in a VMS environment, as ported by VMS programmers.  Time to get away from the computer now.)

Well, that's easy.

Michael F. Cannon has a question for the President:

The rationale for your proposed tax on high-cost health insurance plans is that it would encourage people to purchase less-comprehensive coverage and thereby reduce health care spending.

If that’s a good idea, then why is it bad when insurers raise premiums?

Well, duh!

It's the same reason it's bad when oil companies raise the price of gasoline, but good when the government raises the price of gasoline through higher taxes.

When private companies pass along the increased cost of doing business, it's a wicked conspiracy.

When the government imposes new taxes, it's enlightened central planning.

None of which quite explains why Pres. Beeblebrox wants to encourage us to buy less-comprehensive health plans while at the same time mandating comprehensive health plans.

(Via Insty.)

On the value of opacity

Insty links to an article by Bradley A. Smith suggesting that mandatory disclosure of campaign contributions may not be such a great idea.

I've had the notion for a few months now that, in fact, exactly the opposite approach might be better: rather than mandatory disclosure, consider mandatory anonymity.

Think about it: in order for a donation to influence a politician, the politician needs to know who made the donation.  If all donations were anonymized, there would be no way for a politician to verify a claim that, e.g., DinoPower EnerCorp had made a $50K campaign contribution and that its CEO was thus worthy of special attention.

Yes, this would allow the dreaded Special Interests to make big contributions to the campaigns of favored candidates.  But let's get real here: there are basically two reasons to make a campaign contribution.  You or I might send a little money in hopes of helping to get our favorite candidate get elected.  The big boys, though, hold those big fundraisers in hopes of buying influence.  (Why else do entrenched incumbents in safely-gerrymandered districts need huge campaign funds, anyway?)

If the goal is to get influence, rather than money, out of politics, doesn't it make more sense to keep officeholders in the dark as to where the money came from?

No, I don't have a Grand Plan with all the details worked out.  Just throwing out a thought.

Oy. Such a weekend.

Yesterday was productive, for a sufficiently abstract value of "productive."  At least, by the end of the day, I'd positively identified and fixed a bug that had been bothering me since Thursday:

Test the Project Egbert code in my lab: it works, as far as I can tell.  Run it on the actual toaster in the client's lab: it mostly works, but under certain circumstances it chokes, with a system dump that suggests a bad pointer in an interrupt routine.

Back to my lab: can't duplicate the problem.  Instrument the code, so that a panic dump during an interrupt can report the identity of the interrupt (I don't have stack trace implemented).  It's the I2C handler.  Huh.  Well, there are several I2C devices on the board, mainly used during initialization; the only one that's currently used during normal operation is NVRAM.  The symptoms point to a software problem, but why does it only happen when the board is hooked up to the power circuits?  And why does it trigger on events that don't involve power-circuit activity, and thus shouldn't be inducing noise spikes or any such?

Back to the client's lab.  Futz around for a while.  Determine that it has to be the result of one particular non-critical feature.  Disable that feature, so the tech can work on the toaster without worrying about software crashes.  Confirm that it operates through noisy power-circuit events.  Whew!  After all the effort I put into making that hardware noise-resistant, it would have been embarrassing, to say the least, to have it choke every time there was a power-related event.

Back to my lab.  Spot the bug: a stale bug-fix in the code that de-allocates the buffer after an asynchronous write from a dynamically-allocated buffer.  Remove it.  Still can't duplicate the problem here.  Get busy finishing up implementation of the feature that I'd removed from the client-lab code.  Discover that it doesn't work for a variety of reasons, one of which is that the NVRAM on my board has a valid signature, but one of the stored values had been wrongly initialized by an erly version of the software.  Re-initialize it.  Hey presto!  Now I can duplicate the bug.  Guess the stale bug fix wasn't the bug I was looking for, though it would have caused problems later.

Much further tracking.  End up in the I2C memory write code, where the part that prevents a write from crossing a page boundary (important when dealing with EEPROM) turns out to be subtly wrong, occasionally giving a result of -22 bytes to be written in a chunk instead of 18 bytes, causing a gargantuan write, causing the data pointer (used by the I2C interrupt routine) to go off the end of RAM, causing the data abort trap.  Not the sort of thing you find with Lint, nor yet with any but the most detailed code audit.

All that, and the bug was in a module with my copyright notice on it, not the client's, so fixing it was on my time.  That's life in the consulting lane....  (Well, a bug like that, even if it had been in a module owned by the client, I wouldn't have felt right about charging them for all that time to fix it.  It was a dumb mistake.)

Meanwhile, around lunch time, I'd broken a tooth on a granola bar, and the dentist's office isn't open on weekends, so I can't even see about making an appointment until Monday.

Today... well, it was the sort of day that starts with "wake up with a sinus headache."  Got a few mindless household chores done, but, somehow, I didn't manage to spend the afternoon doing a bunch of writing that I really need to be doing for Project Hyperion.

Tomorrow, call the dentist, take Tinga to the vet, do a bunch of that writing, and get answers to some questions about the next toaster, which we're supposed to be putting together around midweek.

Windows 7: the hate continues

I'd been managing to avoid versions of Windows newer than XP.  This week, however, I was confronted with Windows 7.

Yeah, I'd had a brief exposure to Win 7 on Joy's new laptop, when she had the WLAN problem.  That doesn't really count.

Off to a client site yesterday, to see Egbert ensconced in a real toaster for the first time!  I hauled my old laptop along, 'cause it's got the toolchains on it, and I haven't worked out the details of seeing up the build environment on Windows yet.

The IT department had provided two shiny new laptops for the project.  Actually, one normal-sized laptop and one netbook.  Both from Acer.

Installation of the bare necessities (Subcommander, Ruby, Ruby-Gnome2, and Tera Term Pro 3.1.3) went smoothly.

Now we need serial ports!  Plural.  One RS-232 and one RS-422.  We've got USB serial dongles for both.  Both dongles have been working with XP, and both are simply plug-n-play with Linux.  With Windows 7?  Nope.  Can't find the drivers.  Uninformatively can't find the drivers.

After a while, the IT guy (a Mac type) persuaded Windows to find a driver for the RS-422 dongle.  For the RS-232?  Well, I plugged it into my laptop and used dmesg to learn that it had a PL2303 in it, which was enough information for the IT guy to find a suitable driver on the Web.

So, eventually, we had communication.

Switching back and forth between the two machines - these are factory-fresh machines from the same vendor, now - the user interface is distractingly different.  The main menu is not only awkward (kind of KDE 4 type awkward); it's differently awkward on the two machines.

And the shiny new UI?  Dreadful.  Still has many of the old crocks (such as undersized, non-resizeable file-selector dialog boxes), and it offers no clue where to start on... well... just about anything, starting with calling up a file manager, and progressing to setting the view options on the file manager once you've got one called up.  (Hey, show me the freakin' filename extensions, stupid!)

It's buggy, too.  Plug in a thumb drive, and usually a window pops up asking what to do with it (and, in a case of finally getting it right, it defaults to "open a folder to view files" or whatever they call it).  But... sometimes that window is in the background, and you have to futz around to get to it.

It looks like other things may have changed subtly under the hood, too, 'cause old Tera Term Pro, what we've all been using these many years, exhibits decidedly strange behavior, and needs to be restarted frequently.  Well, mayhap the new Tera Term 4.65 will work better.  Have to try that next time I'm over there.

Also, despite the zippy-fast processors in the new machines, a Ruby-Gnome2 app takes for-freakin'-ever to launch.  On the other hand, starting the same app the second time goes pretty fast.  Is it really taking all that time to load the interpreter and libraries into RAM, and after that, they're cached?

Stay tuned; any day now, I'll be kvetching about OrCAD's horrid UI, with the tiny, fixed-size dialog boxes containing huge lists.  Scroll, scroll, scroll your mouse, halfway down the list....

Update: comments and trackbacks closed, due to persistent spamming.

Encountering the limits of C hackery

Working on one of the comm protocols for Project Egbert today... I've ported the code from the old system... and now it comes time to extend one of the features.

This feature involves a few keywords, which in the old version were always four characters.  Easy handling: pack the characters into a word, and do a switch on it, with multi-character constants (e.g., 'HTRT') as case values.

Now some of the keywords are longer.  Like, seven characters.  Well, I can pack that into a long long, right?  I'm dealing with 64-bit ints all over the place in the new system.

Argh.  Not so fast.  GCC doesn't seem to like more than four characters in a character constant.  It doesn't go to 64 bits, and I can't seem to persuade it to do so.  It puts the last four characters in the low 32 bits, and that's it.

Oh, well.  Strings it is, then.  And strcmp, and if-then-else.

At least it's a fairly short list of keywords.

Searching for the cause of WPS

Surfing the Interwebs on this fine soggy morning (no walkies in this weather!), I see Slashdot linking to this squib about the enduring mystery of SUA (Sudden Unintended Acceleration), also known as WPS (Wrong Pedal, Stupid).

C'mon, guys... let's take a look at one famous panic from the late 90s, the Jeep Cherokee.  I remember that scare particularly, 'cause my garbage scow is such a vehicle, with the offending 4-liter, inline-6 engine.

In the Jeep in question, the accelerator pedal operates a mechanical linkage that opens the throttle.  If the pedal ain't depressed, the throttle is closed, and the engine receives a pittance of air through separate dinky little passages.  (I was acutely aware of this fact at the time of the panic, as the idle-air passages in my vehicle's throttle body had gotten gummed up, casing the engine to cease running at stop lights until I got around to taking it apart and cleaning it.)

And yet there were reported cases of vehicles exactly like mine charging off and smashing into things at high speed with the accelerator released and the brake pedal mashed all the way to the floor.  Because, naturally, of a runaway engine computer.  Which has absolutely no ability to open the throttle.

Definitely what we in the computer business call a PEBKAC, and the auto industry traditionally calls "the nut that holds the wheel."

Yeah, it's at least theoretically possible that the computer in a new drive-by-wire car might command high speed in the absence of driver input.  History, however, suggests otherwise.

(Of course, misplaced floor mats and sticking pedals can create false driver input, so at least some of the incidents may well have resulted from actual mechanical failures.  I'm still skeptical of SUA scares, given the history.)

Were I in the business of making automotive electronics, I'd include a "black box" function to record the last few seconds of pedal positions prior to airbag deployment.  Could be most instructive.

Selling it ain't dumb. Buying it is dumb.

A long-time co-worker of mine, back in the day, remarked thusly, with regard to the Air Force's notorious hammer:

"There's nothing dumb about selling a hammer for $600.  Buying a hammer for $600 is dumb."

Well... I was just looking for something else, and came across an on-line purveyor of medicinal herbs offering to sell...

Wait for it...

Dandelion seeds.

Really.

I wonder how many buyers they have.