Silicon Valley Redneck

Well. That was Interesting.

Over at Toasterco, my understudy had been struggling for a couple of weeks with a manufacturing problem; a batch of controller boards (the crazy complicated, do-everything model with zillions of little SMT components) made by a new contract manufacturer had a high failure rate, and the failure mode made no sense.

Basically, all signs pointed to the I²C bus being wedged LOW (both signals).  This led to various trouble.

So, I spent a couple of hours over there - hey, their lab is air conditioned!  Things continued not making sense.  The SDA and SCL lines were clearly being actively driven low, showing about 0.25V with the normal 10K pullups to 3.3V, and maybe 1.2V with a 330 Ohm pullup.  But then....

The I²C bus has a level translator between 3.3V (the MCU and a few other things) and 5V (a parallel port that really does need to run on 5V).  Were I designing the thing today, that wouldn't be necessary, as there are now 16-bit I²C I/O port chips with split supplies, so the bus can run on one voltage and the I/O on another.  But that's now, and the board was designed then, and it needed the bus bridge.

On the 5V side, there are two pullup resistors, as one might expect.  But.  Wait.  Er...

Know what you get when you place a pair of 0402 resistors next to each other on a 25 mil grid?  Right!  It's kind of hard to tell which pads are what.  And the 0402 resistor footprint in my library at the time had "part goes this way" markings in the assembly drawing, but not on the silkscreen.

So, yeah.  The resistor pair was rotated 90° from its correct orientation.  One resistor was between SDA and SCL, while the other was between 5V and 5V.  Result: SDA and SCL on the 5V side drifted LOW by default, causing the translator chip to drive them low on the 3.3V side.

0225. Mnfxt.

Sometime around 0130, Huckleberry declared Playtime.

He was very persistent about it.

After half an hour of that, I decided I might as well crawl out of bed for a while.  Not sure that was a good idea, really.

Oh, well.  Looks like global financial markets are perking back up a bit.  Speaking of which:

I noticed a headline yesterday to the effect that Soros hadn't shorted the Pound this time.  Really?  So he was so wrapped up in the globalist ruling-class bubble that he truly believed that Brexit was inconceivable?

Funny how the overlords in their shiny palaces never quite get the message that the undifferentiated plebs resent having their lives micromanaged from afar by people they don't know and didn't vote for.

And, closer to home, that (and of course Identity Politics) is how you get things like the Traditionalist Worker Party.  Look at the front page of their web site, and their platform seems almost reasonable in the current political context.  (Mostly wrong, but reasonable.)  Dig a little deeper, and you get to the part where America is for White People, and I'm not White, and they want me to wear these parentheses, just so they know who We are, mind; there's no mention of putting me on the train to Happy Fun Camp.  Yet.

Enough randomness.  Back to sleep.  Maybe.  (Playful fuzzballs aside, I think my metabolism has shifted recently, and maybe I'm burning some fat and liberating stored caffeine.  Yes, I know caffeine isn't traditionally considered fat-soluble, so it's probably something else.  It's 0248, you expect me to make sense?)

Oh, now this is just inexpressibly cute.

Still a-pokin' at that electrical apparatus with the KV30 in it.  (No, it doesn't operate at 30KV.  Different project entirely.)

The analog aspects show definite signs of suboptimality, so naturally I start looking more closely at the power supplies.  Well, the 3.3V power supply, since that's the one that counts.

This dingus runs off an inconveniently high input voltage (though well in SELV territory; it's merely out of bounds for my usual favorite regulators).  I'm using a switcher to get down to 5V, and a little teeny linear to get from 5V to 3.3V.

First thing I check, during bring-up: do I get 5V?  Yes, and well within tolerance.

Next I install the ferrite bead that connects the output of the 5V regulator to the input of the 3.3V regulator, and check again: 3.3V right around nominal, and 5V still healthy.  All this according to the nice accurate bench multimeter.

But, once I start using the ADC, I see unstable measurements.  Since the 3.3V supply is the ADC reference (I have a plan for compensating for supply tolerance, but that comes later, and it can't deal with instability), I stick a scope on the 3.3V supply.

Oh, looky!  A sort of repetitive double-pulse spike, around 100mV peak to peak.  That'll make a right hash of things.

There's... something kind of similar on 5V.  But much less spiky.

Awright.  Replace that bead with a real inductor.  Increase capacitor values.

Now the 5V supply is dang near clean.  But the 3.3V is as spiky as ever.  And it does seem to be synchronized to the 5V switcher.

Tossing a 50mA static load on the 5V rail gets a tidy 1.5MHz ripple on the 3.3V supply, while the 5V looks clean.

The linear regulator is amplifying the noise?

Let's have another look at the layout, shall we?

...Oh.

When I did the layout for the development board, I went with a 2-layer stackup in the interest of getting the boards quick & cheap so's I'd have something to play with.  I also got a little casual with the layout of the linear regulator and its capacitors, and then didn't give it a critical going-over after the autorouter had run traces through that area.

There's a slot in the solder-side (where those two caps live) ground fill, nigh a quarter of an inch long.  The output cap is connected to the plane right near the ground via for the regulator chip: no problem.  The input cap, though, connects right in the middle of the other side of the slot, close to the switcher's noise.

So... maybe rotating the input cap so it shares the output cap's ground connection (close to the chip) will correct the problem?

Nope.  Still all quiet on the 5V front, and 100mV of switching-correlated noise on the 3.3V.

Grumblegrumblegrumble.

Well, that layout issue didn't make a whole lot of sense at 1.5MHz anyway.  At 150MHz, maybe, but I'm not dealing with that here.

Back to pondering.

Update: What the...?  Ripple at the linear reg output cap is lower by almost an order of magnitude.

Um.  One side of the 3.3V trace runs alarmingly close to the switcher's inductor on its way around the end of the board to its destination.  (Another, autorouted trace is even closer.)  Aaaand... yeh.  Horrid noise on that end of the trace.  Which is the end that connects to everything important.

Guess the next thing to try is changing a few of the bypass caps along that path  to something stiffer.  Or, perhaps, cutting the trace and running a patch wire 1/8" south.  Ugh.  Cut & patch is annoying, but I think it's the best next step, really.  And at least the trace is a generous 0.010" wide, not a scrawny 0.006".

Update 2: Rerouting the trace and adding a bonus microFarad downstream each reduced the noise by about a factor of 2, so now it's down around 25mV.  I expect the power to be a fair bit cleaner on the 4-layer life-size board... and there'll certainly be a respin in any case, as the final form factor has yet to be determined, never mind the type of connectors on the ends of the board.

ADC performance now seems consistent with a moderately noisy power supply.  I may add an extra stage of filtering on VDDA in the next iteration, in addition to general cleaning up.

Meanwhile, I can get back to working on the firmware.  Maybe I should first populate one of the piggyback boards, so I can test some real-world I/O functions.

Well. Isn't that special.

If you're debugging an application on the Kinetis KV30 MCU (and, I'm guessing, other members of the family) using the on-chip debugger...

...And you're generating PWM using one of the Flex-Timer Modules...

...And you hit a breakpoint...

...The timer ceases to generate PWM.  You'd think: hardware-type peripheral continues running while the CPU is stopped at the breakpoint.  But you'd be wrong.

So far, I haven't seen it stop with the output HIGH; if the behavior is consistent, with the output always LOW when at a breakpoint, it's possible (for some applications) to design around that, and not have adverse hardware consequences when a breakpoint is encountered.

But, if some hardware design optimization dictates that the "safe" condition is output-HIGH, then hitting a breakpoint could result in a ruptured smoke bushing.

I recall running into some related issue when debugging on an AVR... over a decade ago.  But the FTM is an 8-bit peripheral grafted into a 32-bit MCU, so what should I expect?

(It's also possible that I have something misconfigured at the library level, given that the output is waking up at the default-configured duty cycle and my attempts to change it on the fly don't appear to have any effect.  Once I sort that out, I'll have to address the problem of the high-level library taking an integer percentage for the duty cycle, when I really need to have finer-grained control.)

Update: OK, so I was missing something.  Behavior is configurable; there are four BDM modes, and BDM Mode 3 is what I was expecting; the others are meant to go "safe" at breakpoints.  So that's working now.  Yet to be sorted out: why the duty cycle isn't updating, and/or why some configuration options cause the FTM to generate an unhandled interrupt upon initialization.

Update 2: Awright, got it updating and using full resolution now, mainly by bypassing the peripheral-driver layer after initialization.

Import the necessary:

#include "fsl_ftm_hal.h"

#define FTM0 (g_ftmBase[0])

During initialization:

  // Set the PWM divisor to a known 4096 instead of some calculated value
  FTM_HAL_SetMod(FTM0, 4096);

To set the actual PWM pulse width in clock ticks:

	  FTM_HAL_SetChnCountVal(FTM0, 2, pwm1val);
	  FTM_HAL_SetChnCountVal(FTM0, 4, pwm2val);

	  // Now trigger the update of the actual FTM registers
	  FTM_HAL_SetSoftwareTriggerCmd(FTM0, 1);

Seems to be doing what I want.  The documentation isn't very clear, and the peripheral driver is missing that all-important "change the dang duty cycle" function - if you're using that level of driver, you have to call FTM_DRV_PwmStop and then FTM_DRV_PwmStart, because repeated calls to FTM_DRV_PwmStart don't result in actual changes of duty cycle.  Also, if you're using multiple channels, you have to call FTM_DRV_PwmStop in between the ...Start calls for the channels.  Or something like that.  It's all very confusing, really.

An eventful day, this side of the freeway

Gah.

Woke up a bit on the late side, with assorted felines demanding attention.  Also with a nasty sinus headache.

A hot shower helped with the sinus headache, but left me still feeling decidedly off.  This being Sunday morning, I'd normally head over to my parents' house to do my laundry and take their dog for a brisk walk.  I'd like to do that before the day gets too hot... but...

I'd figured out why the kitchen sink had gotten drippy.  Basically, it needed the nut on the the drain (strainer, whatever it's called) tightened.  I figured I'd disassemble it and apply some fresh putty before tightening it.

So I disassembled that side of the drain plumbing, and commenced to loosen the big nut.

Well, no.  It turned easily enough, but the whole assembly was turning along with it.  I don't seem to have anything suitable for immobilizing the drain body, and didn't feel like fabricating one, so off to the hardware store.

Yreka*!  Special wrench-like thing for grabbing kitchen sink drain bodies.  Back to the kitchen to set about loosenin'...

...No.  I'm nowhere near strong enough.  Were I a little stronger, I expect I'd destroy one wrench or the other before that nut let go.  CLP doesn't help.

At this point, I was getting seriously fatigued, but the sink was by now entirely unusable, so the job must be completed, and I ain't callin' no real plumber on a Sunday.

Out to the shed; fetch the grinder.  I ended up doing this last time, didn't I?  Cats in the bedroom.  Face shield on.  BRZZZZZZZZZZZZ!  Not quite.  BRZZZZZZ!  Ah, here we go.  Fetch a hammer and chisel, and pry the nut apart.  All's well, except for a brand-new slot in the sink's flange, where no slot belongs.

And so I returned to the hardware store for a replacement drain thingy and some JB Weld putty.  Make sure all the bits are going to fit together: OK, with some fiddling and a bit of mix&match.  Jam some epoxy putty into the slot.  While it's curing, put a thick bead of plumber's putty on the new drain's flange, and some PTFE paste on the threads, so maybe this one won't get permanently corroded in place.  Assemble.  Tighten so's most of the putty extrudes; clean up the excess.  Assemble the plumbing.

It appears to be working now.  When I got it back together, about an hour ago, I was beyond exhaustion, and near the point of total collapse.  Rest is good.

After I recuperate a bit more, it'll be time for lunch.  Food is good, but rest first.

Then, in the heat of the afternoon... no, not dog-walking.  Maybe this evening (the laundry still needs doing, and this promises to be another busy work week).

No, the heat-of-the-afternoon project involves fixing the back door so it latches properly again.  It doesn't, at the moment.  Probably calls for yet another trip to the hardware store.  Good thing the door opens inward, or certain furry menaces would be pushing it open at their own convenience.

Update: Back door duly Bubba'ed; should be good for another several years (and, really, the door itself is overdue for replacement).  Laundry put off until tomorrow evening.  I spent a little bit of the evening chasing further performance issues with that embedded TCP/IP stack, and didn't actually solve the remaining problem but came up with a workaround that gives a significant improvement in the time needed to recover from it when it inevitably (but unpredictably) happens.  Other than that... tired.  Need a vacation, but that ain't happening any time soon.

* That's Greek for "I have baked it!", or possibly "I half-baked it!"

Stupid browser changes.

The latest version of Iceweasel (Debian's rebranded Firefox derivative) showed up a couple of days ago.

The UI has changed, and not at all for the better.

The search box no longer has the drop-down search-engine selector.  Instead, you start typing, and then it pops up suggestions from Wikipedia and a bunch of little teeny icons for selecting other search engines.

And: the NoScript menu has lost its main component, showing what domains are allowed and blocked on the current page.  That's gone.  Completely gone, as far as I can tell.  What's left is vastly less useful.  And the toolbar icon is now just an icon, not a status indicator.

Not changed: if you leave the browser running for a while, its CPU usage drifts to 100% or so, and JavaScript and dialogs get obscenely slow.  This has been an annoyance for at least a year.

I keep meaning to migrate almost everything to some other browser, but it seems that all of them (at least the mainstream ones, that don't get unsupported-browser warnings from banks and such) are on the screw-up-the-UI bandwagon.  Still: Chromium seems mostly better behaved, most of the time, though admittedly I haven't tried it with a week's accumulation of open windows-full-of-tabs.  I haven't tried Opera in a while, but perhaps it will be available again after I update to the latest OS release (if memory serves, the latest Opera needs some newer libraries than Debian-Wheezy provides).

Oh, yeah: I don't know how much is the Iceweasel change and how much is the latest change to Twitter's brain-dead presentation (with a notable lack of actual clickable links and lots of JavaScript from a variety of domains), but now if I want to follow anything from a tweet I have to switch browsers.  Because No Actual Links.  And, if there's some blocked JavaScript, I can't tell what is is with the new NoseCrypt UI.

And, that variety-of-domains thing: it's been bothering me since the 90s.  It's bad enough that a lot of sites bring in content from elsewhere, but what imbecile decided that every major site needed separate domains - not just hostnames or subdomains - for stuff like images and other static content?  And why has the practice continued?  I'm talking about the use of, e.g., "gstatic.com" to host static content for google.com; why is it not "static.google.com"?  Trying to save one level of (typically cacheable) DNS lookup, at the cost of totally screwing up protections against cross-site scripting?  Then toss in the casual use of cryptic short domain names in a variety of TLDs ("t.co" comes to mind, but there are zillions of others), and it's really hard to tell what script sources should be allowed.

Also: From the user's perspective, there's no excuse for a URL shortener to use JavaScript when expanding a shortened link.  Clicking on a URL of, e.g., http://makealittleteenyconvenient.link?UniqueLinkID="i3or6jcifjio8qiw4weifi3jdioewqjd" should get an HTTP 308 response (or maybe 307 or 301) pointing to the expanded URL, with no client-side scripting involved at all.  So why do some of these link-click-through-ifiers insist on JavaScript?  I presume that, if there is a purpose, it's one I wish to defeat.

Update: Turns out this was at least partly a Debian issue.  I had NoScript installed as a Debian package; removing the package and installing NoScript from the browser brought back the old, informative menu.

I hate missed interrupts.

Had an interesting Situation last week, on the same project mentioned in yesterday afternoon's post.

Response time to a fairly simple HTTP request was all over the place.  Well, sort of all over.  In my testing, it was somewhere between 2 and 20 milliseconds, mostly at the low end of the range.  At the client site, it was usually just a little over 2 ms, but every once in a while it would be slightly over 200, 300, or 400 ms.

Eh?

And so I headed over there to see what was different.  Firing up Wireshark, we found that there was indeed a 400 ms (in the setup at hand, with a Mac initiating the request) gap in the gadget's response, every once in a while, caused by a complete failure to respond to the request and the Mac retrying the packet.

Fun fact: When PHP's file_get_contents function is used to make an HTTP request, it sends each line of the request as a separate packet.

Anyway, the first thought was that the packet was somehow getting lost, so I spent quite a while, with the customer watching, trying to find Ethernet error indications.

Pro tip: If you set a breakpoint in the Eclipse debugger, and it obstinately happens on the first line of the next function in the source file, then probably the function on which you were setting breakpoints isn't actually used anywhere and has been optimized out, so that all lines within it resolve to the next line that actually has code associated with it.  Eclipse won't tell you that this has happened.

Well, there was no sign of any errors, nor buffer overruns, nor any of those things.  Then I noticed that there was a DUP ACK associated with the event: that both the missed packet and its replacement were in fact being ACK'ed, but not until the replacement came in.

So not a lost packet, but a delay in dealing with it.

Eventually, this led to: the original packet was correctly received, and just sat there, then when another packet arrived they were both processed promptly.

At this point, I took the "process any received packets" code from the interrupt routine in the Ethernet driver and copied it into the 1 ms polling routine.

Ta-daaaaa!  Now everything's processed promptly.

And it explains why I wasn't seeing the problem; there's enough broadcast activity on my network that, if a receive interrupt was missed, there was a pretty high probability of another packet showing up soon and getting things moving again.

I haven't positively confirmed the cause, but the interrupt routine (Somebody Else's Code) appears to be munching the data and then clearing the flags.  Which can be problematic, if another packet arrives during processing (see also: each line of the request sent as a separate packet).

Also, I'm thinking the interrupt routine shouldn't be doing the data-munching anyway.  Maybe it should just clear the flags and kick a semaphore, and munching of received data should happen in task space rather than the interrupt?

Yeah, just a crazy thought.  But... since the TCP/IP stack in question is not task-safe... is it interrupt-safe?  Or could the interrupt-driven data-munching cause problems, beyond the obvious one of spending excessive time in the interrupt handler?

The more I work with this third- and fourth-party code, the more I think I need to find the time to port AGROS to the Cortex M architecture and polish it up for a proper open-source release.

Another USB-induced panic

A few years ago, I ran into a problem with a USB MCU-programmer board not being recognized when plugged into my laptop.  Turns out it was a USB 1 device, and apparently the laptop's USB 2 port wouldn't talk to it directly, despite all the promises of backward compatibility.  Plugging it in via a hub solved the problem.

Fast forward to this morning.  I'm already frazzled, for various reasons, and desperately need a nap... but I also need to get ready to run off to a client site this afternoon for some urgent debugging, and it seems a good idea to have the ST MCU development tools all checked out on my laptop.

So I get everything installed (which was a pain in its own right), carry my copy of the target board, its power supply, and my ST/Link v2 plus spacial adapter out to the table, plug it all together, run st-util to see that all's well:

2015-08-14T11:15:38 INFO src/stlink-common.c: Loading device parameters....
2015-08-14T11:15:38 WARN src/stlink-common.c: unknown chip id! 0xe0042000

(Timestamps not correct, but that was the message.)

Um... oops?  Chip ID 0xe0042000 is, if memory serves, what you get if there's no chip, no target power, or otherwise no response.

So, after much running around, and confirming that it still works on my workstation and on the low-end Linux box in the lab, and taking a lunch break, and deciding to have one last try, I tried again on the workstation, sans target power:

2015-08-14T11:15:38 INFO src/stlink-usb.c: -- exit_dfu_mode
2015-08-14T11:15:38 INFO src/stlink-common.c: Loading device parameters....
2015-08-14T11:15:38 WARN src/stlink-common.c: unknown chip id! 0xe0042000

(Those being the actual time stamps.)

Huh.  That first line.  I wasn't seeing it on the laptop.

Um?

At  my desk, and at the workbench, the handy USB ports are on hubs plugged into the actual back-panel ports.

So I rummage around in the laptop bag for the little portable hub, plug the pod in through that, and hey presto!  I get the exit_dfu_mode line.  With power applied to the target, I even get a proper sign-on, with the chip recognized and everything.

So, if st-util is giving you unknown chip id 0xe0042000, and the pod is plugged directly into the computer... try adding a USB hub.

Yes, I do realize that this makes no sense.  But, then, a lot of things in the USB world make no sense.  Live with it.

Yet another thing needs replacing

My Bluetooth speakerphone, a bargain-basement model to begin with, has never been good about battery management.  If it's not charged for a while, it'll start making "bloop?" noises at random times, which I suppose is meant to be the low-battery indication.  Then, once I've decided to charge it, it turns out to be finicky about the charger; plug it one of the cheap two-hole chargers that I can use for just about anything else, and it apparently doesn't charge, though there's no real indication as to what it thinks it's doing.

Anyway, today it developed a new annoyance.  It made the "pling!" noise that it makes when the switch is turned to the OFF position... and, apparently, turned itself off.  Turning the switch OFF and back ON woke it back up, but then, several minutes later, it did the same thing again.

I don't feel like trying to fix it, given that the slightly upmarket model now costs $20 (plus another $6 to get it delivered tomorrow instead of the day after), so I've ordered the better one.  Apparently the new one, instead of randomly making "bloop?" noises when the battery is low, will randomly announce "Hey, the battery is 37% charged!", or something to that effect.  Anyway, more informative.

Now, here's a crazy thought: why don't Bluetooth speakerphones have some protocol whereby the phone could read the status digitally, and then an app on the phone could report things like battery state, whether it likes the charger, and so on?  Maybe the app could even be used for fiddling with settings, instead of having just the buttons on the speakerphone and some non-obvious way of using them for setup.

Update:  The new one seems pretty decent, though I haven't actually received any calls in the car, so I haven't tested the essential speaker-phone-ness of it.  Gotta figure out how to disable what I assume to be alerts for going in and out of coverage: bloop, bloop (I have the "play a tone when this happens" turned off on the phone, but the speakerphone seems to be making its own sound for the event).  When I switch it off, it emits a two-note sequence which is distractingly the first two notes of the Helston Furry, and I keep expecting Morris dancers.

And speaking of upgrades...

The license key for EAGLE v7 having come through today, I figured I'd install it and have a go.

Oops!  It wants glibc 2.14, and it looks like I have 2.13.  Foo.

Well, there's a new Debian-stable, just came out in June, that probably has more up-to-date libraries.  Many, many pages of instructions for the upgrade-in-place.

Guess I'd better get sundry critical stuff out of the way, and then do the magic dist-upgrade.  Like, later in the week.

Meanwhile, I'll try the new EAGLE in a VM.  VMs are the answer to everything, right?

And maybe I'll try upgrading my laptop before the workstation; not only is it less critical, but it has a smaller hard drive, so perhaps I can back up a disk image for Just In Case.

Update: Possible alternative approach here is to download the source code for the latest glibc, and build it locally.  Then configure the loader, and so on: details.  Got glibc-2.21 compiling now, and if I watch the output scroll by I get seasick.  Csick.  Glibcsick.  One of those things.

Update 2: OK, so that was a bad idea.  I installed the new glibc under /opt, so as not to clobber anything... but after I added it to the ld.so configuration, while I can still launch some third-party applications (and, indeed, ldconfig), things like mv and rm are b0rked.  Guess I gotta break out a bootable CD and fix this....

Update 3: Recovery was complicated by the fact that everything other than boot and swap live in an LVM partition, and the first two bootable CDs I found don't speak LVM.  I was starting to think I'd have to pull the drive and connect it to my laptop, but then I found the Debian-Wheezy netinst disk wherefrom I'd done the installation in the first place, and that had a rescue option that almost got me to a shell on my root filesystem, and from there I could alt-F2 to a shell, muck about on /target, run ldconfig (good thing it wasn't broken!  It wasn't provided by the netinst CD) with -r /target, reboot, and be back on the air.  From glibc malinstall to recovery complete: just about 50 minutes of abject panic.

Gah.  Good thing I hadn't scheduled any real work for today.  Once I get calmed down, there's plenty in the queue, though.

Update 4: In preparation for some of the work in the queue, connect the newly-purchased debug pod, via the newly-built adapter, to the intermediate-model target hardware (the final model, whereon most of the upcoming work must happen, being out for build at the moment).  Seems to work.  Attempt Ethernet communication with the target device: nothin'.  Eep?

Oh.  When I rebooted the workstation, the required IP alias went away; a simple
root@magrat:~# ifconfig eth0:1 192.168.1.1
later, it's working just fine.

Aside from which, I realize that the project I didn't quite get signed off last week actually has two more items that need ticking off; one should just require about half an hour on site, while the other involves, among other things, buying some mildly arcane SMT resistors, which means a Digi-Key order, local OTC sources not having them.  Shoulda done that Friday, really, but on Friday I was rather distracted by family matters.  So: order placed, a little late in the day, and depending on client priorities I may need to make two trips to the site, or just one trip next week.

Update 5: At least I didn't have as bad a day as these guys.  Being able to shut down the computer and spend half an hour sorting out the problem is a luxury.  Most real-world things don't have pause buttons.

Update 6: Fry, fry a hen.  Forget the very-latest glibc; build 2.14, since that's what EAGLE demands.  Also, don't add it to ld.so.conf; instead, try:
eric@magrat:~$ LD_LIBRARY_PATH=/opt/glibc-2.14/lib /xfer/cad/eagle/eagle-lin64-7.3.0.run
Hooray!  It runs the installer!  Then invoke the application with:
eric@magrat:~$ LD_LIBRARY_PATH=/opt/glibc-2.14/lib /opt/eagle-7.3.0/bin/eagle
And it works!

I still gotta upgrade to the current Debian sometime, but it doesn't need to be this week.  (And I should probably try it in a VM first, just to make sure it doesn't break my Ruby stuff.)

Update 7: As a footnote to Update 5, testing & debugging real-time control systems can get Really Interesting when the timescale for things going horribly wrong is in the tens of nanoseconds... especially when there's a variable lag of several seconds or so after the nearest identified precursor event.  On the other hand, such failures can often be repeated as needed, in a controlled setting, without actually destroying anything.

Summer weather, cont'd

Sometime in the early evening, apparently while I was preparing dinner, the office fan died.

This is kinda important, given that the computers put out quite a lot of heat, as do I, so getting the air circulating is something of a priority in this weather.

Fortunately the lab fan (which has been flaky in the getting-started department, owing to sticky bearings, which I already cleaned & oiled once, and the fix didn't last) is currently working, and once started it keeps running, so I've moved that across the hall for now.

Examining the failed fan reveals that the bearings have gotten sticky... but it's beyond sticky bearings preventing it starting up; it failed while running, and attempting a manual-assist start with power applied gets no indication that the motor is even trying.  So, assume a hard failure, and not worth my time, this busy week, to see if it's repairable.

Gotta swing by a store in the morning and buy a brand-new air mover.  Phooey.

Summer weather

This would be a fine day for driving up to Felton and jumping in the river.

Alas, it's actually a day for slaving over a hot soldering iron and/or metalworking tools, as I repair a test station for a client.

The metalworking tools shouldn't be required, but apparently someone overseas either applied very vigorous percussive maintenance or decided to use the Big Wrench to remove a set of standoffs that I'd carefully Loctited in place when last I had my hands on the thing, because those standoffs were never meant to be removed, and there's no plausible reason for removing them.  So, however it happened, there are four standoffs with their male-threaded ends sheared off, and four holes obstructed by Loctited-in-place, sheared-off threaded bits.  This calls for a bunch of rework.  And: More Loctite!

My garage lacks air conditioning.  All things considered, I'd rather be playing in a river.

Taking a cooling-off break.  Tomorrow morning: soldering-iron time.  (Well, mostly solder paste and hot air.  Through-hole connectors get the iron.)  Then some Very Special Glue, and testing.

Also: While I was out & about rounding up stuff for the repair project, I stopped by the local store of a supplier I've been using for a couple of decades or so, and the clerk noted that not only do I have a longstanding account with them, I'm even on Net 30.  Which is interesting, because if I recall correctly that open account was the result of a mixup back in the 90s, and to the best of my knowledge I've never actually qualified for it.  Oh, well: I guess having such an account, in good standing, with a big-name vendor is a good thing?

Annoyance

That toaster controller, with the FPGA in it?  The project with the hard-to-implement spec?

I came to the conclusion that, at a particular time, I need to dump an accumulator.  Actually, I need to duplicate the accumulator, and dump the duplicate; the original needs to operate as before.

This accumulator is tracking the sum of things in a 200-sample-cycle window.  So, there's a 200-stage shift register, and at each cycle, the accumulator adds the current input and subtracts the input of 200 cycles ago.

Dumping the accumulator itself is easy; just needs a couple extra lines in the Verilog.

The shift register that implements the delay?  Well, it's constructed using the Xilinx "Core generator" tool.  I'm offered the option of reconfiguring it with an SCLR input: deal!

And yet, it doesn't appear to work...?

Oh.

SCLR synchronously clears the last stage of the shift register.  Not the whole thing.  Oops.

In fact, the shift register turns out to be built of SRLC32E, which has no clear input.  The final stage is implemented as a single flip-flop with a synchronous reset input.

Rowrbazzle!

This leaves me with a couple of options that spring to mind: either laboriously (or maybe with a script, or with some o' that fancy Verilog trickery that I've managed to forget in the last few years) build up a 200-stage shift register out of flip-flops (and worry about how many are available at this point), or fake it by gating the output to zero for 200 ticks beginning with the reset event.  Maybe the second option makes more sense, despite the evident lameness?

Oooh... tired.

That modest-sized hike someplace shady ended up being slightly longer than planned, and... well, the choice was between someplace shady but steep and any of several places sunny but not particularly steep.  (Several summarily-rejected options were both sunny and steep.)

My legs still being sore from Sunday, I decreed that not-steep was the order of the day.  Around mid-day, it was hot.  And the promised afternoon thundersprinkle utterly failed to materialize.  So: even more tired now, and I got mildly overheated.

I did get some work done on the toaster project today, but that mainly consisted of getting the new test gadget hooked up, determining that Mode A worked, and seeing no indication of Mode B even existing... then looking into the FPGA code for Mode B, sending a couple of probe signals out to test points, and figuring out that my carefully thought out and group-reviewed plan for handling Mode B needs a serious rethink.

Oh, well.  It's one of those wonderfully fun specs that's perfectly clear and unambiguous on paper, while being impossible (or nearly so) to implement exactly in real-time processing.  My plan was for an approximation, and it turns out that there's a simple test case that it doesn't handle at all.  I need to re-consider the cases.  Got several possibilities floating around my head now, so conjuring up a Vision in the next couple of days is likely.

Playing catch-up

Not dead, just busy with one thing and another....

• Woke up at 0301 this morning, according to the alarm clock.  Ambled down the hall.  On my return, the clock said 0303, and yet the alarm light (set for 0400, give or take inaccuracies in both timepieces) was on.  Eh?  Ah: the clock's DST light was off.  Apparently some little varmint had stood on the clock and pressed the dinky button.
• Yesterday's big activity was a getting-in-shape hike with the offspring.  Out at Portola Redwoods State Park, because it's cooler over there, and there are hills, and we could start at the bottom of the hills and slog up.  We did a smallish scenic loop around the developed area, then headed up the Coyote Ridge Trail with the intent of returning via the Tarwater Trail.  I ran out of steam most of the way up, turned around, and took the shortest route back.  Ended up being around seven and a half miles, and an abundance of exercise.  And, clearly, I'm out of shape when it comes to hills.  Oh, and it was hot, and humid, and we got rained on a little.
• It seems that I've been overdoing the low-carb diet.  Point 1: I was pretty much useless Saturday, with symptoms that turned out to be alleviated by rest and food.  Point 2: I noticed yesterday that I'd been misreading the scale, and in fact I've lost about six pounds, not about three, in the last week and a half (the target having been 5-10 pounds by late August).
• Someone of my acquaintance has been dealing with the state health-care-payment bureaucracy, which has somehow gotten the idea that she's two people with slightly different names and the same Social Security number.
• I'm not even going to go into the bizarre dream I vaguely remember from Wednesday night.

Ah, well.  Monday morning!  A fresh new week!  Gotta get some more work done on the current toaster project (having finally finished repurposing an old controller board into a nifty piece of custom test equipment, standing in for a bog-standard bench instrument that doesn't actually seem to exist), and take the kid for a modest-sized* hike someplace shady, and run some errands.

 

* A curious expression, that.  Modest-sized?  How big was Mussorgsky?

Heisenbugs, again

Working on a project that involves a Spartan 6 FPGA.

Features that used to work had recently stopped working.

I hadn't changed the Verilog code for anything close to those features.  (I checked against the repository.)

In tracking down one of them, I created a pathway to the outside, so I could keep an eye on the contents of an internal register.

It promptly started working.

Looks like yet another case of Xilinx's toolchain optimizing out things that really need to be there.

And we don't have any sort of test automation for this project, so regression testing kinda isn't happening.

Well, the full debugging path was rather cumbersome, but I've replaced it with a spurious use of the register contents.  We shall see....

Update: Bouncing back and forth between projects, I find that the networking issue I've been trying to track down (in the mess of third- and fourth-party RTOS+networking code) is also a Heisenbug; dropping just the right sprintf in just the right place makes it work almost every time.  I suspect some wonky race condition, though such are supposed to be impossible in this context.  Grrrrr.

Update 2: Bounce to FPGA project again.  Could it be a timing problem, and not miscompilation?  Timing analysis is verbosely uninformative.  Look back at source code for the section that's not working at the moment.  Um.  That section is clocked at 10 MHz.  The 100 MHz and 300 MHz sections seem to be OK at the moment; how is logic that has 100 freaking nanoseconds between ticks not working?

(Thinks back to the Olden Days when clocking logic at 10 MHz was fast....)
(But at least you could put a scope probe on it!)

Update 3: FPGA problem sorted.  A key signal was crossing from the 300 MHz domain into 10 MHz.  There was a synchronizer, but apparently it wasn't working.  Or maybe the transition detector just wasn't working in the 300 MHz domain, though it was simple enough it ought to have worked.

Meanwhile, on the other project, it turns out that the problem isn't in fact fixed-enough-for-now, and furthermore there's a problem with configuration memory getting corrupted; the latter issue is possibly architectural in nature, and I haven't really looked at that part of the inherited code yet.  Aieeee!  And the first project (the one with the FPGA) has just sprouted a fine crop of spec clarifications and possibly revisions.  Wowf.

Numbers

Off to the lab yesterday for a much-overdue elbow-probing.

Numbers are filtering in.

LDL, 97.  Good!

Some other stuff I wasn't worried about: normal.

Hemoglobin A1c, which I expect will be kind of high: 9.6.  Aieeee!

I was expecting somewhere between 6 and 7, based on how draggy I've been feeling of late.  The actual reading is waaaay high.

So... well, I'll be seeing my actual doctor tomorrow afternoon, but: I really gotta shed the extra 20 pounds.  More exercise, even if I am feeling draggy.  Absolutely, positively No Sweets.  And maybe I should knock off the granola for breakfast and go more for bacon & eggs.  With that low LDL, I don't need to worry about the eggs and b. so much as the carbs.

Update: Financial-market numbers are looking as bad as that A1c number, only in the waaaay-low direction.  Ugh.  I really hope I don't need to cash in any securities any time soon.  And that this isn't the beginning of the Big Slide, which I'm certain The Powers That Be will be frantically trying to postpone until after next year's election, but then they've already been playing games to prop up the numbers for several years now....

Update 2: Doctor concurs with my plan to try lifestyle adjustments before resorting to nasty pills.  Recheck in late August.  Meanwhile, radically lower carbs and more exercise!

Oog.

The matter of the garage-sink drain having risen in priority of late, I decided it was time to Deal With It.  It had been draining quite slowly for some time, and snaking it out couldn't be that much trouble, could it?

It could.

Disconnect wobbly plastic sink's wobbly plastic flexi-drain from the rigid plumbing.  Poke rigid plumbing with cheapie household snake.  Seems to be OK up to the point the snake doesn't want to go further around the bend.  Hm.

Unscrew cleanout plug* between bend and wall.  Poke snake at bend from that side.  Same thing, and between the two attempts, I'm pretty well convinced that the bend is clear, so it's not an accumulation of sand and gravel causing the problem**.

Next: poke the snake the other way, on beyond the wall.  It goes in a fair way easily, then hits some kind of squidgy obstruction.  Eureka?  Pull it back out with a glob of ick.  Repeat, with various wiggling and turning.  Stop getting ick, but the snake now hits a hard obstruction.

Oh, well.  Obstruction could be junction, no?  Put the plug back in and run some water.

Nope.  Drainage has now gone from slow to virtually nonexistent, apparently dominated by a slow leak into the bucket under the drain.

Guess I need to call a real plumber, perhaps Monday afternoon, in among all the other Stuff I Gotta Do (Monday through Wednesday have cluttered schedules already).

Plumbers are expensive, though.  Maybe I'll make a hardware-store run in search of a shorter, better-configured snake?  Don't need more'n 4 feet, and having a decent handle for turning it, and less of a tail getting underfoot, might just make enough of a difference.

Update:  Off to Home Despot; look at snakes; end up with a Kwik-Spin.  Good thing I got something fairly long; after several more probulization cycles, I stopped getting wads of ick 10 feet or so down the pipe.  Actually being able to turn the thing was a big help (the auto-feed feature, not so much in this particular case).

Put the plug back in: it drains properly now!  Good as new!

I didn't try very hard to identify the fibrous component of the ick, but feathers from cage-cleaning over the years would seem the obvious suspect.

One cool shower later, I'm feeling almost human again.  Almost.

And I haven't managed to get the smell off my hands yet.  Maybe baking soda will help.  Or bleach.  Or nitric acid.  Or maybe I should use the time machine and tell myself to have broken out the nitrile gloves.

But, hey: $30 for a snake beats $300*** for a Silicon Valley plumber.

 

* Maybe not intended as such; it could have been the place a washing-machine drain would have connected, had any inhabitant of this house ever owned a washing machine.

** Come to think of it, I believe I'd removed the bend sometime last year and removed such dense crud as had accumulated: dirt, sand, brass, swarf, and whatnot.  So it should have been obvious that the current blockage was downstream.

*** Or however much the neighbors spent having their drain pipes totally re-done, including reaming out the crawlspace to accommodate modern American plumbers.  I think bringing in a team of Cambodian plumbers on H-1B visas might have been cheaper.

"P"DF grumble

So there's yet another exploit for Acrobelfry....

Reminds me I haven't checked for updates in quite a while, what with it being a manual install.

And, look!  I go to Adobe's site, and not only has the Acrobat reader (or whatever they're calling it these days) for Linux not been updated since version 9.5.something, but it's no longer available through the regular download path.  (Version 9.5.5-1 still exists, but only if you know where to look.)

Yes, there are open-source programs for reading PDF files.  They mostly work.  In some cases, they work better than the Adobe product... especially when it comes to printing to a Postscript printer.  (Adobe Acrobat has long been notorious for emitting output that Just Won't Work on a printer that uses Adobe Postscript.  This isn't just a Linux thing; I first noticed it during my time in Corporateland when using a company-issued Windows NT system, back when NT was a thing.)

But... it seems none of them support all the funny features of the "Portable" (ha!) Document Format.

Not that I use any of the fancy unsupported features, but it's a potential work-compatibility issue.

And: none of them support having a gazillion files open in a single window, with a tab bar running along the bottom to select among them.  This is how I use it.  I tend to have quite an assortment of datasheets, manuals, client specs, and other assorted documents sitting open, all in a single window on virtual desktop #11, sandwiched between the web browser (desktop #10, typically several browser windows each with multiple tabs open) and the text editor (jEdit on desktop #12, with, at the moment, 474 files open, and a buffer-list sidebar to manage them).

So, do I need to grab the source code for one of the better open-source viewers, and write a more Adobe-like wrapper for navigation and open-document management?  Office software really isn't my thing; I'm more of a bit-basher, and find GUIs rather baffling.

Actually, that might be a good idea anyway.  Adobe's product has some weird bugs that show up when it's used the way I use it; after a while, the UI becomes horribly and inexplicably sluggish (like taking a minute or so to open a file or print dialog, while not using significant CPU time, so what's it doing?); I don't know if this is a function of time or of the number of files that have been open, but it's highly annoying.

Ferschlugginer third-party software!

Back to arguing with the open-source RTOS/TCP-IP hodgepodge, plus another third-party component which was taken proprietary a couple of years ago (I'm dealing with the last open-source release).

This particular combination is, well, ugh.  Serious case of integration being out of phase, important things not quite working right, and the inter-package interface documentation being pretty much nonexistent.  Also, all available demos are way out of date.

I'll muddle through, I guess.

I was about ready to hang up the AGROS project apart from ongoing support for existing clients, but, seeing the state of certain popular products, I'm having a rethink.

I still need to look at another branch of the open-source RTOS world, for futures, but if that turns out to be anything like what I've been dealing with, I'll consider it worthwhile to give AGROS its long-overdue makeover (much assorted cleanup, port to Cortex M architecture, re-do the configuration program using Qt and possibly a different scripting language, write some device drivers for different MCU families, and document everything - then try to recruit a user group to deal with more device drivers, porting to MIPS and whatnot, and stuff like that).

Because stuff that's dead easy under AGROS - as long as it's on an LPC2xxx platform - is a royal PITA in this other environment.

One of the hangups (other than getting out of the ARM7 world) has been UDP, but I actually did a partial implementation of UDP last summer, it looks like a full implementation will be needed for an existing client this summer, and the main sticky aspect had been the API, which, thanks to the current adventure, I can now see how to do better than what I'm dealing with.  Once UDP is there, DHCP is reasonably easy, and a useful subset of mDNS shouldn't be a big deal apart from wading through all the RFCs that define parts of it.

Update: Just to make my life more and more wonderful, Eclipse and gdbserver sometimes get somehow out of sync, resulting in breakpoints happening in places entirely other than where I set them.  So, as I try to sort through the out-of-phase components of this mess, I start regularly hitting breakpoints on frell.c line 434, when the only breakpoint I have set in frell.c is supposed to be on line 399, in the preceding function.  Growf!  And, no, shutting down Eclipse, gdbserver, the target hardware, and the pod, and then restarting everything, doesn't solve the problem.

Update 2: Optimizing linker can be confusing when debugging.  That breakpoint at line 399?  I'd commented out, for the moment, the only call to that function, in another module, so the code for it disappeared, and what would have been its start address had become that of the following function.  Maybe I need to figure out how to turn that off for debugging purposes.