BunkBlog

The rather useful service Loudtwitter seems to be dead.  Thankfully, the geeks who program for WordPress have a cool tool I can use in its stead. So, you’ll continue to get all my blog posts, and tweets on my blog (and Facebook), and all that plus locked LJ entries on LiveJournal. Myspace’s RSS support remains MIA, so that particular backwater is not on my list of things to worry about. Or, you can just link to the Unhub bar and be done with it.

This has been your overdose of geekery for the day.

Caution: extreme geekery ahead.  Do not continue reading unless you know what hda and hdb are, and the joys of logical volume managers.  Cheers.

Way back last weekend, the DVR I built in 2006 decided to be cranky. I had originally built it with a 250 GB IDE drive, then added a 160 GB IDE drive which I’d salvaged from an older WinBox I replaced.  Sometime after that, I put in a 400 GB SATA drive.  That brought me over 800 GB total capacity, which has been enough for the past couple years. I noticed the temperature was increasing on the hard drives a while ago.

I didn’t worry about it, as the machine worked just fine regardless.  Then. last week, I noticed a little hiccup in the video feed while watching the news. As ever, when the tuners have ever had a problem, I rebooted the DVR to reset the tuners. Turns out, the tuners weren’t the problem. As the machine rebooted, it came up with an error which I’ve never seen before, and then halted the boot. Linux boxes don’t tend to halt their boot processes for much beyond catastrophic failure. I spent that night and part of the next day doing the research I needed to, and ran all the appropriate fixes (LVM activation and E2fsck with alternate master record blah blah). Finally, nearly one day after the machine went down, it was back again.

As you can see from the thin white line near the end of this graph, this is the only significant downtime my DVR has suffered in at least the last year.

I looked through the messages log and saw a lot of errors for hdb that imply some hardware failure. Since I bought that hard drive about 6 years ago and it’s been in constant use for over three years (the DVR never sleeps), it’s not surprising I suppose. Now, of course, I have to take care of this. Since I’ve got the machine set up as an LVM drive, I can add a new hard drive and move the data on to it, then remove the old drive. This is complicated by the fact that I have about 100 GB free, and the dying drive is 160 GB. So, I need to add a new drive before I can remove the old drive. That case is kind of crowded. The joys of vgreduce, pvremove and all manner of other exciting commands are in my future.  I’ll probably get a 500 or 750 GB drive, so the DVR will finally crest the 1 terabyte mark. I’ve already got more than a terabyte on my desktop, so it seems only fair that the DVR get to the same level.

The LVM setup is very confused right now anyway.  I’ve got hda1 (4.5 GB) as a standard ext3 boot and root drive. Simple.  MythTV originally required a “live TV” partition, so I had hda3 (15 GB) as the cache and hda4 (212 GB) as additional storage space (the recordings and such). When I added the 160, that became one big hdb1, still simple. The next upgrade of MythTV removed the necessity to separate live TV from recorded shows, and so the hda3 partition became useless. I added it to the LVM pool, which is a bit silly – hda3 and hda4 are both in there, and not just one merged partition. When I bought the 400 GB drive, that got tacked on as sda1. So, my LVM pool is hda3 hda4 hdb1 sda1. When I get done with the next planned upgrade, it will be hda3 hda4 sda1 sdb1.  Maybe I’ll go ahead and merge hda3 and hda4 while I’ve got the thing down anyway. Pretty much any drive I buy today will be large enough for the 217 GB of hda data and the 160 GB of hdb data, and then I’ll have to delete and recreate the hda3 partition so I can add it back to the LVM pool.  In case you want to sympathize with my plight, here’s the full sequence: physically add sdb, partition sdb1 as LVM, pvcreate sdb1, vgextend VG to include sdb1, pvmove all extents from hda3 hda4 and hdb1 to sdb1, vgreduce hda3 etc. to make them no longer active participants in the LVM, repartition hda3 from the old hda3 and hda4, vgextend to include new hda3, turn off machine and physically remove dying hdb.  Then, it’s back and room for more packrat behavior!

More DVR fun… This morning, one of the five fans in the DVR started making the grinding sound that indicates something has brushed against the fan blades. After two years of never opening the case, I finally cracked that bad boy… yuck. After giving every fan and air duct a good vacuuming (sorry Leo), the machine is now five degrees cooler inside.  Guess that worked.

Sadly, I now have 20 minutes of downtime this morning. Linux geeks are all about the uptime.