Posted by Tom Moertel
Fri, 25 May 2007 21:42:00 GMT
The Kenwood audio-video receiver that forms the core of my
home theater system stopped responding to its remote control. As I
discovered shortly thereafter, having to leave the couch to fiddle
with knobs degrades the “home theater experience.” Clearly, something
had to be done.
I knew the receiver was the culprit because the remote control worked fine
with other components of my system. I figured the IR sensor had gone
bad and did a little Googling for “Kenwood” and “IR sensor” and
“problem”. The results revealed that
many other Kenwood customers had the same
problem.
The cause of the problem, I learned, was that the solder joints which
connect the IR sensor’s leads to the display board eventually fail
because of thermal expansion. That explanation seemed to account for
what I was observing, so I cracked the case in search
of visual confirmation.
First, I found the joints where the IR sensor was connected to the
circuit board. The vertical red line shows where I found them:
Then I examined the joints closely. Sure enough, at least one had completely failed:
The problem confirmed, I moved to the solution phase of the project.
With my soldering iron, I touched-up the joints:
It wasn’t my best work, but it did the job.
Now my receiver is back on speaking terms with its remote control,
and I have returned to the modern world. Life is good
Posted in hardware
Tags hardware, ir, kenwood, receiver, repairs, soldering
38 comments
no trackbacks
Posted by Tom Moertel
Tue, 30 Jan 2007 03:08:00 GMT
I just checked my inbox and noticed the following urgent message from the SMART daemon on my laptop:
The following warning/error was logged by the smartd daemon:
Device: /dev/hda, FAILED SMART self-check. BACK UP DATA NOW!
For details see host's SYSLOG (default: /var/log/messages).
Crap.
Luckily, I recently ordered a new laptop, and it should be arriving tomorrow. Good timing.
Once it arrives, I should be able to transfer my dying laptop’s personality to the new laptop from the back-up copy on my development RAID system. Still, getting things set up just right will probably eat half a day.
Posted in hardware
Tags backup, failure, hard_drive, life, smart
no comments
no trackbacks
Posted by Tom Moertel
Wed, 09 Aug 2006 04:35:00 GMT
If an extended power outage drains your UPS, and your servers are
forced to shut down, will they automatically start up again when the
power is eventually restored? It’s a good question, especially
if your servers are in some distant, unattended server room.
Unless you’ve tested your servers, don’t assume that the answer
is Yes.
Many servers offer a BIOS configuration option that forces them to
automatically power on when they receive line voltage. If your
servers have this option, just set it and you’re done.
Unfortunately, some servers, including a Dell PowerEdge 1600SC
that I’m using, lack this configuration option. When these servers
turn themselves off as the final step of a UPS-controlled
shutdown, they don’t start up again when the power is restored.
Because they were shut down before the power was cut off, they think
they are supposed to remain off when the power is restored. That is,
they remember their on/off status across power outages.
Fortunately, there is a way to make sure these servers automatically
power on: shut them down without powering them off; halt them
instead. That way, when the UPS finally cuts off the supply voltage,
the servers will still be in their “on” state, and they will remember
this state across the outage. Later, when the power is restored, the servers
will automatically restore their pre-outage state and power up.
With Fedora Core Linux and Network UPS
Tools, it’s not difficult to make
sure the servers are halted instead of powered off, but the implementation
isn’t obvious. To spare you the digging, here are the
important bits.
- When the power fails and the UPS-monitoring software decides that
the batteries are almost depleted, it will initiate a server shutdown
using the command defined in the
/etc/ups/upsmon.conf
file. The default command is this:
SHUTDOWNCMD "/sbin/shutdown -h +0"
- The shutdown command will tell the
init process
to enter runlevel 0, which is the prepare-to-halt-the-system runlevel.
- The
init process will stop all of the running
services in an orderly fashion, and then, as the last step, invoke the
final script in the shutdown process:
/etc/rc.d/rc0.d/S01halt.
- The final lines of the
S01halt script will
power off the server. Unless, that is, the file /halt is
present, in which case the script will halt the server instead.
Thus the trick is to make sure that the /halt
file does exist. The trick turns out to be easy to pull off;
just redefine the shutdown command in /etc/ups/upsmon.conf:
SHUTDOWNCMD "/bin/touch /halt; /sbin/shutdown -h +0"
And that’s all there is to it!
Posted in linux, hardware, sysadmin
Tags fedora, halt, hardware, linux, nut, power, shutdown, ups
2 comments
no trackbacks
Posted by Tom Moertel
Tue, 27 Jun 2006 21:41:00 GMT
Recently I wrote about a bad support experience I had with Dell. Today in my inbox was an invitation to participate in a survey about my experience.
I accepted.
My survey responses, which were completely honest, were not glowing. Although Dell ultimately resolved my problem (the cabling shipment arrived the following day), the hoops I was forced to jump through were unacceptable. Dell’s support seemed fundamentally broken, and I had to fight to make it work.
When asked what Dell would need to do before I would feel comfortable recommending them to others, I wrote:
I would need confidence that Dell makes it easy for clued-in technical customers to speak with clued-in support personnel. As things stand, clued-in customers waste too much time on the phone with ineffective support personnel. In my case, I was handed off numerous times and ended up speaking with seven support persons, and only the final person had the knowledge and empowerment to make the situation right for this customer.
I also gave them a link to my article about the experience. It will be interesting to see if anybody reads it.
In any case, I am glad I received the survey invitation. At least it shows that Dell is trying to improve. Further, the survey asked the right questions: I was able to adequately express my dissatisfaction and point out where I thought their process had broken down.
I do hope somebody at Dell figures it out because support is the company’s Achilles heel. HP, in my experience, smokes Dell in this regard.
Update: It looks like my article got the attention of
CMP Media’s
CRN, a source of “vital information for VARs and technology integrators.” Edward F. Moltzen linked to my article in his article of 6 July 2006:
Dell Works, Spends To Get Back Into Good Graces.
Perhaps not entirely coincidentally, on the very same day, a “customer advocate” with Dell’s headquarters in Round Rock, Texas, sent me an email offering help and asking for feedback. Owing to a mix-up, I did not receive his email until he re-sent it on 11 July, but I have since responded with a detailed summary of my experience.
Things are getting interesting.
Update 2006-07-21: It seems the Dell customer advocate was serious
about fixing problems. He reviewed my case and was able to identify a
user-interface problem with Dell’s web site that probably led to a
good part of my difficulties. The problem is that a non-support phone
number is offered in a portion of the support section of Dell’s web
site. Customers, like me, who call the phone number are connected to
people trained to handle pre-invoice issues, not support. Oops.
As of this writing, the UI problem still exists, but I trust that
it will be solved soon.
Posted in hardware, business
Tags dell, support
12 comments
no trackbacks
Posted by Tom Moertel
Fri, 23 Jun 2006 18:56:00 GMT
Yesterday I received a new server from Dell to replace the server I
decommissioned earlier this
week.
My initial impression was favorable: the build quality looked good,
the power supply seemed up to the task, and the heat sinks and
blowers seemed almost absurdly beefy.
And then I noticed the drive bays.
Dell, doin’ it’s own thing, server style
You need to understand that Dell engineers like to do their own thing on
occasion.
Sometimes they will use power supplies that have weird mounting
configurations. Or snap-in blowers that are hard to source. Or, in
this case, drive bays that require goofy mounting hardware.
On this server there are two non-standard things about the drive bays:
- They require proprietary, snap-in drive carriers (but for
servers this is fairly common).
- They are located so close to the edge of the server’s case that
special low-profile, right-angle power and SATA data cables are
required for drive mounting. This is basically a non-stop train to
Goofy Town.
Now, here’s the head scratcher. Somebody at Dell was smart enough to
fill each of the bays with a proprietary drive carrier. That person
realized that if Dell didn’t provide the carriers, the bays were
pretty much useless, and customers would likely be upset because they
couldn’t actually use the drive bays they had just purchased as part of their shiny new servers.
Likewise, somebody was smart enough to provide the special
low-profile, right-angle power cable required for each bay. But
nobody thought to provide the special low-profile, right-angle SATA
cable required for each bay. Oops.
If it were a standard cable, I could understand the omission. In this
case, however, the cable is effectively proprietary and thus should
have been considered an essential part of the bay itself, just like
the carrier and power connecter are, and provided out of the box.
Dell’s phone support: “You are in a maze of twisty little passages, all alike”
So I called Dell, using the phone number on my order screen, to get
the required cabling. Once Dell picked up, the problem was solved
with a simple, 17-step process, requiring only about two and a half hours
in phone-maze hell:
- After phone-tree surfing, I ended up talking with Temi. She didn’t
know the part number for the cable, so she said she would have
somebody call me back.
- An hour later, Scott – in sales – called me back. He said he really
couldn’t sell me the cables because “there was essentially no resale
value to them.” But he said maybe “parts” could help me out. So he
transferred me to Manuel in the parts department.
- Manuel was able to narrow the selection down to four potential cables.
But he didn’t know which of the four I needed. So he said he would
conference in a tech specialist, who would be able to pick the right
cable.
- I ended up back at the main menu of the phone tree. Manuel
was not on the line. Oops.
- Once again, I surfed the phone tree to business customer service.
- This time I was connected to Cathy. I explained the situation.
She said she couldn’t help me but would transfer me to somebody
who could.
(At this time, I had been on the phone for one solid hour.)
- George picked up. He seemed clued in. After I explained the
situation, I could sense that he got it: It is not cool
to ship a customer a server with effectively unusable drive bays.
- Unfortunately, George said he was not the right person to take care
of the issue. (I got the feeling he was in the support group for
big-money enterprise customers and that my small company didn’t quite
make the cut.) He said he would give me the
exact phone number and extension to call to speak with the people
who could get the job done.
- When he gave me the
number, I noticed it was the same number that had been on my order
screen and had started my mad quest through Dell’s customer-support,
phone-tree hell. When I informed George of this, he seemed surprised.
In that case, he said, he would personally transfer me to a “resolution
specialist” who had the clout to get things done. Further, he
assured me, he would make sure the specialist understood the
situation before he handed off the call. Cool.
- George conferenced in Erica and briefly explained the situation.
OK, she said, she would help me out. George said goodbye, and I
thanked him.
- Erica, now in charge, asked me what I wanted her to do. I said,
figure out what the right part is, and send a shipment to me. Erica
said that she didn’t know what the right part was, but she could
transfer me to parts, and they could probably help me out. I said,
no way, I had already talked to parts – about an hour and a half ago
– and now that I was speaking with a resolution specialist I
didn’t want to be de-escalated.
- At this point, everything fell apart. Erica said that she couldn’t
get me the parts. All she could really do, in fact, was arrange for the
server to be picked up for a refund. This blew my mind.
- Staying calm, I pointed out the absurdity of the situation:
“Wouldn’t it make
more sense to conference in the right group and just have them
send me the parts? Think about it,
you just sold me the server. Now you’re telling me that the solution
is to send the server back for a refund? If I do that, it will be
as if I had not done business with Dell in the first place. Are you
absolutely certain that the best solution Dell has to offer is
effectively the same as not doing business with Dell? Doesn’t that
strike you as absurd? Aren’t you empowered to do something that
makes a little more sense, both for me and for Dell? George told
me that you were a ‘resolution specialist’ who had the power to
make things right for customers. Isn’t that the case?”
- Apparently, the situation struck somebody as absurd because at
that moment a gentleman by the name of Michael broke into the
conversation. He thanked Erica for her help and said that he would
be taking over the call.
- After Erica left the conversation, Michael explained that he and
George (from step 10) had been monitoring the conversation since
George’s hand-off, just to make sure the
situation was handled properly. Because the call seemed to be
headed in the wrong direction, they felt it was time to take
the call back and make things right themselves.
- Michael – who seemed like a no-nonsense kind of guy – said
that he was going to find out what I needed, make sure it was
in stock, and get it to me. And that’s exactly what he did.
In about three minutes, he had confirmed the part number of
the correct cabling, verified that it was in stock,
and then handed the call over to George,
who (1) arranged for the shipment, (2) gave it some kind of insane
priority that he said would get it fulfilled before the shift
change in the next hour,
and (3) got me a tracking number. I thanked George for his help,
and he gave me his direct line, just in case I ever needed it.
- Problem solved.
All in all, I am not happy with Dell’s support. Even though Michael and George kicked ass on behalf of this customer – note to Dell: you need more guys like them – it was too late to undo the damage caused by nearly two hours of ineffective prior support.
I have some more thoughts that I will share later, especially regarding the comparative merits of HP’s support.
Until then, does anybody have any other entertaining phone-support tales to tell?
Update:
See
Dell-support follow-up survey
to read about how my problems with Dell support ended up getting the attention of a business magazine and Dell headquarters.
Posted in hardware, business
Tags dell, phone, support, tech
3 comments
no trackbacks
Posted by Tom Moertel
Sat, 18 Feb 2006 02:58:00 GMT
I am happy to report that I am typing this post on my new homebrew
workstation. It sure does feel snappy!
Vitals: AMD Opteron 165 (dual core), 4-GB ECC RAM, 500-GB
RAID5 storage (hot-swap trays), Fedora Core 4 Linux (workstation
install). I went with the AMD Opteron because of the on-chip memory
controllers and better I/O architecture.
Here’s a snapshot taken halfway through the assembly process:
The heat-pipe system that AMD provided to remove heat from the Opteron 165
reminds me of the exhaust systems on top-fuel funny cars:
Dig that shiny copper!
Posted in hardware
3 comments
no trackbacks
Posted by Tom Moertel
Fri, 17 Feb 2006 07:23:00 GMT
Tonight while building a new workstation, I needed to update the BIOS
on the motherboard, a Tyan Tomcat
K8E. Tyan, however,
offers only floppy-based BIOS flashing software to do the job. Worse,
the software requires me to boot into DOS first, using a DOS boot
floppy that is neither provided nor lying around the office (I’m
a Linux guy).
One more thing: it turns out that my new floppy drive is junk.
Thus we arrive at tonight’s problem: If you do not have a floppy drive, how can you flash a motherboard’s BIOS when its manufacturer provides only a DOS-floppy-based BIOS flasher?
Fortunately, the problem can be solved. In case you ever need
the solution, here it is.
Disclaimer: This recipe worked fine for me, but might not for you. If you follow these instructions, you do so at your own risk and assume all responsibility for whatever happens, even if your computer catches on fire or your pants explode. You have been warned.
First, download a bootable floppy image from the FreeDOS
Project. The one you want is the 2.88-MB
ODIN image because it has
about 1.5 MB of free space, enough to hold the contents of the BIOS
flasher’s floppy.
Second, mount the floppy image so that you can edit it:
mkdir /tmp/image
mount -o loop /path/to/odin2880.img /tmp/image
Third, copy the BIOS flasher and associated files into the mounted
floppy image. I just unziped Tyan’s archive directly into
the image:
unzip /tmp/tyan_2865_301.zip -d /tmp/image
Fourth, unmount the image.
umount -d /tmp/image
Fifth, create a bootable CD-ROM from the floppy image.
cd /tmp
mkdir boot_cd
mv /path/to/odin2880.img boot_cd
mkisofs -o odin-cdrom.img -b odin2880.img -c boot.catalog boot_cd
cdrecord -v -eject odin-cdrom.img
Finally, reboot your PC using the CD-ROM and flash away! (Note: If FreeDOS asks, you don’t want to use extended memory or anything like that because BIOS flashers don’t like it. You want old 8086-style
unprotected memory.)
Posted in hardware, hacks
no comments
no trackbacks
Posted by Tom Moertel
Tue, 15 Nov 2005 19:23:00 GMT
Recently I had Verizon’s fiber-optic service “FiOS” installed
at my home. The installation process took about a half day
and involved placing the following boxes around my house:
- optical network terminal (ONT, installed outside of house)
- battery backup unit (BBU, installed in basement)
- power adapter (plugged into household electrical outlet)
The ONT was installed next to my old POTS
junction box:
The ONT acts like a miniature central office. To my house it provides
four POTS lines for voice service and one 10/100 Mbps Ethernet port for
data service. The ONT accepts a single fiber-optic cable that
connects all of these services back to Verizon’s central office.
As part of the installation process, Verizon moved my POTS lines from
copper over to the ONT’s POTS interfaces. Verizon wanted to remove my
copper-based service altogether, but I forbade them from doing so
because I have non-Verizon business lines that I want to keep on
copper, which competitive carriers can use to offer me service.
(Verizon is not required to share its fiber cables with competitive
carriers.)
If you look closely at the ONT, you’ll see that it also is capable of
handling video service:
(At present Pennsylvania’s cable-franchise laws prevent Verizon
from offering video service, but I’m sure Verizon’s lobbyists are
working to change that situation.)
Unlike copper wires, fiber-optic cables do not carry power. The ONT,
therefore, must be powered from my home’s electrical service. If the
power goes out, the battery backup unit (BBU) will supply power for
the ONT’s voice services for about four hours.
VoIP users beware: When the household power fails, the ONT’s data
services will be dropped immediately in order to conserve the BBU’s
battery. This seems pretty lame to me, but Verizon confirmed this
behavior when I called them to ask about it. If you need data service
during a power failure, make sure your ONT is powered via a UPS
under your control.
To provide data service to my house, the installer ran a CAT-5 cable
from the ONT’s 10/100 Ethernet port into my house, where it plugs into
a D-Link 4-port “Ethernet Broadband Router,” provided by Verizon for
free. Although the provided router has NAT and firewall features, I
placed a Linux-based firewall between it and the rest of my home
network as an added precaution.
I have been using the service for several days now, and here is my
verdict:
It’s just broadband.
Practically speaking, I can’t tell any difference between FiOS and my
Adelphia cable-modem service. I ordered 5-Mbps service from both
providers, and both services provide about 5 Mbps down, which is
faster than fast enough for me. The FiOS service has slightly lower latency – I
can ping www.google.com in about 9 ms – and that’s a nice plus.
The big benefit of FiOS is competition: Verizon’s price is about $10/month
less than Adelphia’s. When I called Adelphia to cancel my service,
their representative attempted to change my mind by offering me a
3-month promotional discount and trying to sell me extra television
channels.
I passed.
Posted in reviews, hardware, networking
Tags fios, networking
41 comments
no trackbacks
Posted by Tom Moertel
Sat, 12 Nov 2005 02:42:00 GMT
The main network switch in my home office is an HP ProCurve
4000m, which
has been running non-stop for over half a decade. It is a great
switch, and even though it is getting old, it is still dependable.
A while ago I noticed that the 4000m’s fault indicator was
lit. So I logged into the switch and checked the log: fan
1 was dead. The switch has built-in redundancy (three fans),
and so I didn’t worry about it, but I did call HP ProCurve
tech support.
The woman I spoke with was friendly and helpful. I told her what was
wrong, and she said a new fan array would
be on my doorstep within 48 hours. No charge. (I guess the
ProCurve warranty really is worth something.)
Today, I installed the array. This meant opening up the switch,
which is a fun thing to do. If you are curious about what is inside
of a 4000m, I took photos of the operation.
During the process, I recalled why I love old-style HP engineering:
- The replacement parts came with clear instructions
that showed me how to remove the old array and install the new one.
They were easy to follow and didn’t leave anything to guess.
- The 4000m is solid – inside and out.
- The electrical components are top quality.
- The industrial engineering is superb. For example, all of the user-removable
screws have non-stripping torx heads and are designed not to fall
out and get lost; instead they remain attached to the module or
panel you are removing. (See this photo of removed modules
to see how the screws stay in place.)
Everything about the process made me think, wow, this is really well engineered.
The thing is, I know, as I sit here and watch the blinking LEDs on my
now-restored 4000m, that my next network switch will probably be a
Dell.
As much as I love the ProCurve engineering, the Dell price is
compelling. Even if I expect the Dells to fail twice as often (and
the Dell warranties are comparatively lame), I can buy twice as
many Dells and keep spares on the shelf – and still save money
compared to the equivalent ProCurve equipment.
I find the situation somewhat sad. I am an engineering guy to the
core. So when I go for the cheaper product because it is so darn cheap,
I know that much of the market will do likewise. That bodes ill
for HP. Like HP’s calculators, the ProCurves too may pass into
history.
Posted in photography, hardware, engineering
Tags 4000m, engineering, hardware, hp, networking, photos, switch
11 comments
no trackbacks
readers
