When Is The Best Not The Best?

On June 11, 2007, in Games, by peterb

I had what started off as an extremely productive weekend. I got a lot of work done at my job, did some house stuff, changed the oil on the car, and was otherwise efficient and in a good mood. As a reward, I decided to play some more Forza 2, a game I’m reviewing for PTD Magazine. I’ve been enjoying Forza, and approached my Xbox 360 with some enthusiasm. This should have been a warning sign. Because, inevitably, as soon as I turned the machine on, 3 blinky red lights appeared, and the machine was officially as dead as a doornail.

The machine was one year and one month old. Translated, that means the machine was one month out of warranty. 45 minutes on the phone with Microsoft customer service in Durban or Chennai or somewhere resulted in them agreeing to drop the repair fee from $140 to $65.

Now, there are a few directions in which I could take this article. First would be the observation that I would rather simply not use an Xbox 360 than pay $140 to get it fixed, and that from Microsoft’s perspective, asking for that much seems sort of, well, goofy. You’re already losing money on the console. The only way you reach profitability is if I buy games. But this isn’t really a topic about which there’s very much to say. Philosophically, it’s not actually surprising to be asked to pay money to fix a machine that’s out of warranty.

Perhaps a little more interesting is that they dropped the price simply because I bitched about it. That seems to me to be a poor customer service experience by definition. It’s used up 45 minutes of their support time, and I’m left with that “just negotiated to buy a new car” sort of feeling. Maybe if I’d stayed on the phone for an hour they’d have dropped the price by another 25%? Will someone who reads this article who did pay $140 to fix their machine now be furious and feel ripped off by Microsoft? It just seems to me that by removing certainty from the equation you’re making the entire experience more hateful, and squandering company goodwill on something that, in the long run, is pretty picayune. Either make the transaction completely painless from the beginning (“Here’s your new Xbox 360,”) or stick to your guns (“We’re sorry that you don’t want to pay the repair fee, sir. Have a nice day.”) But making the transaction turn into a negotiation worthy of an Algerian souk just seems pennywise and pound foolish.

But the most interesting question to me is that of consumer electronics and reliability. Now, I’m picking on Microsoft here because I own a console that blew up, but I presume that once 10 or 20 people buy PS3′s we will see similar reliability problems with those machines as well. Consider how unusual it is to hear of nonportable consumer electronic devices that just stop working a year after purchase. How old is your DVD player? How old is your stereo receiver? 2 years old? 5? 10? I still have VCRs that work after 20 years, and those have more moving parts than an Xbox. As long as we’re talking about game consoles, I still have Atari 2600s that work just fine. Yet the Xbox 360 has developed such a reputation for bursting into (virtual) flame that its failure mode has garnered a nickname and a Wikipedia article.

This is the collision of two worlds: the world of computer hardware and software, where the people developing on the “bleeding edge” expect some number of problems, and the world of consumer electronics, where things are supposed to just work. I’m extremely unimpressed with any company pushing products that lean towards the former rather than the latter.

I understand the technical reasons why the Xbox 360 is prone to hardware failure: Microsoft chose a design where they crammed a bunch of extremely high end equipment into as small a case as they could manage. Their engineering decisions create unreasonable amounts of heat, and unreasonable amounts of heat lead to an unreasonably low mean time between failures. That there are technical reasons for the console bursting into flame whenever a goldfinch looks at it sideways doesn’t make it acceptable.

In the meantime, we can compare this to the Nintendo Wii, which, through conservative design and engineering puts out nearly no heat, and certainly has not (yet) garnered a reputation for being unreliable.

All of which leads me to the question: what good is the most advanced graphics subsystem in the world if you’re incapable of packaging it in a way that performs reliably for an acceptable period of time?

 

18 Responses to “When Is The Best Not The Best?”

  1. Dr. Click says:

    I’m a little confused here. *You* started to haggle with *them*, and then didn’t like that they decided to indulge your desire to haggle?

    What you’re basically demanding is that they save you from yourself.

  2. peterb says:

    I’m not demanding anything. I’m perfectly capable of being glad to save $70, while still being irritated that I had to spend 45 minutes on the phone to Chennai to get there.

  3. BÃ¥rd says:

    If you had lived in Norway, you would have had a 5 year warranty that makes Microsoft bleed

  4. Duncan says:

    Your comparison of a console to a DVD Player, Receiver, or VCR is an unfair one. With the exception of the VHS machine (who’s technical lifespan has generated it’s fair share of lemons), a system like the Xbox 360 does have more moving parts, more technical requirements, and a harder life in general. Most receivers and DVD players don’t have hard drives to manage. Let alone graphics processors that are demanded to create visually stunning imagery every moment the device is running.

    The physical stress your console endures is akin to a mid- to high-end computer running nothing but Oblivion it’s whole life. And who hasn’t ever encountered hardware failures in the first year of owning a computer?

    Yeah, the fact that they shoehorned a space heater into that tiny case doesn’t help. But no one had created a system that does what the 360 is expected to do, under the conditions the 360 is expected to endure. Ever. These are design hurdles that are hard to test for while under a shipping deadline from management. Some failures are expected, especially if there was an initial production issue.

    Microsoft should be covering these kind of manufacturing issues for the life of the system. They should also be attempting to fix these issues, and I wouldn’t be surprised if they aren’t (or haven’t already). I’d expect to see a percentage of older systems continue to encounter these problems, but fewer and fewer as newer production runs enter circulation.

  5. psu says:

    I think the point is that the machine was designed to requirements that are too stringent for long term reliability. If they can’t make the machine run well then maybe they did something wrong, hmm?

    But, FWIW, out of the last 5 computers I have bought or otherwise acquired, none had any internal hardware failure. I did have a couple of laptop screens go bad thought.

  6. Alex Groce says:

    Duncan: I think this means “the 360 was not ready for prime time.”

    I mean, it was “okay” given technology that the early cell phones were the size of a brick and had five minutes of battery life. But there’s a reason nobody bought them then. The 360 is clearly better than that, but maybe this suggests “less like a PC is good.”

  7. Dr. Click says:

    Duncan:

    Please get back to us when you identify the DVD and VCR that doesn’t have moving parts.

    Thanks.

  8. Nat says:

    It’s fine to say “wow, this design is really complicated and hard to get right”.

    When you combine that with “this is a finished product, please pay us $400 for it”, it suddenly gets a lot harder to get away with the but-this-stuff-is-haaaaaard excuse.

    If it’s too hard to make a stable product with a given set of characteristics, the right answer is *not* to make it anyway and then call the customers suckers for expecting it to, y’know, work in exchange for their money.

  9. Nat says:

    Also: I’d claim that the steady stream of apologists explaining that we shouldn’t expect complicated pieces of technology to work are PART OF THE PROBLEM.

    I mean, why should manufacturers go through the time and expense and effort of real QA and testing if we’re just going to shrug our shoulders and say “eh, these things just break, that’s how it is” when their poorly-engineered pieces of crap inevitably fail?

    To hell with that. In almost any other part of the market, the idea that a FOUR HUNDRED DOLLAR purchase shouldn’t reasonably be expected to be reliable would be completely insane.

    We should expect better than this.

  10. Duncan says:

    Dr. Click – Most DVD Players only have the DVD Drive, not a hard drive. Most recievers have at most a single servo to move the large volume knob (some don’t even have this fancy feature anymore), and I did make an exception for VCRs. But most Xbox 360s aren’t suffering from mechanical failure of moving parts anyway.

    Nat- When was the last time a mass produced product was stress tested after a full-scale batch was assembled, but before shipping to distributors? I would venture: never. You create a prototype, you use a turn-key solution to generate a beta batch. You stress test the daylights out of the beta. Then you have to implement full-scale production. But the means, and sometimes the suppliers and parts themselves, are different from the small scale batch. Yes, you can do some testing with the mass produced units, but you won’t be able to do the same stress testing that the beta units went through. You can’t afford to, because that’s a lot of money sitting around not being turned into profit.

    My bet would be that the units they stress tested were a lot hardier units simply because they were made in a smaller batch, with choice parts. They were also more expensive units. As much as 2-10 times as expensive (depending on parts and assembly). Not a cost you’d want to pay as a consumer. These are simply the realities of mass production. The test unit is usually a sturdier unit, and you can’t predict all of the minor changes that affect the unit when it shifts to mass production.

    Once they hit the shelves you just have to deal with it.

  11. Nat says:

    If the units they stress tested were “hardier units” made “with choice parts”, then it’s really not a particularly good test, is it?

    You can concoct theories all you like about how it’s simply impossible to avoid problems like this, but the simple fact is that we don’t see this kind of pain and failure rates in many other manufactured products. If I go buy a Kitchenaid mixer, I don’t expect it’s going to blow up within two years, and they have plenty of moving parts.

    What you’re saying is really that reliability costs money. That’s not surprising. I claim that the answer shouldn’t be “consumers should suck it up so manufacturers can make more money”.

  12. Dr. Click says:

    Now I have really lost the thread of Duncan’s argument, which appears to boil down to concluding that testing is impossible because testing gold-plated prototype designs doesn’t tell you anything about the field performance of shit-plated production designs. The claim is that Microsoft tested ruggedized prototypes of the Xbox (“hardier units” etc.), but went on to use (sub-)standard parts in mass production. The conclusion I would draw is that the problem with the Xbox isn’t that there’s some fundamental difficulty in testing, but is that Microsoft made a deliberate decision to cut corners on production quality, and ended up delivering sub-standard boxes.

  13. Duncan says:

    What I am trying to say is that there are unexpected and subtle design changes from stage to stage in the manufacturing process. And these are not theories. I deal with this kind of thing (on a much smaller scale) at work.

    When you build a beta unit you are using a smaller run of circuit boards, produced at one time. You are using a smaller stock of parts, from (usually) your choice of suppliers, and you are very careful about how the units are produced.

    Then you test. And the units go through changes if needed, and you test again. Eventually you wind up with a design and assembly process that passes your tests.

    But things (sometimes small and unavoidable things) change as you scale up production. Perhaps a part is not available from a certain supplier, so you switch suppliers. Same part, same specs, different realities. And small differences make all the difference. Especially if you are running your hardware hard. Maybe the test batch had a thermal compound that worked just that much better then the spec said. But in full production you had to switch to a different supplier (due to cost, or availability, or production requirements) and the new stuff is right on the spec, and you lose that buffer zone that the original test units had.

    That kind of stuff is extremely hard to predict, because sometimes there is very little explanation for the new failure.

    And for the record: Xbox 360 hardware failure has very little to do with mechanical failure, usually it’s a heat issue affecting the Motherboard, Graphics Processor, or CPU. And as far as analogies go, the expensive Kitchenaid mixer would be a $2500 dollar Alienware computer. The Xbox 360 is a high end console, but not the be-all-end-all of computing power. It’s more like a Cuisinart mixer (and yeah, those fail sometimes).

  14. Doug says:

    Hmmm… my PS3 doesn’t have any of those problems. I was talking with the gamestore guys about theirs and they said the overheat protection works great. It gets too hot, it shuts off. They said it did have one problem, which was the part that can tell whether it is talking to a real TV as opposed to a recording device can burn out. But if it does, Sony replaces it real fast with as little hassle as possible.

  15. Chris Hanson says:

    What I find interesting about all of this is that they worked so hard to cram everything into a form factor that’s fundamentally inconvenient. Imagine how much more reliable it would be if they had the same parts in a larger enclosure – such as a standard 17.5-inch AV enclosure! They’d not only have better internal airflow and reliability, people would probably be more likely to buy the unit as a digital entertainment hub rather than just a game console.

    I know this is one of the reasons I’m still waiting to get an Xbox 360. It’ll be the most inconvenient item in my AV setup, especially with an external HD-DVD unit…

  16. [...] It seems like just the other week that I ranted about the Xbox 360 design being unacceptably fragile. And what’s the news this week? Microsoft is extending Xbox 360 warranties for the “red ring of death” to 3 years (even retroactively!), refunding service fees for that repair already paid by customers, and taking a one billion dollar (plus) charge to cover the costs [...]

  17. [...] For those following along at home, my Xbox 360 Died The Death about a month ago and I shipped it off to Microsoft for repair service. When I sent it they had not yet announced the repair program, so they billed me $70. [...]

  18. [...] of the machines will fail catastrophically after just a couple of years of light duty. Of course, correctly observed (twice!) that the “big” consoles were aggressively designed to capture a “high [...]