Microsoft Defends Surface Family, Fires Back At Flawed Consumer Reports Defect Testing Methodology

Consumer Reports (CR) has really gotten under the Microsoft's with its claims that the Surface line of devices is unreliable. CR has already stated that it will not award the coveted "Recommended" status to the new Surface Book 2 and Microsoft is still rankled by the decision. Microsoft has stepped up again and is offering up some new data on its Surface range and is stating that fewer than 1 in 100,000 new Surface devices have reliability issues.

The row between the Redmond, Washington software giant and the publication started earlier this year after a CR survey of 90,000 subscribers found that 25% of Microsoft laptops and tablets gave owners problems by the end of the second year of ownership. The rub for some folks is that the CR survey doesn’t appear to have quantified the sort of problems seen to help determine if the issues were user error caused by downloading willy nilly while not running any anti-virus protection or a legitimate problem with the hardware or software on the Surface device.

Microsoft’s Ryan Gavin, GM or Surface, has challenged the findings, saying that the Surface devices get more reliable with each new generation. He said, " One of the things you're seeing is the reliability of our products over time, with every generation getting better and better and better." Gavin also claims in the new Surface devices, including the Surface laptop and Studio, reliability issues have only cropped up in a fraction of the devices. "We're talking about incidents per device of less than 0.001%," he said.

Gavin has an issue with the methodology that CR uses, adding, "We thought it was unfortunate the report that Consumer Reports put out, partly used the methodology that is consistent with how you evaluate home appliances and applied it to laptops. That would be like [asking] 'Has your dishwasher behaved unexpectedly in the past two years?' and if the answer was 'Yes', you'd be deemed unreliable."

The numbers that Microsoft is quoting are for newer Surface devices including the Surface Laptop. CR's survey that stripped the devices of their recommended ratings were for a broad range of Surface products, including older tablet models. The takeaway here is that Microsoft seems to be claiming that any reliability issues were found were for older tablets.