BringBack914 - MSN autos doesn’t test vehicles for reliability, and neither does Consumer Reports. CR gets its data from owners via a voluntary survey of subscribers, who define whether or not they consider a problem they encountered to be serious. MSN gets its data from mechanics’ repair files to show what problems they are encountering as well as the cost to repair. Other sites use the exact same data that MSN uses (I believe it might be CarMD doing the actual reporting). I haven’t seen a single one filtering the data to make it preferable for someone buying ad space.
So what does all that mean?
Well, CR has the lack of paid advertising claim, but they also rely on people buying subscriptions to make ends meet, and to sell subscriptions, they have a need to make the ratings look important - hence the restructuring, or compression, of ratings over the years to the point where you can have a solid black rating now and still have been what was above average 15 years ago. They also have flawed survey methodology where you have a self-selected population of respondents defining their own responses. You can’t tell from their rating if that is a $50 problem or a $5000 problem. While they have a lot of people responding, there are thousands of model/year combos they’re trying to evaluate, so there are not a lot of responses on many models, bringing the statistical significance of two differently rated vehicles into question.
On the MSN side, you have the impression of ads working against them, though there is no evidence to support that claim. You have a smaller number of people providing data (about 3,000, IIRC), but each person comes into contact with potentially hundreds of vehicles. That’s still a smaller sample, meaning they might miss some things. But they are less likely to classify an ordinary wear item as a serious reliability issue. They also provide dollar cost estimates for the repair, so you can tell if it is a $50 or $5000 problem.
Is one better than the other? Not really. They BOTH have their uses. I use CR to find the vehicles with solid black ratings and avoid them. But to tell the difference between an average rating in CR with an excellent rating? I turn to MSN and others. Then I’ll know if it is a cheap repair or an expensive one, or if there really seems to be a difference at all. You’ll also catch reports that make no sense - like the Ford Fusion’s 2.5L I-4 coming in as more reliable per CR than the Mazda6’s 2.5L I-4, when in fact they are the same engine.