I wholeheartedly agree with the “eat your own dog food” approach. As Joel pointed out, while the QA process may find all the out-and-out bugs, it doesn’t necessarily identify items such as usability issues and limitations. It also forces the developer to take a step back and consider issues from a different perspective, the customer's perspective, the one that actually matters.

I would just like to add that if at all possible it’s also beneficial to involve professional services and support in this process. Consider their input at the design stage and then again during testing before it’s too late. Their input is especially important in the case of updates to an existing product.

Dan

-- Anonymous, May 06, 2001

Answers

Eating your own dog food is a great way of guerilla usability testing. And , like you said QA doesnt solve the problem, though I've noticed a lot of head-scratching in QA wether something is a bug or not.

On the other hand, there is no substitute to starting out with a good customer experience plan. Just like software is first designed and then iteratively improved by testing and debugging, customer experience is best done by planning first and then testing, so you know what you're testing against.

-- Anonymous, May 12, 2001