“The way a bug was found and fixed in the Feed Validator is not disturbing, I actually think it was an inspiring proof that all the aspects of its QA process worked:
- There is a public feedback channel (several, indeed) for a problem to be reported to: Brian sent a message to the W3C QA mailing-list, asking whether someone could make sense of the problem he was facing
- The tool is implementing public specifications: I was able to look at the RSS taxonomy module spec, and compare its prose with the implementation in the Feed validator
- The validator is open source: 10 minutes of browsing around clear code was enough to find the issue, and create a patch, which was promptly reviewed and applied by Sam Ruby, one of the maintainers of the validator.
- There is a test suite, to which I submitted a revised test case: now that the bug is fixed, we know that it will never appear again without being spotted automatically.
Time between original feedback and applied patch: about 24 hours.
There is nothing shocking about this bug, but the speed at which it was processed and fixed. Maybe that was lucky, I just happened to have a bit of time to look at the issue and the bug was easy to fix. Other, more complex bugs in tools that we (W3C’s QA Tools development effort) maintain are not so lucky, and indeed Brian is right in pointing out that we could use more help and resources to make our tools better. But I can not agree with his slightly provocative title that
Validators Donâ€™t Always Work. The Feed validator works, and so does its Quality Assurance process, as demonstrated in the prompt fixing of a small bug in its implementation of a faulty, not widely used specification.
Validators are extremely important tools for the adoption of technologies, and it is perfectly normal to be concerned about their quality. This is why finding bugs is good news, and the best use of one’s energy is not to worry about them, but to help find them, report them, patch them and build regression test cases for them.
Nothing and noone is perfect, that’s why we have QA.”
This was a great example of how things have changed for the better and evolved over time at the W3C when handling issues and bugs but I would also like to stress that there was nothing wrong with the way otherÂbugs and issues were handled in the past.