In a benchmarking study selected products or services are compared against
each other. Benchmarking can be done with competing products or with earlier
version of a new product. Benchmarking can be done between two or more
products. A benchmarking study consists of usability and user experience
benchmarking. The study is usually done using both subjective and objective
usability metrics. The study can be arranged either in laboratory environment
or in the field. Also expert evaluation is possible, and recommended e.g. as a
part of evaluating the out of the box experience.
In the benchmarking study the test users carry out identical tasks with the
products or the services that are benchmarked against each other. Different
kinds of methods are used to gather data to gain numerical grades to the
tested services or products. The evaluation report presents the benchmarking
grades and analyses the differences between the benchmarked products.