The dilemma of Parity in Products Tests...

Why does the quality of our favorite products gets worse with time?

Any loyal consumer of a category notices, through a sufficiently long time, if there is a fall of quality of their favorite brand or even of many brands in the market. The manufacturers, naturally, seek better finance results and are constantly seeking ways of obtaining equivalent formulas with lower costs. However, the way in which these new formulations are tested can lead to deceiving results.

A product is normally tested against benchmarks of the market (habitually the current formulation of the brand and one or two competitors). This is the best practice and it is the recommended. However, if we are in a very competitive market, it is possible that the benchmarks are already, themselves, in a sensory level much distant from the ideal one. Successive cycles of reformulation may make the benchmark of today already much inferior to the formulation of, let’s say, 10 years ago.

We take as an example a morning cereal, that went through reformulations once a year, within the last ten years. At each reformulation, it was tested against the previous formulation, always with parity of results. The changes were, therefore, insignificant from one year to the other and the new formulation always took the place of its previous one. However, if we test the current product against the formulation of 10 years ago, possibly, these individual insignificant changes, accumulated through this long period, have generated an enormous sensory gap between the original and the current product. And this gap is more than enough for the consumer to notice the decline of the product and, eventually, to abandon the brand.

How to face this dilemma? First, choose benchmarks that are really demanding. The current market test continues being essential. I suggest testing against the current formulation and against competitors whose sensory quality is undeniable and recognized.

Testing the product without the brand is very important. As we know and have already published in this blog, the brand interferes with the sensory perception of the consumer and may make the technically average products to be well evaluated and excellent products to have a mediocre performance.

We should, as well, have at least part of the sample (or a booster), of heavy users or consumers preferring the brand. The heavy users are the ones more able to notice subtle changes in the formulation of their favorite brand.

If the expected changes between products are very subtle, consider a comparative design with measurements of preference (sequential-monadic or proto-monadic). And be careful with the sampling size: we should not forget that small samples invite to the parity of statistical results.

Finally, it is really important to compare the results with normative data. We want to know if the product that we test is really good and not only the “less bad” among a roll of (which may or may not be) mediocre products.

The loss of value of the brand may occur imperceptibly through time and one of the factors for that is the decrease in quality and of the perceived value of the product. Well-done product tests may help making the correct decisions and avoiding that to happen.

Rodrigo TONI

General Manager

Former Regional CEO of Southeast Asia and Former CEO IPSOS Brasil He worked for RI/TNS, Nielsen, among others Specialized in Management at IESE (Spain) and INSEAD (Singapore) Specialized in Survey Methodology, SRC, University of Michigan (USA). Social Scientist (USP) and Agronomist Engineer (USP)