How relevant is the lack of reciprocity in pairwise comparisons? An experiment with AHP

Most pairwise comparison (PC) methods typically require the explicit elicitation of only half of the comparisons, and infer the rest by assuming reciprocity in the decision maker’s comparisons. However, this may imply losing useful information contained in the additional comparisons that could be made, and which might be different from the first ones. This study assesses how relevant the lack of reciprocity may be in an experimental setting, and to what extent the information included in the additional comparisons may influence results. Our experiment shows that decision makers display substantial levels of irreciprocity and inconsistency, and that they generally prefer preference vectors calculated without assuming reciprocity in their comparisons. According to our results, our main conclusion is that, in general, decision makers should be requested all the comparisons in a PC matrix.