Evaluation Criteria

Negotiation competitions are all about assessing and comparing negotiation performance. Importantly, this comparison takes place under the critical eyes of the participants themselves, who have vested interest a fair evaluation of their performance. It is important for them that the assessment of their negotiation skills is conducted in a well-ordered, fair and convincing manner. Therefore, a transparent evaluation based on convincing principles is needed in negotiation competitions. Thus, after having intensive discussion and deliberation on this point throughout the history of The Negotiation Challenge, we have developed an evaluation methodology, which has been broadly accepted by its participants.

The Negotiation Challenge 2019 in Kyoto

The first and arguably most important evaluation criterion of success is, of course, the negotiated outcome. After all, it is a genuine purpose of many negotiations. Yet, a precondition for using the outcome to rank negotiators is the ability to measure it. In many negotiation simulations, e.g. concerning prices, costs or salaries, measurement and comparison are easy. However, in other simulations, the outcome of a negotiation includes another dimension, which is beyond the substance of the negotiation. This dimension is the relationship between the parties. For example, how trustworthy were the negotiators? Did the parties act fairly during the process? What is the relationship between the negotiators after the negotiation? These issues are involved in many negotiation and are a genuine part of the outcome.

Despite a good result, if it leads to mistrust or if one of the parties feels cheated, the negotiation is certainly not as good as if this was not the case. For this reason, in simulations, in which a relationship between the parties is important, a comprehensive ranking should not only be based on the substantive outcome, but it has to consider also the relational outcome.

Thus, to consider this, one must better define what the relational outcome is so that it can be measured more accurately. Seemingly, it is more complex than just trust and fairness. It has various dimensions. One tool that can be used in this regard is the “Subjective Value Inventory (SVI)“ that Curhan & Elfenbein proposed in 2007. It is a scale capturing various dimensions of negotiation outcome using sixteen questions spread across four categories. Based on the SVI, the relational outcome can indeed be measured and compared. Still, despite its usefulness, the SVI has certain limitations that are inherent to every questionnaire-based measurement of a complex phenomenon.

To mitigate these limitations, it is useful to include the negotiation process itself in the performance evaluation. This is intuitive since performance is an attribute of the process and the outcome only a result of that. A good negotiator is doing good things to get a good result. Conceptually then, the process drives the outcome. Thus, based on the research insights of negotiation success drivers, which link process and outcome in negotiations, in negotiation competitions we use judges, who evaluate these drivers as they observe the negotiations being conducted. Though we readily admit that a subjective element certainly exists to using judges, the complexity and the qualitative nature of the process does not allow otherwise for objective (numerical) procedures. Indeed, human judgment is needed to assess such important dynamics as fairness, the subtle interactions within and between the teams or their ability to listen actively and to contribute to a joint understanding of the case.

For the purpose of The Negotiation Challenge, we developed a negotiation competency model, which is thoroughly described in Smolinski and Xiong 2020. In our model, we clustered a variety of negotiation skills and attitudes into four broader categories: language and emotionality, negotiation intelligence, relationship building, and moral wisdom. The logic of such an arrangement follows the ease with which one may observe these skills and attitudes during negotiations. Language and emotionality are the first and most easily observable part of negotiators’ behavior and create a direct impression about negotiators’ style and personality. This category is then followed by the concrete skills and tactics a negotiator uses, summarized in our model under negotiation intelligence. Finally, the third and last categories, relationship building and moral wisdom, are more difficult to observe because negotiators’ motivations and values usually hide behind their various language patterns and skills. These four categories are distinct but complementary, together constituting our negotiation competency model.

Finally, it is important to note that the total score given to the teams on the evaluation form does not necessarily have to be the arithmetic average of the evaluation categories. Rather, the detailed evaluation criteria should only guide the judging process and be used to help each judge justify their impression and evaluation of performance. In fact, after the negotiation, the judges gather and use these forms to compare and calibrate their individual evaluations with each other and then agree on and compose the final team rankings for each of the roles. This is a very important step in the evaluation process, which typically results in long and intensive discussions due to the fact that different reference points may be used while assessing the negotiators’ performance. However, it ultimately leads to a consensual ranking supported by all involved judges.

Although non-scoreable complex negotiation rounds evaluated by the judges have become an integral part of The Negotiation Challenge, performing well in them is not enough to win the competition. That is, to prevail in The Negotiation Challenge, the best teams need to perform and rank well in every round. Underperformance in any of the rounds radically reduces the chances of making it to the final. The Great Negotiators are complete negotiators.