11. Consistency

Rigorous application of a rating system calls for simultaneous solution of its performance formula for each player in its active pool over a specified period of time.  This condition is approximated by so-called continuous or periodic ratings, which are essentially sequential in nature.  Continuous ratings are calculated event by event, while periodic ratings are calculated for calendar periods.  Performance ratings are incorporated, with some loss of precision, into established ratings. Whatever the customary method of calculation, the existence of at least one simultaneous solution for typical data indicates a system's consistency.  The first International Rating List was previously cited as an example of simultaneous solution (Applications), but the details of its approach should be noted.  Calculations were "continued until successive values of the differences showed little or no significant change," eight iterations in all [E2].  While this produced a more or less self-consistent set of ratings, there is no assurance that the solution was exact.  An exact solution, determining whether a system of formulas is consistent, is more convincingly arrived at by matrix manipulation. 

The consistency of the systems under discussion can be tested very simply using the data of the following hypothetical tournament.

Table 1.

Single Round Robin for Four Players

As seen from its matrix representation, the system of linear formulas for this tournament has an infinite number of solutions, with the rating of player D as a free variable.  A particular solution is arrived at by assigning D a rating and calculating the other ratings accordingly.  For example, with D rated .5, the solution set is {.75,  .75,  0, .5}.  The system of Berkin formulas would normally be homogeneous and requires special treatment to get a meaningful solution.  The rating of D has been arbitrarily set to .5.  The progressive system follows the performance formula of the Berkin System.  The system of dual ratio formulas has a unique solution.  Note that the ratings of A and B, for whom W > L, are calculated by formula [9.1].  Finally, there is the matrix representation of the system of Elo formulas, calculated by [4.2].  Because the logistic function is used, matrix manipulation in this case is not precise.  Note that formulas were multiplied through by 3, the number of opponents, to avoid repeating decimals.  It is clear, despite the imprecision, that the system has no solution, and manipulation does not  proceed beyond the echelon form.

The Elo System, in conclusion, is the only one among the four examined to be inconsistent. This clearly has implications for a simultaneous solution. Whether it also affects the accuracy of sequential ratings is more difficult to determine, though results of the test below suggest as much.