Test-Retest Reliability of Badminton Matches
The term test-retest reliability describes the reliability of a test when performed multiple times. We treat badminton matches as measurements which one of the players or pairs is stronger, and ask the question whether this test result is replicated in the next match between the same players, that means if the next match is won by the same player or pair.
If all matches were always won by the same player or pair, we would expect 100% of the pairs to have the same winner. If, on the other hand, matches were totally random, we would expect half of the pairs to feature the same winner. In reality we expect some players to be better than others, so we also expect that repeated matches will more often have the same winner than not. If we assume that for a pair of matches both matches are won with the same probability \(p\) by one player or pair, then the probability for both matches being won by the same player or pair is given by \(P=p^2 + (1-p)^2\), where the first summand is the case where this player wins both matches, while the second summand is for him losing both matches.
All Matches
First we take every match that was properly finished into account. For every match we search for the successive match with the same players and teams. If there was one, it was then counted if both matches were won by the same player or pair. If a pair of players or teams competed multiple times, each pair of successive matches was counted seperately. So the first and second match were counted as one pair of matches, the second and third as one pair, and so on.
The results are shown in the following table.
| Discipline | Pairs of Matches | With the Same Winner | Percentage |
|---|---|---|---|
| All | 98303 | 69152 | 70.35% |
| Men’s Singles | 42052 | 28951 | 68.85% |
| Women’s Singles | 29338 | 21271 | 72.50% |
| Men’s Doubles | 10202 | 7012 | 68.73% |
| Women’s Doubles | 8647 | 6300 | 72.86% |
| Mixed Doubles | 8064 | 5618 | 69.67% |
The number of pairs of matches in the doubles disciplines is lower as there are fewer doubles matches in general and also because pairs might split up. For this analysis, we demand that both pairings are the same.
We find that in only about 70% of the pairs of matches, both matches are won by the same player or pair. If we use this value to determine the probability \(p\) for a single match, we get that this average match is won with a probability of around 82% by the better player. We can also see that numbers are higher in women’s singles and women’s doubles and lower in men’s singles and men’s doubles.
Only Matches with Odds
If we perform the same analysis, but only take matches into account that had betting offered on them, we get the following results:
| Discipline | Pairs of Matches | With the Same Winner | Percentage |
|---|---|---|---|
| All | 20019 | 13280 | 66.34% |
| Men’s Singles | 6321 | 3958 | 62.62% |
| Women’s Singles | 5476 | 3748 | 68.44% |
| Men’s Doubles | 3122 | 2054 | 65.79% |
| Women’s Doubles | 2538 | 1792 | 70.61% |
| Mixed Doubles | 2562 | 1728 | 67.45% |
We see all percentages go down. The percentage of pairs with the same winner across all disciplines goes down to 66%. If we again calculate the probability for a single match, we get that the probability to win a single match was about 79% for the better player or pair. In the table, we can also see a difference between the disciplines: the doubles disciplines are lower by about 2-3%, while the singles disciplines drop by 4-6%.
Interpretation
Smaller differences in strength should correspond to a lower percentage of pairs of matches with the same winner. Because if one player is much stronger than his opponent, it is very likely that he will win both matches. However if both players are equally strong, then we would expect the matches’ outcome to be dependent on the players’ state on the day of the match, or even random, and we would only expect 50% of the pairs to be won by the same player.
Also another influence is the strength development. If players or teams meet again, several years might have passed and one of the players might have improved. For example if in the first match an experienced player easily defeats a young and inexperienced player, they might meet again after some years when the young player is in his prime and now more than a match for the other player.
In the data we see differences in the percentages for the different disciplines. We can thus conclude that strength differences are lower in men’s singles and doubles. We can even sort the disciplines, going from the discipline with the lowest strength differences, men’s sinlges, to the one with the largest strength differences, women’s doubles.
We can also see that matches with odds are more competitive than matches without. Probably the database will contain many matches with lower-ranked players that give a lot of one-sided matches.
By Point Ratio
When asking how likely the next match will be won by the same player or pair, we can also distinguish between matches that were won with different point ratios. So, in each pair of matches, we will sort by the percentage of points the winner won in the first match. For example, if the first match was won 21-10 21-11, the winner won 66.7% of the points. We can then count how many pairs of matches with a given point percentage in the first match were won by the same player.
We can also calculate the expected probability for the winner to also win the next match. We can take the point percentage and use it as the expected probability to win a rally and then calulate the probability to win a match.
The following two plots show the results, first for all matches and then only for matches with odds. The black solid line shows the expected percentage as calculated from the point percentage of the first match of the pair. This line is the same in all plots. The bars show the data, where the bar’s color shows the number of pairs of matches in the corresponding bin. The mapping from colors to number of pairs of matches is given by the column on the right. The size of the bar is given by the binomial error for the percentage of won second matches.
We can clearly see that the actual percentage of won succesive matches is far closer to 50% than expected. For a point percentage of 70%, which corresponds to a clear 21-9 21-9 victory, the expected probability is very close to 100%, the observed percentage of won matches is only about 90%. We can therefore say, that there is a regression toward the mean-effect present in the data. In the first match, it is quite likely that the percentage of won points was too extreme, i.e. when a match ended 21-9 21-9 as mentioned above, it is more likely that the loosing player was stronger then the points he scored indicate. In the next match he is thus expected to return to his usual form and perform better. It is then not as likely that the loosing player was weaker than the 18 points indicate. This not only applies to clear victories. If two equally strong players compete, the winner will always have over performed winning more points than he was expected to. In the next match it is then returned to equal strength and equal probabilities to win.
The next plots show the same results split up by disciplines. The gray line always shows the result for all disciplines combined and is included for comparison.
We can see that the percentages of won second matches are lower in men’s singles and larger in women’s doubles. For the other disciplines differences are not as large.
Conclusion
The test-retest reliability is smaller than one could expect. Only two-thirds of pairs of matches with odds are won by the same player or pair. However, note that lots of matches are played between players who are at a similar level. For these matches, a much lower percentage is to be expected.
We also find that probabilities for the winner of the first match also winning the second match don’t rise as sharp as one would expect by a naive approach. Even after clear victories there are still pairs of matches where the next match is won by the loser of the first match. This discrepancy can at least partly be explained by a regression toward the mean, other factors might be the form of the players on each day or changes in strengths between the two matches.
For a bettor it implies that the outcome of the previous match should not be given too much weight. Even loosing more than one previous match does not necessarily mean that the outcome of the next match is certain. Keep that in mind, the next time head-to-head statistics are shown before a match.