Comments on Proposed Judging and Scoring Reforms

by Sandra Loosemore
May 4, 2002

"Openness would accomplish more than forestalling an arms race. As it did in science, it would reveal error and expose abuse. Men performed in secrecy, behind closed doors and guarded borders and silenced printing presses, what they were ashamed or afraid to reveal to the world."

-Richard Rhodes, "The Making of the Atomic Bomb"

"The very fact that knowledge is in itself the basis for civilization points directly to openness as the way to overcome the present crisis."

-Niels Bohr, Letter to the United Nations (1950)

Introduction

I had hoped to make this article a detailed technical analysis of the proposals for changes to the scoring and judging systems being put forth by the ISU Council and the USFSA for vote at the ISU Congress in June. However, I have been unable to get a copy of the actual text of either proposal. The ISU completely ignored my request, and the USFSA said their proposal would not be released until after the upcoming Governing Council meeting. So I can provide only general comments based on what has been reported in press releases and news articles.

I have to begin by emphasizing something that both the ISU and USFSA seem to have forgotten: the problems afflicting skating in the wake of the controversy at the Salt Lake City Olympics have nothing to do with the scoring system. The problems are lack of public accountability and transparency in the way the current judging and scoring rules are applied, which make it possible for some judges and officials to try to manipulate competition results behind the scenes. If skating is to regain its credibility as a sport, we need less secrecy, not more. I therefore feel that the secrecy surrounding both scoring system proposals are a bad sign, that neither the ISU nor USFSA really understands the nature of the problem they are facing. All of these radical proposals to change the scoring system might give the impression that they are busily at work trying to "fix" the sport -- but they're not fixing the things that are really broken.

The ISU Proposal

In the immediate aftermath of the controversy involving the outcome of the pairs competition at the 2002 Olympic Winter Games, ISU President Ottavio Cinquanta outlined his proposal for a new system of judging, marking, and scoring figure skating. According to ISU Communication 1150, this proposal was endorsed by the ISU Council.

The gist of the proposal is:

blah blah blah I do not know on what basis Mr. Cinquanta considers himself qualified to design new judging criteria for figure skating. He is not himself a figure skater or a figure skating judge, and has in fact admitted publicly "....blah blah blah....".

Likewise, I do not know what credentials Mr. Cinquanta might have as a designer of algorithms for scoring figure skating. I suspect he has none.

Any responsible computer scientist should be aware of the inherent risks of relying on a computational procedure that cannot be independently verified or reconstructed by outside observers. These risks include both deliberate attempts to modify the data or program used to process the marks (including viruses, trojan horses, and similar attacks), and accidents such as programming errors in the scoring software itself as well as loss of data from hardware or systems software failures.

As an example of a past problem of this nature, at the Grand Prix Final in December, 2001, a bug in the scoring software resulted in the arena scoreboard announcing the wrong team as the winner of the competition. To those in the audience who had been keeping score themselves, and to the ISU officials in charge of accounting, it was obvious that something was wrong. If the original marks and intermediate results are not displayed, how can anyone independently verify that the scoring software is performing the correct computations to arrive at the final placements?

In addition, it is hard to see how anyone would think that keeping the entire marking process and scoring algorithm secret and hidden from the general public would increase the accountability of judges, and dissuade them from "cheating".

The USFSA Proposal

Conclusions

SkateWeb
 Home © 1994-2010 SkateWeb