KDD Cup  

Home Page
KDD Cup 2008
KDD Cup 2007
KDD Cup 2006
KDD Cup 2005
KDD Cup 2004
KDD Cup 2003
KDD Cup 2002
KDD Cup 2001
KDD Cup 2000
KDD Cup 1999
KDD Cup 1998
KDD Cup 1997
SIGKDD

KDD Cup 1998: Performance Evaluation Criteria

The CUP is aimed at recognizing the most accurate, innovative, efficient and methodologically advanced data mining tools in the marketplace.

The participants will again be evaluated based on the performance of their algorithm on the validation or hold-out data set. The KDD-CUP program committee will consider the following metrics in their evaluations:
  • Lift curve or gains table analysis listing the cumulative percent of targets recovered in the top quantiles of the file
  • Receiver operating characteristics (ROC) curve analysis and the area under the ROC curve
  • Several statistical tests to ensure the robustness of the results.
Last year, the performance in the top 10 percent of the file was considered as a measure of precision while the performance in the top 40 percent of the file was considered as a measure of stability and marketing coverage. The average performance up to the 40th percentile was also looked at as a measure of overall performance.