Contest


UCR Insect Classification Contest

Organized by:

Yanping Chen
Eamonn Keogh
Gustavo E. A. P. A. Batista

Unofficial Results are here www.cs.ucr.edu/~eamonn/CE/CE_CONTEST_unofficial_results.pdf

Official results will be announced soon.

Download the details of the contest in PDF or Powerpoint format.

Have a question?

If you are a contestant (or potential contestant), please email: UCR.insect.contest2012@cs.ucr.edu
If you are with the media, please email: eamonn@cs.ucr.edu

FAQ (see the full contest details in the PDF or Powerpoint first)

Q) Do you accept international participants in the contest or it is only limited to USA ?
A) Anyone in the world is welcome to enter, except current or former students of the organizers, or people with obvious COIs with the organizers.

Q) Can I use a windows executable instead of Matlab?
A) Before we answer, let us tell you why we originally choose to do things this way. First, we did not want to have to deal with dozens of variations of code that may or may not run on our machines. Second, we are running this contest with the goal of making a strong benchmark freely available to the world, to hasten progress on a socially important problem. Having said that, we having gotten many requests about this. Here is our compromise:
- First, note that you can call an .exe from Matlab, using the function system(command)
- Given this, you are allowed to send us two files, an .exe and a matlab m-file that invokes it.
- Assuming your team is lead by Jones from Intel, the files must be called Intel_Jones.exe and Intel_Jones.m
- The m-file must list the team as before, and it can evoke the .exe as often as needed.
- We will place both files in the current working folder, and run our testing code as explained before. If your code does not work, we may spent upto one hour of our time emailing you etc, trying our best to make it work. After that we will simply ignore your entry.
- Your two code files must be accompanied by source code, or very detailed pseudocode that can be made publicly available.

Current Overall Leader
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_DominikSchnitzer	99.8%	93.29%	Achieved in week 4

Week 1:
Team Name	Leave-One-Out¹	Hold Out²	Notes
Columbia_Ellis	96.4%	92.53%
OFAI_DominikSchnitzer	94.2%	91.04%
UA_tatti	93.8%	90.36%
OFAI_JanSchlueter	91.2%	88.67%
DAI_Spiegel	85.6%	85.18%
SibSAU_Mangalova	86.8%	84.44%
DLR_cerra	69.4%	66.91%

Week 2:
Team Name	Leave-One-Out¹	Hold Out²	Notes
Columbia_Ellis	92.4%	91.47%
OFAI_DominikSchnitzer	98.6%	90.87%
OFAI_JanSchlueter	95.0%	90.62%
DAI_Spiegel	92.0%	89.51%
SibSAU_Mangalova	92.4%	89.47%
TEIC_Potamitis	92.4%	88.91%
DLR_cerra	92.6%	88.42%

Week 3:
Team Name	Leave-One-Out¹	Hold Out²	Notes
Columbia_Ellis	93.4%	91.89%
OFAI_DominikSchnitzer	95.2%	90.91%
OFAI_JanSchlueter	92.6%	90.18%
TEIC_Potamitis	93.8%	89.96%
UCLouvain_vankeerberghen	86.6%	84.60%

Week 4:
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_DominikSchnitzer	99.8%	93.29%
Columbia_Ellis	95.2%	92.67%
OFAI_JanSchlueter	97.0%	90.27%
SibSAU_Mangalova	93.0%	89.71%

Week 5:
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_JanSchlueter	94.4%	90.78%

Week 6:
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_JanSchlueter	92.6%	90.93%
ZIB_Schaefer	94.0%	90.40%
DAI_Spiegel	92.8%	89.09%
TEIC_Potamitis	100.0%	87.69%

Week 7:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	94.8%	92.31%

Week 8:
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_DominikSchnitzer	99.6%	93.16%
ZIB_Schaefer	96.8%	90.64%
TEIC_Potamitis	98.8%	88.80%

Week 9:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	95.8%	92.27%

Week 10:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	95.8%	92.27%
ICMC_IgorBraga	92.6%	89.27%

Week 11:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ICMC_IgorBraga	94.8%	89.91%
TEIC_Potamitis	99.8%	87.33%
SibSAU_Mangalova	99.6%	85.84%

Week 12:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	96.2%	91.71%
TEIC_Potamitis	99.6%	91.29%
ICMC_IgorBraga	92.8%	89.49%
SibSAU_Mangalova	97.6%	85.89%

Week 13:
Team Name	Leave-One-Out¹	Hold Out²	Notes
TEIC_Potamitis	100.0%	92.18%
ICMC_IgorBraga	93.2%	89.36%

Week 14:
Team Name	Leave-One-Out¹	Hold Out²	Notes
OFAI_DominikSchnitzer	96.8%	92.82%
TEIC_Potamitis	100.0%	91.87%
ZIB_Schaefer	95.4%	91.29%
ICMC_IgorBraga	98.4%	90.53%
TUT_Virtanen	95.8%	89.56%

Week 15:
Team Name	Leave-One-Out¹	Hold Out²	Notes
TEIC_Potamitis	100.0%	92.76%
ZIB_Schaefer	95.4%	90.04%
ICMC_IgorBraga	100.0%	89.71%
TUT_Virtanen	92.4%	88.6%

Week 16:
Team Name	Leave-One-Out¹	Hold Out²	Notes
TEIC_Potamitis	100.0%	92.62%
ZIB_Schaefer	95.4%	90.93%
ICMC_IgorBraga	100.0%	89.73%

Week 17:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	92.6%	92.38%
NUS_Yu	93.0%	89.98%
IMSU_Kondrashkin	88.2%	87.93%
TUT_Virtanen	94.4%	87.73%

Week 18:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	96.8%	92.04%
OFAI_JanSchlueter	99.2%	91.24%
MSU_RyzhkovAlex	88.4%	87.87%
CMU_ChunChen	(abandoned, too slow)

Week 19:
Team Name	Leave-One-Out¹	Hold Out²	Notes
ZIB_Schaefer	96.6%	92.09%
NUS_Yu	92.4%	90.56%
TUT_Virtanen	90.2%	85.69%

Most Weeks on Top of the Leaderboard
Team Name	Number of weeks	Notes
Columbia_Ellis	3	weeks 1, 2, 3
OFAI_DominikSchnitzer	16	weeks 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19

1: Why are we telling you your leave-one-out accuracy? Since you can also compute this, it allows you to check to see that we are using your code correctly.

2: This is accuracy(D1_public| D1_evaluation)

Acknowledgments: Thanks to the Bill and Melinda Gates Foundation, UCR Chancellor, Timothy P. White (Chancellor's Strategic Initiatives) and Vodafone Americas Foundation for funding the development of the sensors and the data collection. Thanks to Agenor Mafra-Neto and his team at ISCA Technologies for technical assistance. We have other people to thank after the contest is over, so as not to compromise the contest. Thanks to Itai Cohen for permission to use the insect images on this page.