ISBI 2015 Call for Challenges

The ISBI 2015 Conference is soliciting proposals for scientific challenges. The aim of the challenges is to accelerate the pace of research by providing quantitative comparisons of competing approaches to cutting-edge problems, using standardized datasets.

Each challenge needs an organizer.  The organizer will be responsible for providing training and testing data, defining the tasks, specifying the metrics, managing entries, and conducting the studies and on-site presentations. ISBI will assist in advertising challenges, but the organizer is ultimately responsible for challenge participation and success.

The format of each challenge will be determined by the challenge organizer.  We strongly encourage on-site evaluations and scientific presentations by challenge participants.  We also encourage the investigation of alternative formats for hosting grand challenges, repetition of previous challenges to track how the field has advanced, and challenges that address machine learning and other base technologies as well as challenges that address end-user applications.  Challenge organizers will have the opportunity to present summaries of their challenges during the main conference.

Challenges are tentatively scheduled for the day prior to ISBI oral sessions, April 16, 2015, at the New York Marriott at Brooklyn Bridge, NY, USA.   Challenges may run for a half or full day.

Important Challenge Dates

Challenge Proposal Deadline:                                                                      September 10, 2014

Notification of Acceptance:                                                                           September 24, 2014

Challenge Announcements (Website, Tasks, Data, and Metrics):                September 30, 2014

How to Submit a Challenge Proposal

To propose a challenge, please submit a proposal in PDF format to the ISBI 2015 Challenge Co-Chairs:

Stephen Aylward            <stephen.aylward@kitware.com>

Badri Roysam                  <broysam@central.uh.edu>

Please include the following in the proposal:

1. Contest title and abstract
2. Name, affiliation, and contact information for the primary organizer as well as the names and affiliations of the co-organizers.
3. Description of and motivation for the problem being addressed by the challenge
4. Description of the dataset that will be used and how it will be allocated for training and testing
5. Description of the challenge tasks and schedule
6. Details on the evaluation metrics
7. Estimated number of participants

When designing a challenge, please consider the following:

• The dataset should be interesting, available, and sufficiently large.
• The tasks should be interesting and novel, but also accessible without too much domain specific knowledge.
• The evaluation metrics should be clear and easy to apply.  Supplying implementations in C++ or Python is ideal.

Reference Material

Previous ISBI Challenges

• http://www.biomedicalimaging.org/2012/index.php/programme/isbi-challenges
• http://www.biomedicalimaging.org/2013/program/isbi-challenges/
• http://www.biomedicalimaging.org/2014/program/challenges/

Grand-challenge hosting options
(These links are provided for convenience.  The services provided by the following are not endorsed by ISBI):

• http://www.grand-challenge.org/index.php/Main_Page
• http://challenge.kitware.com/midas
• http://www.insight-journal.org/
• https://www.codalab.org/