Submission and Evaluation

Submission

The Cup participants shall submit their programs in C++ or Java source code creating a single executable, which can be called with filename(s) for the input dataset, a starting points filename, and a filename for writing the program results into a file. You are not allowed to use libraries other than those listed below for your program. If you think that a specific library makes sense for your submission, please write an email to the mailing list, we will then decide, whether we accept the library or not. Libraries should be open source. Details on the exact submission rules will be published on the Submission and Evaluation page soon. Using libraries or tools that are not listed in the following section leads to immediate disqualification.

Submission Portal

Participants shall submit their solution as a single .zip file via EasyChair:

https://easychair.org/conferences/?conf=giscup18

The submission must contain:

  • The original source code and all dependencies (submission of the source code is mandatory to ensure originality of the submitted work).
  • A readme.txt file containing information on how to compile and run the submitted code. You may optionally include a brief description of the main idea behind your submission.
  • A contact.txt file containing the names and email addresses of the authors.

Evaluation System

The challenge will be evaluated on an up to date desktop machine. You can expect a multicore CPU and enough RAM for holding the dataset.

Allowed libraries / tools

  • C++ 11 or Java

  • All libraries from Boost including the Boost Graph Library

  • All libraries to assist with reading input files

Additional libraries can be requested by mailing the contest chairs.

Evaluation Criteria

All submissions are evaluated on efficiency and correctness on the given sample datasets as well as larger datasets. The datasets used to evaluate each submission will be similar to the sample datasets, but larger. We will test all submission with increasing size of spatial network, starting points, and number of controllers. You are not allowed to serialize any information between different runs. We will use the total elapsed wall clock time as shown in the sample submission. For breaking ties, we will first look into the scalability behavior by varying each input parameter for larger and larger datasets. Finally, we break ties on code stability, quality, and readability.

Mailing List / Google Group

Challenges similar to the one posted here often lead to questions concerning various details of the challenge and evaluation rules. We will use a public Google Group for communicating any questions that can be answered as well as the corresponding answers to all interested people. Therefore, please try to pose questions only in a form that does not leak too much of your ideas.

https://groups.google.com/forum/#!forum/giscup-2018

We will also post notifications for important updates to the challenge to this group.