Submission

Note:

Now Evaluation Set Phase: 2024.3.13 — 2023.3.27 23:59(AOE)
1. Due to the limits of codalab, you have to participate in a new competition on codalab for the evaluation phase leaderboard.
2. Follow the steps mentioned below, make sure to register on CodaLab using the same institutional email address you used when registering in the ChatCLR challenge.
3. Then, set your team name on CodaLab, and make sure it's the same as the one that you use to register in the ChatCLR challenge, and then send applications to participate.

note: make sure to use the same e-mail address and teamname in the ChatCLR challenge and CodaLab, or you may not be allowed to submit the results in the competition. If you'd like to change e-mail address, please contact with organizers.

seteamname.png

  • STEP 2: Apply to join in the competition and wait for approval.

Tips: just check the box ( you don't need to download the term as you have signed it when you register in ChatCLR challenge ), and we'll permit access at that night.

register

  • STEP 3: Submit your result and check the scores

  • Make sure your submitted zip file as following directory structure:

    |submission.zip

    ​ ||--submission.txt

  • Make sure every row of the submission use the following format, which is a key-word separated by a space character:

segment_id 文字文字文字 （only Chinese characters included in the text part without any punctuation）

The segment_id remains consistent across eval_far_segments.zip within the evaluation set, named as follows:

< Speakers ID >_< Room ID >_< Speakers IDs >_< Configuration ID >_< Index > _< Start time >-< end time >

  • Here are the submission example and the whole segment_id list for evaluation set.

  • When the "Status" in the submission table turns to "Finished", you can refresh the page the check the score below. You can also click the button named “View scoring output log” to check if you miss any segments in your submission, for example:

    CCER: 90.49

    In your submission, some segments are missing. Please check and submit again!
    missing segments:
    S193_R11_S193194_C01_I1_000115-000152
    S193_R11_S193194_C01_I1_000320-000780
    S193_R11_S193194_C01_I1_000824-001208

  • Other information

  • Make sure a. your submitted file, includes all videos in the evaluation set. b. your submission file is in required format, or the submission will fail.

  • For each submitted score, it will be directly on the leader board if it's your best score so far.

  • During the evaluation set phase, every group is allowed to 5 times per day. Due to the problem of CodaLab, there is a small probability that the result will not be obtained for a long time. Please wait patiently or submit at another time.

  • If you meet any problems during submission, please contact us.