Task 1 Leaderboard
| Rank | Team | Score | Technical Report |
|---|---|---|---|
| 1 | AI lab, Xiaomi Corporation | 0.058 | [paper] |
| 2 | Duke Kunshan University | 0.071 | [paper] |
| 3 | Beike | 0.091 | [paper] |
| 4 | Harbin Institute of Technology, Shenzhen | 0.101 | / |
| 5 | Mobile Phone Department, Xiaomi Corporation | 0.108 | [paper] |
| 6 | Nanyang Technological University | 0.109 | [paper] |
| 7 | Tongji University | 0.113 | / |
| 8 | Northwestern Polytechnical University | 0.121 | / |
| 9 | Lenovo | 0.122 | / |
| 10 | Beijing University of Civil Engineering and Architecture | 0.123 | [paper] |
| 11 | Toshiba Europe Ltd | 0.241 | / |
| 12 | Chung-Ang University | 0.254 | / |
| 13 | Baseline | 0.322 | / |
| 14 | South China University of Technology | 0.402 | / |
| 15 | Japan Advanced Institute of Science and Technology | 0.498 | / |
| 16 | Shanghai University | 0.898 | / |
Submission
Note:
Now Evaluation Set Phase: 2022.01.01 00:00:00 — 2022.01.24 00:00:00 (AOE)
1.Due to the limits of codalab, you have to participate in a new competition on codalab for the evaluation phase leaderboard.
2.Follow the steps mentioned below, make sure to use the same e-mail address on misp2021 to register on codalab, and then send applications to participate.
3.Follow the steps mentioned below, set your teamname on codalab，and make sure it's the same as the one that you use to register in the misp2021.
- STEP 1: Register on codalab
note: make sure to use the same e-mail address and teamname in the misp2021 and codalab, or you may not be allowed to submit the results in the competition. If you'd like to change e-mail address, please contact with organizers.
- STEP 2: Apply to join in the competition and wait for approval.
tips: just check the box ( you don't need to download the term as you have signed it when you register in misp2021 competition ), and we'll permit access at that night.
- STEP 3: Submit your result and check the scores
Make sure your submitted zip file as following directory structure:
|summission.zip
||--transcript.txt
Make sure every row of the transcript use the following format, which is a key-word separated by a space character:
utterance_id 1or0
Here is a submission example and the whole utterance_id list for eval set provided. Note the number of segments in truth ground is smaller than those directly extracted from the transcript(.textgrid files), since some of the segments labeled in .textgrid files lack relevant signal or video segments in wav or mp4 files for early shutdown of recording ice in the recording process.
When the status turns to "Finished" you can refresh the page the check the scores below. You can also click the button named “View scoring output log” to check if you miss any segments in your submission, for example:
stdout.txt
In your submission, some segments are missing. Please check and submit again!
Miss segments:
0
1
2
- Other information
Make sure a. your submitted transcript, includes all segments in the development or evaluation set. b. your submission file format, or the submission will fail.
For each submitted score, it will be directly on the leader board if it's your best score so far.
During the evaluation set phase, every group is allowed to 3 times per day. For the development set phase, there is no limit.
If you meet any problems during submission, please contact with misp2021 administrators.