Task 1 Leaderboard
| Rank | Team | DER(%) | Technical Report |
|---|---|---|---|
| 1 | Wuhan University - Alibaba | 8.82 | [paper] |
| 2 | Shanghai Jiaotong University | 10.82 | [paper] |
| 3 | Northwestern Polytechnical University | 10.9 | [paper] |
| 4 | Chongqing Jiaotong University | 13.05 | / |
| 5 | Beijing Institute of Technology | 13.43 | / |
| 6 | Vivo | 13.73 | / |
| 7 | Xiamen University | 13.88 | / |
| 8 | Baseline | 13.88 | / |
| 9 | A-star | 13.92 | / |
| 10 | Johns Hopkins University | 14.33 | / |
Submission
Note:
Now Evaluation Set Phase: 2023.01.15 — 2023.02.05 (AOE)
1.Due to the
limits of codalab, you have to participate in a new competition on codalab
for the evaluation phase leaderboard.
2. Follow the steps mentioned below, make sure to use the same Institutional e-mail address on misp2022 to register on codalab, and then send applications to participate.
3. Follow the steps mentioned below, set your team name on codalab，and make sure it's the same as the one that you use to register in the misp2022.
- STEP 1: Register on codalab
note: make sure to use the same e-mail address and teamname in the misp2022 and codalab, or you may not be allowed to submit the results in the competition. If you'd like to change e-mail address, please contact with organizers.
- STEP 2: Apply to join in the competition and wait for approval.
tips: just check the box ( you don't need to download the term as you have signed it when you register in misp2022 competition ), and we'll permit access at that night.
- STEP 3: Submit your result and check the scores
Make sure your submitted zip file as following directory structure:
|submission.zip
||--submission.rttm
Make sure every row of the transcript use the following format, which is a key-word separated by a space character:
SPEAKER < Session_ID > 1 < start_time > < duration_time > <NA> <NA> < speaker_ID > <NA> <NA>
One example:
SPEAKER R10_S186187188_C11_I1 1 2.8955 0.9352 <NA> <NA> 186 <NA> <NA>
Here is a submission example. Please note that you need to save the results of all sessions into one rttm file. And the Session_ID of the result must be exactly the same as the following Session_ID list. Here is the Session_ID list.
When the "Status" in the submission table turns to "Finished", you can refresh the page the check the score (DER) below. You can also click the button named “View scoring output log” to check the detailed score (FA, MISS, SPKERR).
- Other information
Make sure a. your submitted rttm file, includes all sessions in the evaluation set. b. your submission file ia in required format, or the submission will fail.
For each submitted score, it will be directly on the leader board if it's your best score so far.
During the evaluation set phase, every group is allowed to 5 times per day. Due to the problem of CodaLab, there is a small probability that the result will not be obtained for a long time. Please wait patiently or submit at another time.
If you meet any problems during submission, please contact us. (Please indicate whether you participated in track1 or track2).