This challenge consists of two phases. The submission and evaluation procedures for each phase are as follows.

📌 Submission¶

• Each team is allowed a maximum of 2 submissions.
• The submission must be a single .json file containing report results for all TIFF files.

Submission File Format (.json)¶

The file should follow a list-based JSON format as shown below:

[  
  {  
    "id": "---",  
    "report": "---"  
  },  
  {  
    "id": "---",  
    "report": "---"  
  },

... ,

  {  
    "id": "---",  
    "report": "---"  
  }    
]

*Submissions with invalid JSON format will not be scored. Please ensure your file is properly formatted.

📌 Evaluation Metric¶

The final score is calculated as follows:
ranking score = 0.15 × (ROUGE + BLEU) + 0.4 × KEY score + 0.3 × EMB score

cf. KEY score: Jaccard similarity between sets of extracted keywords
cf. EMB score: Cosine similarity of sentence embeddings generated by a pre-trained language model
(*The weights were determined based on consultation with clinical experts)

You can evaluate the generated results by referring to the following GitHub repository:

Evaluation code

The leaderboard will reflect each team's best score among their submissions.