Asiya has been designed to assist both system and metric developers by offering a rich repository of metrics and meta-metrics.
We compiled a rich set of specialized similarity metrics operating at different linguistic levels (lexical, syntactic and semantic). We also studied how the scores conferred by different metrics may be integrated into a single measure of quality, without having to adjust their relative importance.
Asiya is an evolution, extension, refactoring, and a replacement for its predecessor, IQMT.
Technical Requirements (optional):
- Individual metrics (BLEU, NIST, GTM, ROUGE, METEOR, ...)
- Linguistic processors:
- C&C Tools
- Charniak Parser
- MINIPAR dependency parser
Asiya allows for evaluation of translation quality aspects at different linguistic levels. Metric scores can also be combined into a single measure of quality
- Jesús Giménez and Lluís Màrquez. Asiya: An Open Toolkit for Automatic Machine Translation (Meta-)Evaluation. The Prague Bulletin of Mathematical Linguistics. Number 94, pages 77-86, 2010.