Authors:
Jesús Giménez and Lluís Màrquez
References:
Description:
A a common workbench on which automatic MT evaluation metrics can be robustly used and combined for the purpose of MT system development. Current version includes a rich set of metrics operating at different linguistic levels (lexical, shallow syntactic, syntactic, and shallow semantic).
Functionality:
Technology:
Perl
Technical Requirements:
Optional:
- Individual metrics (BLEU, NIST, GTM, ROUGE, METEOR, ...)
- Linguistic processors:
- C&C Tools
- SwiRL
- BIOS
- Charniak Parser
- MINIPAR dependency parser
- SVMTool
Modules:
Metrics:
Innovation:
IQmt allows for evaluation of translation quality aspects at different linguistic levels. Metric scores can also be combined into a single measure of quality, either through QARLA, of via arithmetic mean.
Development:
Publications:
- Jesús Giménez and Enrique Amigó. IQMT: A Framework for Automatic Machine Translation Evaluation. Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC'06). Genoa, Italy, 22-28 May. 2006. Departament Research Technical Report (LSI-07-29-R).
- Jesús Giménez and Lluís Màrquez. Linguistic Features for Automatic Evaluation of Heterogeneous MT Systems. Proceedings of WMT 2007 (ACL'07).
- Jesús Giménez and Lluís Màrquez. A Smorgasbord of Features for Automatic MT Evaluation. Proceedings of the 3rd ACL Workshop on Statistical Machine Translation (shared evaluation task).
Contact: lluism@lsi.upc.edu
