Supplementary Material for
On the Elements of an Accurate Tree-to-String Machine Translation System
Graham Neubig, Kevin Duh
Proceedings of ACL 2014 (PDF)

This is the supplementary material for the paper "On the Elements of an Accurate Tree-to-String Machine Translation System" presented at ACL 2014. In summary, the paper showed the following three elements are very important to obtain high accuracy when using tree-to-string translation systems, with experiments on patent data for English-Japanese and Japanese-English:

• Parsing: It is important to use more accurate parsers, and also important to consider multiple parse candidates as input by using forests.
• Alignment: It is important to have accurate alignments, which allows for the extraction of more and better rules. We substituted GIZA for Nile.
• Search: It is important to have a good search algorithm, where we substituted standard cube pruning (CP) for hypergraph search (HS).

Unfortunately the data used in the paper is from NTCIR, which requires permission to use (although this permission is free), so I cannot provide the data to reproduce the experiments. I can, however provide data from the Kyoto Free Translation Task and scripts to achieve similar results. The files and results can be found below:

Data and (messy) Scripts

Results

All the following results are for English-Japanese with the Travatar decoder. For reference a phrase based system with Moses using Nile got a score of BLEU=22.23, RIBES=68.00.