Refinements in hierarchical phrase-based translation systems
The relatively recently proposed hierarchical phrase-based translation model for statistical machine translation (SMT) has achieved state-of-the-art performance in numerous recent translation evaluations. Hierarchical phrase-based systems comprise a pipeline of modules with complex interactions. In this thesis, we propose refinements to the hierarchical phrase-based model as well as improvements and analyses in various modules for hierarchical phrase-based systems. We took the opportunity of increasing amounts of available training data for machine translation as well as existing frameworks for distributed computing in order to build better infrastructure for extraction, estimation and retrieval of hierarchical phrase-based grammars. We design and implement grammar extraction as a series of Hadoop MapReduce jobs. We store the resulting grammar using the HFile format, which offers competitive trade-offs in terms of efficiency and simplicity. We demonstrate improvements over two alternative solutions used in machine translation. The modular nature of the SMT pipeline, while allowing individual improvements, has the disadvantage that errors committed by one module are propagated to the next. This thesis alleviates
Pino, Juan Miguel
statistical machine translation ; hierarchical phrase-based translation ; natural language processing ; language modeling ; natural language generation
Refinements in hierarchical phrase-based translation systems. The relatively recently proposed hierarchical phrase-based translation model for statistical machine translation (SMT) has achieved state-of-the-art performance in numerous recent translation evaluations. Hierarchical phrase-based systems comprise a pipeline of modules with complex interactions. In this thesis, we propose refinements to the hierarchical phrase-based model as well as improvements and analyses in various modules for hierarchical phrase-based systems. We took the opportunity of increasing amounts of available training data for machine translation as well as existing frameworks for distributed computing in order to build better infrastructure for extraction, estimation and retrieval of hierarchical phrase-based grammars. We design and implement grammar extraction as a series of Hadoop MapReduce jobs. We store the resulting grammar using the HFile format, which offers competitive trade-offs in terms of efficiency and simplicity. We demonstrate improvements over two alternative solutions used in machine translation. The modular nature of the SMT pipeline, while allowing individual improvements, has the disadvantage that errors committed by one module are propagated to the next. This thesis alleviates
hierarchical phrase based translation system relatively_recently propose hierarchical phrase based translation model statistical_machine translation_smt achieve state_of_the_art performance numerous recent translation evaluation hierarchical phrase based_system comprise module complex_interaction propose refinement hierarchical phrase based_model well improvement analyse various module hierarchical phrase based_system take opportunity increase_amount available training_data machine_translation well exist framework distribute compute order build good infrastructure extraction estimation retrieval hierarchical phrase based grammar design implement grammar extraction series job store result grammar use format offer competitive trade off term efficiency simplicity improvement alternative solution use machine_translation modular nature smt allow individual improvement disadvantage error commit module propagate next