The Use of Rules in Machine Translation – Fundamentals of Natural Language Processing

The Use of Rules in Machine Translation

Rule-based machine translation uses a large number of built-in language rules and millions of bilingual dictionaries for each language pair. The software reads the text and turns it into a transitional representation, which is then used to make text in the target language. This method needs huge dictionaries with information about morphology, syntax, and meaning, as well as large sets of rules. The program uses these complicated sets of rules to change the grammar of the source language into the grammar of the target language.

Translations are based on massive dictionaries and complex linguistic concepts. One can improve the quality of an out-of-the-box translation by adding their own terms to the translation process. In this scenario, you’d generate user-defined dictionaries that change the system’s defaults. Most of the time, there are two steps: an initial investment that makes a big difference and doesn’t cost much and a continuous investment that makes a difference over time. While rule-based MT helps businesses reach the necessary quality threshold and beyond, the process can be time-consuming and expensive.

Statistics in Machine Translation Technology

Statistical machine translation uses statistical translation models whose parameters are based on the study of both monolingual corpora and bilingual corpora. Statistical translation models are easy to make, but the technology depends a lot on existing multilingual corpora. For a specific topic, you need at least two million words and a lot more for general language. In theory, it is possible to meet the quality criterion, but most businesses don’t have the huge amounts of existing multilingual corpora needed to build the necessary translation models. Also, statistical machine translation uses a lot of CPU and needs a complicated hardware setup to run translation models at an average level of performance.

Semantic Language Modeling

Language can be difficult to understand because there are so many different ways to say the same thing. For instance, a driver might ask, “Where can I get gas near here?” or “Where is the closest gas station?” All of these terms basically mean the same thing. To figure out what the driver wants, you need to understand the meaning of the words being used. A car company could teach a language model to understand statements like these and give the right satellite navigation directions in response.

A semantic language model is a method that ranks the likelihood of words in a phrase based on how they make sense together. When the semantic language model is used in a conversational system, the dialog state and domain semantics can be added on the fly to help guide the speech recognizer during the decoding process. We talk about one of these applications that uses a semantic language model to handle spontaneous speech in a reliable way. Even though the semantic language model can be made without data, data-driven machine learning approaches can help it a lot. An example-based method is also presented here to demonstrate a possible approach.