Open-source resources and standards for Arabic word structure analysis : fine grained morphological analysis of Arabic text corpora
Morphological analyzers are preprocessors for text analysis. Many Text Analytics applications need them to perform their tasks. The aim of this thesis is to develop standards, tools and resources that widen the scope of Arabic word structure analysis - particularly morphological analysis, to process Arabic text corpora of different domains, formats and genres, of both vowelized and non-vowelized text. We want to morphologically tag our Arabic Corpus, but evaluation of existing morphological analyzers has highlighted shortcomings and shown that more research is required. Tag-assignment is significantly more complex for Arabic than for many languages. The morphological analyzer should add the appropriate linguistic information to each part or morpheme of the word (proclitic, prefix, stem, suffix and enclitic); in effect, instead of a tag for a word, we need a subtag for each part. Very fine-grained distinctions may cause problems for automatic morphosyntactic analysis â particularly probabilistic taggers which require training data, if some words can change grammatical tag depending on function and context; on the other hand, finegrained distinctions may actually help to disambiguate other words in the local
Sawalha, Majdi Shaker Salem
Open-source resources and standards for Arabic word structure analysis : fine grained morphological analysis of Arabic text corpora. Morphological analyzers are preprocessors for text analysis. Many Text Analytics applications need them to perform their tasks. The aim of this thesis is to develop standards, tools and resources that widen the scope of Arabic word structure analysis - particularly morphological analysis, to process Arabic text corpora of different domains, formats and genres, of both vowelized and non-vowelized text. We want to morphologically tag our Arabic Corpus, but evaluation of existing morphological analyzers has highlighted shortcomings and shown that more research is required. Tag-assignment is significantly more complex for Arabic than for many languages. The morphological analyzer should add the appropriate linguistic information to each part or morpheme of the word (proclitic, prefix, stem, suffix and enclitic); in effect, instead of a tag for a word, we need a subtag for each part. Very fine-grained distinctions may cause problems for automatic morphosyntactic analysis â particularly probabilistic taggers which require training data, if some words can change grammatical tag depending on function and context; on the other hand, finegrained distinctions may actually help to disambiguate other words in the local
open_source resource standard_arabic word structure analysis fine grain morphological_analysis arabic_text corpus morphological text_analysis many text application need perform_task aim_develop standard tool resource widen scope arabic_word structure analysis particularly morphological_analysis process arabic_text corpus different_domain format genre non text want morphologically tag arabic corpus evaluation exist morphological highlight shortcoming show require tag assignment significantly complex arabic many language morphological add appropriate linguistic information part morpheme word prefix stem_suffix effect instead tag word need part fine_grained distinction may cause problem automatic morphosyntactic analysis particularly probabilistic require training_data word change grammatical tag depend function context hand distinction may actually help disambiguate word local