Automatic lexicon extraction from comparable, non-parallel corpora

Date of Publication

2004

Document Type

Master's Thesis

Degree Name

Master of Science in Computer Science

College

College of Computer Studies

Department/Unit

Computer Science

Thesis Adviser

Rachel Editha O. Roxas

Defense Panel Chair

Lolita Reyes

Defense Panel Member

Allan B. Borra
Rachel Editha O. Roxas

Abstract/Summary

An automated approach of extracting bilingual lexicon (or dictionary) from comparable, non-parallel corpora is developed, implemented and tested. The corpora used are of bilingual domains containing 381,553 English and 92,610 Tagalog terms, with corresponding 4,817 and 3,421 distinct root words, respectively. The terms in the resulting lexicon are grouped into their respective senses. For the 100 test words (50 high frequency words, HFW and 50 low frequency words, LFW), 50.29% (HFW) and 31.37% (LFW) of the expected translations in all clusters were generated (called recall test). 56.12% (HFW) and 21.98% (LFW) of the expected translations within clusters were generated (called precision test). The overall results represented by the F-measure (a combination of recall and precision), show that 10.65% of the expected translations for the 100 test words were generated. Inclusion of several natural language resources (e.g. lexicon expansion to include alternate senses, word per word lexicon translation, larger comparable corpora), improvement of preprocessing techniques (e.g. stemming and part of speech tagging for Tagalog), and other enhancements (e.g. smoothing of sparse data and disambiguation techniques) would improve the overall performance of the system.

Abstract Format

html

Note

With: Technical manual and User's manual

Language

English

Format

Print

Accession Number

TG03689; CDTG003689

Shelf Location

Archives, The Learning Commons, 12F Henry Sy Sr. Hall

Physical Description

3 v. in 1 ; 28 cm. + computer optical disc.

Keywords

Lexicology--Data processing; Computational linguistics; Algorithms

This document is currently not available here.

Share

COinS