Zero-shot Cross-lingual POS Tagging for Filipino

• Main Authors: Layacan, Jimson Paulo, Flores, Isaiah Edri W., Tan, Katrina Bernice M., Estuar, Ma. Regina Justina, Montalan, Jann Railey E., De Leon, Marlene M
• Format: text
• Published: Archīum Ateneo 2024
• Subjects: Computer Engineering; Computer Sciences; Electrical and Computer Engineering; Engineering
• Online Access: https://archium.ateneo.edu/discs-faculty-pubs/427; https://archium.ateneo.edu/context/discs-faculty-pubs/article/1429/viewcontent/2024.fieldmatters_1.9.pdf
• Institution: Ateneo De Manila University

Supervised learning approaches in NLP, exemplified by POS tagging, rely heavily on the presence of large amounts of annotated data. However, acquiring such data often requires significant amount of resources and incurs high costs. In this work, we explore zero-shot cross-lingual transfer learning to address data scarcity issues in Filipino POS tagging, particularly focusing on optimizing source language selection. Our zero-shot approach demonstrates superior performance compared to previous studies, with top-performing fine-tuned PLMs achieving F1 scores as high as 79.10%. The analysis reveals moderate correlations between cross-lingual transfer performance and specific linguistic distances–featural, inventory, and syntactic–suggesting that source languages with these features closer to Filipino provide better results. We identify tokenizer optimization as a key challenge, as PLM tokenization sometimes fails to align with meaningful representations, thus hindering POS tagging performance.