Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

📅 👥 2 authors arXiv:2611.14480v1

Authors

Abstract

In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.

Paper Information

arXiv ID:
2611.14480v1
Published:

Related Categories