Machine Translation

Allegro’s in-house Machine Translation (MT) engine is purpose-built for e-commerce, addressing the challenge of communicating product information across multiple languages at scale and reduced cost. E-commerce platforms like Allegro must translate millions of product offers, descriptions, and customer interactions to enable seamless cross-border commerce.

Off-the-shelf solutions often struggle with domain-specific terminology, leading to inaccuracies that can confuse customers, misrepresent products, and erode trust. In contrast, our MT system accurately translates industry-specific terms, jargon, and unique e-commerce dialects while maintaining high scalability—ensuring every offer, product description, and help article resonates with non-native speakers as if written in their native language.

By leveraging state-of-the-art machine learning techniques and a vast linguistic corpus, Allegro’s MT engine enhances accessibility for target markets like Czech and Slovak speakers. Beyond meeting immediate business needs, this project contributes to the broader MT community by developing methodologies and datasets tailored to specialized domains like e-commerce.

Our in-house Machine Translation engine offers several distinct advantages:

1. Tailored Accuracy: Unlike general-purpose solutions, our MT system is finely tuned to handle Allegro’s diverse linguistic data, including both standard Polish and e-commerce-specific dialects. By incorporating domain-specific training data, the system delivers translations that are more accurate and contextually appropriate.

2. Cost Efficiency: Off-the-shelf solutions often involve significant recurring costs, particularly for platforms with extensive and growing multilingual content. Our in-house solution significantly reduces these costs by leveraging internal resources, such as our data center’s downtime for large-scale batch processing.

3. Scalability: The MT system is designed to grow with Allegro. As we expand to new markets, we can efficiently adapt the model to additional languages without incurring prohibitive costs. Innovation: Our approach includes cutting-edge techniques like model distillation, quantization, and backtranslation, allowing us to achieve near-state-of-the-art accuracy in a lightweight, production-ready format.

4. Ecosystem Contribution: We’ve released some of our models, such as Herbert and plT5, as open source, encouraging collaboration and innovation within the NLP community.

By focusing on these unique aspects, our MT engine not only provides a competitive advantage for Allegro but also sets a benchmark for how machine translation can be applied in niche domains.

Over the past year, we have achieved several milestones in the development of our MT engine:

1. Domain-Specific Adaptation: We advanced our training pipeline to include a comprehensive process:

• Pretraining on general bilingual datasets.
• Fine-tuning on e-commerce-specific data filtered from general corpora.
• Adaptation to Allegro’s unique linguistic data using backtranslation.
• Placeholder-aware training to incorporate additional linguistic knowledge and enhance MT quality.
• Improved model security (e.g., reducing the number of profanity hallucinations).

2. Model Optimization: By employing model distillation, we reduced the complexity of our neural models without sacrificing accuracy. This allowed us to deploy the MT engine on CPUs, making it cost-effective and resource-efficient.

3. Scalable Infrastructure: We optimized our data center operations to perform large-scale translations during off-peak hours, effectively translating millions of offers and products “almost for free.”

4. Innovative Evaluation Metrics: To ensure translation quality, we combined human assessments with automated metrics such as BLEU, chRF, and COMET. We also use MOS (Mean Opinion Score) for qualitative evaluation tracking agreement between annotators and actively working with them on improving alignment of human scores, creating a reliable benchmark for comparing models.

5. Empirical Success: Our MT engine now matches or exceeds commercial translation engines performance for Allegro’s specific downstream tasks, achieving remarkable accuracy in translating complex e-commerce texts.

6. Open Collaboration: We’ve continued to support the European NLP community by sharing research results during conferences and meetups, fostering collaboration and innovation in specialized translation domains.

These innovations have positioned our MT engine as a cornerstone of Allegro’s international expansion strategy, enabling seamless user experiences across linguistic boundaries.

The implementation of machine translations enables us to support merchants from around the globe, helping them seamlessly enter foreign markets supported by Allegro. At the same time, it empowers customers to understand product details effortlessly and, when needed, communicate effectively with sellers from other countries using Allegro’s integrated translation system.

Machine Translation (MT) Service: Efficiently manages dynamic content from consumers and merchants, including products, offers, and reviews, with an impressive 2.582 billion words translated in 2024 alone.

Cost Efficiency: Achieved significant savings of millions of zlotys throughout the year by reducing dependency on commercial translation engines, enabling more cost-effective and scalable operations.

Translation Quality – on example of Hungarian: 1/ Acceptable Translations Rate (ATR) (scale 0–100%):

• Offer Titles & Product Names: 18.8% higher quality than leading commercial competitors.
• Offer/Product Descriptions: 11% better performance than alternative solutions.

https://allegro.pl/