That's Fresh! Newsletter

FROM THE AI WORLD

Have you heard of LlaMantino, the Italian version of Meta's LLM, Llama-2, developed by the University of Bari? This model was created to improve text generation and linguistic comprehension skills in Italian, a language that is greatly underrepresented in the training datasets used to train Llama models (considering that only 11% of the training data is in a language other than English).

To create LlaMantino, researchers adopted two strategies: the use of the language adaptation technique QLoRA, which allows for the transfer of linguistic knowledge from general pre-trained models (like Llama) to models for a specific language, and the enrichment of the training dataset with Italian texts. Thanks to these techniques and the use of the Leonardo HPC supercomputer, several models of LlaMantino have been developed, now available on Hugging Face.

I believe that the LlaMantino project represents an exciting step forward in adapting pre-trained language models for the Italian context. This not only provides more proficient models in this language but also opens up new opportunities for understanding the mechanisms of language learning, sparking interest in the potential adaptation of this knowledge to other languages and linguistic contexts.

Italian text generation

The developers of LLaMAntino explored various tuning approaches to ensure a high-quality text generated in Italian suitable for common tasks.

CLEARBOX AI

Detecting money muling

In this use case, our synthetic data generation improved money muling detection models performances, often characterized by high data imbalance.

BLOG POST

Creating train and test datasets

This step lays the groundwork for training models effectively, ensuring they can learn from one set of data and then be evaluated on a separate, unseen one.

WEEKLY MEME

Your pals may also like...