Further investigating transformer models in recommender systems
Transformer architecture shows promise in RecSys applications with techniques like Bert4Rec. In 2021 NVIDIA introduced the Transformers4Rec (T4Rec) framework using the HuggingFace Transformers library to bridge NLP and the realm of RecSys. Their research focused on session-based next-click predictions on smaller e-commerce, and news datasets.
We adapted T4Rec to the larger Ekstra Bladet News Recommendation Dataset (RecSys 2024 challenge), and explored model performance with different configurations, including the addition of textual embeddings, and metadata of the articles, as well as the impact of different dataset sizes.
Our results showed significant improvements in model performance from the addition of vectorized textual representations and validated the assumptions regarding larger data volumes from NLP. In our comparison of model bases we found that GPT-2 outperforms XLNET.