Song Lyric Meaning Generator Using Transformer (Case Study: Drake's Song Lyrics)

The limited availability of structured and credible sources that interpret Drake’s song lyrics makes it difficult for listeners to fully grasp the meaning and emotional depth within his music. This study aims to develop a web-based lyric meaning generator capable of automatically interpreting Drake’s lyrics using the Transformer architecture. The system employs the LLaMA 3 model, which is fine-tuned through Low-Rank Adaptation (LoRA) to improve training efficiency while maintaining high semantic accuracy. The curated dataset consists of Drake’s song lyrics, their corresponding interpretations, and metadata such as album and reference sources. Data preprocessing and supervised fine-tuning were conducted using the Hugging Face framework in Google Colab. A  gradio-based web application was implemented with a Retrieval-Augmented Generation (RAG) mechanism to enhance contextual relevance during inference. Evaluation metrics, including Semantic Similarity and ROUGE-L, were applied to measure the model’s ability to produce coherent and contextually aligned interpretations. The results demonstrate that the fine-tuned LLaMA 3 model effectively generates accurate lyric meanings while reducing computational cost. Overall, this study highlights the potential of Transformer-based models to bridge the gap between music and natural language understanding, particularly in analyzing metaphorical and emotion-rich song lyrics.

Keywords: Song lyric meaning generator, Transformer, LLaMA 3, Drake, NLP, LoRA, Fine-tuning, RAG

Copyright (c) 2025 Journal of Software Engineering and Multimedia (JASMED)

This work is licensed under a Creative Commons Attribution 4.0 International License.

[1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention Is All You Need,” Advances in Neural Information Processing Systems, vol. 30, 2017.

[2] F. Rahman and S. Ratna, “Perancangan E-Learning Berbasis Web Menggunakan Framework Codeigniter,” Technol. J. Ilm., vol. 9, no. 2, p. 95, 2018, doi: 10.31602/tji.v9i2.1370.

[3] Y. Agrawal, R. G. R. Shanker, and V. Alluri, “Transformer-Based Approach Towards Music Emotion Recognition from Lyrics,” in Advances in Information Retrieval, D. Hiemstra, M.-F. Moens, J. Mothe, R. Perego, M. Potthast, and F. Sebastiani, Eds., Lecture Notes in Computer Science, vol. 12657. Cham, Switzerland: Springer, 2021, pp. 160–174, doi: 10.1007/978-3-030-72240-1_12.

[4] W. Duan, Z. Zhang, Y. Yu, and K. Oyama, “Interpretable Melody Generation from Lyrics with Discrete-Valued Adversarial Training,” in Proceedings of the 30th ACM International Conference on Multimedia (MM ’22), Lisbon, Portugal, 2022, pp. 6973–6975, doi: 10.1145/3503161.3547742.

[5] C. Wu et al., “LLaMA Pro: Progressive LLaMA with Block Expansion,” Jan. 2024, [Online]. Available: http://arxiv.org/abs/2401.02415

[6] C. Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, pp. 1–67, 2020.

[7] Y. Mao et al., “A survey on LoRA of large language models,” Front. Comput. Sci., vol. 19, no. 7, pp. 1–144, 2025, doi: 10.1007/s11704-024-40663-9.

[8] C. M. V. de Andrade, W. Cunha, D. Reis, A. S. Pagano, L. Rocha, and M. A. Gonçalves, “A Strategy to Combine 1stGen Transformers and Open LLMs for Automatic Text Classification,” Aug. 2024, [Online]. Available: http://arxiv.org/abs/2408.09629

[9] S. Naseri, S. Reddy, J. Correia, J. Karlgren, and R. Jones, “The Contribution of Lyrics and Acoustics to Collaborative Understanding of Mood,” May 2022, [Online]. Available: http://arxiv.org/abs/2207.05680

[10] A. Han and R. Park, “Deep-Hop Rapper: Using LSTM and Transformer for Rap Lyric Generation Stanford CS224N {Custom, Default} Project.” [Online]. Available: https://deepbeat.org/.

[11] R. Zhang et al., “LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention,” Mar. 2023, [Online]. Available: http://arxiv.org/abs/2303.16199

[12] N. Weir, A. Poliak, and B. Van Durme, “Probing Neural Language Models for Human Tacit Assumptions,” Proc. 42nd Annu. Meet. Cogn. Sci. Soc. Dev. a Mind Learn. Humans, Anim. Mach. CogSci 2020, pp. 377–383, 2020.

[13] E. Hu et al., “Lora: Low-Rank Adaptation of Large Language Models,” ICLR 2022 - 10th Int. Conf. Learn. Represent., pp. 1–26, 2022.

[14] F. Dennis Heraldi and F. Zakhralativa Ruskanda, “EasyChair Preprint Effective Intended Sarcasm Detection Using Fine-Tuned Llama 2 Large Language Models Effective Intended Sarcasm Detection Using Fine-tuned Llama 2 Large Language Models,” 2024.

[15] M. Ventura and M. Toker, “TRBLLmaker -- Transformer Reads Between Lyrics Lines maker,” Dec. 2022, [Online]. Available: http://arxiv.org/abs/2212.04917.

[16] H. Touvron et al., “Llama 2: Open Foundation and Fine-Tuned Chat Models,” 2023, [Online]. Available: http://arxiv.org/abs/2307.09288.

[17] E. M. Bender, T. Gebru, A. McMillan-Major, and S. Shmitchell, “On the dangers of stochastic parrots: Can language models be too big?,” FAccT 2021 - Proc. 2021 ACM Conf. Fairness, Accountability, Transpar., pp. 610–623, 2021, doi: 10.1145/3442188.3445922.