{"id":3880,"date":"2025-12-15T11:21:15","date_gmt":"2025-12-15T11:21:15","guid":{"rendered":"https:\/\/jita-au.com\/?p=3880"},"modified":"2025-12-15T14:02:18","modified_gmt":"2025-12-15T14:02:18","slug":"an-evolutionary-overview-of-large-language-models-from-statistical-methods-to-the-transformer-era","status":"publish","type":"post","link":"https:\/\/jita-au.com\/index.php\/2025\/12\/15\/an-evolutionary-overview-of-large-language-models-from-statistical-methods-to-the-transformer-era\/","title":{"rendered":"An Evolutionary Overview of Large Language Models: From Statistical Methods to the Transformer Era"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Vol. 15 No. 2 (2025): JITA - APEIRON<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Boris Damjanovi\u0107, Dragan Kora\u0107, Dejan Simi\u0107, Negovan Stamenkovi\u0107<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

An Evolutionary Overview of Large Language Models: From Statistical Methods to the Transformer Era<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Review paper
DOI: https:\/\/doi.org\/10.7251\/JIT2502145D<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\tDownload Article PDF<\/span>\n\t\t\t\t\t<\/span>\n\t\t\t\t\t<\/a>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Abstract<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

While the early evolution of large language models (LLMs), including shift from statistical approaches to the Transformer\narchitecture, illustrates their historical impact on the processing of natural language; however, the latest research in neural networks\nhas enabled the faster and more powerful rise of language models grounded in solid theoretical foundations. These advantages,\ndriven by advances in computing systems (e.g., ultra-powerful processing and memory capabilities), enable the development of\nnumerous new models based on new emerging technologies such as artificial intelligence (AI). Thus, we provide an evolutionary\noverview of LLMs involved in the shift from the statistical to deep learning approach, highlighting their key stages of development, with a\nparticular focused on concepts such as self-attention, the Transformer architecture, BERT, GPT, DeepSeek, and Claude. Finally, our\nconclusions present a reference point for future research associated with the emergence of new AI-supported models that are\nirreversibly transforming the way an increasing number of human activities are performed.\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t

\n\t\t\t\t
\n\t\t\t\t\t

Keywords: Artificial intelligence, large language models, Transformer architecture, self-attention.<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

Paper received:<\/b> 7.11.2025.
Paper accepted:<\/b> 12.11.2025.<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\n\t\t\t