An enhancement of Q-learning algorithm applied for social learning application - Muring, John Carlo D.; Villadores, John Marlie D. (2025)
ABSTRACT: This study focuses on enhancing the traditional Q-learning algorithm by incorporating Feature-rich State Representation. Dynamic Reward Engineering and Simplified Neural Network within a model with simulated social learning environment which is later then applied to its application counterpart. Comparative experiments demonstrate that the enhanced Q-learning model significantly outperforms its unenhanced version. Visual analysis of different episode counts ranging from 10-1000 shows the difference in the effectiveness in learning in a given dynamic environment. The traditional Q-learning algorithm showed that in terms of its learning trajectory it suffers from over simplistic reward design in a dynamic environment and it tends to be incapable of providing more contexts in states leading to state-space complexity which has earlier convergence. This reflects its simplicity and incapability of having meaningful learning, as it tends to stabilize its learned Q-values after only 10 episodes. On the other hand, experimental results demonstrated that Feature-Rich State Representation and Dynamic Reward Engineering significantly outperformed traditional Q-learning methods. The lower mean Q-value in the enhanced model indicated more precise learning, while the traditional model exhibited inflated Q-values due to inadequate state differentiation. Additionally, the enhanced model demonstrated significantly lower variance (0.012) and standard deviation, ensuring consistent learning behavior over time, whereas the traditional approach showed greater fluctuations, leading to instability, while the reward design resulted in a controlled reward growth (0.49 vs. 0.53) and ensured steady progression. Furthermore, NN-based Q-learning required higher peak memory compared to the Q-table method, but it effectively optimized memory allocation, preventing poor memory usage, making it more suitable for complex social learning environments without the full computational complexity and demands of Deep Q-Networks. This integration tests the ability of the model to handle intricate social dynamics. This study contributes to the field by presenting a robust simulation framework that bridges the gap between traditional reinforcement learning methods and the evolving demands of real-world scenarios.