[ Log In | Register]
! Skip Navigation Links
Skip Navigation Links
Image of Vibration Testing and System Dynamics
ISSN: 2475-4811 (print)
ISSN: 2475-482X (online)
Journal of Vibration Testing and System Dynamics

C. Steve Suh (editor), Pawel Olejnik (editor),

Xianguo Tuo (editor)

Pawel Olejnik (editor)

Lodz University of Technology, Poland

Email: pawel.olejnik@p.lodz.pl

C. Steve Suh (editor)

Texas A&M University, USA

Email: ssuh@tamu.edu

Xiangguo Tuo (editor)

Sichuan University of Science and Engineering, China

Email: tuoxianguo@suse.edu.cn

Modified Gradient Descent Approach Involving Caputo Fractional Derivative with Metaheuristic Optimizer

Journal of Vibration Testing and System Dynamics 8(4) (2024) 443--453 | DOI:10.5890/JVTSD.2024.12.006

Rinki Sharma, Priyanka Harjule

Department of Mathematics, Malaviya National Institute of Technology, 302017 Jaipur, India

Download Full Text PDF

 

Abstract

In this paper, a modified Gradient Descent algorithm is investigated, which involves Caputo fractional derivative and a metaheuristic optimizer. Mathematical formulation for convergence analysis of the proposed algorithm is done in this study. Further, this article aims to enhance the initialization for better performance of existing fractional Gradient Descent algorithm. In this study the significance of using fractional derivative in Gradient Descent is also examined. Caputo fractional derivative is preferred due to its memory and non-locality properties. Metaheuristic algorithms are efficient in providing approximate results in fewer iterations. The fractional Gradient Descent algorithm with metaheuristic initialization is given for faster convergence. Further, the fractional Gradient Descent algorithm with a firefly optimizer is applied on the quadratic loss function. Results shows that this hybridization of the firefly optimizer and fractional order Gradient Descent converges in fewer iterations as compared to the existing algorithms.

References

  1. [1]  Abd Elaziz, M., Dahou, A., Abualigah, L., Yu, L., Alshinwan, M., Khasawneh, A.M., and Lu, S. (2021), Advanced metaheuristic optimization techniques in applications of deep neural networks: a review, Neural Computing and Applications, 33, 14079-14099.
  2. [2]  Ruder, S. (2016), An Overview of Gradient Descent Optimization Algorithms, arXiv preprint arXiv:1609.04747.
  3. [3]  Gori, M. and Tesi, A. (1992), On the problem of local minima in backpropagation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(1), 76-86.
  4. [4]  Cetin, B.C., Burdick, J.W., and Barhen, J. (1993), Global descent replaces gradient descent to avoid local minima problem in learning with artificial neural networks, In IEEE International Conference on Neural Networks, 836-842.
  5. [5]  Hamid, N.A., Nawi, N.M., Ghazali, R., and Salleh, M.N M. (2012), Solving local minima problem in back propagation algorithm using adaptive gain, adaptive momentum and adaptive learning rate on classification problems, In International Journal of Modern Physics: Conference Series, World Scientific Publishing Company, 9, 448-455.
  6. [6]  Dauphin, Y.N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., and Bengio, Y. (2014), Identifying and attacking the saddle point problem in high-dimensional non-convex optimization, Advances in Neural Information Processing Systems, 27, 1-9.
  7. [7]  Wang, Y., He, Y., and Zhu, Z. (2022), Study on fast speed fractional order gradient descent method and its application in neural networks, Neurocomputing, 489, 366-376.
  8. [8]  Hai, P.V. and Rosenfeld, J.A. (2021), The gradient descent method from the perspective of fractional calculus, Mathematical Methods in the Applied Sciences, 44(7), 5520-5547.
  9. [9]  Wang, J., Wen, Y., Gou, Y., Ye, Z., and Chen, H. (2017), Fractional-order gradient descent learning of BP neural networks with Caputo derivative, Neural Networks, 89, 19-30.
  10. [10]  Bao, C., Pu, Y., and Zhang, Y. (2018), Fractional-order deep backpropagation neural network, Computational intelligence and neuroscience, 2018, 1-11.
  11. [11]  Özdemir, N. and Karadeniz, D. (2008), Fractional diffusion-wave problem in cylindrical coordinates, Physics Letters A, 372(38), 5968-5972.
  12. [12]  Harjule, P. and Bansal, M.K. (2020), Fractional order models for viscoelasticity in lung tissues with power, exponential and mittag–leffler memories, International Journal of Applied and Computational Mathematics, 6(4), 119.
  13. [13]  Baleanu, D., Machado, J.A.T., and Luo, A.C. (Eds.), (2011), Fractional Dynamics and Control, Springer Science \& Business Media.
  14. [14]  Harjule, P., Gurjar, A., Seth, H., and Thakur, P. (2020), Text classification on Twitter data, In 2020 3rd International Conference on Emerging Technologies in Computer Engineering: Machine Learning and Internet of Things (ICETCE) (pp. 160-164), IEEE.
  15. [15]  Agarwal, B., Harjule, P., Chouhan, L., Saraswat, U., Airan, H., and Agarwal, P. (2021), Prediction of dogecoin price using deep learning and social media trends, EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, 8(29), e2-e2.
  16. [16]  Harjule, P., Tokir, M. M., Mehta, T., Gurjar, S., Kumar, A., and Agarwal, B. (2022), Texture enhancement of medical images for efficient disease diagnosis with optimized fractional derivative masks, Journal of Computational Biology, 29(6), 545-564.
  17. [17]  Li, C. and Deng, W. (2007), Remarks on fractional derivatives, Applied Mathematics and Computation, 187(2), 777-784.
  18. [18]  Khan, S., Ahmad, J., Naseem, I., and Moinuddin, M. (2018), A novel fractional gradient-based learning algorithm for recurrent neural networks, Circuits, Systems, and Signal Processing, 37, 593-612.
  19. [19]  Sheng, D., Wei, Y., Chen, Y., and Wang, Y. (2020), Convolutional neural networks with fractional order gradient method, Neurocomputing, 408, 42-50.
  20. [20]  Li, C., Qian, D., and Chen, Y. (2011), On Riemann-Liouville and caputo derivatives, Discrete Dynamics in Nature and Society, 2011, 1-16.
  21. [21]  Wei, Y., Chen, Y., Gao, Q., and Wang, Y. (2019), Infinite series representation of functions in fractional calculus, In 2019 Chinese Automation Congress (CAC) (pp. 1697-1702), IEEE.
  22. [22]  Oldham, K. and Spanier, J. (1974), The Fractional Calculus Theory and Applications of Differentiation and Integration to Arbitrary Order, Elsevier.
  23. [23]  Samko, S.G., Kilbas, A.A., and Marichev, O.I. (1993), Fractional Integrals and Derivatives (Vol. 1), Yverdon-les-Bains, Switzerland: Gordon and breach science publishers, Yverdon.
  24. [24]  Abdel-Basset, M., Abdel-Fatah, L., and Sangaiah, A.K. (2018), Metaheuristic algorithms: A comprehensive review, Computational Intelligence for Multimedia Big Data on the Cloud with Engineering Applications, 185-231.
  25. [25]  Yang, X.S. (2010), Nature-Inspired Metaheuristic Algorithms, Luniver press.
  26. [26]  Grigoletto, E.C. and de Oliveira, A.R.L. (2020), Fractional order gradient descent algorithm, Proceeding Series of the Brazilian Society of Computational and Applied Mathematics, 7(1),
  27. [27]  Altan, G., Alkan, S., and Baleanu, D. (2023), A novel fractional operator application for neural networks using proportional Caputo derivative, Neural Computing and Applications, 35(4), 3101-3114.
  28. [28]  Shin, Y., Darbon, J., and Karniadakis, G.E. (2021), A caputo fractional derivative-based algorithm for optimization, arXiv preprint arXiv:2104.02259.
  29. [29]  Chen, Y., Gao, Q., Wei, Y., and Wang, Y. (2017), Study on fractional order gradient methods, Applied Mathematics and Computation, 314, 310-321.
  30. [30]  Hijazi, H., Kandil, N., Zaarour, N., and Hakem, N. (2019), Impact of initialization on gradient descent method in localization using received signal strength, In ITM Web of Conferences (Vol. 27, p. 01003), EDP Sciences.

Copyright © 2011-2024 L & H Scientific Publishing. All rights reserved. Wildcard SSL Certificates