[ Log In | Register]
! Skip Navigation Links
Skip Navigation Links
Image of Vibration Testing and System Dynamics
ISSN: 2475-4811 (print)
ISSN: 2475-482X (online)
Journal of Vibration Testing and System Dynamics

C. Steve Suh (editor), Pawel Olejnik (editor),

Xianguo Tuo (editor)

Pawel Olejnik (editor)

Lodz University of Technology, Poland

Email: pawel.olejnik@p.lodz.pl

C. Steve Suh (editor)

Texas A&M University, USA

Email: ssuh@tamu.edu

Xiangguo Tuo (editor)

Sichuan University of Science and Engineering, China

Email: tuoxianguo@suse.edu.cn

CombOpNet: a Neural-Network Accelerator for SINDy

Journal of Vibration Testing and System Dynamics 9(1) (2025) 1--20 | DOI:10.5890/JVTSD.2025.03.001

Siyuan Xing$^1$, Qingyu Han$^2$, Efstathios G. Charalampidis$^{3,4}$

$^1$ Mechanical Engineering Department, California Polytechnic State University, San Luis Obispo, CA, 93407- 0403, USA

$^2$ Electric Engineering Department, California Polytechnic State University, San Luis Obispo, CA, 93407-0403, USA

$^3$ Department of Mathematics and Statistics, San Diego State University, San Diego, CA 92182-7720, USA

$^4$ Computational Science Research Center, San Diego State University, CA 92182-7720, USA

Download Full Text PDF

 

Abstract

In the present work, we develop and assess a compact neural-network architecture called hereafter the Combinatorial Operation Neural Network (CombOpNet) which is designed to mitigate the curse of dimensionality of Sparse Identification of Nonlinear Dynamics (SINDy). Within CombOpNet, nonlinear terms in dynamical equations are represented by a chain of multiplication of univariate functions. These functions correspond to neurons of a multi-layer neural network which allows the construction of nonlinear terms in dynamical equations by employing forward propagation. This way, the CombOpNet can form and perform summations of all possible combinations of univariate functions now using far fewer weights, which themselves can reconstruct the elements in the coefficient matrix of SINDy. If $n$ and $p$ denote the number of dimensions and order of nonlinearity, respectively, the present method reduces the number of SINDy coefficients from $n\binom{n+p}{n}$ to $\mathcal{O}(n^3p)$. This reduction facilitates the scalability of SINDy, making SINDy applicable to multidimensional systems such as power grids and climate models. We discuss the implementation of CombOpNet in Tensorflow, and demonstrate its applicability to several complex nonlinear dynamical systems with an eye towards solving high-dimensional problems.

Acknowledgments

This material is based upon work supported by the Research, Scholarly \& Creative Activities (RSCA) Program awarded by the Cal Poly division of Research, Economic Development \& Graduate Education and Chrones Endowed Professorship (SX), and the US National Science Foundation under Grant No. DMS-2204782 (EGC). SX thanks Y.C. Lai and B.G. Anderson for useful discussions.

References

  1. [1]  Brunton, S.L. and Kutz, J.N. (2019), Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge University Press.
  2. [2]  Kutz, J.N., Brunton, S.L., Brunton, B.W., and Proctor, J.L. (2016), Dynamic Mode Decomposition, Society for Industrial and Applied Mathematics.
  3. [3]  Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., and Yang, L. (2021), Physics-informed machine learning, Nature Reviews Physics, 3, 422-440.
  4. [4]  Goodfellow, I., Bengio, Y., and Courville, A. (2016), Deep Learning, MIT Press.
  5. [5]  Brunton, S.L., Proctor, J.L., and Kutz, J.N. (2016), Discovering governing equations from data by sparse identification of nonlinear dynamical systems, Proceedings of the National Academy of Sciences, 113, 3932-3937.
  6. [6]  Guruwacharya, N., Chakraborty, S., Saraswat, G., Bryce, R., Hansen, T.M., and Tonkoski, R. (2024), Data-driven modeling of grid-forming inverter dynamics using power hardware-in-the-loop experimentation, IEEE Access, 12, 52267-52281.
  7. [7]  Katsoulakis, M.A. and Vilanova, P. (2020), Data-driven, variational model reduction of high-dimensional reaction networks, Journal of Computational Physics, 401, 108997.
  8. [8]  Song, J., Tong, G., Chao, J., Chung, J., Zhang, M., Lin, W., Zhang, T., Bentler, P.M., and Zhu, W. (2023), Data driven pathway analysis and forecast of global warming and sea level rise, Scientific Reports, 13, 5536.
  9. [9]  Beam, E., Potts, C., Poldrack, R.A., and Etkin, A. (2021), A data-driven framework for mapping domains of human neurobiology, Nature Neuroscience, 24, 1733-1744.
  10. [10]  Bongard, J. and Lipson, H. (2007), Automated reverse engineering of nonlinear dynamical systems, Proceedings of the National Academy of Sciences, 104, 9943-9948.
  11. [11]  Schmidt, M. and Lipson, H. (2009), Distilling free-form natural laws from experimental data, Science, 324, 81-85.
  12. [12]  Udrescu, S.M. and Tegmark, M. (2020), Ai feynman: A physics-inspired method for symbolic regression, Science Advances, 6(16), eaay2631 (16 pages).
  13. [13]  Petersen, B.K., Landajuela, M., Mundhenk, T.N., Santiago, C.P., Kim, S.K., and Kim, J.T. (2021), Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients, Proceeding of the International Conference on Learning Representations, 1-26.
  14. [14]  Donoho, D.L. (2006), Compressed sensing, IEEE Transactions on Information Theory, 52, 1289-1306.
  15. [15]  Baraniuk, R.G. (2007), Compressive sensing [lecture notes], IEEE Journals $\&$ Manazine, 24, 118-121.
  16. [16]  Wang, W.X., Yang, R., Lai, Y.C., Vassilios, K., and Grebogi, C. (2011), Predicting catastrophes in nonlinear dynamical systems by compressive sensing, Physical Review Letters, 106, 154101 (4 pages).
  17. [17]  Zhang, L. and Schaeffer, H. (2019), On the convergence of the SINDy algorithm, Multiscale Modeling $\&$ Simulation, 17, 948-972.
  18. [18]  Brunton, S.L., Proctor, J.L., and Kutz, J.N. (2016), Sparse identification of nonlinear dynamics with control (sindyc), IFAC-PapersOnLine, 49, 710-715.
  19. [19]  Rudy, S.H., Brunton, S.L., Proctor, J.L., and Kutz, J.N. (2016), Data-driven discovery of partial differential equations, IFAC-PapersOnLine, 49, 710-715.
  20. [20]  Rudy, S., Alla, A., Brunton, S.L., and Kutz, J.N. (2019), Data-driven identification of parametric partial differential equations, SIAM Journal on Applied Dynamical Systems, 18, 643-660.
  21. [21]  Messenger, D.A. and Bortz, D.M. (2021), Weak SINDy for partial differential equations, Journal of Computational Physics, 443, 110525.
  22. [22]  Mangan, N.M., Brunton, S.L., Proctor, J.L., and Kutz, J.N. (2016), Inferring biological networks by sparse identification of nonlinear dynamics, IEEE Transactions on Molecular, Biological, and Multi-Scale Communications, 2, 52-63.
  23. [23]  Kaheman, K., Kutz, J.N., and Brunton, S.L. (2020), SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 476, 20200279 (25 pages).
  24. [24]  Jacobs, M., Brunton, B.W., Brunton, S.L., Kutz, J.N., and Raut, R.V. (2023), HyperSINDy: Deep generative modeling of nonlinear stochastic governing equations.
  25. [25]  Hoffmann, M., Fr\"ohner, C., and Noé, F. (2019), Reactive SINDy: Discovering governing reactions from concentration data, The Journal of Chemical Physics, 150, 025101.
  26. [26]  Kaiser, E., Kutz, J.N., and Brunton, S.L. (2018), Sparse identification of nonlinear dynamics for model predictive control in the low-data limit, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474, 20180335 (25 pages).
  27. [27]  Zolman, N., Fasel, U., Kutz, J.N., and Brunton, S.L. (2024), RL: Interpretable and efficient model-based reinforcement learning, arXiv:2403.09110, ICS.
  28. [28]  Fukami, K., Murata, T., Zhang, K., and Fukagata, K. (2021), Sparse identification of nonlinear dynamics with low dimensionalized flow representations, Computer Methods in Applied Mechanics and Engineering, 926, A10 (35 pages).
  29. [29]  Prokop, B. and Gelens, L. (2023), Data-driven discovery of oscillator models using SINDy: Towards the application on experimental data in biology, bioRxiv, pp.2023-08.
  30. [30]  Bramburger, J.J. and Kutz, J.N. (2020), Poincaré maps for multiscale physics discovery and nonlinear floquet theory, Physica D: Nonlinear Phenomena, 408, 132479.
  31. [31]  Bramburger, J.J., Dylewsky, D., and Kutz, J.N. (2020), Sparse identification of slow timescale dynamics, Physical Review E, 102, 022204.
  32. [32]  Saqlain, S., Zhu, W., Charalampidis, E.G., and Kevrekidis, P.G. (2023), Discovering governing equations in discrete systems using PINNs, Communications in Nonlinear Science and Numerical Simulation, 13, 107498 (12 pages).
  33. [33]  Zheng, P., Askham, T., Brunton, S.L., Kutz, J.N., and Aravkin, A.Y. (2019), A unified framework for sparse relaxed regularized regression: SR3, IEEE Access, 7, 1404-1423.
  34. [34]  Cortiella, A., Park, K.C., and Doostan, A. (2021), Sparse identification of nonlinear dynamical systems via reweighted $l_1$-regularized least squares. Computer Methods in Applied Mechanics and Engineering, 376, 113620 (31 pages).
  35. [35]  Naozuka, G.T., Rocha, H.L., Silva, R.S., and Almeida, R.C. (2022), SINDy-SA framework: Enhancing nonlinear system identification with sensitivity analysis, Nonlinear Dynamics, 110, 2589-2609.
  36. [36]  Cortiella, A., Park, K.C., and Doostan, A. (2022), A priori denoising strategies for sparse identification of nonlinear dynamical systems: A comparative study, Journal of Computing and Information Science in Engineering, 23, 01004 (23 pages).
  37. [37]  Egan, K., Li, W., and Carvalho, R. (2024), Automatically discovering ordinary differential equations from data with sparsee regression, Communications Physics, 7.
  38. [38]  Chen, Z., Liu, Y., and Sun, H. (2021), Physics-informed learning of governing equations from scarce data, Nature Communications, 12, 6136 (13 pages).
  39. [39]  Kaheman, K., Brunton, S.L., and Kutz, J.N. (2022), Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data, Machine Learning: Science and Technology, 3, 015031 (28 pages).
  40. [40]  Hirsh, S.M., Barajas-Solano, D.A., and Kutz, J.N. (2022), Sparsifying priors for bayesian uncertainty quantification in model discovery, Royal Society Open Science, 9, 211823 (20 pages).
  41. [41]  Zhang, S. and Lin, G. (2018), Robust data-driven discovery of governing physical laws with error bars, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 474.
  42. [42]  Gel\ss, P., Klus, S., Eisert, J., and Sch\"{u}tte, C. (2019), Multidimensional approximation of nonlinear dynamical systems, Journal of Computational and Nonlinear Dynamics, 14, 061006 (15 pages).
  43. [43]  Champion, K., Lusch, B., Kutz, J.N., and Brunton, S.L. (2019), Data-driven discovery of coordinates and governing equations, Proceedings of the National Academy of Sciences, 116, 22445-22451.
  44. [44]  Tibshirani, R. (1996), Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society. Series B (Methodological), 58, 267-288.
  45. [45]  Kingma, D.P. and Ba, J. (2015), Adam: A method for stochastic optimization, ICLR (Poster).
  46. [46]  Zhu, M. and Gupta, S. (2018), To prune, or not to prune: Exploring the efficacy of pruning for model compression, 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings, OpenReview.net.
  47. [47]  Noack, B.R., Afanasiev, K., Morzynski, M., Tadmor, G., and Thiele, F. (2003), A hierarchy of low-dimensional models for the transient and post-transient cylinder wake, Journal of Fluid Mechanics, 497, 335–363.
  48. [48]  Holmes, P., Lumley, J.L., and Berkooz, G. (1996), Turbulence, Coherent Structures, Dynamical Systems and Symmetry, Cambridge Monographs on Mechanics, Cambridge University Press.
  49. [49]  Shimizu, T. and Moroika, N. (1980), On the bifurcation of a symmetric limit cycle to an asymmetric one in a simple model, Physics Letters A, 76, 201-204.
  50. [50]  de Silva, B., Champion, K., Quade, M., Loiseau, J.C., Kutz, J., and Brunton, S. (2020), Pysindy: A python package for the sparse identification of nonlinear dynamical systems from data, Journal of Open Source Software, 5, 2104.
  51. [51]  Carbone, V. and Veltri, P. (1992), Relaxation processes in magnetohydrodynamics - a triad-interaction model, Astronomy and Astrophysics, 259, 359-372.
  52. [52]  Lorenz, E. (1995), Predictability: a problem partly solved, Seminar on Predictability, 4-8 September, 1, 1-18.

Copyright © 2011-2024 L & H Scientific Publishing. All rights reserved. Wildcard SSL Certificates