Ваш браузер устарел.

Для того, чтобы использовать все возможности сайта, загрузите и установите один из этих браузеров.

скрыть

Article

  • Title

    PREDICTIVE CONTROL METHODS IN TASKS OF SEARCHING SADDLE POINTS

  • Authors

    Smorodin А.

  • Subject

    INFORMACION TECHNOLOGY. AUTOMATION

  • Year 2020
    Issue 3(62)
    UDC 517.9/519.6
    DOI 10.15276/opu.3.62.2020.10
    Pages 80-90
  • Abstract

    The article presents new methods for searching critical points of a function of several variables, including saddle points. Such problems are found in various fields of theoretical and practical science, for example, saddle-point construction lens design, machine and deep learning, problems of convex optimization and nonlinear programming (necessary and sufficient conditions for the solution are formulated using saddle points of the Lagrange function and proved in the Kuhn-Tucker theorem. When training neural networks, it is necessary to repeat the training process on large clusters and check the network's trainability at different loss functions and different network depth. Which means that thousands of new calculations are run, where each time the loss function is optimized on large amounts of data. So any acceleration in the process of finding critical points is a major advantage and saves computing resources. Many modern methods of searching saddle points are based on calculating the Hessian matrix, inverting this matrix, the scalar product of the gradient vector and the current vector, finding the full Lagrangian, etc. However, all these operations are computationally “expensive” and it would make sense to bypass such complex calculations. The idea of modifying the standard gradient methods used in the article is to apply fixed-point search schemes for nonlinear discrete dynamical systems for gradient descent problems. It is assumed that these fixed points correspond to unstable equilibrium positions, and there are large units among the multipliers of each equilibrium position. The averaged predictive control methods are used. Results of numerical modeling and visualization are presented in the form of two tables, which indicate basins of attraction for each critical point in each scheme, and statistical data by the convergence rates.

  • Keywords numerical methods for finding saddle points, controlled nonlinear discrete systems, basins of attraction
  • Viewed: 116 Dowloaded: 2
  • Download Article
  • References

    Література

    1. Bociort F., Maarten van Turnhout, Marinescu O. Practical guide to saddle-point construction in lens de-sign. Proc. SPIE 6667. Current Developments in Lens Design and Optical Engineering VIII, 666708 (18 September 2007). DOI: https://doi.org/10.1117/12.732477.

    2. A Fast Saddle-Point Dynamical System Approach to Robust Deep Learning / Esfandiari Y., Balu A., Ebrahimi K, Vaidya U., Elia N., Sarkar S. arXiv:1910.08623.

    3. Adolphs L., Hadi D., Lucchi A., Hofmann T. Local Saddle Point Optimization: A Curvature Exploita-tion Approach. Proceedings of Machine Learning Research. 2019. Volume 89. P. 486–495.

    4. Ott E., Grebodgi C., Yorke J.A. Controlling chaos. Phys. Rev. Lett. 1990. 64. 1196–1199.

    5. Chen G., Dong X. From chaos to order: Methodologies, Perspectives and Application. Singapore : World Scientific, 1998, 760 p.

    6. Jackson E.A. Perspectives of Nonlinear Dinamics. Vol. I, II. Cambridge Univ. Press, Cambridge, 1980, 1990 Chaos II, ed. Hao Bai-Lin. World Sci., 1990.

    7. Yang D., Zhou J. Connections among several chaos feedback control approaches and chaotic vibration control of mechanical systems. Commun. Nonlinear Sci. Numer. Simulat. 2014. 19. 3954–3968.

    8. Miller J.R., Yorke J.A. Finding all periodic orbits of maps using Newton methods: sizes of basins. Physica D. 2000. 135. 195–211.

    9. Ypma T.J. Historical Development of the Newton-Raphson Method. SIAM Rev. 1995. 37. 531–551.

    10. Polyak B.T. Stabilizing chaos with predictive control. Automation and Remote Control. 2005. 66 (11). 1791–1804.

    11. Dmitrishin D., Iacob E., Stokolos A. Average predictive control for nonlinear discrete dynamical sys-tems. Advances in Systems Science and Applications. 2020. 20(1). P. 27–49.

    12. Vieira de S.M., Lichtenberg A.J. Controlling chaos using nonlinear feedback with delay. Phys. Rev. 1996. E 54. 1200–1207.

    13. Morgul O. Further stability results for a generalization of delayed feedback control. Nonlinear Dynam-ics. 2012. 1–8.

    14. Dmitrishin D., Skrinnik I., Lesaja G., Stokolos A. (2019). A new method for finding cycles by semilin-ear control. Physics Letters A. 2019. 383, 16. P. 1871–1878. DOI: https://doi.org/10.1016/j.physleta.2019.03.013.

    References

    1. Bociort, F., Maarten van Turnhout, & Marinescu, O. (2007). Practical guide to saddle-point construc-tion in lens design, Proc. SPIE 6667. Current Developments in Lens Design and Optical Engineering VIII. DOI: https://doi.org/10.1117/12.732477.

    2. Esfandiari, Y., Balu, A., Ebrahimi, K, Vaidya, U., Elia, N., Sarkar, S. (2020). A Fast Saddle-Point Dy-namical System Approach to Robust Deep Learning, arXiv:1910.08623.

    3. Adolphs, L., Hadi, D., Lucchi, A., & Hofmann, T. (2019). Local Saddle Point Optimization: A Curva-ture Exploitation Approach. Proceedings of Machine Learning Research, 89, 486–495.

    4. Ott, E., Grebodgi, C., & Yorke, J.A. (1990). Controlling chaos. Phys. Rev. Lett. 64, 1196–1199..

    5. Chen, G., & Dong, X., (1999). From chaos to order: Methodologies. Perspectives and Application. World Scientific, Singapore.

    6. Jackson, E.A. (1990). Perspectives of Nonlinear Dinamics. Vol. I, II, Cambridge Univ. Press, Cam-bridge, 1980, 1990 Chaos II, ed. Hao Bai-Lin. World Sci.

    7. Yang, D., & Zhou, J. (2014). Connections among several chaos feedback control approaches and cha-otic vibration control of mechanical systems. Commun. Nonlinear Sci. Numer. Simulat, 19, 3954–3968.

    8. Miller, J.R., & Yorke, J.A. (2000). Finding all periodic orbits of maps using Newton methods: sizes of basins. Physica D, 135, 195–211.

    9. Ypma, T.J. (1995). Historical Development of the Newton-Raphson Method. SIAM Rev., 37, 531–551.

    10. Polyak, B.T. (2005). Stabilizing chaos with predictive control. Automation and Remote Control. 66 (11), 1791–1804.

    11. Dmitrishin, D., Iacob, E., & Stokolos, A. (2020). Average predictive control for nonlinear discrete dy-namical systems. Advances in Systems Science and Applications, 20(1), 27–49.

    12. Vieira de S.M., & Lichtenberg, A.J. (1996). Controlling chaos using nonlinear feedback with delay. Phys. Rev. E 54, 1200–1207.

    13. Morgul, O. (2012). Further stability results for a generalization of delayed feedback control. Nonlinear Dynamics, 1–8.

    14. Dmitrishin, D., Skrinnik, I., Lesaja, G., & Stokolos, A. (2019). A new method for finding cycles by se-milinear control. Physics Letters A, 383, 1871–1878. DOI: https://doi.org/10.1016/j.physleta. 2019.03.013.

  • Creative Commons License by Author(s)