История разработки и перспективы системной инженерии нейросетевых архитектур
Ключевые слова:
нейросетевые архитектуры, история нейронных сетей, системная инженерия, машинное обучение, многокритериальная оценкаАннотация
В статье дана классификация типовых архитектур искусственных нейронных сетей, а также рассмотрены ключевые исторические этапы их развития. Выявлены базовые функциональные свойства и представлен метод многокритериальной оценки потенциальной эффективности нейросетевых архитектур. Предлагаются несколько подходов к системной инженерии нейронных сетей. Показано, что наряду с хорошо зарекомендовавшими себя принципами многослойности, иерархичности, рекуррентности на первый план выходят вопросы обеспечения модульности, структурной адаптивности, а также вычислительной эффективности нейросетевых моделей.
Библиографические ссылки
W.S. McCulloch and W. Pitts, “A logical calculus of the ideas immanent in nervous activity,” Bulletin of Mathematical Biology, vol. 5 (4), pp. 115-133, 1943.
M.G. Quiles and R.A.F. Romero, “A computer vision system based on multi-layer perceptrons for controlling mobile robots,” Proc. of 18th International Congress of Mechanical Engineering, pp. 1-8, 2005.
A.L. Hodgkin and A.F. Huxley, “Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo.,” The Journal of Physiology, vol. 116 (4), pp. 449-472, 1952.
G.H. Rutherford, Z.D. Mobille, J. Brandt-Trainer, R. Follmann and E. Rosa, “Analog implementation of a Hodgkin–Huxley model neuron,” American Journal of Physics, pp. 918-923, 2020.
F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psychological Review, vol. 65 (6), pp. 386-408, 1958.
M.K. Bugeja and S.G. Fabri, “Multilayer perceptron adaptive dynamic control of mobile robots: experimental validation,” Proc. of the 2nd European Robotics Symposium EUROS 2008, pp. 165-174, 2008.
S. Linnainmaa, “The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors,” Master's Thesis (in Finnish). Univ. Helsinki, 1970.
Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard and L.D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1 (4), pp. 541-551, 1989.
V.P. Gladun, “Formation of concepts by growing learning nets,” Kibernetika, vol. 2, 1970, pp. 99-104 (In Russ.).
Y.V. Selyavskiy, “Dynamic growing pyramidal networks for assessing innovative projects feasibility in metallurgy,” Journal of Legal and Economic Studies, vol. 2, pp. 246–250, 2017 (In Russ.).
K. Fukushima, “Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position,” Biological Cybernetics, vol. 36 (4), pp. 193-202, 1980.
C. Lin, J. Jhang, “Intelligent traffic-monitoring system based on YOLO and convolutional fuzzy neural networks,” Proc. of IEEE Access, vol. 10, pp. 14120-14133, 2022.
J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc Natl Acad Sci USA, vol. 79 (8), pp. 2554-2558, 1982.
Z. Li and B. Deng, “A networked smart home system based on recurrent neural networks and reinforcement learning,” Systems Science & Control Engineering, vol. 9 (1), pp. 775-783, 2021.
S.E. Fahlman, G.E. Hinton and T.J. Sejnowski, “Massively parallel architectures for AI: NETL, Thistle, and Boltzmann machines,” Proc. of AAAI-83109, vol. 113, pp. 1-10, 1983.
O. Alemi, W. Li and P. Pasquier, “Affect-expressive movement generation with factored conditional Restricted Boltzmann Machines,” Proc. of 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 442-448, 2015.
L. Chua and L. Yang, “Cellular neural networks: Theory,” IEEE Transactions on Circuits and Systems, vol. 35, pp. 1257-1272, 1988.
Z. Szlavikt, R. Tetzlaff, A. Blug and H. Hoefler, “Visual inspection of metal objects using cellular neural networks,” Proc. of Int’l Workshop on Cellular Neural Networks and Their Applications, pp. 1-5, 2006.
G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of Control, Signals, and Systems, vol. 2 (4), pp. 303-314, 1989.
Z. Dlugosz and R. Dlugosz, “Nonlinear activation functions for artificial neural networks realized in hardware,” Proc. of 2018 25th International Conference "Mixed Design of Integrated Circuits and System" (MIXDES), pp. 381-384, 2018.
L.K. Hansen and P. Salamon, “Neural network ensembles,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, pp. 993-1001, 1990.
M. Aljasim and R. Kashef, “E2DR: A deep learning ensemble-based driver distraction detection with recommendations model,” Sensors, vol. 22 (5), pp. 1858, 2022.
T. Martinetz and K. Schulten, “A "Neural Gas" network learns topologies,” Artificial Neural Networks, T. Kohonen, K. Makisara, O. Simula, J. Kangas (Eds.). Elsevier, pp. 397-402, 1991.
H. Sasaki, T. Fukuda, M. Satomi, N. Kubota and N. Kubota, “Growing neural gas for intelligent robot vision with range imaging camera,” Proc. of 2009 International Conference on Mechatronics and Automation, pp. 3269-3274, 2009.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9 (8), pp. 1735-1780, 1997.
F. Nicola, Y. Fujimoto and R. Oboe, “A LSTM neural network applied to mobile robots path planning,” Proc. of 2018 IEEE 16th International Conference on Industrial Informatics (INDIN), pp. 349-354, 2018.
W. Maass, “Networks of spiking neurons: The third generation of neural network models,” Neural Networks, vol. 10, pp. 1659-1671, 1997.
G. Tang and K.P. Michmizos, “Gridbot: an autonomous robot controlled by a Spiking Neural Network mimicking the brain’s navigational system,” ICONS '18: Proceedings of the International Conference on Neuromorphic Systems, pp. 1-8, 2018.
M. Gori, G. Monfardini and F. Scarselli, “A new model for learning in graph domains,” Proc. of the 2005 IEEE International Joint Conference on Neural Networks, vol. 2, pp. 729-734, 2005.
B. Platten, M. Macfarlane, D. Graus and S. Mesbah, “Automated personnel scheduling with reinforcement learning and graph neural networks,” Proc. of RecSys in HR’22: The 2nd Workshop on Recommender Systems for Human Resources, pp. 1-10, 2022.
D.P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv:1312.6114, pp. 1-14, 2013.
X. Chen, J. Xu, R. Zhou, W. Chen, J. Fang and C. Liu, “TrajVAE: A Variational AutoEncoder model for trajectory generation,” Neurocomputing, vol. 428, pp. 332-339, 2021.
J. Sohl-Dickstein, E.A. Weiss, N. Maheswaranathan and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” Proc. of the 32nd International Conference on Machine Learning, vol. 37, pp. 1-10, 2015.
I. Reutov, “Generating of synthetic datasets using diffusion models for solving computer vision tasks in urban applications,” Procedia Computer Science, vol. 229, pp. 335-344, 2023.
C.R. Qi, H. Su, K. Mo and L. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” arXiv:1612.00593, pp. 1-19, 2016.
V. Mayer, Q. Feng, J. Deng, Y. Shi, Z. Chen and A. Knoll, “FFHNet: Generating multi-fingered robotic grasps for unknown objects in real-time,” Proc. of 2022 International Conference on Robotics and Automation (ICRA), pp. 762-769, 2022.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, pp. 5998-6008, 2017.
H. Kim, Y. Ohmura and Y. Kuniyoshi, “Transformer-based deep imitation learning for dual-arm robot manipulation,” Proc. of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 8965-8972, 2021.
S. Sabour, N. Frosst and G.E. Hinton, “Dynamic routing between capsules,” arXiv:1710.09829, pp. 1-11, 2017.
X. Zhang, L. Zheng, Z. Tan and S. Li, “Loop closure detection based on residual network and capsule network for mobile robot,” Sensors, vol. 22 (19), pp. 7137, 2022.
M. Raissi, P. Perdikaris and G.E. Karniadakis, “Physics informed deep learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations,” arXiv:1711.10561, pp. 1-22, 2017.
A.K. Sahoo and I. Klein, “MoRPI-PINN: A Physics-informed framework for mobile robot pure inertial navigation,” arXiv:2507.18206, pp. 1-9, 2025.
R.M. Hasani, M. Lechner, A. Amini, D. Rus and R. Grosu, “Liquid time-constant networks,” Proc. of AAAI Conference on Artificial Intelligence, vol. 35, pp. 7657-7666, 2020.
M. Chahine, R. Hasani, P. Kao, A. Ray, R. Shubert, M. Lechner, A. Amini and D. Rus, “Robust flight navigation out of distribution with liquid neural networks,” Science Robotics, vol. 8 (77), pp. 1-15, 2023.
S. Ren, K. He, R. Girshick and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” arXiv:1506.01497, pp. 1-14, 2015.
R. Sapkota and M. Karkee, “Comparing YOLO11 and YOLOv8 for instance segmentation of occluded and non-occluded immature green fruits in complex orchard environment,” arXiv:2410.19869, pp. 1-17, 2024.
L. Fu and S. Li, “A new semantic segmentation framework based on UNet,” Sensors, vol. 23 (19), pp. 8123, 2023.
P. Smolensky, “Information processing in dynamical systems: Foundations of harmony theory,” Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations, pp. 194-281, 1986.
T. Kohonen, “Self-organization and associative memory,” Springer Series in Information Sciences. 3d ed, 1989.
V. Di Massa, G. Monfardini, L. Sarti, F. Scarselli, M. Maggini and M. Gori, “A comparison between recursive neural networks and graph neural networks,” Proc. of the 2006 IEEE International Joint Conference on Neural Networks, pp. 778-785, 2006.
Загрузки
Опубликован
Выпуск
Раздел
Лицензия
Copyright (c) 2025 Системная инженерия и инфокоммуникации
Это произведение доступно по лицензии Creative Commons «Attribution-ShareAlike» («Атрибуция — На тех же условиях») 4.0 Всемирная.