Loading...

Performance Comparison of Kolmogorov-Arnold Networks and Neural Networks

Eshtehardian, Mohammad | 2025

0 Viewed
  1. Type of Document: M.Sc. Thesis
  2. Language: Farsi
  3. Document No: 58385 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Hossein Khalaj, Babak; Yassaee Maybodi, Mohammad Hossein
  7. Abstract:
  8. Neural networks have become a primary tool in machine learning, capable of learning complex mappings by applying fixed nonlinear activation functions (e.g., ReLU or tanh) to linear combinations of data. However, a fundamental challenge lies in the mathematical interpretation of their internal workings. As an alternative, Kolmogorov-Arnold Networks (KANs) have been introduced, founded on the Kolmogorov-Arnold representation theorem. By employing learnable activation functions on the edges of the network, KANs offer an inherently more interpretable model architecture. The main objective of this thesis is to provide a theoretical and analytical comparison between Kolmogorov-Arnold Networks and classical neural networks. The research focuses on analyzing the behavior of these two architectures in the overparameterized regime and investigating the convergence rate of the gradient descent algorithm. In this context, it is demonstrated that under specific conditions, KANs achieve desirable convergence even when only a subset of their parameters are trained. This finding highlights a path toward reducing computational complexity compared to prior methods
  9. Keywords:
  10. Neural Networks ; Over-Parameterized Neural Networks ; Gradient Descent Algorithm ; Convergence Rate ; Tangent Kernel ; Kolmogorov-Arnold Networks

 Digital Object List

 Bookmark

No TOC