Loading...

Design of Memory Structure in Machine Learning Based on Human Learning System

Bidokhti, Amir | 2022

0 Viewed
  1. Type of Document: Ph.D. Dissertation
  2. Language: Farsi
  3. Document No: 58519 (05)
  4. University: Sharif University of Technology
  5. Department: Electrical Engineering
  6. Advisor(s): Ghaemmaghami, Shahrokh
  7. Abstract:
  8. Machine learning, artificial intelligence and cognitive science have been very popular in recent years. In the meantime, thanks to the significant increase in computing power of computers and easy access to large amounts of data, deep neural networks have been favored by researchers in recent years. With the help of these networks, high accuracy methods have been proposed in solving image processing, speech processing, time series processing, etc. Deep neural networks consist of multiple layers of neurons that perform layer-by-layer processing. In recent years, ideas have been proposed to use specific structures in neural systems and add external elements such as memory to these systems. Especially, with the introduction of memory networks, neural Turing machine (NTM) and differentiable neural computer (DNC), a group of neural networks able to learn the algorithm of converting the two from the input and output sequences have been proposed. They are very effective in solving certain problems in natural language processing (NLP). Being fully differentiable, these schemes enable us to train them using the error backpropagation method, and thus they will be useful for a wide range of artificial intelligence problems. The proposed scheme in the current research is considered as an extension of the above schemes because it proposes a separate memory module inspired by mechanisms and structures of memory in the human brain. The proposed memory is based on a graph and keeps the relationships between the contents of the memory, and on the other hand, with a subsystem called memory consolidation network, it can simulate the role and performance of sleep in improving the quality of memory and the overall performance of the system. Applying the proposed scheme to synthetic algorithmic tasks shows that the proposed scheme has a lower error than the basic schemes of LSTM and NTM and converges faster. Also, our experiments on the bAbI question answering tasks show that the proposed scheme is capable (both with and without memory consolidation network) to successfully solve 16 out of 20 tasks and the average error among these tasks is 2.8%
  9. Keywords:
  10. Machine Learning ; Graph Neural Network ; Cognitive Science ; Memory Augmented Neural Networks ; Human Learning System ; Memory Structures ; Brain-Inspired Learning

 Digital Object List

 Bookmark

No TOC