Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks
Shallow feed-forward networks are incapable of addressing complex tasks such as natural language processing that require learning of temporal signals. To address these requirements, we need deep neuromorphic architectures with recurrent connections such as deep recurrent neural networks. However, th...
Saved in:
Main Authors: | , , , , , , , , , , , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2021
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/152915 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-152915 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1529152021-10-30T20:11:05Z Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks John, Rohit Abraham Acharya, Jyotibdha Zhu, Chao Surendran, Abhijith Bose, Sumon Kumar Chaturvedi, Apoorva Tiwari, Nidhi Gao, Yang He, Yongmin Zhang, Keke K. Xu, Manzhang Leong, Wei Lin Liu, Zheng Basu, Arindam Mathews, Nripan School of Materials Science and Engineering School of Electrical and Electronic Engineering Interdisciplinary Graduate School (IGS) Energy Research Institute @ NTU (ERI@N) HealthTech NTU Engineering::Materials Engineering::Electrical and electronic engineering Optogenetics Signal Noise Ratio Shallow feed-forward networks are incapable of addressing complex tasks such as natural language processing that require learning of temporal signals. To address these requirements, we need deep neuromorphic architectures with recurrent connections such as deep recurrent neural networks. However, the training of such networks demand very high precision of weights, excellent conductance linearity and low write-noise- not satisfied by current memristive implementations. Inspired from optogenetics, here we report a neuromorphic computing platform comprised of photo-excitable neuristors capable of in-memory computations across 980 addressable states with a high signal-to-noise ratio of 77. The large linear dynamic range, low write noise and selective excitability allows high fidelity opto-electronic transfer of weights with a two-shot write scheme, while electrical in-memory inference provides energy efficiency. This method enables implementing a memristive deep recurrent neural network with twelve trainable layers with more than a million parameters to recognize spoken commands with >90% accuracy. Ministry of Education (MOE) Published version We would like to acknowledge the funding from MOE Tier 1 grants: RG87/16, RG 166/ 16 and MOE Tier 2 grants: MOE2015-T2-2-007, MOE2015-T2-2-043, MOE2017-T2-2- 136 and MOE Tier 2 grant MOE2016-T2-1-100. 2021-10-21T08:57:59Z 2021-10-21T08:57:59Z 2020 Journal Article John, R. A., Acharya, J., Zhu, C., Surendran, A., Bose, S. K., Chaturvedi, A., Tiwari, N., Gao, Y., He, Y., Zhang, K. K., Xu, M., Leong, W. L., Liu, Z., Basu, A. & Mathews, N. (2020). Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks. Nature Communications, 11, 3211-. https://dx.doi.org/10.1038/s41467-020-16985-0 2041-1723 https://hdl.handle.net/10356/152915 10.1038/s41467-020-16985-0 32587241 2-s2.0-85086844183 11 3211 en Nature Communications 10.21979/N9/SQ7XOF © 2020 The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/ licenses/by/4.0/. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Materials Engineering::Electrical and electronic engineering Optogenetics Signal Noise Ratio |
spellingShingle |
Engineering::Materials Engineering::Electrical and electronic engineering Optogenetics Signal Noise Ratio John, Rohit Abraham Acharya, Jyotibdha Zhu, Chao Surendran, Abhijith Bose, Sumon Kumar Chaturvedi, Apoorva Tiwari, Nidhi Gao, Yang He, Yongmin Zhang, Keke K. Xu, Manzhang Leong, Wei Lin Liu, Zheng Basu, Arindam Mathews, Nripan Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
description |
Shallow feed-forward networks are incapable of addressing complex tasks such as natural language processing that require learning of temporal signals. To address these requirements, we need deep neuromorphic architectures with recurrent connections such as deep recurrent neural networks. However, the training of such networks demand very high precision of weights, excellent conductance linearity and low write-noise- not satisfied by current memristive implementations. Inspired from optogenetics, here we report a neuromorphic computing platform comprised of photo-excitable neuristors capable of in-memory computations across 980 addressable states with a high signal-to-noise ratio of 77. The large linear dynamic range, low write noise and selective excitability allows high fidelity opto-electronic transfer of weights with a two-shot write scheme, while electrical in-memory inference provides energy efficiency. This method enables implementing a memristive deep recurrent neural network with twelve trainable layers with more than a million parameters to recognize spoken commands with >90% accuracy. |
author2 |
School of Materials Science and Engineering |
author_facet |
School of Materials Science and Engineering John, Rohit Abraham Acharya, Jyotibdha Zhu, Chao Surendran, Abhijith Bose, Sumon Kumar Chaturvedi, Apoorva Tiwari, Nidhi Gao, Yang He, Yongmin Zhang, Keke K. Xu, Manzhang Leong, Wei Lin Liu, Zheng Basu, Arindam Mathews, Nripan |
format |
Article |
author |
John, Rohit Abraham Acharya, Jyotibdha Zhu, Chao Surendran, Abhijith Bose, Sumon Kumar Chaturvedi, Apoorva Tiwari, Nidhi Gao, Yang He, Yongmin Zhang, Keke K. Xu, Manzhang Leong, Wei Lin Liu, Zheng Basu, Arindam Mathews, Nripan |
author_sort |
John, Rohit Abraham |
title |
Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
title_short |
Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
title_full |
Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
title_fullStr |
Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
title_full_unstemmed |
Optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
title_sort |
optogenetics inspired transition metal dichalcogenide neuristors for in-memory deep recurrent neural networks |
publishDate |
2021 |
url |
https://hdl.handle.net/10356/152915 |
_version_ |
1715201493582217216 |