Optoelectronic graded neurons for bioinspired in-sensor motion perception

Mennel, L. et al. Ultrafast machine vision with 2D material neural network image sensors. Nature 579, 62–66 (2020).

Article  CAS  Google Scholar 

Jang, H. et al. In-sensor optoelectronic computing using electrostatically doped silicon. Nat. Electron. 5, 519–525 (2022).

Article  Google Scholar 

Chai, Y. In-sensor computing for machine vision. Nature 579, 32–33 (2020).

Article  CAS  Google Scholar 

Choi, C. et al. Curved neuromorphic image sensor array using a MoS₂–organic heterostructure inspired by the human visual recognition system. Nat. Commun. 11, 5934 (2020).

Article  CAS  Google Scholar 

Zhou, F. et al. Optoelectronic resistive random access memory for neuromorphic vision sensors. Nat. Nanotechnol. 14, 776–782 (2019).

Article  CAS  Google Scholar 

Seung, H. et al. Integration of synaptic phototransistors and quantum dot light-emitting diodes for visualization and recognition of UV patterns. Sci. Adv. 8, eabq3101 (2022).

Article  CAS  Google Scholar 

Jayachandran, D. et al. A low-power biomimetic collision detector based on an in-memory molybdenum disulfide photodetector. Nat. Electron. 3, 646–655 (2020).

Article  Google Scholar 

Chai, Y. Silicon photodiodes that multiply. Nat. Electron. 5, 483–484 (2022).

Article  Google Scholar 

Zhou, F. & Chai, Y. Near-sensor and in-sensor computing. Nat. Electron. 3, 664–671 (2020).

Article  Google Scholar 

Li, X. et al. Power-efficient neural network with artificial dendrites. Nat. Nanotechnol. 15, 776–782 (2020).

Article  CAS  Google Scholar 

Wan, T. et al. In-sensor computing: materials, devices, and integration technologies. Adv. Mater. 9, 2203830 (2022).

Article  Google Scholar 

Kim, M. et al. An aquatic-vision-inspired camera based on a monocentric lens and a silicon nanorod photodiode array. Nat. Electron. 3, 546–553 (2020).

Article  Google Scholar 

Simonyan, K. & Zisserman, A. Two-stream convolutional networks for action recognition in videos. Adv. Neural Inf. Process Syst. 27, 568–576 (2014).

Google Scholar 

Ye, H. et al. Evaluating two-stream CNN for video classification. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval 435–442 (Association for Computing Machinery, 2015).

Liao, F. et al. Bioinspired in-sensor visual adaptation for accurate perception. Nat. Electron. 5, 84–91 (2022).

Article  Google Scholar 

Jung, D. et al. Highly conductive and elastic nanomembrane for skin electronics. Science 373, 1022–1026 (2021).

Article  CAS  Google Scholar 

Song, Y. M. et al. Digital cameras with designs inspired by the arthropod eye. Nature 497, 95–99 (2013).

Article  CAS  Google Scholar 

Lee, M. et al. An amphibious artificial vision system with a panoramic visual field. Nat. Electron. 5, 452–459 (2022).

Article  Google Scholar 

Ayers, J., Davis, J. L. & Rudolph, A. Neurotechnology for Biomimetic Robots (MIT Press, 2002).

Webb, B. Robots with insect brains. Science 368, 244–245 (2020).

Article  CAS  Google Scholar 

de Ruyter van Steveninck, R. & Laughlin, S. The rate of information transfer at graded-potential synapses. Nature 379, 642–645 (1996).

Article  Google Scholar 

Tuthill, J. C., Nern, A., Holtz, S. L., Rubin, G. M. & Reiser, M. B. Contributions of the 12 neuron classes in the fly lamina to motion vision. Neuron 79, 128–140 (2013).

Article  CAS  Google Scholar 

Zheng, L. et al. Network adaptation improves temporal representation of naturalistic stimuli in Drosophila eye: I dynamics. PLoS ONE 4, e4307 (2009).

Article  Google Scholar 

Miall, R. The flicker fusion frequencies of six laboratory insects, and the response of the compound eye to mains fluorescent ‘ripple’. Physiol. Entomol. 3, 99–106 (1978).

Article  Google Scholar 

Kelly, D. & Wilson, H. Human flicker sensitivity: two stages of retinal diffusion. Science 202, 896–899 (1978).

Article  CAS  Google Scholar 

Uusitalo, R. & Weckstrom, M. Potentiation in the first visual synapse of the fly compound eye. J. Neurophysiol. 83, 2103–2112 (2000).

Article  CAS  Google Scholar 

Nikolaev, A. et al. Network adaptation improves temporal representation of naturalistic stimuli in Drosophila eye: II mechanisms. PLoS ONE 4, e4306 (2009).

Article  Google Scholar 

Hu, W., Wang, T., Wang, X. & Han, J. I_h channels control feedback regulation from amacrine cells to photoreceptors. PLoS Biol. 13, e1002115 (2015).

Article  Google Scholar 

Laughlin, S. B., de Ruyter van Steveninck, R. R. & Anderson, J. C. The metabolic cost of neural information. Nat. Neurosci. 1, 36–41 (1998).

Article  CAS  Google Scholar 

Juusola, M., French, A. S., Uusitalo, R. O. & Weckström, M. Information processing by graded-potential transmission through tonically active synapses. Trends Neurosci. 19, 292–297 (1996).

Article  CAS  Google Scholar 

Schuetzenberger, A. & Borst, A. Seeing natural images through the eye of a fly with remote focusing two-photon microscopy. Iscience 23, 101170 (2020).

Article  CAS  Google Scholar 

Liu, K. et al. An optoelectronic synapse based on α-In₂Se₃ with controllable temporal dynamics for multimode and multiscale reservoir computing. Nat. Electron. 5, 761–773 (2022).

Article  CAS  Google Scholar 

Warland, D., Landolfa, M., Miller, J. P. & Bialek, W. in Analysis and Modeling of Neural Systems (ed Eeckman, F. H.) 327–333 (Springer, 1992).

Jiang, J. et al. Defect engineering for modulating the trap states in 2D photoconductors. Adv. Mater. 30, 1804332 (2018).

Article  Google Scholar