LAUSR

Multimode-fused sensing in the somatosensory system helps people obtain comprehensive object properties and make accurate judgments. However, building such multisensory systems with conventional metal-oxide-semiconductor technology presents serious device integration and circuit complexity challenges. Here, we report a multimode-fused spiking neuron (MFSN) with a compact structure to achieve human-like multisensory perception. The MFSN heterogeneously integrates a pressure sensor to process pressure and a NbOx -based memristor to sense temperature. Using this MFSN, multisensory analog information can be fused into one spike train, showing excellent data compression and conversion capabilities. Moreover, both pressure and temperature information are distinguished from fused spikes by decoupling the output frequencies and amplitudes, supporting multimodal tactile perception. Then, we fabricate a 3×3 MFSN array and feed the fused frequency patterns into a spiking neural network for enhanced tactile pattern recognition. Finally, we simulate a larger MFSNs for classifying objects with different shapes, temperatures and weights, validating the feasibility of our MFSNs for practical applications. Our proof-of-concept MFSNs enable the building of multimodal sensory systems and contribute to the development of highly intelligent robotics. This article is protected by copyright. All rights reserved.