Brain-Inspired Computing: A Systematic Survey and Future Trends

Brain-inspired computing (BIC) is an emerging research field that aims to build fundamental theories, models, hardware architectures, and application systems toward more general artificial intelligence (AI) by learning from the information processing mechanisms or structures/functions of biological nervous systems. It is regarded as one of the most promising research directions for future intelligent computing in the post-Moore era. In the past few years, various new schemes in this field have sprung up to explore more general AI. These works are quite divergent in the aspects of modeling/algorithm, software tool, hardware platform, and benchmark data since BIC is an interdisciplinary field that consists of many different domains, including computational neuroscience, AI, computer science, statistical physics, material science, and microelectronics. This situation greatly impedes researchers from obtaining a clear picture and getting started in the right way. Hence, there is an urgent requirement to do a comprehensive survey in this field to help correctly recognize and analyze such bewildering methodologies. What are the key issues to enhance the development of BIC? What roles do the current mainstream technologies play in the general framework of BIC? Which techniques are truly useful in real-world applications? These questions largely remain open. To address the above issues, in this survey, we first clarify the biggest challenge of BIC: how can AI models benefit from the recent advancements in computational neuroscience? With this challenge in mind, we will focus on discussing the concept of BIC and summarize four components of BIC infrastructure development: 1) modeling/algorithm; 2) hardware platform; 3) software tool; and 4) benchmark data. For each component, we will summarize its recent progress, main challenges to resolve, and future trends. Based on these studies, we present a general framework for the real-world applications of BIC systems, which is promising to benefit both AI and brain science. Finally, we claim that it is extremely important to build a research ecology to promote prosperity continuously in this field.


I. I N T R O D U C T I O N , M O T I V A T I O N , A N D O V E R V I E W
Mimicking the biological nature of the brain to build more general artificial intelligence (AI) as powerful as the human brain has been a dream of human beings for several tens of years.Although artificial neural networks (ANNs), the mainstream neural networks in deep learning, such as multilayer perceptrons (MLPs) [1], convolutional neural networks (CNNs) [2], and recurrent neural networks (RNNs) [3], have achieved great success in a number of fields in science and engineering [4], [5], [6], [7], [8], [9], they are difficult, if not impossible to be the right path to reach the dream, due to the fact that they only exploit the concept of the arithmetic operation of a single neuron instead of dynamic properties of the neural networks, that is to say, the internal structures/functions of the brain at the single neuron, synapse, neural circuit, network, and system levels have been ignored.Geoffrey Hinton, one of the originators of deep learning, believed that the key to breaking the current AI technologies lies in building a bridge between AI and the human brain [10].However, a fundamental issue is how to enhance AI models by leveraging advanced research achievements in neuroscience.This motivation has led to a flurry of research into brain-inspired computing (BIC).
The term "BIC" is not new and has been appearing in research articles for about 20 years [11], yet so far, its definition remains unclear.Wakuya [11] believes that an intuitive, nonlogical, way of thinking governs the brain and claims that modern computers can only account for the logical aspects of human computation, while the brain works differently.Therefore, at the very beginning, BIC mainly referred to the nonlogical aspect of computing models [12], whose structure is inspired by biological nervous systems.However, at that time, the method of learning in MLPs such as backpropagation (BP) [13] is also considered to be a BIC methodology, which does not appear to be accurate now.Later, BIC became well known by the TrueNorth chip [14], a non-von Neuman architecture neuromorphic chip with programmable spiking neurons and configurable synapses, and since then, it has been considered to be another term for "neuromorphic computing" [15], [16], [17], [18] describing devices and systems that mimic some functions of the biological neural systems, which was coined in the late 1980s by Mead [19].Actually, the two concepts of "neuromorphic computing" and "BIC" are not quite the same.They should not be regarded as two identical concepts.
In this survey, we consider BIC as an emerging research field that aims to build fundamental theories, models, hardware architectures, and application systems toward more general AI by learning from not only the information processing mechanisms but also the structures and functions of biological nervous systems.Here, one good candidate to capture the information processing mechanisms and the structures/functions of biological nervous systems is the spiking neural networks (SNNs) [20], [21], [22].
Although SNNs are also "artificial" networks in a sense, they are natural to represent the multiscale dynamic properties of neural systems and contain the units, including dendrite, synapse, soma, and axon coming from the field of computational neuroscience.Therefore, we treat SNNs and ANNs as two independent and different concepts, and the concept of BIC can be restated to build more advanced theoretical models, training algorithms, hardware architectures, and application systems based on SNNs for breaking the technical bottleneck of current AI.Based on this concept, the research field of BIC is broader than that of neuromorphic computing, which mainly replicates the way neurons are organized, communicated, and learned at the hardware level.However, definitely, neuromorphic computing can be one of the most important ways to realize the objective of BIC.In this sense, BIC can be easily distinguished from current deep learning technologies built on ANNs, in which each neuron is a multiplication and accumulation (MAC) unit followed by a nonlinear function, no matter how deep, wide, and complicated the network is.
It is well known that the great success of deep learning is largely driven by advances made in systematic learning theories such as the stochastic gradient descent methods, various benchmark tasks and datasets, and friendly programming tools, for instance, Tensorflow [46], Pytorch [47], and efficient processing platforms such as graphics processing units (GPUs) [48] with high processing parallelism and memory bandwidth.Similarly, nowadays, the prosperity of BIC depends on building four components of BIC infrastructures, i.e., modeling/algorithm, hardware platform, software tool, and benchmark data, as seen in Fig. 1.The development of each component will be around their respective key considerations or mainstream technologies.In the following, we claim that the co-design of these four components is becoming a ubiquitous trend in the field of BIC, as shown in Fig. 2.
On the modeling/algorithm side, existing schemes cover both the single neuron at the cell level and SNN models at the network level as well as their training mechanisms.We clarify that SNNs equals ANNs plus neuronal dynamics.This implies that ANNs and SNNs could share the same network topology and the difference is that neurons in SNNs are characterized by differential equations due to neuronal dynamics, where the spikes are dependent on such dynamics.However, the range of neuronal dynamics varies a lot.It can be as simple as the first-order differential equation (e.g., the leaky integrate-and-fire (LIF) model) [49] or as complicated as a set of differential equations (e.g., the Hodgkin-Huxley (H-H) model) [50], even the dynamics existing not only in the soma but also in dendrites [51], [52], [53].Regarding the learning algorithms, current schemes can be divided into three categories, i.e., unsupervised learning [25], [54], ANNto-SNN conversion [55], [56], [57], [58], [59], [60], [61], [62], and direct training algorithms [21], [63], [64], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75], [76].For the development of truly brain-inspired models/algorithms, we mainly focus on the field of SNNs in this survey.Apart from the SNNs, there are other models inspired by the brain such as liquid state machine [77], echo state networks [78], and continuous attractor neural networks (CANNs) [79].On a more general level, models or algorithms [80], [81] developed by borrowing from the brain's learning rules and organizational structure are all in the category of the BIC field.For the sake of being more focused, we do not discuss these works in this survey.
How to build more biologically plausible methodologies to define or solve some tasks that cannot be done or done well in the current AI models will become a significant task to be solved urgently.We suggest that learning from more bio-plausible neurons such as the multicompartment neuron models or refined neuron models [51], [52], [53] is of great potential, while one has to consider the following four aspects: bio-plausibility, effectiveness, efficiency, and trainability.
Based on the above four components, a BIC framework to obtain a full-stack solution for enhancing its applicability in practice is desired.The high-level models and algorithms optimization provides guidance for the co-design of convenient hardware and software, the low-level hardware design provides feedback for the co-design of efficient software and algorithms, and the benchmark datasets provide various tests and verification scenarios.This co-design will be more and more ubiquitous in building applicationoriented BIC systems.What is more, BIC is quite interdisciplinary field that consists of many different domains, such as computational neuroscience, statistical physics, chip design, material science, computer science, and AI.Hence, BIC not only learns widely from various domains but also in turn inspires and impacts these domains.For example, benefiting from the processing-in-memory architecture, BIC chips can make the same applications that traditional von Neumann processors (e.g., CPUs and GPUs) must consume very high energy.The corresponding design inspiration of BIC chips stems from the colocating of computation in computational neuroscience.Apart from hardware, BIC models and algorithms always draw their nutrients from biology.The effectiveness and efficiency of BIC intelligence in various scenarios have further stimulated the interest of scientists in exploring the mysteries of biological intelligence.In this context, taking BIC as the turning point and leveraging the advantages of multidisciplinary cross-integration to promote the common development of various domains and realize more general AI definitely forms a valuable and important topic.We believe that this shall be of great interest to the research communities in both AI and brain science.In this way, readers shall benefit a lot from obtaining a clear picture of BIC.
However, the chaos of tremendous works and divergent methods in this field highly require top-down guidance on the method selection for researchers, especially beginners.There is an urgent requirement to do a comprehensive survey for BIC research to help correctly recognize and analyze such bewildering methodologies.What is the key issue to enhance the development of BIC?What roles do the current mainstream technologies play in the general framework of BIC? Which techniques are truly useful in real-world applications?These questions are quite interesting but largely remain open.Moreover, insightful opportunities and challenges are also highly expected for the entire community.In this survey, we will systematically summarize most of the existing methodologies for each of the four components, and describe their main challenges and future trends.Based on the co-design of learning algorithms, hardware chips, software tools, and benchmark datasets, we will present a general framework of BIC systems, which is promising to benefit both AI and brain science.We believe that what we have done in this survey will be of interest to scientists and engineers working on computational neuroscience, AI, neuromorphic chips and systems, computer science, and so forth, for building a research ecology to promote prosperity continuously.
Finally, the organization of this article is summarized as follows.Section II briefly gives some preliminary knowledge.In Sections III-VI, the recent status of BIC models and algorithms, hardware platforms, software tools, and benchmark data will be summarized in turn.A framework for BIC systems will be proposed in Section VII.Section VIII summarizes this article, clarifies some general misunderstanding, and forces many possible opportunities and challenges as well.

II. B R I E F P R E L I M I N A R I E S , C O N C E P T S , A N D M A I N C H A L L E N G E S
It is well known that the convolution operation in deep neural networks (DNNs) is inspired by the receptive field [168], [169], [170], which is one of the basic concepts originated from neuroscience.Due to this fact, the boundary between the concept of "BIC" and its counterpart "deep learning" becomes blurred with the prosperity of DNNs.To clarify the concept of BIC, we will introduce several basic concepts in the related research fields in Fig. 3. Specifically, we will start by briefly introducing the concepts of ANNs and SNNs that mainly originated from deep learning and computational neuroscience, respectively.What makes SNNs different from ANNs lies in that SNNs exploit neural dynamics commonly observed in the brain.We will also differentiate DNN accelerators designed for supporting ANNs and neuromorphic chips for supporting SNNs.Later, two definitions of BIC: Classic/Generalized BIC, will be introduced.At the same time, the scope of BIC will be compared with two emergent research fields, i.e., brain for AI and AI for brain.Classic BIC is a subset of brain for AI, and it focuses on SNN models and neuromorphic chips; its related applications are also inspired by computational neuroscience, while Generalized BIC considers both fields of brain for AI and AI for brain.Last but not least, we will illustrate the interdisciplinary feature of BIC and point out that the main challenge is how BIC models and systems can exploit the advanced achievements of computational neuroscience to achieve more general AI.

A. Artificial Neural Networks
A neuron is the basic unit of a neural network that receives signals from the connected pre-neurons, conducts a nonlinear transformation, and then produces an output signal that multicasts to post-neurons.The connection can be termed as synapse, and the connection efficacy as weight (W ), the neuronal signal as activation (x), and the nonlinear transformation as activation function (φ(•)).The output of neuron i denoted as yi can be written as where bi is a bias.All networks connected using the above neuron model can be called ANNs.There are three typical ANN models distinguished by different network structures: 1) MLPs; 2) CNNs; and 3) RNNs.

B. Spiking Neural Networks
SNNs can be considered as ANNs by substituting each single neuron or synaptic weight with spiking neuronal dynamics, i.e., where the range of neuronal dynamics varies a lot.It can be as simple as the first-order differential equation or as complicated as a set of differential equations, even the dynamics existing not only in the soma but also in dendrites.
As illustrated in Fig. 4, ANNs and SNNs share the same network topology, and the difference is that the neurons in SNNs are characterized by differential equations due to neuronal dynamics.With the rich dynamic properties, the spikes in SNNs are dependent on the concept of neural circuits that can be easily introduced to BIC, which nowadays is becoming a promising energy-efficient alternative to traditional neural computing and has permeated into a myriad of application domains such as image and speech recognition, object detection, autonomous driving, and other intelligence-related real-world applications.

D. Neuromorphic Computing
Neuromorphic computing is also known as neuromorphic engineering [15], [16], [17] based on the use of very large-scale integrated circuit (VLSI) systems to mimic biological functions of the nervous system.This survey mainly refers to any device that uses hardware neurons to do computation.Recently, the term "neuromorphic" has been used to describe analog, digital, analog-digitalmixed VLSI, and software systems that implement models of neural systems [41].Neuromorphic algorithms usually refer to SNNs, and neuromorphic computing is generally thought of as running SNNs on neuromorphic chips [16].In this survey, "neuromorphic" is a subset of Classic BIC in the sense that it is more related to hardware, which relies on hardware neurons for computing.

F. Brain for AI
Brain for AI aims to enhance AI technologies by getting inspiration from: 1) the signal transmission and learning rules in the nervous system [25], [183], [184]; 2) the structures and functions of the brain [185], [186]; or 3) the mental or cognitive processes of human beings [187], [188], in order to: 1) reduce the resources and energy consumed by AI [42], [87], [189], [190], [191]; 2) achieve comparable performance in tasks that are rudimentary to the brain but difficult for traditional AI [29], [192], [193]; 3) establish general principles or frameworks for the next generation of AI; or 4) tackle the existing problem in the field of deep learning [81], [194].Classic BIC is a subset of brain for AI because the former mainly focuses on the computing/learning capability of the system, while the latter can additionally refer to the general concept related to AI technologies.

G. AI for Brain
AI for brain can power brain science with the help of AI technologies so that we can: 1) explain complex phenomenons in the brain using an AI framework [195]; 2) have a better understanding of the structures and functions of the brain [196], [197] (also, AI can strengthen brain imaging techniques and facilitate research on brain structures or functions [198]); 3) predict cognition, development, and mental health [199], [200], [201]; or 4) control behaviors and mental processes [202].

H. Generalized BIC
The scope of Generalized BIC considers both fields of brain for AI and AI for brain.Having this in mind, most emerging techniques, such as Classic BIC, brain-computer interface [203], [204], brain atlas [205], [206], [207], NeuroImage [208], [209], and various applications related to the combination of AI and brain [210], [211], [212], belong to the Generalized BIC, whose concept must keep pace with the development of science and engineering and change constantly.

I. BIC Versus Deep Learning
There is currently a lack of systematic discussion on the differences between BIC and deep learning.We need to clearly differentiate two concepts, "BIC" and "deep learning," the latter of which has flourished in various disciplines.Note that deep learning is a broader concept, and the typical networks used are ANNs (CNNs, RNNs, Transformers, and so on), in which every neuron is an MAC unit followed by a nonlinear function, no matter how deep, wide, and complicated the network is.As ANNs are the most commonly used models in deep learning, we simplify the differences between BIC and deep learning here as the difference between BIC and ANNs.Thus, in this survey, deep learning is mainly considered to be built on ANNs, and we simplify the differences between BIC and deep learning here as the differences between BIC and ANNs in three points.inherent scalability, and spike/event-driven computation, stem from their choice to incorporate bioplausible neurons and synapses as primary computational units.The importance of SNN algorithms is therefore seen in conjunction with the unique computational properties of SNN-enabled neuromorphic chips.Thus, BIC models generally adopt asynchronous and sparse accumulation operations, thus being quite energy-efficient on neuromorphic hardware.This is the reason why AI chips optimize high computing power more, while neuromorphic chips focus more on low energy.

J. Main Challenge of BIC
The main challenge is how BIC systems can exploit the advanced achievements of computational neuroscience to bridge the gap between AI and neuroscience.Specifically, the challenge is how to execute co-design better of the four components of BIC, i.e., model/algorithm, hardware chips, software tools, and benchmark datasets, by learning from mechanisms, structures/functions of biological neural systems, and building a research ecology to promote the prosperity of BIC continuously.These four components play as the infrastructures of BIC systems, which will be reviewed in the following sections.of a neuron mainly includes four parts: the dendrite, synapses, the soma, and the axon.The dendrite collects synaptic input signals generated from other neurons and passes them to the soma.The soma generates a spike (i.e., action potential) when the incoming signal updates the neuron membrane potential and makes it cross a certain threshold.The spike propagates along the axon without attenuation and transmits the signal to downstream neurons through synapses at the axon terminal.

A. From Single-Neuron Model to SNN
Biological plausibility and computational complexity are our primary concerns in modeling a single neuron.Herz et al. [214] abstract single-neuron modeling into five levels based on a particular goal.The first three levels (levels 1-3) focus on fine structures and physiological details, which are mainly used in the field of computational neuroscience.The latter two levels (levels 4 and 5) focus on abstract structure and computational efficiency, which are commonly used in the AI field.The current situation is that there is a gap between AI and computational neuroscience, that is, models in the AI field at levels 4 and 5 are almost impossible to utilize the findings on neuronal models at levels 1-3.Identifying what details of the neuronal dynamics, electrophysiology, neurochemistry, and regulatory mechanisms related to AI are important and what can be disregarded are critical to the modeling of BIC.
In this survey, we introduce neuronal models from temporal and spatial perspectives, as shown in Fig. 5.In terms of spatial complexity, single-compartment models simply consider the modeling of the soma, completely ignoring the existence of the dendrite and the axon.Among them, the LIF model [215], the H-H model [50], and the Izhikevich model [216] are the most famous ones.By contrast, a multicompartment model takes into account the morphology of a cell so that it is no longer a point neuron [51], [52], [53].The dendritic structure of the neuron is simplified as a small set of sparsely connected compartments, while the axon is often neglected due to its negligible influence on the output.Input streams are injected into a fixed subset of the dendritic compartments, whose distances to the soma might be different from one another.Signals are propagated between neighboring dendritic compartments unidirectionally or bidirectionally, in a nonlinear way.The soma gets inputs from those compartments adjacent to it, processes information as did in a point neuron model, and generates output signals.Multicompartment models are more biologically plausible than single-compartment ones.Also, since there are only a small number of compartments in such a model, it is feasible to use this model in the framework of AI.Finally, with the highest spatial complexity, a detailed neuron model [217] tries to finely reproduce the morphology of a real neuron using hundreds (or even thousands) of compartments, in order to simulate the signal transmission process within a biological neuron as closely as possible.Such a costly model can hardly make sense in the scope of AI.In terms of temporal property, neuronal models can be classified into two categories according to whether they exhibit temporal dynamic characteristics.For instance, ANN neurons produce outputs without the temporal dimension [218], [219].Spiking models, on the other hand, simulate the dynamics of membrane potential (and perhaps other state variables) in the soma (and dendritic compartments), whose outputs are spike trains within a time duration.
The most popular spiking neuron model is the LIF model, which incorporates membrane potential integration, a leaky term, and threshold-based firing while neglecting specific ionic currents and morphological details where we denote u as the membrane potential, τm as the membrane potential time constant, urest as the resting potential, I as the input current, Rm as a finite leak resistance, and t f as the spiking time.The LIF model could achieve a lower power consumption by coding signals in binary events.
Another widely used single-compartment model, the ANN point neuron, tries to reduce computational complexity at the expense of discarding temporal dynamics (see (1) and Fig. 5).The signals are coded in continuous values, and the ANN propagates information only in the spatial domain, especially in feedforward networks.As a special case, binary ANN neurons transmit 0-or-1 values between each other by replacing the continuous activation function with a binarization function.
Fig. 6 compares the biological features of neuronal models and their energy costs.Typically, models with higher bio-plausibility will consequently consume more energy, so the points representing these models are distributed around the diagonal of the plot.On the one side, an LIF neuron model not only contains more biological features than a typical ANN neuron but also consumes lower energy since the replacement of the costly MAC operation with the efficient ac (accumulation) operation, though one must admit that the current von Neumann computers can better perform dense matrix-vector multiplications for ANNs in AI tasks.However, a binary ANN neuron can be considered the simplest spiking neuron without temporal dynamics, and it consumes the least energy cost.On the other side, more complicated neurons, including the fine neuron models, contain more bio-plausibility characteristics but need higher computational complexity and energy cost.Future research may focus on the dashed circle in the bottomleft corner of Fig. 6 to boost computational efficiency and reduce power consumption or pay attention to the topright circle to design models with more biological features for accomplishing tasks difficult for traditional AI.
2) Embedding Neuronal Models Into SNN: To further realize the goal of BIC, it is necessary to embed the aforementioned neuron models into a neural network.Fig. 7 shows three commonly used neural network architectures, including MLPs, RNNs, and CNNs, in which the corresponding basic layers are named fully connected (FC) layer, recurrent layer, and convolutional (Conv) layer.Networks with other topologies may also be used.Although these networks typically use ANN neurons as their basic unit, we may simply replace the neurons with spiking models by adding temporal dynamics.Then, we can build the corresponding SNNs.Clearly, the challenge in choosing a neuron model used in a neural network is to take the tradeoff between biological plausibility and computational complexity.For example, morphologically realistic models, as in Fig. 5(a), can approximate the dynamics of a real neuron quite well, but their high dimensionality will cause a huge computational burden.Accordingly, fine neuron models are not suited for large-scale networks.In contrast, reduced compartmental models can greatly alleviate the problem of the high computational cost while still mimicking the real interaction between the dendrite and the soma.They can demonstrate their strengths in tasks that are natural for the human brain but hard for AI.Eventually, with an extremely low energy consumption due to its simplified morphology and event-based nature, the LIF model is used in most of the current SNNs.In summary, the choice of neuron model in a network might depend on the task setting and the researchers' fields of interest.

B. Learning Algorithms
The training of ANNs is a data-dependent process of optimizing key parameters of the network for a specific task, in which the learning algorithm undoubtedly plays a crucial role.The gradient descent algorithm combined with error BP is the core of the current ANN optimization

Fig. 8. Milestones for SNN algorithms. The development of SNNs can be roughly divided into three stages: early spiking neurons and simple networks, multilayer SNNs, and now mainstream deep SNNs. On the other hand, from the perspective of how to train SNNs, the main technical routes of SNNs can be divided into three categories: unsupervised learning, ANN-to-SNN conversion, and direct training.
theory, and its series of variants have evolved from vanilla SGD [220] to ADAM [221] and AMSGrad [222].In addition, the introduction of normalization methods [223] and distributed training [224] have enabled the implementation of large-scale and high-performance ANNs, which are widely used in practical AI scenarios.In contrast, there are no recognized core learning algorithms or techniques in the field of SNNs.The learning algorithms exhibit significant diversity due to the varying degrees of emphasis between biological plausibility and task performance, as well as different neuron models and coding schemes adopted in a network.
Current algorithms for SNNs can be divided into two categories (Fig. 8): unsupervised learning derived from biological synaptic plasticity and supervised learning algorithms incorporating deep learning methods.The latter can be further divided into ANN-to-SNN conversion and direct training algorithms.Generally speaking, unsupervised learning algorithms focus on bio-plausibility, ANN-to-SNN conversion algorithms mainly consider effectiveness, and direct training algorithms pay more attention to efficiency and trainability.In the following, we will introduce each of the methods, and their comparative studies are summarized in Tables 1 and 2.
1) Unsupervised Learning: Synaptic plasticity is thought to be the biological basis of neural memory and learning.SNNs allow a type of bioinspired learning that depends on the relative timing of spikes between pairs of directly connected neurons, such as Hebbian learning and spike-timing dependent plasticity (STDP).Different from the layer-bylayer BP, the information required for weight modification is locally available, which can be easily implemented in distributed computing and online learning.A common description that fits well with observations in [226] can be formulated as where ∆wij denotes the weight change; ti and tj denote the presynaptic and postsynaptic spiking times, respectively; τ + and τ − denote the constants affecting the scale of time window; and a + and a − correspond to long-term potentiation (LTP) and long-term depression (LTD), respectively.The correlated spiking of presynaptic and postsynaptic neurons can result in strengthening or  weakening synapses, depending on the temporal order of spike [see Fig. 9(a)].One goal of SNN algorithms is to verify the potential role of synaptic plasticity rules in the construction of intelligent systems, and it has been shown that network-level learning can be a result of time-correlated synaptic dynamics.At the level of individual neurons, Guyonneau et al. [227] find that in synaptic simulation under the STDP rule, a targeted spike train by a population of afferent neurons can elicit fast recognition and selective response in a single postsynaptic neuron.Diehl and Cook [54] demonstrate a two-layer SNN consisting of only a single layer of excitatory neurons and its one-to-one corresponding layer of inhibitory neurons, where the excitatory connections from the input are trained by STDP and the subsequent inhibitory layer ensures lateral inhibition and competition between neurons.After completing unsupervised training, excitatory neurons are able to respond selectively to the input features, obtaining an accuracy of 95% on MNIST.Masquelier and Thorpe [228], [229] design an STDP-based feedforward SNN mimicking the ventral visual pathway in the brain.The network gradually develops selectivity for common features and shortens the delay required to excite the neurons, and eventually, the spikes containing important feature information of images will be issued quickly and can be further used for classification tasks.In addition, research on applying STDP rules on spiking CNNs has been reported [25], which contains multiple The Bienenstock-Cooper-Munro (BCM) rule [231] is another synaptic plasticity rule proposed in 1982 based on experiments in the visual cortex.The original Hebbian learning rule does not include a decay or enhancement mechanism for synaptic connections, resulting in instability in the constructed models.Therefore, the BCM rule assumes that neurons have a threshold that dynamically adapts to the historical weight changes and is utilized to determine the tendency of synaptic changes, so the connections can eventually reach a steady state.The subsequent synaptic weight association training (SWAT) [232] incorporates the variable threshold in the BCM rule to form a negative feedback regulation for the shape of the STDP window, enhancing stability during training.
2) ANN-to-SNN Conversion: ANN-to-SNN conversion emerges as an alternative approach to the training of high-performance SNNs so as to further exploit the power-efficient neuromorphic computing based on already developed deep learning fruits.The basic idea underlying the method is that continuous activation values of ANNs using ReLU as the nonlinearity can be approximated by the average firing rates in SNNs [27], [56].Therefore, after proper modifications on weights of an offline trained ANN, similar input-output mapping can be obtained through its spiking counterpart (see Fig. 9).In essence, the training relies on BP performed in the ANN, and thus, it avoids the differentiation difficulty faced by the direct training of SNNs.Converted SNNs maintain a minimal gap with ANNs in terms of accuracy and are feasible to be extended to large-scale datasets and emerging network structures, which have been validated on VGG, GoogLeNet, ResNet, and large EfficientNet [24], [57], [61], [233], [234], [235], [236], obtaining progressively more refined conversion loss on ImageNet.
In order to make the model more adaptive to spike activities after conversion, certain structural constraints are imposed on the original ANN model and modules suitable for SNNs should be considered [57].The nonleaky IF model with the soft-reset mechanism [62], [237] has been adopted as a regular practice to alleviate the accuracy degradation.Another focus in the conversion procedure is the balancing between thresholds and weights [61], which is called weight normalization [56], [57] or analyzed as the flooring/clipping error [238] as well, since there is not a threshold term in the pretrained ANN.The threshold can be set data dependently to the maximum activation or a percentage thereof [57], [61], or adaptively fine-tuned after conversion [238].A new perspective from the initialization of membrane potentials has been presented [239], [240], which further matches the activation function of ReLU and the multistep function in a converted SNN.The balance is critical to guarantee appropriate firing rates in the converted SNN and is expected to influence the most tradeoff between accuracy and latency.As more nuanced balancing techniques and all-round conversion pipelines have been proposed, the number of timesteps required has dropped significantly from thousands to fewer than a hundred, making nearly lossless conversion possible.
Overall, ANN-to-SNN conversion allows rapid transformation of ANN breakthroughs into the SNN field and serves as an efficient approach to leverage neuromorphic platforms, but it also has its inherent limitations.In addition to the performance degradation caused by the constraints imposed on the original ANN, low-latency converted SNNs are an ongoing research challenge since it still takes a long simulation length to complete an inference compared to the direct training of SNNs, which might result in extra latency and energy consumption.Most converted SNNs are only applicable to static images and not suitable for neuromorphic stream data.Besides, conversion methods pay more attention to narrowing the performance gap between ANNs and SNNs, rather than exploring the inherent dynamics or the uniqueness of SNNs, so their role in driving brain-inspired intelligence is relatively limited.
3) Direct Training: Inspired by the immense success of gradient descent-based algorithms in ANNs, researchers have never stopped considering its application for endto-end direct training of SNNs.Direct training methods can be classified into two categories based on the coding schemes used, i.e., rate coding and temporal coding.It is customary to utilize an explicitly iterative version of (3) for its implementation in the mainstream machine learning frameworks and to express the threshold-gated nonlinearity by (5) where oi[t] and ui[t] denote the output spike and membrane potential of the ith neuron at the tth timestep, respectively, and Θ denotes the Heaviside step function controlled by the threshold u th and represents the nondifferentiable dynamics arising from the binary format of spiking output.Rate coding-based methods work around the nondiffentiability of the firing function with the surrogate derivative and compute the gradients with respect to the spike activations, while temporal coding-based methods focus on the timings of existing spikes and compute the gradients with respect to the spike timings [241].
Direct training algorithms exhibit a range of specific function forms employed as surrogates.In the study by Lee et al. [21], the spiking input exerts a continuous influence on the membrane potential through a low-pass filter, while abrupt alterations in the membrane potential are categorized as noise and disregarded.Jin et al. [74] introduce the HM2-BP algorithm, a hybrid approach that breaks down error BP in SNNs into two components: microscopic changes in postsynaptic potential resulting from synaptic inputs and macroscopic BP of the loss function defined by rate coding.Wu et al. [68], [76] propose spatiotemporal BP (STBP) that considers both the spatial and temporal credit assignment in roll-out SNNs based on both the iterative LIF model and several approximated derivatives of spike activities.Some recent studies have suggested the concept of a learnable surrogate gradient, a mechanism capable of dynamically modulating the function's shape and smoothness during the training process, which aims to mitigate the discrepancies that can arise from the estimation of gradients [248], [249].
In temporal coding-based methods, a spike response kernel is often utilized to describe how the spike event of a neuron affects another, which enables us to simulate SNNs without explicit integration, and the individual firing timing, instead of the 0/1 firing pattern, is regarded as the state variable of neurons [241], [250], [251], [252], [253] such as where tj ∈ Ti,j = {τ |t last i < τ < t, oj[τ ] = 1} denotes the spiking times of the jth afferent neuron and t last i is the last spiking time of the ith neuron.The derivative of the spiking time with respect to the membrane potential, i.e., σ(ui[t]) = (∂ ti/∂ui[ ti]), is taken into consideration in the chain of BP, implying that the gradient propagates only at the time when a spike occurs, whereas it happens throughout the entire time window in rate coding-based methods.SpikeProp [253] is the pioneer in this category, in which expressions of spiking times for hidden units are linearized, allowing for analytical computing to approximately hidden layer gradients.EventProp [250] is the first proposed for a continuous-time SNN to backpropagate errors at spiking times and compute the gradient in an event-based, temporally, and spatially sparse fashion.Although the rate coding-based methods do yield welldefined gradients, they may suffer from certain limitations [246].Since the generation and removal of spikes cannot be described through the learning of spiking times, the network can be relatively fragile and requires a good initialization state.Allowing only a single spike per trial is a common neuronal constraint in this approach.Besides, a complex sorting algorithm is needed to configure a clear logic chain of gradient BP, which hinders the applicability of this approach to deep networks.
Directly trained networks excel in encoding information efficiently, necessitating significantly fewer timesteps compared to conversion networks, which can be particularly appealing for the implementation on power-efficient neuromorphic hardware.In addition, they are inherently better suited for processing spatiotemporal data from the emerging address event representation (AER)-based sensors, a task that is less applicable for ANNs and converted SNNs [254].Unfortunately, one prominent problem of the directly trained SNNs lies in the limited scale of models.The capacity of neural networks is surely crucial for their success, but earlier directly trained SNNs often experience severe accuracy degradation and are confined to shallow structures and simple tasks.Inspired by the representation power of deep ANNs, more emphasis has been placed on the design of SNN-oriented network structures and training.Emerging works, such as normalization techniques [63], [255], [256], spiking residual learning [64], [65], and attention-based SNNs [66], [257], are gradually narrowing the performance gap compared to ANNs.Models, such as SpikeGPT and SpikingBert [258], [259], [260], align with the current trend of seeking intelligence from large language models and demonstrate the immense potential of large-scale SNNs for more complicated tasks.They offer competitive performance with nonspiking models while reducing computational complexity and energy consumption.

C. Key Considerations
Note that neuroscience has long been an important driver of the progress in AI [265]; thus, developing true BIC models/algorithms is particularly critical, and how to build more biologically plausible methodologies to define or perform some tasks that cannot be done or done well by current AI models will become the main task to be solved urgently [266].We propose that leveraging the neural dynamics inspired by the structure/function of a single neuron or neural systems is of great potential [267], for example, the multicompartment neuron models or even refined neuron models.To this end, four key aspects have to be considered as illustrated in Fig. 10, including great development potential based on bio-plausibility, development goal oriented to the performance needs of practical application scenarios, the inherent low-energy development advantage brought by sparse spike activities, and the development bottleneck caused by the trainability of large-scale SNNs.
1) Bio-Plausibility: As previously indicated, both BIC and conventional AI draw inspiration from neuroscience, with the former being more biologically plausible due to its spiking neural dynamics.Traditional AI has made incredible progress in recent years with a variety of practical application scenarios [268], [269], [270], [271].However, these feats conceal certain serious flaws, such as huge energy consumption and poor robustness, resulting in an insurmountable bottleneck period in the development of traditional AI research [16], [23], [272].Fortunately, these limitations are easily surmounted by the human brain, which points to a promising direction for the development of machine intelligence.Since existing BIC algorithms can only simulate the somatic dynamics of biological neurons, it is hopeful that these bottlenecks in traditional AI can be overcome if more neuroscience mechanisms are organically incorporated into BIC algorithms to enhance their bio-plausibility.For instance, modeling spiking neurons can benefit greatly from the dynamics of biological neurons.BIC systems may work more like the human brain with more accurate modeling of the dynamics of finegrained neuronal structures such as synapses, dendrites, and axons.The use of attention processes, which aid humans in focusing on crucial information, is another such example.By implementing attention processes in BIC algorithms, the task performance is significantly improved, and the energy cost is significantly decreased [66], [69], [273], [274].This is because the attention suppresses spike firing from noisy neurons in SNNs, which is consistent with the observation in neuroscience [275], [276].Moreover, the long short-term memory mechanism is critical for time-series applications.Slow after-hyperpolarizing spiking neurons [277] can easily perform this function, while it is difficult for traditional AI.In short, bio-plausibility offers infinite possibilities for the realization of general AI through BIC.
Besides overcoming the bottlenecks of traditional AI, bio-plausible learning algorithms can also deepen our understanding of the brain.Backpropagation, the learning algorithm widely considered the cornerstone of conventional AI, is unlikely to be the way the human brain learns due to its strict constraint of weight symmetry in forward and backward pathways, as well as the requirement of a global teaching signal and precise gradient calculation.To tackle the problem of weight symmetry, feedback alignment is proposed, which sets weights as random matrices in the feedback pathway [278].The success of feedback alignment on shallow networks and small datasets reveals that the weight symmetry requirement of BP can be relaxed, and it is possible that the biological brain may assign blame in a BP-like, but less constrained manner.For a solution to the problem of global gradient-based signal, Lillicrap et al. [279] propose neural gradient representation by activity differences (NGRAD) in a review.In this framework, weight updates are driven by the difference between locally generated neural activities.For example, difference target propagation [280] uses the gap between neural activities and target activities to optimize synaptic weights in a layer-wise style.The target activities are generated using the target of the subsequent layer, ensuring locality while not introducing accurate gradient computation.Another work builds microcircuits with reduced compartmental neurons and transmits error signals back through the apical compartment [53].Synaptic weights are updated according to the predictive error between distinct compartmental potentials [51].In addition, a second line of research turns to contrastive predictive coding for assistance [281], aiming to learn robust representations in a local self-supervised manner.Greedy InfoMax (GIM) greedily maximizes the mutual information between its consecutive output by optimizing each encoder using the module-local InfoNCE loss [282].CLAPP further combines this idea with synaptic plasticity, using predictive dendritic input and broadcasted modulation factors to bring contrastive predictive coding to a biological setting [230].These works provide novel views on how the brain can solve credit assignment problems without calculating gradient and transmitting it backward along the whole network.Notably, Hinton recently proposed the forward-forward learning algorithm, aiming to revolutionize the learning of neural networks [283].The forward-forward algorithm is a greedy multilayer learning framework inspired by Boltzmann machines and noise contrastive estimation.By training the network to maximize "goodness" on positive data and minimize "goodness" on negative data in a layer-wise manner, the method can be used even when the precise details of the forward pass are unknown.This algorithm has the advantage of locality and flexibility and is a promising model of learning in the cortex.We believe that BIC systems can be considered as streamlined models of the brain, and bio-plausible learning rules have the potential to imply how the brain learns and works.
2) Effectiveness: The expectation for BIC is to effectively and efficiently achieve machine intelligence.People are most concerned about the effectiveness of BIC or AI in diverse real-world applications.Traditional AI systems, such as Deepmind's AlphaGo [271] and AlphaZero [284], have achieved astounding victories in complicated strategy games, defeating the best human players and becoming a legendary event.Since then, the AI area has received an unheard-of amount of attention, and its rich application scenarios and superior task performance have consistently been updated.By contrast, BIC has received far less attention because of the lack of excellent task performance and convincing application scenarios.There are lots of works focusing on how to make BIC algorithms more effective [16].Drawing on research in neuroscience is one of the main lines, such as modeling of spiking neurons [70], [285] and learning inspired by synaptic plasticity [286], [287].Another important path is to draw nutrients from the development of traditional AI, including the design of SNNs with increasing network scales [63], [64], [65] and more effective training based on mature BP algorithms [76], [246].The growth of these fields has driven the notable advancements of BIC.Some recent state-of-the-art results show that the performance of BIC on some temporal classification tasks is able to surpass traditional AI [288].
3) Efficiency: Energy efficiency is the prominent advantage of BIC systems, which stems from the event-driven computing and sparse spike activities of SNNs.With spikebased encoding, the synaptic ac operation is around five times less expensive than the inherent MAC operation in conventional AI [289].On the other hand, the fewer spikes are generated, the fewer times ac operations are performed.Thus, a sparse firing regime is the key to achieving the energy advantage [277].Realizing improved performance with fewer spikes is valuable and challenging, but the relationship between spike activities and performance is complex and has not been adequately explored.Some known impact factors include the modeling of spiking neurons, the network structure and the scale, the dataset size, training techniques, the penalty function, and so on.For example, incorporating learnable membrane time constants [70], fusing long short-term memory units [277], or rectifying membrane potential distributions at the neuron level [290] can play a role in regulating spike activities and task accuracy.Parameter regularization, such as activity regularization or network compression [247], [291], [292], is also a commonly employed method that drops spike activities at the cost of task performance.Another notable approach is data-dependent processing, which adjusts the spike response based on the input, such as directly masking unimportant information in the temporal dimension [69], to reduce spikes while maintaining task performance.The current practice shows that BIC has great potential for the effectiveness and efficiency of concurrent processing, but this requires ongoing efforts at both the theoretical and algorithmic levels.

4) Trainability:
Trainability is one of the main bottlenecks in the development of BIC.SNNs are challenging to train because of the complicated temporal dynamics of spiking neurons and the nondifferentiability of spike activities [183], [246].As a result, SNN models' performance and scales are constrained for a very long time.However, the network scale has been shown to be a key factor in enhancing the performance and scalability of deep learning models.As mentioned, the three main training approaches for SNNs at the moment are: conversionbased learning, which first trains an ANN before converting it to the corresponding SNN counterpart [239], [293]; error-driven spike-based direct training, which uses surrogate gradients for error BP [76], [246]; and neuroscience-oriented local learning rules, which update weights between presynaptic and postsynaptic neurons when an asynchronous spike is triggered [54], [229].Many advanced learning algorithms have emerged to promote the rapid development of BIC in recent years.For example, the state-of-the-art conversion-based method can greatly drop the simulation steps while achieving the best performance [293], the direct training method expands the SNN scale to more than 100 layers [64], [65], and the hybrid benefits of local and global learning are prominent in various tasks such as few-shot learning and continual learning [286].
On the other hand, when developing more physiologically plausible fine-grained spiking neuron models, trainability is one of the significant challenges to overcome.The most well-known LIF neuron is already a compromise between the complicated dynamics of biological neurons and the reduced mathematical form, as illustrated in Figs. 5 and 6.However, it has been reported quite recently that the training algorithms for LIF-based SNNs are gradually matured and stabilized. 1 It is still impossible to effectively train networks with more complicated neuron models, such as HH neuron models or multicompartment models.When using networks with richer neuronal dynamics to solve tasks that cannot be done or done well in the current AI models at present, trainability becomes a major concern to be addressed.In a nutshell, there is an urgent need to address the hardware cost, time commitment, and training complexity, which continue to be significantly higher than those of traditional AI [66], [112].

IV. H A R D W A R E P L A T F O R M S
As claimed in Fig. 3 and Section II, at the hardware level, BIC can be considered as neuromorphic computing.In order to keep better consistent with the literature, in this section, we do not distinguish between "braininspired" and "neuromorphic."We know that hardware platforms provide computing power for brain-inspired systems, which play a vital component in the BIC ecology.Efficient hardware platforms can significantly accelerate the evaluation of BIC models, thus facilitating the exploration of novel models and their applications in reality.
Note that BIC hardware platforms contain neuromorphic sensors and neuromorphic chips.In this section, we first discuss typical neuromorphic sensors and then give an overview and make a comparison of neuromorphic chips.Generally, neuromorphic sensors are implemented for realizing partial sensing functions of biological sensory organs involving the retina, cochlea, and skin (see Fig. 11).Since the development of neuromorphic sensors mainly focuses on the field of vision, we mainly focus on the discussion of the following two typical neuromorphic cameras.

A. Neuromorphic Cameras
Neuromorphic cameras with spiking out use spatiotemporal 1-bit points to encode the light intensity [300].In general, these bioinspired cameras can be classified into two categories, including dynamic vision sensors (DVSs) [160] and time-based image sensors [301].As illustrated in Fig. 12, we present the visual sampling mechanisms about these two types of neuromorphic cameras (e.g., DVS and Vidar [294]).We also show representative examples by mapping asynchronous visual streams into event images and spike images.
1) Dynamic Vision Sensors: DVSs, namely, event cameras [145], are sensitive to the scenario dynamics and directly respond to light changes at microsecond temporal resolution.In fact, the dynamic sensing principle can be regarded as the differential sampling of the light intensity.As a result, these silicon retinas work in a completely different way than conventional cameras, which acquire a stream of asynchronous events instead of providing a sequence of structured frames.Some representative event cameras include DVS128 [160], DAVIS346 [149], Prophesee Gen4 [295], OV60B10 [296], Tianmouc [297], ATIS [298], and so on.Due to the advantages of high temporal resolution (us), high dynamic range (HDR), little redundancy, low latency, and low power consumption, event cameras are gaining more and more attention in the computer science and neuroscience communities.So far, event cameras are the mainstream of neuromorphic cameras in research and industrial applications, because of 2) Time-Based Image Sensors: Time-based image sensors [301] in the latter type, taking the integrating sampling model, generate a spike when the accumulation of photons for a pixel reaches a prefixed threshold.These bioinspired cameras usually encode the light intensity into the instantaneous frequency or interspike intervals for each pixel.This frame-free imaging paradigm brings the ability to reconstruct fine images using spike frequency or interspike intervals.These typical vision sensors contain ATIS [298], octopus retina [299], Vidar Gen1 [294], and so on.For example, Vidar Gen1 has a high temporal sampling frequency of 40 000 Hz, which is suitable to deal with highspeed vision tasks.
Overall, since the first commercial event camera (i.e., DVS128 [160]) developed by the iniVation company in 2008, a series of neuromorphic cameras has merged with more advanced CMOS technology.As illustrated in Table 3, we compare some representative neuromorphic cameras with the specific performance parameters.From Table 3, there is a clear trend to increase the spatial resolution, transmission bandwidth, and fill factor while reducing chip size and pixel size.To enable asynchronous readout of spatiotemporal events, the transport protocol of most neuromorphic cameras utilizes the AER [302].To acquire static information, some event cameras (e.g., DAVIS346, ATIS, and OV60B10) are implemented with grayscale readout.Time-based image sensors (e.g., octopus retina and Vidar Gen1) can reconstruct fine texture via spike frequency or interspike intervals [303], but they fail to directly obtain dynamic information from the readout circuits.
Neuromorphic cameras can serve as good examples, suggesting that the simplifications made by BIC systems for computational purposes might bring them advantages and empower them to outperform biological systems.The circuits within each pixel are based on CMOS technology, and there is no substantial difference from those found in traditional cameras.While these circuits can partially mimic the signal processing in the retina, they fall short of replicating the intricate nonlinear dynamics of biological neurons.Moreover, the retina comprises multiple types of neurons interconnected in a complicated manner [304], but most existing neuromorphic cameras overlook this connective property for the sake of simpler implementation.Nevertheless, these simplifications prove to be beneficial.Thanks to the lightweight implementation of light-intensity sampling and interneuron wiring, the advantage of electronic devices in terms of processing speed compared to biological systems can be fully leveraged.Modern integrated circuits can handle communication cycles six orders of magnitude smaller than the interspike interval of a biological neuron, thus having the potential to transcend the speed limitations of biological vision.Consider Vidar as an example [294]; its photoelectric conversion process takes merely 10 ns, while that of a human retina occurs on the scale of microseconds or milliseconds [305].This suggests that simplifications made by BIC systems can sometimes empower them to outperform biological systems.  of the convolution operation in ANNs, various dataflow architectures with high data reuse rates between processing elements (PEs) have been explored in the processing unit.The memory hierarchy is similar to that of CPUs/GPUs, including off-chip DRAM and on-chip buffers at different levels.The early design of deep learning chips mainly considers single chips yet seldom considers interchip links.This situation gradually changes due to the increasing need for high-performance servers and data centers in recent years.

B. Neuromorphic Chips
Distinct from deep learning chips, as shown in Fig. 13, BIC chips target the emulation of the brain-inspired SNNs from the beginning.In the brain cortex, computation and memory are integrated together other than explicitly separated in von Neumann architectures.Inspired by this, most BIC chips adopt non-von Neumann decentralized manycore architectures, where each core has local computation and memory resources that are tightly coupled.BIC chips present massive computational parallelism and high memory locality without access to off-chip memory.Furthermore, since it is impossible to put all neurons and synapses of a human brain onto a single chip, the design of BIC chips considers much on the interchip communication for pursuing strong scalability: scaling a core up to a chip, to a board, to a server, and finally to a brain simulator [307].The design of deep learning accelerators considers more on improving performance, while BIC chips emphasize more on efficiency.In the following, we individually review existing BIC hardware from the following aspects of functionality, architecture, and implementation perspectives.An overview of the BIC hardware taxonomy in this work can be found in Fig. 14, and more detailed features of typical BIC hardware are provided in Table 4.
2) From Functionality Perspective: The functionality of a chip is determined by its instruction set and the supported models.The primary model supported by early BIC hardware is SNNs while being extended to cover ANNs in several modern BIC chips.In addition, learning ability is also an important functionality that will be discussed.
a) Supporting SNNs: BIC chips, also termed neuromorphic chips, stem from the concept of neuromorphic engineering proposed by Mead [19].The early neuromorphic hardware lies at the small-scale circuit level until some classic functional chips come out, including Neurogrid [95] from Stanford University, BrainScaleS [89] from Heidelberg University, SpiNNaker [82] from Manchester University, TrueNorth [14] from IBM, ROLLS [90] and DYNAPs [96] from ETH Zürich, Darwin [85] from Zhejiang University, Loihi [42] from Intel, ODIN [91], and MorphIC [92] from Universite catholique de Louvain.The primary target model of these hardware platforms is brain-inspired SNNs.Owing to the nature of event-driven computation and sparse activities of SNNs, the computational circuits of neuromorphic chips typically exhibit lower switching rates and smaller memory access bandwidth, hence consuming

(c) From the architecture perspective, some BIC chips adopt decentralized manycore architectures, while others adopt ANN accelerator-like architectures, and some of them adopt far-memory computing, while others adopt near-memory or in-memory computing. (d) From the implementation perspective, some BIC chips only demonstrate small-scale systems, while others demonstrate large-scale systems, and some of them use analog-digital-mixed circuits, while others use fully digital circuits.
Vol. 112  much lower power than conventional processors.This advantage can be further enhanced through techniques such as clock gating.The latency can also be improved if the chip only processes validated spike events by skipping zeros [102].
b) Supporting spiking and ANNs: The ultimate goal of supporting SNNs to realize powerful and efficient brain intelligence is cool, which is the original motivation of the BIC community.Unfortunately, nowadays, people still have quite limited knowledge of the brain, even if researchers have achieved many breakthroughs in divergent domains.Incorporating more brain features into the modeling of SNNs to make them more powerful and then strengthen their link to real-world tasks is of course very important for BIC.However, it cannot hide the current dilemma that SNNs have not yet presented superior results beyond execution efficiency compared to ANNs.Although the current comparison between SNNs and ANNs on deep-learningoriented benchmarks is a little unfair [254], the criticisms of SNNs from the application perspective are indeed ongoing [42].
To this end, some groups propose to develop crossparadigm platforms that combine SNNs and ANNs.Tianjic [87], [88] from Tsinghua University is the first BIC chip that invents hybrid models and architectures, which can support SNNs, ANNs, and hybrid neural networks (HNNs) [309].This paves a new way at the current stage for pushing BIC systems up to a higher intelligence level by merging the advantages of both SNNs and ANNs.For example, an HNN is able to achieve high accuracy from ANNs and rich dynamics, high efficiency, and high robustness from SNNs [72], [254], [310].The Tianjic team conducts many investigations on cross-paradigm comparison [72], [254], neural modeling [309], learning algorithms [286], hardware platforms [87], [88], and applications [311], [312] to build the ecology for such kind of hybrid BIC route.Recently, the hybridization idea has been broadly borrowed by the latest generation of BIC chips such as BrainScale 2 [84], SpiNNaker 2 [97], Loihi 2 [108], and the chips in [308] and [313].c) Supporting learning rules: Learning ability is a vital function of the brain.Most BIC chips only support the inference stage of neural networks, while the learning stage must be accomplished on GPUs in advance.However, the GPU architecture is tailored for general-purpose or ANN-oriented workloads, rather than SNN-dominated neuromorphic workloads.The learning of SNNs on GPU is inefficient and hard to optimize [93].To solve this challenge, some works design BIC chips that can support learning rules.For example, ROLLS, ODIN, and MorphIC support the spike-driven synaptic plasticity (SDSP) rules [90], [91], [92], Loihi adds learning modules for STDP rules [42], and FlexLearn further extends to a broader scope of supported synaptic plasticity rules [101].In SpiN-Naker and BrainScaleS, STDP learning has been demonstrated through time stamp recording and learning circuits [89], [94].Furthermore, in the new generation of the two chips, more flexible learning rules can be realized by virtue of the embedded programmable units [84], [97].Recently, BPTT learning has been applied to SNNs and demonstrated much higher accuracy compared to bio-plausible synaptic plasticity rules [68], [76].Several works, such as H2Learn [103] and SATA [104], design specific architectures for BPTT learning of SNNs.In the future, the incorporation of learning rules will be increasingly critical for BIC chips to explore large and complex neuromorphic models.
3) From Architecture Perspective: With the target functionality, designing a computing architecture to realize the functionality is the next important step.An abstraction of the architecture can be found in Fig. 15.In the family of BIC chips, the typical designs feature the decentralized manycore architecture with near-memory computing or inmemory computing in each core.Recently, the design of ANN accelerators has also been adapted for performing SNN workloads.
a) Near-memory-computing architecture: At the coarse-grained level, the classic decentralized manycore architecture of BIC chips without off-chip memory well integrates the computing and memory together, which can be regarded as a non-von Neumann architecture.While at the fine-grained level, the organization of the computation and memory resources within each core can be different.If the PEs and on-chip memory are physically separated, the architecture can be called near-memory-computing architecture.Here, "near" means that the distance between memory and computing in a core is close even though they are separated.
Most early neuromorphic chips can be categorized into near-memory-computing architectures.For example, all of TrueNorth, Tianjic, Darwin, Loihi, ODIN, and Mor-phIC adopt separate but close memory and computing in each core.For SpiNNaker, the off-chip memory is closely packed on each computing die, which can also be regarded as near-memory computing compared with common von Neumann architectures.Recently, a similar architecture in the BIC domain has been borrowed by researchers for designing ANN accelerators.For example, the emerging IPU [314] from Graphcore, Goya and Gaudi from Habana [315], Hanguang [316] from Alibaba, Simba [317] from NVIDIA, and NorthPole [318] from IBM apply the decentralized manycore architecture to their ANN accelerators for high throughput.The organizations of each core in these chips also belong to the family of near-memorycomputing architectures.This implies that researchers in different fields are learning from each other in the design of modern intelligent chips.
b) In-memory-computing architecture: Besides the near-memory-computing architecture, the PEs and on-chip memory in each core can be physically fused together, which can be called in-memory-computing architecture.In this kind of architecture, the matrix operation of the synaptic integration is performed in the synaptic memory itself.In practice, there are usually two categories of memories for computation: based on traditional memories or based on emerging memories.Notice that although we emphasize more on performing SNN workloads in this survey, the following techniques are also applicable to ANN workloads because the matrix operation on the memory array is intrinsically general for various neural network models.
Traditional memories, such as SRAM, DRAM, and Flash, can be redesigned to support some logic operations.For example, Liu et al. [319] design a 6T-SRAM cell with a transmission gate formed by two additional transistors for binary-weight vector-matrix multiplications (VMMs) and then implement SNNs by building a synapse array and neuron blocks.In [320], a novel SRAM-based SNN chip supporting on-chip unsupervised learning has been proposed through the software-hardware co-design of the STDP algorithm.Guo et al. [321] use an embedded nonvolatile floating-gate cell array working in the subthreshold domain for analog VMM, modified from a NOR flash memory.Wu et al. [322] further introduce Poisson neurons that exploit the back-hopping oscillation in STT-MRAM based on a 1T-1M array for binary-weight VMM, enabling the operation of SNNs.One of the advantages of using traditional memories is making the simulation and fabrication easier since the ecology of traditional memories is mature.Emerging memories here mainly refer to memristor-based memory devices [323], [324], [325].A crossbar of memristors is a natural VMM engine [326], [327], [328], which is just the primary workload in neural networks.In detail, the synaptic weights are stored on the crossbar, and the multiplication with the presynaptic inputs is also computed on the same crossbar, fusing the computing and memory in this way.For example, Zhang et al. [329] design memristor-based synapses and neurons and build full-memristor SNNs that can learn with the STDP rule.Limited by the fabrication process, current memristor-based neuromorphic chips still lie at small scales.Memristor-based BIC hardware is a large field with a lot of material hierarchies and architecture designs, which can be found in some published surveys and is not the focus of this work.
c) ANN accelerator variants: Similar to the idea of borrowing from neuromorphic chips to the design of ANN accelerators (e.g., the aforementioned IPU, Habana, and Hanguang), the design of neuromorphic chips can also borrow architectural ideas from ANN accelerators.Specifically, the dataflow architecture with high data reuse between PEs is primarily designed for high-precision matrix multiplication in ANNs.A simple way to modify the PE array for SNNs is replacing the multipliers with selectors, which pass the synaptic weights for accumulation if the input spike as the selection signal is valid while passing zero weights otherwise [330].
Based on the above design similar to a spiking version of Eyeriss, SpinalFlow [102] further reorganizes the input spikes of each layer and enables the skipping of invalid spikes (i.e., zero inputs) by only sending nonzero inputs into the PE array.This architecture can achieve significant acceleration since the spike activities of SNNs are quite sparse.SATA [104] combines multipliers and selectors in a PE array to allow the processing of both the high-precision matrix operation and the spiking matrix operation, which supports the matrix operations in both forward and backward passes.With tailored vector units for other operations beyond the matrix operation, SATA is eventually able to execute BPTT learning for SNNs.Different from SATA, H2Learn [103] designs a lookup table (LUT)-based engine for the spiking matrix operation in the forward pass and a PE array for the high-precision and sparse matrix operation in the backward pass, thus also enabling BPTT learning for SNNs.These ANN accelerator variants for SNNs once again reflect the mutual learning trend between deep learning and neuromorphic computing.
4) From Implementation Perspective: Implementing an architecture design to get a fabricated chip is the key step to reaching real-world applications.From the implementation perspective, the tradeoff becomes more complicated because many factors, including application scenarios, PPA, and programmability, should be comprehensively considered.In this section, we coarsely discuss two categories: small-scale chips and large-scale chips.For simplicity, we exclude the designs without tapeout.a) Small-scale chips: For edge applications, low power consumption and real-time response are usually the two rigid demands for BIC chips.In this situation, small-scale designs are preferred compared to redundant large-scale ones.Overall, the fabrication cost of small-scale BIC chips can be low, and there is no need to consider the scaling up and the complete ecology, thus making it easier to explore emerging technologies.
For example, ROLLS [90] exploits subthreshold analog signals of transistors to naturally simulate the dynamics of spiking neurons, which is actually a historical approach at the root of neuromorphic engineering [307].The chip has only 256 neurons, 256 × 256 short-term synapses, and 256 × 256 long-term synapses.Even though analog circuits enjoy fast response and low power consumption, they are not stable enough to carry signals over a long distance.Therefore, the analog neurons usually work with a digital routing fabric for communication between neurons, forming an analog-digital-mixed chip.Because analog circuits are difficult to program and sensitive to process, voltage, and temperature (PVT) variations, they are not the mainstream of modern large-scale BIC chips, although popular in some early designs.
DYNAPs [96] follows the analog-digital-mixed solution while further exploring a hierarchical routing network that has three levels: L0 (intracore), L1 (intercore), and L2 (interchip).The intrachip communication uses a multicast technique, the intercore communication uses a tree topology, and the interchip communication uses a grid topology.In this way, the low latency of the tree communication and the high bandwidth of the grid communication are combined.There are four cores in DYNAPs and 256 neurons for each core.ODIN [91] is a fully digital implementation of ROLLS with the same number of neurons.MorphIC [92] is an enhanced version of ODIN with four cores and 512 neurons for each core.The routing network of MorphIC is almost the same as that of DYNAPs, but it additionally demonstrates 1-bit synaptic weights that can be learned through stochastic SDSP.In addition, DYNAPs have been digitized, scaled up, and then productized as DYNAP-CNN and DYNAP-SE by SynSense [45] for always-ON devices with fast response (several milliseconds) and ultralower power consumption (several milliwatts).Furthermore, a neuromorphic chip can be integrated with an event camera for building a sensing-memory-computingintegrated solution, such as Speck [331], also developed by SynSense.
Compared to commonly used synchronous digital circuits with a global clock, asynchronous digital circuits are considered to be closer to the operation mode of the brain.Hence, in recent years, many asynchronous SNN chips for edge computing have emerged.For instance, asynchronous SNN chips supporting on-chip learning are designed in [332], [333], and [334] for various edge scenarios, all demonstrating high energy efficiency.Zhang et al. [335] introduce an asynchronous neuromorphic hardware architecture search (ANAS) method as well as a configurable asynchronous neuromorphic hardware simulator to optimize both the numerical and nonnumerical design space of asynchronous neuromorphic chips.Despite higher energy efficiency, the lack of mature design methods and EDA tools has hindered the widespread adoption of asynchronous digital circuits, which might be a critical issue that needs to address in the future.
b) Large-scale chips: The primary motivation for fabricating large-scale BIC chips is to build a brain simulator for exploring artificial general intelligence such as the brain.Thus, the design principles of large-scale BIC chips are quite different from the small-scale ones.There are several principles that must be considered for large-scale BIC chips [307].
1) Scalability: The chip can be scaled up to a board, to a server, and finally to a brain simulator, which needs a flexible and high-speed routing infrastructure.2) Programmability: The chip can be easily programmed by users without much knowledge about hardware, which needs an easy-to-use programming framework and an efficient compiler.3) Reliability: The chip must be reliable in the generation of computed results, which needs stable circuits and signals.

4) Compatibility:
The chip should be compatible with the mainstream general-purpose processors for constructing a practical intelligent system, which needs standard communication interfaces.
Neurogrid [95] and BrainScaleS [89] are two early BIC platforms that use analog-digital-mixed circuits.Neurogrid adopts the aforementioned subthreshold signals of transistors to simulate neural dynamics, while BrainScaleS adopts the above-threshold signals for significant acceleration via much larger currents.The new generation of BrainScaleS, i.e., BrainScaleS 2 [84], further incorporates two plasticity processing units (PPUs) to allow more neural models and learning rules.
As aforementioned, analog circuits are hard to program and sensitive to PVT variations, and most of the recent BIC chips are implemented with fully digital circuits.SpiN-Naker [82] is a digital neuromorphic platform that acts as one of the mainstay platforms (the other is BrainScaleS) for BIC in Europe.Each SpiNNaker board has 48 chip multiprocessors (CMPs), and each CMP has 18 ARM cores and an off-die SDRAM.The new generation of SpiNNaker, i.e., SpiNNaker 2 [97], further incorporates a numerical accelerator for common functions such as exponentials, log or random functions, and a multiply-and-accumulate array for matrix operations.By hybridizing the ARM cores and the accelerators, SpiNNaker 2 can support both SNNs and ANNs.Different from the use of general-purpose processors in SpiNNaker, TrueNorth [14] is the first fully digital ASIC chip for large-scale SNNs.By leveraging the eventdriven processing and sparse activities of SNNs, TrueNorth adopts asynchronous circuit design to realize low power consumption (tens of milliwatts per chip).Following the digital ASIC solution of TrueNorth, Darwin [85] scales down the number of spiking neurons in each chip until its second generation for large-scale networks comes out.With a similar chip-level architecture, Loihi [42] further adds specific circuits to implement several synaptic plasticity rules such as STDP as aforementioned, while Tianjic [87], [88] adds an additional datapath for ANNs with minimal area cost by reusing resources to a great extent.Following the direction of Tianjic, Kuang et al. [308] unify the ANN and SNN paradigms within the LIF neuron framework and demonstrate a hybrid neuromorphic chip.
In recent years, it is increasingly clear that the difficulties met by neuromorphic chips, even including machine learning chips during industrialization, lie in the full-stack usability other than the performance results emphasized by academic papers.To improve the programmability, many teams develop software toolchains to ease the high-level model design and the low-level deployment on hardware, such as BrainScaleS OS [336], [337] for BrainScaleS, SpiNNTools [338] for SpiNNaker, Corelet [339] and Compass [340] for TrueNorth, Darwin-S [341] for Darwin, and Lava [108] for Loihi.

5) Challenges and Trends:
Even though there are many BIC hardware platforms, as aforementioned, they are indeed meeting some challenges in practice and consequentially presenting several interesting trends.First, to break the intrinsic restrictions within a single domain, the designs of ANN accelerators and BIC chips are beginning to borrow functional and architectural ideas from each other.For example, some ANN accelerators utilize the decentralized manycore architecture from the BIC community, and some BIC chips reversely adopt architectures such as ANN accelerators.Second, some emerging BIC chips begin to integrate sensors to form a solution that tightens sensing, memory, and computing together.Third, except for the high bio-plausibility and low-power advantages, BIC chips have not yet proved their effectiveness in solving intelligent tasks with superior performance compared to GPU and ANN accelerators.This challenge is also a common problem met by the entire BIC community.Finally, distinct from early BIC chips that focus on the chip design itself, recent BIC chips pay more attention to BIC systems, software toolchains, and application pools.This reflects the fact that the BIC community has begun to care about the user experience and tries to make the system easier to build, program, and deploy, which has been particularly evidenced by the new chips from several big teams in both academia and industry over the world.

V. S O F T W A R E T O O L S
Current brain-inspired software can be subsumed under three categories according to their usage and infrastructure: neuromorphic toolchain, algorithm programming platform, and brain network simulator.In the following, we review existing software tools in each category separately and conclude with the challenges and trends.An overview of the typical software tools is displayed in Table 5.

A. Neuromorphic Chip Toolchains
Neuromorphic chips are arising in recent years to benefit from the efficiency of BIC such as TrueNorth [41] and Loihi [42].Neuromorphic toolchains are aimed to facilitate the high-level model design and compile the programs to the low-level executable codes described by computational primitives supported by hardware.As shown in Fig. 16, the compilation for neuromorphic chips generally comprises four steps [312].First, the neural network model is translated into a computational graph.Second, the computational graph is converted into a primitive graph.Then, a mapper transforms the primitive graph into a mapping graph.Finally, a code generator generates the low-level codes to execute on the chip.
Toolchains for CPUs and GPUs have been developing for decades and have been quite mature.However, design toolchains for neuromorphic chips are posed with extra challenges.Neuromorphic chips are relatively nascent, which features decentralization and in-memory computing.Therefore, they require distributed storage and intercore/chip dataflow, complicating the programming process.We review the toolchains of some typical neuromorphic chips as follows.
HICANN [89] uses PyNN [123] as the Python interface to specify the model architecture, the evaluation of the output, and the experimental form for deployment on hardware.As biological networks have a graph-based representation, the configuration data are computed similar to  the process of deploying a design written with HDL onto a field-programmable gate array (FPGA).BrainScaleS [89] software tries to provide a unified electronic specification of experiments also by virtue of PyNN [123] and translate the biological design into hardware primitives.BrainScaleS2 [84] uses a modified GCC compiler as a toolchain to support vector instructions.Based on this compiler, the hardware is compatible with the C++ standard library.Thereby, the hardware abstraction library (HAL) is workable both on the chip and the host system.The software of Neurogrid [95] has a GUI interface for interactive visualization and an HAL to translate simulated networks into the hardware space.
Darwin [85] provides an operation system Darwin-S that is hierarchical and modular with its own model definition language.The application IDE includes a model development toolkit and a debug tool for user convenience.The team of TrueNorth [41] develops a native Corelet language for model definition along with the programming environment and a Compass software for simulating the chip architecture.It also supports a variety of composable algorithms for developing applications.The toolchain of Loihi [42] consists of a Python API to design SNN topologies and customize the learning rule.A compiler and a runtime library are used to translate the model into primitives for executing on the chip.Loihi 2 [108] is the second generation of Loihi.It provides an open-source software, Lava, which is hardware-agnostic.Lava does not target at specific chips and is claimed to be portable across both conventional and neuromorphic processors.
Braindrop [106] provides an automatic synthesis software for translating abstracted computation into executable codes on the chip.The software comprises a front-end interaction with the Nengo [342] software and a back-end interface with the driver software.SpiN-Naker [107] provides a configuration software SpiNNTools based on PyNN descriptions or other formats.It takes the definition of the neural circuit and generates routing information.A neural network and the node topology are deemed as a graph, which can be configured by software and then uploaded to the node mesh.Tianjic [87], [88] is the first attempting at integrating ANNs and SNNs into a hybrid neuromorphic chip with a manycore architecture.
They develop a software framework for the unified description and conversion of various neural networks.
Recent neuromorphic chips also provide their toolchains.The software toolchain developed for the NeuRRAM chip [343] enables quick implementation of various machine learning applications.It provides Python-based APIs at different levels: low-level APIs for basic operations of each chip module, middle-level APIs for essential operations required in neural network layers, and high-level APIs for complete implementations of neural network layers.This design allows software developers, even those not familiar with the NeuRRAM chip, to deploy their machine learning models on it, facilitating a range of machine learning applications.The toolchain of NorthPole [318] automatically orchestrates computation, memory, and communication tasks on a specialized hardware array to ensure optimal utilization and prevent resource collisions.It strategically manages neural network layer processing with spatial and temporal efficiency, enabling existing applications to seamlessly transition to this advanced architecture.
The described software automatically orchestrates computation, memory, and communication tasks on a specialized hardware array to ensure optimal utilization and prevent resource collisions.It strategically manages neural network layer processing with spatial and temporal efficiency, enabling existing applications to seamlessly transition to this advanced architecture.
To sum up, current neuromorphic toolchains are often concurrently designed, which typically comprise an enormous amount of codes.They are often highly coupled with the hardware configuration and thus hard to quickly migrate to other chips, impeding the fast evaluation of algorithms on different hardware platforms.The development of a general-purpose toolchain is crucial to accelerate the evolution of neuromorphic systems.Loihi 2's Lava software represents a positive step in this direction, yet the field anticipates further advancements to mature and thrive.

B. Algorithm Programming Platforms
Algorithm programming platforms are designed to streamline the implementation of SNNs.Typically constructed upon conventional general-purpose computing hardware, these platforms are tailored to be compatible with prevalent deep learning frameworks, tapping into the wealth of advancements in computer science.
Presently, popular deep learning frameworks, such as PyTorch and TensorFlow, cater primarily to ANNs, lacking optimizations that cater to the unique attributes of SNNs.When developing BIC algorithm programming platforms, several key factors must be considered.First, they should support various neuron models and neuromorphic datasets.Second, they should make full use of the characteristics of SNNs to accelerate when executing on GPUs.Finally, interfaces for deployment on neuromorphic chips are expected.
So far, a number of SNN-focused algorithm programming platforms have emerged.SpykeTorch [109] and SIN-ABS2 target at convolutional SNNs.SpykeTorch [109] is designed to simulate convolutional SNNs with at most one spike per neuron and the rank order encoding scheme.It enables easy implementation of learning rules and is generic and capable of reproducing results of various studies.Similarly, SINABS 2 implements several spiking convolutional layers and provides APIs to compare with conventional CNNs.BindsNet [110], PySNN,3 Spy-Torch, 4 SpikingJelly, 5 Norse [111], and SNNtorch [112] further extend support to general SNNs.BindsNet [110] is particularly suited for machine learning and reinforcement learning with an interface to the OpenAI gym [113].PySNN 3 is geared toward correlation-based learning methods.SpyTorch 4 proposes a new surrogate gradient method named SuperSpike to smoothen spike signals.SpikingJelly 5 incorporates many neuromorphic datasets such as CIFAR10-DVS [114], ASL-DVS [115], and DVS Gesture [116], which can be easily loaded by users.Norse [111] tries to introduce the sparse and event-driven characteristics of SNNs and supports many typical neuron models.SNNtorch [112] establishes some online variants of BP that are more biologically plausible.
These platforms inherit the user-friendly, modular, and dynamic qualities of PyTorch, thus supporting CPU or GPU deployment.Consequently, they enable efficient SNN simulation with significant parallelism.
For interfacing with neuromorphic chips, SpikingJelly 5 supports deployment on a neuromorphic chip named Lynxi HP300. 6Combined with the CTXCTL software, SINABS 2 can also port models onto the DynapCNN chip [117].PyNCS [118] does not target specific hardware.It features modularity, portability, and expandability by providing a front end to specify the network architecture and a set of Python APIs for interfacing with hardware, thus decoupling high-level design and bottom-level descriptions.
The importance of algorithm programming platforms for neuromorphic algorithm development is well-recognized.Ideally, such software should be intuitive enough for beginners yet versatile enough for crafting new models.Users are able to engage in high-level programming without delving into the complexities of hardware-level intricacies.Nonetheless, core functionalities must be realized in lowlevel languages and must be finely tuned to capitalize on specific hardware infrastructures.
In summary, while the initiatives in existing algorithm programming platforms lay a promising foundation, there remains significant room for advancement.Future efforts will likely focus on broadening the deep learning framework to further enhance functionality and optimize performance.

C. Brain Network Simulators
Brain network simulators aim at simulating biological neural networks with the support for diverse neural activities and synaptic models.They are mainly based on SNNs, which are considered to be more bio-plausible and energy efficient than ANNs.The simulation can be conducted in various scales ranging from individual molecules and compartment models to the whole brain system.In contexts where specialized hardware is not widely available, software simulators become crucial for verifying hardware performance, testing modifications, and developing BIC algorithms.Furthermore, they have applications in the analysis of brain diseases.Ideally, brain network simulators should have the following features.First, they should support a wide range of neuron models.Second, they should enable modular development and guarantee efficient execution.Third, they should have high parallelism and should be compatible with distributed processors or clusters to support large-scale simulation.
Pioneering brain simulators, such as GEneral NEural SImulation System (GENESIS) [119], Neuron [120], NEST [121], and Brian [122], have established foundational frameworks.GENESIS [119] is a general-purpose simulator for realistic neural systems, e.g., subcellular components and complex models of individual neurons.NEURON [120] uses sections to model individual neurons.Conventionally, users are supposed to create compartments manually, while in NEURON, the sections are split into individual compartments automatically.The primary scripting language, along with a Python interface, is provided.Parallelization is supported based on the MPI protocol, making it possible to deploy on a multicore processor.NEST [121] features optimization for large-scale networks.It represents spikes in continuous time and supports the combination of different types of neuron models in a network.It can also take advantage of multiprocessor computers and clusters.The implementation is based on C++ with a Python interface named PyNEST.It also enables the development of a build-in script language named SLI.Brian [122] is intuitive and efficient for designing new models, especially for those with single-compartment neurons.Users can arbitrarily customize neuron models by writing mathematical equations.Therefore, it is especially suitable for the use of teaching.
PyNN [123] serves as a unifying interface for various brain network simulators, promoting efficiency through code reuse.Brian2 [125] uses mathematical equations to describe every aspect of neural models, extending the scope of Brian, and making it possible to be independent of computing devices.However, it is still criticized for its steep learning curve.CARLsim 6 [344] is a user-friendly and powerful GPU-optimized library designed for simulating extensive SNN models with intricate biological fidelity.It enables the deployment of Izhikevich neuron models with realistic synaptic behavior across conventional x86 CPUs as well as popular GPUs.Offering a programming interface reminiscent of PyNN within C/C++, CARLsim 6 facilitates granular control over the configurations and parameters at the synapse, neuron, and network layers.
Auryn [126], ANNarchy [127], GeNN [128], Brian2GeNN [129], and BSIM [130] focus on increasing the simulation speed.Auryn [126] is a simulator for recurrent SNNs with synaptic plasticity, which increases the parallelism to reduce latency.ANNarchy [127] defines neuron and synapse models with equation-oriented mathematical descriptions.The definition is then utilized to generate C++ codes to efficiently execute on parallel hardware.GeNN [128] is a code generation framework, which aims to facilitate the use of graph accelerators for computational models of large-scale neuronal networks.BSIM [130] utilizes the cross-population parallelism to make full use of GPU resources, the sparsity-aware load balance to deal with the activity sparsity, and the dedicated optimization to support multiple GPUs.The DeepDendrite framework [345], empowered by the dendritic hierarchical scheduling (DHS) method, propels neuron simulation forward, effectively narrowing the gap between it and AI.This innovative method markedly elevates simulation speeds, thereby enabling an extensive exploration of computational brain principles and fostering the evolution of AI algorithm development.With its potential applications, the framework holds promise in advancing fields of neuroscience and AI, specifically in the realms of spatial pattern analysis and image classification tasks.
There are also some large-scale brain simulators worth mentioning in recent years.The MONET simulator [131] released in 2020 targets human-scale brain simulation using supercomputers.The authors propose a tile partitioning method that reduces the network communication cost drastically, thus guaranteeing the scalability and efficiency of MONET.A large-scale brain simulator called BrainCog [346] was released in 2023, aiming at realizing brain-inspired intelligence.It supports various BIC and brain simulation models at multiple scales and diverse types of cognitive functions such as perception, decisionmaking, and reasoning.BrainPy [132] released in 2022 focuses on the modeling of brain dynamics, which is indispensable for mining neural mechanisms under brain functions.It provides a general-purpose programming framework to ease and facilitate user development based on the just-in-time (JIT) compilation.The work in [134] utilizes the common deep learning framework TensorFlow to simulate biologically detailed large-scale models for area V1, termed "V1 Simulator," on a single GPU.The DeepDendrite framework [345], utilizing the DHS method, accelerates neuron simulation, bridging the gap between it and AI.This method significantly boosts simulation speeds, facilitating the exploration of brain computational principles and AI algorithm development.The framework shows promising applications in neuroscience and AI, such as spatial pattern analysis and image classification tasks.
Regarding the works of brain network simulators for the applications in brain disease analysis, the Virtual Brain (TVB) [124] is a multiscale neuroinformatics platform for whole-brain simulation, which borrows methods from statistical physics to reduce the complexity at the micro level and accomplish the macro-level organization.Stefanovski et al. [347] tap into the potential of using TVB for understanding the mechanisms of Alzheimer's Disease.Meier et al. [348] also reveal that TVB has the potential to improve the effectiveness of deep brain stimulation treatment for Alzheimer's disease.TVB also provides a cloud service in EBRAINS [349].
Generally, most current simulators focus on providing an easy-to-use toolkit for building BIC models with versatile functions and accelerating the computation.There are several functional advantages to using these software tools.First, some simulators [121], [131] focus on large-scale brain simulation, and the ultimate goal is to achieve a human-level simulation, where large-scale network construction, splitting, and simulation on multicluster supercomputers are supported.The achievement has been attained for the human cerebellar (68 billion neurons) [131], cat cerebellum (1 billion neurons) [357], and macaque cortex (4 million neurons) [353].In the opposite direction, some of them focus on detailed simulation of a neuron or sub-neuron structures such as multicompartment models, receptors, and even protein structures, which offer more fine-grained simulation.In another aspect, several software tools focus on speeding up the simulation speed, especially with GPU acceleration, such as NeuroGPU [359] and GeNN [128].A realtime simulation (the simulation can be executed faster than biological time with a time_step7 of 0.1 or 1 ms) is the main pursuit.Currently, real-time simulation within 1-mm 2 macaque visual cortex is achievable [352]; however, large-scale simulation in real time is still challenging.As illustrated in Table 6, MONET is 578× slower than the human cerebellar [131] and NEST is 600× slower than a 1B neuron balanced neural network [356].It is hard to achieve high simulation speed, large scale, and fine simulation granularity, and the current situation is shown in Table 6.In addition, several simulators, such as NEST 8 and NEURON,9 provide a library with rich models, including neuron models, synaptic models, and network models along with related computational neuroscience publications [360], which helps the neuroscience community.Related to the large-scale human brain simulation, the concept of digital twin brain (DTB) is introduced recently, which aims to bridge the gap between biological and AI by simulating the human brain's structure and function [361].It helps personalize medical interventions, for example, in neurorehabilitation and neurosurgery, through multiscale brain modeling [362].A DTB computing platform can also simulate a personalized biological brain structure along with the whole human brain scale proposed in [363].
Except for the above achievements, there are still several challenges concerning the development of brain network simulators.First, users are required to have a certain degree of computational neuroscience foundation, which can be formidable to users without relevant backgrounds.Second, many simulators are not universal enough, which leads to a lack of cross-platform compatibility and optimization.Third, it is difficult to establish a task-level connection between brain simulation and machine learning at present.Finally, they are designed to execute on von Neumann architectures, failing to take advantage of the high efficiency of neuromorphic hardware.

D. Challenges and Trends
BIC stands at the forefront as a transformative paradigm for the forthcoming wave of AI advancements, transcending the limitations traditionally imposed by Moore's law.The research community has witnessed a significant uptick in the development of BIC-oriented software, a testament to its burgeoning potential.Yet, several formidable challenges remain, pointing to the pivotal trends on the horizon.
First, a pressing requirement exists for the creation of general-purpose software tools.This necessitates the decoupling of software from hardware, hinging on the compatibility with neuromorphic systems.Traditional computing systems, anchored in Turing completeness and the von Neumann architecture, boast a well-established hierarchy.Zhang et al. [364] propose a concept named "neuromorphic completeness," which mitigates the necessity for hardware completeness and formulates a new system hierarchy.This includes a Turing-complete software abstraction layer and flexible neuromorphic architectures.Their pivotal work lays the theoretical groundwork for the design of general-purpose software tools, stimulating further advancements in this domain.
Second, contemporary software tools should be optimized to resonate with the characteristics of brain-inspired models and algorithms, such as sparse spiking and eventdriven processes.These distinctive attributes introduce additional complexities in software design, including irregular memory access patterns and numerical precision issues-subjects that call for in-depth investigation and experimentation.
Third, existing software tools, particularly algorithmic programming platforms and brain network simulators, are mainly deployed on standard hardware such as CPUs and GPUs.It is expected that future research will synergize these tools with the innate benefits of neuromorphic hardware.The booming of deep learning owes much to the advent of high-performance computing platforms such as GPUs.Programming frameworks for deep learning, including PyTorch and TensorFlow, have achieved seamless hardware integration.A similar level of sophisticated software tool development is highly sought after in the BIC community to accelerate the advancement of algorithms and applications.Developing a user-friendly software toolchain plays a crucial role in promoting BIC killer applications to the public, which is also crucial for the establishment of the BIC application ecosystem.

VI. B E N C H M A R K D A T A S E T S
In this section, we first review the current status of neuromorphic datasets.Then, we broadly classify existing neuromorphic datasets from two perspectives, followed by a comprehensive overview of neuromorphic dataset properties.Finally, we discuss the challenges and future trends of neuromorphic datasets.

A. Current Status
Large-scale datasets play a dominant role in the era of deep learning [365], which allows monitoring progress with quantitative benchmarks and provides data-driven possibilities for learning algorithms.Current neuromorphic datasets are obviously not comparable to traditional framebased datasets (e.g., ImageNet [366] and KITTI [367]) in computer vision in terms of task diversity, data size, and scenario complexity.There may be two reasons: 1) neuromorphic sensors (e.g., DVS [160], DAS [161], and tactile sensors [162]) are considerably more expensive than standard sensors (e.g., traditional cameras) and 2) asynchronous spatiotemporal events [368] present sparse points in three dimensions, and thus, large-scale hand-annotated labels are not easy to obtain as conventional frames.However, BIC techniques [16], [272] are booming to process information with extreme energy, leading to a dramatic increase in the number of neuromorphic datasets.

B. Categorization
1) From Dataset Generation Perspective: Simulated Datasets and Real-World Datasets: From the dataset generation perspective, existing neuromorphic datasets can be roughly divided into two categories (i.e., simulated datasets and real-world datasets).The first category is the simulated dataset that converts frame-based datasets into the neuromorphic domain [145].The first part of them [135], [136], [137], [138], [369] adopts eventbased simulators, e.g., ESIM [139] and V2E [140], to convert large-scale video datasets into asynchronous spatiotemporal events.For example, Rebecq et al. [135] use ESIM [139] to simulate dynamic events triggered by random camera motion from MS-COCO [141] images.Gehrig et al. [136] implement an event-based sampling module in CARLA [370], which renders high-frame-rate images and converts them into dynamic events using ESIM [139].Li et al. [137] utilize V2E [140] to convert videos into dynamic events for object detection, and they directly use existing large-scale annotated labels from KITTI [367] dataset.Lin et al. [138] propose an omnidirectional discrete gradient algorithm to convert frames into event streams.The second part of them (e.g., N-MNIST [142], N-Caltech101 [142], N-UCF50 [143], CIFAR10-DVS [114], and N-ImageNet [144] uses event cameras to record images from popular frame-based datasets on an LCD monitor.For instance, the conversion of the CIFAR10-DVS [114] dataset is implemented via a repeated closed-loop smooth movement of images.The N-ImageNet [144] dataset is recorded by a moving event camera in front of an LCD monitor using programmable hardware.These simulated datasets allow reducing costs and advancing the development of event-based algorithms.However, those conversion strategies fail to capture dynamic changes in high-speed or low-light real-world scenarios, which is exactly what event cameras are good at. The second category refers to the real-world dataset that contains event data by directly recording various realworld objects.Generally, they can be categorized into classification tasks and regression tasks [145].The first group mainly comprises event-based datasets for object recognition (e.g., DVS-Gesture [116], N-CARS [146], and ALS-DVS [115]) and action recognition (e.g., DVS-PAF [147] and DHP19 [148]).In the second group, neuromorphic datasets for regression tasks include image reconstruction (e.g., CED [149] and BS-ERGB [150]), object detection (e.g., PKU-DDD17-CAR [151], 1Mpx Automotive Detection [152], and PKU-DAVIS-SOD [371]), object tracking (e.g., FED240hz [153] and VisEvent [154]), depth estimation (e.g., MVSEC [155] and DESC [156]), and SLAM (e.g., UZH-FPV [158] and VECtor [159]) and so on.As shown in Table 7, most of the existing real-world datasets are for object recognition tasks, and few for complex regression tasks, especially scene segmentation datasets with pixellevel annotation.
2) From Dataset Modality Perspective: Single Modality and Multimodality: Neuromorphic datasets can also be classified into single-modality and multimodality from the dataset modality perspective.Several datasets in the first type only contain asynchronous spatiotemporal events from a single neuromorphic sensor (e.g., DVS [160], DAS [161], or tactile sensor [162]).Most datasets in the neuromorphic community [145] are eventbased vision datasets due to the widespread attention and the rapid application of event cameras.Recently, several neuromorphic datasets for speech and touch have also been released.For example, the N-TIDIGIS18 [163] dataset is recorded by playing the audio files from the TIDIGITS dataset to the dynamic audio sensor (DAS, i.e., CochleaAMS1b).The ST-MNIST [164] dataset comprises large-scale handwritten digits obtained by handwriting on a neuromorphic tactile sensor (NTS) array.Other datasets in the second type provide hybrid heterogeneous sensing streams from multiple neuromorphic sensors.For instance, the GRID [166] visual-audio lipreading dataset is recorded using two bioinspired silicon multimodal sensors (i.e., DVS [160] and DAS [161]).A visual-tactile event-based dataset [165] is built for intelligent power-efficient robot systems using a neuromorphic fingertip tactile sensor and an event camera.Besides, some works [151], [153], [154], [155], [156], [158], [159], [167], [372] attempt to integrate neuromorphic sensors with other sensing modalities (e.g., light detection and ranging (LiDAR), RGB-D camera, infrared camera, IMU, and GPS) for intelligent robots in challenging scenarios.These emerging multimodal datasets involving neuromorphic sensors will stimulate research for robust perception (see Table 8).For better visualization, we show three representative datasets from the modality perspective in Fig. 17.

C. Dataset Properties
1) Sparse Events: Inspired by biological nervous systems, neuromorphic hardware systems implement neuronal and synaptic computational operations using eventdriven communication (e.g., AER [302]).As a result, neuromorphic sensors (e.g., DVS [160], DAS [161], and tactile sensors [162]) can sense dynamic changes with asynchronous spatiotemporal events in real time.Taking DVS for instance, an event can be described as a tuple ⟨x, y, t, p⟩, including four components: spatial coordinates, timestamp t, and polarity p. Intuitively, all pixels generate asynchronous discrete and sparse points in three dimensions [368].

2) Spatiotemporal Features (Hybrid Representation):
Neuromorphic sensors present a novel paradigm shift in the acquisition of sensing information [145].Hence, most existing deep learning techniques designed for frames cannot be directly applied to asynchronous spatiotemporal events.This poses a key question: what is the best way to fully exploit spatiotemporal features from event streams to maximize the performance in given tasks?Generally, it is necessary to convert discrete points into successive measurements using a kernel function [373], which can adopt handcrafted functions or neural network architectures.According to the literature, these event rep-  resentation strategies can be classified into image-like representations, handcrafted descriptors, DNNs, and SNNs.The early attempts directly map asynchronous events into image-like representations (e.g., event images [374] and voxel grids [375]).Besides, some effective spatiotemporal descriptors (e.g., time surfaces [376]) are extracted from asynchronous events, while they are time-consuming and strongly depend on the types of moving objects.Some end-to-end learning representations (e.g., EST [373] and Matrix-LSTM [377]) are generated by deep learning models.Biologically interpretable SNNs [246], [378], [379], [380] are utilized to fully exploit spatiotemporal eventbased information with extremely low power computation.However, little work has been explored in the hybrid representation from multiple heterogeneous event streams.

D. Challenges and Trends
BIC is an emerging technology in the era of deep learning.In fact, it is unfair to directly compare task diversity, data size, and scenario complexity of the two types of datasets in BIC and deep learning.However, an essential problem is to investigate what the key properties that BIC datasets have inspired by biological nervous systems are.Regarding the real-word application, a major trend for BIC/neuromorphic datasets is to build more challenging scenarios (e.g., high speed or low light) to highlight the advantages of neuromorphic sensors over conventional sensors.It is also suggested that providing more various open-source neuromorphic datasets will open up an opportunity for the BIC community.

VII. F R A M E W O R K O F B I C S Y S T E M S
Fig. 18 presents several milestones for BIC hardware and software.Early studies mainly focus on the investigation of a BIC algorithm or a BIC chip, while modern ones pay more attention to the full-stack solution for enhancing the applicability in practice.The design of a BIC system must consider a complete framework, including algorithms, software, hardware, and potential applications, as illustrated in Fig. 19.BIC hardware and software usually act as the intermediate link between highlevel algorithms/applications and low-level systems.The design of a BIC chip targets the optimal implementation of an instruction set that can support the involved algorithms.In this way, the software can focus on programming neural models and mapping models onto the chip instruction set with hardware restrictions, which no longer needs many hardware details.The decoupling of software  and hardware significantly matters in the framework of BIC systems by enabling the independent and iterative development of software tools and hardware [364].For a large-scale system, the hardware platform aims at maximizing the computing power of chips via a fast communication hierarchy and an efficient workload scheduling strategy between many chips and even servers.The scheduling strategy can be integrated into the software tools as a part of the operation system.
The construction of a BIC system in recent years presents two directions, big systems or small systems, with different requirements (see Fig. 20).Big BIC systems, such as BIC servers in the cloud, emphasize more on high scalability and high throughput, which are critical for increasing the system scale to deploy multiple large models and the operation speed of these large models.The chip design of big systems often needs stronger programmability and reliability, and the software design needs better generalizability.In contrast, small BIC systems emphasize more on high efficiencies, such as low power, low latency, and compact size, which are usually applied in real-time and low-power mobile devices and robots at the edge.Because small BIC systems do not care much about the complexity of scaling up the system and the need for high programmability and generalizability, we can see more explorations of novel learning rules, circuit designs, and fabrication processes in the design of small BIC systems.The above transition from a single algorithm or chip design to a full-stack solution pushes the design work of a BIC system up to a complicated project.The modern design of a BIC system requires comprehensive collaborations between different disciplines, including neuroscience, AI, computer architecture, integrated circuits, and software engineering.This situation increases the difficulty in developing a practically usable BIC system by an academic team.For large BIC systems, the challenge becomes more severe due to the high scaling complexity and more considerations of the practicability.
In detail, the construction of a large BIC system usually needs the following considerations.
1) A complete instruction set, which can avoid unsupported operators to a great extent.Users might give up a product when they meet operators that cannot be executed on the system, even if these operators are only a small minority.2) Heterogeneous SoC integration, which can include extra units such as CPU cores, DDR interfaces, image or video encoders/decoders, and sensor interfaces, beyond the core design for improving the programmability and usability in practical tasks.3) High-speed communication interface such as SerDes and PCIE interfaces, which can provide sufficient bandwidth between chips and boards for reducing the system-level latency.4) Advanced fabrication process, which can bring high chip performance and yield.5) High reliability, which is important for reducing the system crashes.6) User-friendly software tools, which should be general enough for easy deployment of large-scale neural models.7) A sustainable application ecology, which reflects the motivation for using the system to develop applications and finally represents the vitality of the system.However, solving these challenges is very costly and human labor demanding, which is probably unaffordable for the academic realm.
In the future, a feasible route might be that an academic team develop a prototype system with innovative techniques, while an industry team further makes the corresponding product.

VIII. D I S C U S S I O N A N D C O N C L U S I O N
In this article, we provided a systematic survey on BIC from four components of BIC infrastructures: model/algorithm, hardware platform, software tool, and benchmark datasets, based on which we claimed some basic related concepts in this research field.For each component, we summarized the recent advances, key considerations, and mainstream technologies, as well as future trends.Finally, the framework of BIC systems is presented, and we pointed out that the co-design of these four components is a ubiquitous trend for the future development of BIC.To truly realize the potential of BIC, the entire field needs to make systematic efforts.The BIC research still has a long way to go, and we discuss BIC's current and longrange vision here.

A. Future Directions at Current Stage
At the current stage, we provide several potential applications and scenarios where BIC characteristics can be desirable: 1) Lightweight Intelligence: SNNs feature event-driven computing, which can avoid high-precision multiplication operations and benefit from sparse spike activities.Such lightweight computing results in reduced resource overhead and power consumption when running on BIC systems.This is beneficial for edge intelligence with limited resources and power supply.In particular, event-driven SNN algorithms combined with asynchronous neuromorphic chip design can maximize the advantages of low-power computing in neuromorphic computing [331].
2) Multitask Agents: An agent, such as a robot or a drone, usually performs multiple tasks for processing different sensor signals and then making reliable decisions.The manycore decentralized architecture of BIC chips is naturally suited for running multiple intelligent models owing to the decoupled computing, memory, and control between cores.This mitigates the memory conflicts when running multiple threads on von Neumann architectures, thus improving the execution performance for multitask workloads.
3) Hybrid Sensing for Complex Environments: The frame images collected by conventional cameras have rich texture information, while the spike events collected by DVS cameras enjoy HDR and low latency.Integrating hybrid sensing technologies can combine complementary advantages for high-quality imaging and comprehensive decisions in complex environments as motion blurs, poor light conditions, bad weather, and sudden anomalies.

4) Hybrid Computing for Flexible Configuration:
ANNs feature high-precision neural coding and intensive computing, while SNNs feature binary neural coding and sparse computing.On different datasets for variable applications, ANNs and SNNs usually present different accuracy results and computing costs.Integrating hybrid computing paradigms can combine complementary advantages for flexible configuration of the application accuracy and computing cost by tuning the resource ratio between ANNs and SNNs.

5) Bioinspired Large Language Models:
The emergence of large Language models (LLMs) heralds the beginning of a new era of AI.One of the biggest concerns is that current LLMs consume a vast amount of energy.The human brain performs much more complex tasks than LLMs and comprises a larger network (with 86 billion neurons and trillions of synapses) but only consumes 20 W of power.Thus, BIC systems are promising approaches to solving the energy problem.Drawing from both computational neuroscience and computer science is key to success.Neuroscience has long been an essential driver of progress in AI, but current AI is very different from how the human brain works.One example is that current artificial neurons are oversimplified.Biological neurons have rich dendritic nonlinear computing units and soma dynamics.These complex dynamics make the expression ability of a single biological neuron much higher than that of artificial neurons.Current ANNs use low-complexity artificial neurons and supersized networks to achieve machine intelligence.Another possible bioinspired way is a combined approach of highcomplexity spiking neurons and networks of a certain size.Thus, drawing extensively on the dynamics of dendrites and somas in biological neurons, it is hoped that a bridge can be constructed from computational neuroscience to large-scale models.
6) Algorithm-Software-Hardware Co-Design: Constructing a robust BIC ecosystem and elucidating the application advantages of BIC requires a collective effort across the entire field.Taking spike-based neuromorphic computing as an example, neuromorphic paradigms for vision and auditory bionic perception exhibit high sensitivity and low energy consumption advantages.Benefiting from brain-inspired event-driven processes and spatiotemporal dynamics, neuromorphic algorithms are particularly wellsuited for processing neuromorphic data.Applicationalgorithm co-design is fundamental to unlocking killer applications in neuromorphic computing.The complex spatiotemporal dynamics render the training of neuromorphic algorithms significantly more challenging than that of traditional AI.A comprehensive software toolchain can aid researchers by reducing the costs associated with algorithm design.The effective implementation of the advantages of neuromorphic algorithms is contingent upon the support of robust neuromorphic hardware, and a complete software toolchain can reduce the costs of algorithm deployment.Therefore, the development and enhancement of BIC systems are inseparable from the synergistic design of algorithms, software, and hardware.

B. Exploring the Essence of BIC
It is well admitted that modeling and algorithms are the driving force of BIC, which tends to build new models and algorithms by learning from mechanisms, structures, and functions of biological neural systems.In neuroscience and physics, substantial progress has been accomplished in characterizing information-processingrelated neural dynamics on different scales.Generally speaking, there are three levels of learning from the microscopic scale (e.g., single-neuron models [215], [381]), mesoscopic scale (e.g., stochastic network models of neural populations and circuits [382], [383]), eventually to macroscopic scale (e.g., mean-field neural mass models of brain regions [384], [385], and models of entire brain networks [124], [386], [387]), and new properties persistently emerge and previous properties may vanish during the cross-scale transmission of neural dynamics, which can be discovered in coarse-grained flows by applying the renormalization group [388], [389], [390].
Although the existing BIC framework stems from learning the brain and seems to share similar terminologies with neuroscience, the learning itself is still limited to phenomenological simulation with computational simplification.In this sense, the research of BIC remains at an early stage of modeling single neurons and constructing cluster architectures by naive connections among single neurons, THAT is to say, existing works mainly leverage the single neural model and hardly learn from the neural circuits or higher levels of the brain structures and functions.The complex topology and special dynamic phenomena coming from neural connections may be the critical point for the brain to coordinate the body's myriad functions and behaviors and achieve human intelligence.However, BIC researchers still lack the understanding to apply them to practical modeling.Similarly, for the brain simulations based on either software or BIC chips, with a reductionism belief, existing schemes attempt to reproduce whole-brain functions by simply increasing the number of modeled neurons or complicating the iteration rules among neurons [17], [391].While such an idea is practical in engineering, it conflicts with the fact that brain functions are not the plain sums of the dynamics of single neurons, irrespective of how complex the modeled single neurons are [381], [383].
Certainly, it is not necessary for BIC designs to precisely model the intricate physical nature of the brain.However, accelerating progress in current AI by investing in fundamental research in neural computation will be of great potential.More and more scientists believe that research in this direction may reveal the basic ingredients of intelligence and catalyze the next revolution in AI [265].We suggest that BIC may learn a lesson from the invalidation of reductionism in neuroscience and develop real multiscale architectures for cross-scale neural dynamics to emerge.In summary, how can BIC learn from the brain at a microscopic scale, mesoscopic scale, and macroscopic scale simultaneously, and how can BIC exploit cross-scale neural dynamics, and develop corresponding theories, models, architectures, and hardware systems to deal with realworld applications are of great potential for the long-term future research.

Fig. 1 .
Fig. 1.Four components of BIC infrastructures: model/algorithm, hardware platform, software tool, and benchmark data.The prosperity of BIC depends on the construction of these components.

Fig. 2 .
Fig. 2. Overview of BIC systems: co-design of four components of BIC becoming more and more ubiquitous in building an end-to-end BIC system, by combining computer science and neuroscience.

Fig. 4 .
Fig. 4. Illustration of SNN = ANN + Neuronal Dynamics.(a) ANN architecture: the structure of neural networks can be very different types, such as MLPs, CNNs, and RNNs.Neuronal dynamics mainly consider two parts.(b) Soma dynamics.(c)Dendritic dynamics.The soma dynamics can be reflected in the change of the membrane potential, denoted as u(t).When the membrane potential exceeds a threshold v, the soma fires a spike.The dendrite geometry can be divided into many compartments, and each compartment can be regarded as an equivalent circuit containing capacitance and resistance (cable theoretical model), receiving signals from different sources.The subfigure shows a compartment of dendritic dynamics.

Fig. 5 .
Fig. 5. Single-neuron models.From left to right, neuronal models are more efficient but less biologically plausible.(a) Dendrites of neurons are modeled as thousands or hundreds of compartments, aiming to mimic the fine structure of neurons.(b) Modeling of the dendrites and soma of the neuron.The model above takes dynamics into account, the model below does not.(c) Model reduces the neuron structure to point neurons and only soma dynamics are considered or not.

Fig. 6 .
Fig. 6.Relationship between biological features of neuronal models and their implementation efficiency.The two dashed circles mark two important research directions: improving biological plausibility and boosting computational efficiency.

Fig. 9 .
Fig. 9. Schematics of SNN algorithms.(a) Classic window of unsupervised STDP for synaptic modifications.Data adapted from [225].(b) Conversion pipeline from pretrained ANNs to SNNs.(c) Error BP through both spatial and temporal domains in direct training and surrogate derivatives for the nondifferentiable spike function.

1 )
Difference From Deep Learning Chips: Deep learning chips focus on the execution optimization of ANNs for better PPA compared to conventional general-purpose processors.The researchers of deep learning chips usually come from the computer architecture community, which is heavily affected by the design of CPU/GPU processors.Therefore, the architectures of deep learning chips are usually the variants of von Neumann ones, featuring an elaborated processing unit and the corresponding memory hierarchy[306].Benefitting from rich data reuse patterns 560 PROCEEDINGS OF THE IEEE | Vol.112, No. 6, June 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Fig. 14 .
Fig. 14.Taxonomy of BIC hardware in this work.(a) BIC chips are discussed from functionality, architecture, and implementation perspectives.(b) From the functionality perspective, some BIC chips only support SNNs, while others further support HNNs, and some only support inference, while others further support learning.

574
PROCEEDINGS OF THE IEEE | Vol.112, No. 6, June 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

Table 1
Pros and Cons of Training Methods of SNNs

Table 2
Summary of SNN Algorithms and Their Performance on Different Benchmarks

Table 3
Representative Neuromorphic Cameras With the Specific Performance Parameters the excellent ecological environment created by the open source of datasets, codes, and software tools.

Table 4
Overview of Typical Neuromorphic Hardware

Table 5
Overview of BIC Software

Table 6
Several Simulations Performed on Brain Simulators

Table 7
Comparison With Representative Datasets Involving Event Cameras

Table 8
Comparison With Representative Neuromorphic Datasets From the Modality Perspective 572 PROCEEDINGS OF THE IEEE | Vol.112, No. 6, June 2024 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.