Garrett Kenyon

Retinal ganglion cells (RGCs) summate inputs and forward a spike train code to the brain in the form of either maintained spiking (sustained) or a quickly decaying brief spike burst (transient). We report diverse response transience values across the RGC population and, contrary to the conventional transient/sustained scheme, responses with intermediary characteristics are the most abundant. Pharmacological tests showed that besides GABAergic inhibition, gap junction (GJ)–mediated excitation also plays a pivotal role in shaping response transience and thus visual coding. More precisely GJs connecting RGCs to nearby amacrine and RGCs play a defining role in the process. These GJs equalize kinetic features, including the response transience of transient OFF alpha (tOFFα) RGCs across a coupled array. We propose that GJs in other coupled neuron ensembles in the brain are also critical in the harmonization of response kinetics to enhance the population code and suit a corresponding task. [Display omitted] •RGC response transience values form a continuum•A single retinal pathway carries signals to both sustained and transient RGCs•GJ mediated lateral excitation fine-tunes RGC response kinetics•tOFFα RGC GJs equalize kinetic features for cells in the coupled array Molecular neuroscience; Cellular neuroscience; Sensory neuroscience

Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. While the exact mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown, there are high-level neural mechanisms of perception that we can incorporate in a novel computational model - ones that utilizes lateral and top down feedback in the form of hierarchical competition. We demonstrate that these neural mechanisms provide the foundation of a novel classification framework that rivals traditional supervised learning in computer vision. Additionally, we show that the innate priors built into our architecture support out of distribution generalization on the application of face detection.

Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of over-lap between the event representations and the pre-training data used to initialize the weights of the object detectors. Then, we apply event-to-video conversion models that convert event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection Fig. 1. Comparison of the same VisDrone-VID [1] scene techniques on our overhead target task. These results suggest using various input representations. Top Left: Event Count that better alignment between event representations and exist-Map. Top Right: FireNet [2] Gray-scale Frame. Bottom: ing large pre-trained models may result in greater short-term Original RGB Frame. performance gains compared to end-to-end event-specific architectural improvements.

Audio classification aims at recognizing audio signals, including speech commands or sound events. However, current audio classifiers are susceptible to perturbations and adversarial attacks. In addition, real-world audio classification tasks often suffer from limited labeled data. To help bridge these gaps, previous work developed neuro-inspired convolutional neural networks (CNNs) with sparse coding via the Locally Competitive Algorithm (LCA) in the first layer (i.e., LCANets) for computer vision. LCANets learn in a combination of supervised and unsupervised learning, reducing dependency on labeled samples. Motivated by the fact that auditory cortex is also sparse, we extend LCANets to audio recognition tasks and introduce LCANets++, which are CNNs that perform sparse coding in multiple layers via LCA. We demonstrate that LCANets++ are more robust than standard CNNs and LCANets against perturbations, e.g., background noise, as well as black-box and white-box attacks, e.g., evasion and fast gradient sign (FGSM) attacks.

Neuromorphic processors have garnered considerable interest in recent years for their potential in energy-efficient and high-speed computing. The Locally Competitive Algorithm (LCA) has been utilized for power efficient sparse coding on neuromorphic processors, including the first Loihi processor [4, 5]. With the Loihi 2 processor enabling custom neuron models and graded spike communication, more complex implementations of LCA are possible [9]. We present a new implementation of LCA designed for the Loihi 2 processor and perform an initial set of benchmarks comparing it to LCA on CPU and GPU devices. In these experiments LCA on Loihi 2 is orders of magnitude more efficient and faster for large sparsity penalties, while maintaining similar reconstruction quality. We find this performance improvement increases as the LCA parameters are tuned towards greater representation sparsity. Our study highlights the potential of neuromorphic processors, particularly Loihi 2, in enabling intelligent, autonomous, real-time processing on small robots, satellites where there are strict SWaP (small, lightweight, and low power) requirements. By demonstrating the superior performance of LCA on Loihi 2 compared to conventional computing device, our study suggests that Loihi 2 could be a valuable tool in advancing these types of applications. Overall, our study highlights the potential of neuromorphic processors for efficient and accurate data processing on resource-constrained devices.

We consider the problem of computing a sparse binary representation of an image. To be precise, given an image and an overcomplete, non-orthonormal basis, we aim to find a sparse binary vector indicating the minimal set of basis vectors that when added together best reconstruct the given input. We formulate this problem with an L2 loss on the reconstruction error, and an L0 (or, equivalently, an L1) loss on the binary vector enforcing sparsity. This yields a so-called Quadratic Unconstrained Binary Optimization (QUBO) problem, whose solution is generally NP-hard to find. The contribution of this work is twofold. First, the method of unsupervised and unnormalized dictionary feature learning for a desired sparsity level to best match the data is presented. Second, the binary sparse coding problem is then solved on the Loihi 1 neuromorphic chip by the use of stochastic networks of neurons to traverse the non-convex energy landscape. The solutions are benchmarked against the classical heuristic simulated annealing. We demonstrate neuromorphic computing is suitable for sampling low energy solutions of binary sparse coding QUBO models, and although Loihi 1 is capable of sampling very sparse solutions of the QUBO models, there needs to be improvement in the implementation in order to be competitive with simulated annealing.

Sparse coding has long been thought of as a model of the biological visual system, yet previous approaches have not employed it as a method to model the activity of individual neurons in response to arbitrary images. Here, we present a novel model of primary cortical neurons based on a biologically-plausible sparse coding model termed the locally-competitive algorithm (LCA). Our hybrid LCA-CNN model, or LCANet, is trained on a selfsupervised objective using a standard image dataset and regression models are trained to predict neural activity based on a modern neurophysiological dataset containing the responses of hundreds of neurons to natural image stimuli. Our novel sparse coding model better represents the computations performed by biological neurons and is significantly more interpretable than previous models.