US20240365025 - PROGRAMMABLE EVENT OUTPUT PIXEL
BAE SYSTEMS Information and Electronic Systems Integration Inc
A neuromorphic focal plane array ROIC device for temporal and spatial synchronous and asynchronous image event processing comprising a plurality of pixels, each pixel comprising an input section comprising a Sample and Hold (SH) component; a low offset buffer/comparator section comprising a Switched Capacitor Filter (SCF); and a digital event output section comprising an analog pixel bus whereby temporal and spatial image data are synchronously and asynchronously processed.
View attachment 77473
This document appears to be a
patent application for a
neuromorphic focal plane array (FPA) Read Out Integrated Circuit (ROIC) designed for high-speed, low-power, event-based image processing. Below is an analysis of its
key points, innovations, and applications.
1. Purpose & Innovation
The disclosed invention improves
event-based image processing using
neuromorphic technology applied to
Focal Plane Arrays (FPA). The key innovations include:
- Spatio-temporal event detection: The system can detect changes in both time (temporal) and space (spatial) within an image.
- Neuromorphic processing: Uses principles inspired by biological neural networks to reduce power consumption and increase efficiency.
- High-speed, low-power design: Essential for applications where large amounts of image data must be processed quickly.
- Event-driven architecture: Unlike traditional image sensors that capture entire frames, this system only processes significant "events" (changes in a scene), reducing data bandwidth.
2. Technical Contributions
A. Neuromorphic FPA & Read-Out Integrated Circuit (ROIC)
- The FPA consists of pixels that receive and process electromagnetic radiation.
- A detector circuit in the analog detection layer processes pixel outputs.
- A digital event processing layer refines the event data.
B. Event Processing Mechanism
- Uses asynchronous logarithmic output and synchronous integrated signal output.
- Implements a threshold-based detection system, where a comparator compares pixel outputs to programmable thresholds.
- Includes a Switched Capacitor Filter (SCF) for offset calibration and spatial comparisons.
- Uses Buffered Direct Injection (BDI) for improved signal fidelity.
C. Pixel-Level Processing & Analog Bus (ABUS)
- Each pixel can process and share data with its neighbors via horizontal and vertical switches.
- This allows for distributed event detection and efficient tracking of changes.
- The ABUS enables pixels to communicate, transferring event data without needing full-frame readout.
D. Multi-Mode Operation
- Can switch between different processing modes based on real-time requirements.
- Features programmable sensitivity tuning for different operational scenarios.
3. Applications
The system is
suitable for a wide range of high-performance imaging tasks, including:
- Military & Defense
- Hypersonic detection and tracking (e.g., missile warning systems).
- Naval and aerospace surveillance (e.g., infrared threat detection).
- Industrial & Scientific
- Low-light imaging with reduced power consumption.
- Autonomous vehicle vision systems.
- Medical Imaging
- Could be adapted for high-speed medical diagnostics using neuromorphic vision.
4. Potential Impact
This technology represents a
significant leap forward in
neuromorphic imaging. By integrating
spatial and temporal event detection at the pixel level, it enables
faster, more efficient image processing while reducing
size, weight, and power (SWAP)—a critical factor in defense and aerospace applications.
Conclusion
This patent presents a
novel and sophisticated approach to
neuromorphic image processing, making it highly suitable for
high-speed, low-power, real-time event detection. The
combination of analog and digital processing layers,
adaptive pixel interactions, and
programmable thresholds makes it a
powerful tool for defense, surveillance, and advanced AI-driven imaging systems.
Note the digital neuromorphic processing layer
The US DARPA has selected three teams of researchers to develop event-based (neuromorphic) infrared (IR) camera technologies.
www.airforce-technology.com
FENCE programme manager Whitney Mason said: “Neuromorphic refers to silicon circuits that mimic brain operation; they offer sparse output, low latency, and high energy efficiency.
“Event-based cameras operate under these same principles when dealing with sparse scenes, but currently lack advanced ‘intelligence’ to perform more difficult perception and control tasks.”
Researchers from Raytheon, BAE, and Northrop will work to develop an asynchronous read-out integrated circuit (ROIC) with low-latency and a processing layer that integrates with the ROIC to detect relevant ‘spatial and temporal signals’.
According to DARPA, the ROIC and processing layer will jointly enable an integrated FENCE sensor to operate on less than 1.5W of power.
Mason added: “The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels.”
microelectronicscommons.org
www.purdue.edu
