F.E.N.C.E

[uiux]

https://www.darpa.mil/news-events/2021-07-02

DARPA Announces Research Teams to Develop Intelligent Event-Based Imagers

DARPA today announced that three teams of researchers led by Raytheon, BAE Systems, and Northrop Grumman have been selected to develop event-based infrared (IR) camera technologies under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. Event-based – or neuromorphic – cameras are an emerging class of sensors with demonstrated advantages relative to traditional imagers. These advanced models operate asynchronously and only transmit information about pixels that have changed. This means they produce significantly less data and operate with much lower latency and power.

https://govtribe.com/opportunity/federal-contract-opportunity/fast-event-based-neuromorphic-camera-and-electronics-fence-hr001121s0001

A. Background
The term neuromorphic engineering was coined by Carver Mead1 and refers to the development of silicon circuits that mimic brain operation; its hallmark traits are low latency, sparse output, and extreme energy efficiency. Event-based (or neuromorphic) cameras operate on the principle of sparse output, only responding to changes in the scene, with accompanying low latency (on the order of tens of microseconds) and low power (on the order of milliwatts) for small format cameras in sparse scenes.Event-based imagers are an emerging class of sensors with major demonstrated advantages relative to traditional cameras. Because they operate asynchronously and only transmit data from pixels that have changed, they have been shown to produce over 100x less data in sparse scenes relative to traditional focal plane arrays (FPAs).2 This leads directly to 100x lower latency at 100x lower power. State-of-the-art visible event-based cameras have been developed by iniVation in Switzerland and by Prophesee in France for applications that include autonomous vehicles, robotics, augmented/virtual reality, and video gaming.Despite their inherent advantages, existing event-based cameras are not currently compatible with DoD applications as DoD scenarios are highly cluttered and dynamic.

The FENCE program seeks to develop an integrated event-based infrared (IR) FPA with embedded processing to overcome these challenges. See U//FOUO addendum for more detail.

B. Program Description

The Fast Event-based Neuromorphic Camera and Electronics (FENCE) program seeks to develop and demonstrate a low latency, low power, event-based camera and a new class of signal processing and learning algorithms that use combined spatial and temporal (spatio-temporal) information to enable intelligent sensors for tactical DoD applications. FENCE will develop an infrared neuromorphic imager consistent with military requirements. The sole technical area (TA) will develop an asynchronous read-out integrated circuit (ROIC) capable of very low latency and power operation. It will also develop a low power processing layer that integrates with the ROIC to identify salient spatio-temporal signals. The ROIC and the processing layer together will enable an integrated FENCE sensor to operate at low power (< 1.5 W).

DARPA developing brain-mimicking IR cameras

The DARPA US research and development agency has announced that it is developing ‘event-based’ infra

www.inavateonthenet.net

DARPA developing brain-mimicking IR cameras

The brain-like cameras are being developed by a team of researchers led by Raytheon, BAE Systems and Northrop Grumman. The project is being developed under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program, operating asynchronously and transmitting information about pixels that have changed. This kind of camera will produce less data and operate with lower latency and power.

Get Ready for the Future of Artificial Intelligence in the War Realm

DARPA and several industry partners such as Raytheon Intelligence & Space, Northrop Grumman and BAE Systems are now fast-tracking a technological system engineered to find and transmit only images or pixels that have “changed” in order to pinpoint moments of relevance.

nationalinterest.org

Get Ready for the Future of Artificial Intelligence in the War Realm

DARPA and several industry partners such as Raytheon Intelligence & Space, Northrop Grumman and BAE Systems are now fast-tracking a technological system engineered to find and transmit only images or pixels that have “changed” in order to pinpoint moments of relevance.


---

WIPO - Search International and National Patent Collections

US11063667 - Systems, devices, and methods for optical communication

A technology is described for optical communication. An example of the technology can include receiving an event stream containing indications of independent events detected by pixels in an event camera. An event may be a change in brightness detected by a pixel in the pixel array, and the pixel independently generates an indication of the event in response to detecting the event. The event stream can be demultiplexed into a plurality of communication streams containing related events associated with a plurality of communication sources. The events contained in a communication stream can be aggregated based in part on an event proximity and an event time that associates an event with other events contained in the event stream. The plurality of communication streams can be demodulated to extract optically transmitted information from the plurality of communication streams, which can be sent to a data Consumer

Theoptically-modulated signals cause asynchronous pixel activations to occur and an associated change of brightness to be detected, each of which is encapsulated as an event by associated readout electronics and tagged with a corresponding pixel location and timestamp (e.g., E(x,y,t)). One or more event streams are sent to the event processors (e.g., event processor(s) 104 such as shown in FIG. 1),

US20200249080A1 - Optical information collection system - Google Patents

An optical information collection system includes a neuromorphic sensor to collect optical information from a scene in response to change in photon flux detected at a plurality of photoreceptors of the neuromorphic sensor. A sensor stimulator stimulates a subset of the plurality of...

patents.google.com

US20200249080 - OPTICAL INFORMATION COLLECTION SYSTEM
Applicants
THOMAS EDMUND PETTY
NORTHROP GRUMMAN SYSTEMS CORPORATION

Abstract(EN)
An optical information collection system includes a neuromorphic sensor to collect optical information from a scene in response to change in photon flux detected at a plurality of photoreceptors of the neuromorphic sensor. A sensor stimulator stimulates a subset of the plurality of photoreceptors according to an eye movement pattern in response to a control command. A controller generates the control command that includes instructions to execute the eye movement pattern

 

[SilentioAsx]

Please tell me Akida is at the heart of these sorts of developments !

NDA’s make me nervous. I need to start seeing contracts rolling in. There are so many applications being uncovered already. I can see why Sean appears frantic to get ourselves established asap as the ‘standard’ or ‘go to’ in neuromorphic technology .

 

[uiux]

Xia Receives $1.1M DARPA Grant to Develop High Dynamic Range Cameras : Institute for Applied Life Sciences : UMass Amherst

Qiangfei Xia, professor of electrical and computer engineering, was recently awarded a grant of approximately $1.1 million grant from the Defense Advanced Research Projects Agency (DARPA) under a program to develop a camera that can track fast events more quickly and efficiently by only...

www.umass.edu

Xia Receives $1.1M DARPA Grant to Develop High Dynamic Range Cameras

Xia’s award is part of the multimillion dollar effort led by BAE Systems under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program launched by DARPA. Neuromorphic cameras are an emerging class of sensors that only measure the motion of what they are recording, such as changes in brightness. Each pixel operates independently, producing significantly less data yet operating with much lower latency and at lower power.

The objective of FENCE, according to DARPA(link is external), is to develop and demonstrate the camera and learning algorithms that use combined spatial and temporal (location in space and time) information to enable intelligent sensors for tactical applications ranging from autonomous vehicles and robotics to IR search and tracking.

According to Xia, the UMass team will provide design, development, and fabrication expertise in spatio-temporal algorithms, integrated circuit design, and neural processing. Xia will share the expertise from his Nanodevices and Integrated Systems Lab(link is external) with BAE for the program.

The BAE team will develop an asynchronous read-out integrated circuit with low latency. The read-out and processing layers will enable an integrated FENCE sensor to operate on less than 1.5 watts of power.

---

Memristive crossbar arrays for brain-inspired computing - Nature Materials

Memristive devices show great potential as artificial synapses and neurons, yet brain-inspired computing can be realized only by integrating a large number of these devices into reliable arrays. This Review discusses the challenges in the integration and use in computation of large-scale...

www.nature.com

Three-dimensional memristor circuits as complex neural networks - Nature Electronics

A three-dimensional circuit composed of eight layers of monolithically integrated memristive devices is built and used to implement complex neural networks, demonstrating accurate MNIST classification and effective edge detection in videos.

www.nature.com

 

[uiux]

https://www.darpa.mil/attachments/U_RDTE_MJB_DARPA_PB_2023_APR_2022_FINAL.pdf

Title: Fast Event-based Neuromorphic Camera and Electronics (FENCE)

Description: The Fast Event-based Neuromorphic Camera and Electronics (FENCE) program will develop and demonstrate a low latency, low power event-based infrared (IR) camera to enable intelligent sensors for tactical DoD applications. Event-based imagers are an emerging class of sensors with major demonstrated advantages relative to traditional cameras. State-of-the-art visible event-based cameras have been shown to produce over two orders of magnitude less data in optimal conditions relative to traditional framing cameras because they transmit data only from pixels that have changed. This leads directly to two orders ofm agnitude lower data latency and a commensurate reduction in power consumption. Despite their inherent advantages, existing event-based cameras are not compatible with DoD applications because DoD applications regularly face conditions that are not naturally sparse, where issues such as clutter and noise would cause a large percentage of the event-based pixels to change simultaneously. When this happens, today's event-based cameras do not perform significantly better than traditional cameras.

FENCE will develop an infrared event-based imager consistent with military requirements. FENCE will develop a four-megapixel
asynchronous read-out integrated circuit (ROIC), co-designed with a 3D integrated processor that will intelligently remove noise
and clutter to maintain low power and latency operation even when faced with all of the pixels firing simultaneously. If successful,
this new class of sensors enabled by FENCE will be capable of responding to fast moving targets and discriminating dim targets in
noisy conditions.

FY 2022 Plans:

B. Accomplishments/Planned Programs ($ in Millions) FY 2021 FY 2022 FY 2023
- Conduct ROIC preliminary design review.
- Simulate timing accuracy and power of the ROIC.
- Conduct processor layer preliminary design review.
- Perform initial analysis of relevant system parameters including power and latency.

FY 2023 Plans:

- Measure ROIC power and timing fidelity.
- Conduct critical design review of processor layer.
- Fabricate processor layer in advanced node silicon.
- Measure processing layer power consumption.

FY 2022 to FY 2023 Increase/Decrease Statement:
The FY 2023 decrease reflects the shift from ROIC design and simulations to processing layer design and fabrication.


---

Three-dimensional integrated circuit - Wikipedia

en.m.wikipedia.org

 

[uiux]

Contracts for August 30, 2022

Today's Defense Department contracts valued at $7.5 million or more are now live on Defense.gov.

www.defense.gov

Contracts For Aug. 30, 2022​

DEFENSE ADVANCED RESEARCH PROJECTS AGENCY

Raytheon Co., El Segundo, California, was awarded a $16,271,803 modification (P00003) to previously awarded, cost-plus-fixed-fee contract HR0011-21-C-0134 to exercise the Phase 2 Option for the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. The modification brings the total cumulative face value of the contract to $25,113,974 from $8,842,171. Work will be performed in Goleta, California (56%); El Segundo, California (16%); Cambridge, Massachusetts (13%): McKinney, Texas (10%); Tempe, Arizona (2%); Tewksbury, Massachusetts (2%); and New York, New York (1%), with an expected completion date of June 2024. Fiscal 2022 research, development, test and evaluation funds in the amount of $6,350,000 are being obligated at time of award. The Defense Advanced Research Projects Agency, Arlington, Virginia, is the contracting activity.

Northrop Grumman Systems Corp., Linthicum Heights, Maryland, was awarded an $8,706,668 modification (P00003) to previously awarded cost-plus-fixed-fee contract HR0011-21-C-0132 to exercise the Phase 2 Option for the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. The modification brings the total cumulative face value of the contract to $14,336,501 from $5,629,833. Work will be performed in Linthicum Heights, Maryland (75%); Baltimore, Maryland (13%); San Diego, California (3%); and Stanford, California (9%), with an expected completion date of June 2024. Fiscal 2022 research, development, test and evaluation funds in the amount of $3,300,000 are being obligated at time of award. The Defense Advanced Research Projects Agency, Arlington, Virginia, is the contracting activity.

 

[uiux]

Contracts for August 30, 2022

Today's Defense Department contracts valued at $7.5 million or more are now live on Defense.gov.

www.defense.gov

Contracts For Aug. 30, 2022​

DEFENSE ADVANCED RESEARCH PROJECTS AGENCY

Raytheon Co., El Segundo, California, was awarded a $16,271,803 modification (P00003) to previously awarded, cost-plus-fixed-fee contract HR0011-21-C-0134 to exercise the Phase 2 Option for the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. The modification brings the total cumulative face value of the contract to $25,113,974 from $8,842,171. Work will be performed in Goleta, California (56%); El Segundo, California (16%); Cambridge, Massachusetts (13%): McKinney, Texas (10%); Tempe, Arizona (2%); Tewksbury, Massachusetts (2%); and New York, New York (1%), with an expected completion date of June 2024. Fiscal 2022 research, development, test and evaluation funds in the amount of $6,350,000 are being obligated at time of award. The Defense Advanced Research Projects Agency, Arlington, Virginia, is the contracting activity.

Northrop Grumman Systems Corp., Linthicum Heights, Maryland, was awarded an $8,706,668 modification (P00003) to previously awarded cost-plus-fixed-fee contract HR0011-21-C-0132 to exercise the Phase 2 Option for the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. The modification brings the total cumulative face value of the contract to $14,336,501 from $5,629,833. Work will be performed in Linthicum Heights, Maryland (75%); Baltimore, Maryland (13%); San Diego, California (3%); and Stanford, California (9%), with an expected completion date of June 2024. Fiscal 2022 research, development, test and evaluation funds in the amount of $3,300,000 are being obligated at time of award. The Defense Advanced Research Projects Agency, Arlington, Virginia, is the contracting activity.

So the above funding was to exercise the phase 2 option (green)

I am interested in the processor PDR & functionality simulation (yellow) and the other processor related items in phase2

 

[DollazAndSense]

https://comptroller.defense.gov/Portals/45/Documents/defbudget/fy2024/budget_justification/pdfs/03_RDT_and_E/RDTE_Vol1_DARPA_MasterJustificationBook_PB_2024.pdf

Title: Fast Event-based Neuromorphic Camera and Electronics (FENCE)

Description: The Fast Event-based Neuromorphic Camera and Electronics (FENCE) program will develop and demonstrate a low latency, low power event-based infrared (IR) camera to enable intelligent sensors for tactical DoD applications. Event-based imagers are an emerging class of sensors with major demonstrated advantages relative to traditional cameras. State-of-the-art visible event-based cameras have been shown to produce over two orders of magnitude less data in optimal conditions relative to traditional framing cameras because they transmit data only from pixels that have changed. This leads directly to two orders of magnitude lower data latency and a commensurate reduction in power consumption. Despite their inherent advantages, existing event-based cameras are not compatible with DoD applications because DoD applications regularly face conditions that are not optimal, where issues such as clutter and noise cause a large percentage of the event-based pixels to change simultaneously. When this happens, today's event-based cameras do not perform significantly better than traditional cameras. FENCE will develop an infrared event-based imager consistent with military requirements. FENCE will develop a four-megapixel asynchronous read-out integrated circuit (ROIC), co-designed with a 3D integrated processor that will intelligently remove noise and clutter to maintain low power and latency operation even when faced with all of the pixels firing simultaneously. If successful, this new class of sensors enabled by FENCE will be capable of responding to fast moving targets and discriminating dim targets in noisy conditions.

FY 2023 Plans:
- Measure ROIC power and timing fidelity.
- Conduct critical design review of processor layer.
- Fabricate processor layer in advanced node silicon.
- Measure processing layer power consumption.

FY 2024 Plans:
- Integrate components into full focal plane array (FPA).
- Measure integrated processor layer power consumption.
- Perform initial FPA functionality testing.

FY 2023 to FY 2024 Increase/Decrease Statement: The FY 2024 decrease reflects moving from design and fabrication to integration and testing.