This is pretty interesting.
NVIDIA Orin Brings Arm and Ampere to the Edge at Hot Chips 34
By
Patrick Kennedy
-
August 23, 2022
NVIDIA Orin For 2022 Self Driving Cars
At Hot Chips 34 we got to see the NVIDIA Orin at a new level of detail during one of the talks.
This is NVIDIA’s Arm-based edge CPU with an integrated Ampere GPU for the edge markets.
Note: We are doing this piece live at HC34 during the presentation so please excuse typos.
NVIDIA Orin Brings Arm and Ampere to the Edge at Hot Chips 34
NVIDIA Orin has 12x Arm Cortex-A78AE cores using Arm v8.2. It incorporates an Ampere GPU, LPDDR5 memory, and has more I/O connectivity.
The real goal of the platform is to bring NVIDIA AI inference to the edge in various form factors from automotive to robotics.
HC 34 NVIDIA Orin Overview
The Arm Cortex-A78AE cores operate at 2.2GHz and are arranged in three clusters each with 2MB of L3 cache per cluster in the 12-core configuration.
HC 34 NVIDIA Orin CPU Complex
These are not official results, but for some perspective, this performance is very high. The SPEC CPU 2017 rate integer at 39.36 is very good. An 80-core 3.0GHz Ampere Altra Arm data center
CPU is 301 and a 64-core AMD Milan-X is
around 440. Again, those are official results compared to Orin that is estimated.
HC 34 NVIDIA Orin CPU Performance
Orin a safety island complex with Arm Cortex-R52 cores.
HC 34 NVIDIA Orin Safety Island
Here are the specs on that. The lockstep pairs is perhaps the most interesting and important part.
HC 34 NVIDIA Orin Safety Island CPU And Memory
Another big feature is that Orin uses Ampere GPU IP. The embedded products skipped Turing so they went from Volta in Xavier to Ampere in Orin.
HC 34 NVIDIA Orin Ampere GPU
This GPU can split into two GPUs using MIG, like higher-power Ampere GPUs. This is a feature of the chip but during the talk NVIDIA said this is not supported in software and that is why we probably had not heard of it before.
HC 34 NVIDIA Orin Ampere GPU 2
NVIDIA has its Deep Learning Accelerator with an emphasis on INT8 for AI inferencing since that is a key capability of this line. Training happens on bigger chips and systems. FP16 was removed in this generation due to power efficiency. DLA is designed for well-understood AI inference models and running at a lower power and lower area overhead. As a result, FP16 was removed in favor of INT8 optimization.
HC 34 NVIDIA Orin Next Gen DLA
Here are the new Orin features:
HC 34 NVIDIA Orin Net Features Compared To Xavier DLA
Something that I did not know before this talk was that there is a multi-Orin topology available. This can happen either with PCIe x4 links or with 10GbE.
HC 34 NVIDIA Orin Multi Orin
There has been a lot of talk about AV1 video encode/ decode. Orin has this along with being able to support functions for cameras and it even has 10GbE.
HC 34 NVIDIA Orin Additional Enhancements
Performance is up with the more modern platform, but the main focus is really on accelerators on a platform like this.
HC 34 NVIDIA Jetson AGX Orin Performance V Xavier
In terms of energy efficiency, the new chips are faster and use more power, but they end up being more efficient.
HC 34 NVIDIA Jetson AGX Orin Performance V Xavier MLPerf
The NVIDIA Jetson Orin is the line that will power robotics and has a 10W-60W TDP range.
HC 34 NVIDIA Jetson AGX Orin And NX Orin
For automotive, there is the Drive Orin Automotive. This is the one that scales to four Orin.
HC 34 NVIDIA Drive Orin Automotive
This is a big step function for NVIDIA and also an area where having more compute and energy efficiency is always welcome.
Final Words
We actually have a video for the NVIDIA Jetson AGX Orin that was going to go live next week but we are probably going to add in a bit from the HC34 presentation. This video was used as a bit of a test for a slightly new angle that took some time (and failures) to get together.
NVIDIA Jetson AGX Orin Developer Kit
Stay tuned for more on this as it is a very exciting platform.
At Hot Chips 34, we got a deeper-dive into NVIDIA Orin, the company's Arm CPU and Ampere GPU AI edge platform
www.servethehome.com