NVIDIAJetson AGX Orin 64GB Developer Kit

An interesting little computer. Runs an ARM variant of Ubuntu, which is pre installed on the eMMC flash drive. Shared RAM like a Mac, so it will support running fairly large models. It's not completely clear how to go about upgrading it to the latest (you use the software developer kit that is a download from Nvidia, with both a USB connection and Ethernet connection to the computer running the SDK). If you want to run an LLM on it, you will need to reinstall the OS to an NVMe drive (not included).

All I can say is that this tiny cube truly satisfies my needs.Of courxe, it may not be strong enough for your needs, but if, like me, you are doing deep experimentation on machine reading, this one satisfies.

Great for local AI! I have Llama 3.1 70B running on an NVIDIA NeMo container powering the AI of Home Assistant. I no longer need Alexa, or any internet, a total game changer for smart homes. I warn you though this incredibly powerful machine is pretty complicated, making it all work (as a layman) demands true commitment.

I am revising my original review listed below and downgrading my review to one star. NVIDIA gas finally made available a version of their software that ships with Ubuntu 22, but it is not stable. After executing "apt upgrade" video drivers for the Orin lock up the Orin on reboots, making it unusable. I re-imaged my Orin twice and had the same issue. This is not a new problem for NVIDIA, as I observed the forums for th Oring. A company of NVIDIA's stature should be able to ship more reliable software for their hardware. There are far better options for $2K to the Orin, don't waster your money.Original review------------------Most reviewers of hardware drool the Orin hardware specs, and this device has impressive specs. Impressive hardware specs do not matter if there is limited support for the libraries. The NVIDIA Orin is frozen in time in terms of the most important libraries. For example, I use Python 3.11 for most things, but the only version that ships with this product is 3.8. NVIDIA provides versions of Python libraries, like Pytorch, that are needed to take advantage of the Orin GPU, but if you are stuck on Python 3.8, you will be limited in your ability to perform many if not most AI programming tasks. To that end, for support of Large Language Models (LLMs), you are better off running an Intel processor based machine with a GPU board. From a cost standpoint this is certainly cheaper.I realize this device is oriented towards image processing in support of IoT or Robotics projects. If you are processing images from an array of cameras, the Orin is a decent solution. As a platform for running local LLMs look elsewhere.Lastly, buyers need to be aware that if this Orin is to be re-flashed, or re-imaged, you really need to have an Intel based Linux machine, running Ubuntu 20!. I upgraded my Orin using the utility provided for this purpose. It is not easy and something only the most experienced pros can handle, IMHO.

Received the authentic Nvidia Orin AGX, and it runs just fine. Now the real work begins...

This is a very powerful AI system. It's setup was very straight forward and it comes equipped with the full nVidia AI development stack. It runs Ubuntu 20.04 pre installed, and includes Docker. 64gb of main memory is more than sufficient. I added a 4TB SSD for additional storage. In addition to being a powerful AI system, it is also a fully loaded Linux workstation, and the nVidia GPU allows it to run large LLMs. If you are serious about getting involved with state of the art AI development, this system is an excellent choice.

The memory is shared between GPU and CPUs, but at 64 GB, there is plenty of room to run 70B parameter models. It is also great for fine tuning smaller models, something that is cost prohibitive with CSPs. The community behind the software projects for Jetson is really amazing; for customized copilots you can get up and running in a few hours using really high quality containers from the Nvidia community. My experience is based on fitting out the unit with 2TB of NVMe.

Fantastic little boxes. I was able to build out a couple prototype projects using advanced computer vision object models.... but because the developer kits do not come with a base plate, the motherboard and chipsets are (unreasonably) exposed to the world.In my case, the unit was placed inside a carryon piece of luggage to get to a customer demo. I got to the demo, but damage to the circuity rendered the unit unusable.