Nvidia SDKs

Nvidia provides many Software Development Kits (SDKs) that simplify any new engineer's interactions with GPUs. It is leaps and bounds above its competitors AMD and Intel in providing resources to hobbyists or individual engineers. (Nevertheless, all three companies actively engage in collaborations with universities and other corporations). SDKs are like coding libraries but designed to specifically suit the capabilities of an Nvidia system.

The JetPack SDK is probably the best place to start when approaching a new Nvidia system. It allows new users to set up their embedded devices with an operating system and proprietary software NVIDIA Jetpack is an SDK package equipped with several tools to assist in the building of AI applications, providing support to the AGX Xavier, TX2, TX1, and Jetson Nano. It is also referred to as the Nvidia SDK Manager.

The Nvidia SDK Manager requires two hardware components. First, the user must have a host computer running Ubuntu 16.04 or 18.04, a working Internet connection, and a Jetson device. The host device must have a minimum of 40GB of free disk space whereas the Jetson must have at least 25GB of free disk space. Once a USB 2.0 connection has been established between the two devices, the Jetson can be easily set up with the host computer. 

The SDK has a GUI which guides the user on setting up the Jetson as well as any additional packages that need to be installed. This is the only way to put an operating system onto the Jetson. The SDK manager can be used later to update the SDK installation on the system.

Next, we will explore some other SDKs that Nvidia provides. These SDKs should give you a general idea of the various high-level applications that can be tackled using a GPU.

Deepstream SDK

The DeepStreak SDK is a computer-vision library that includes AI-based video processing with sensor processing. Deepstream relies on the Gstreamer paradigm for multimedia processing. DeepStream is free of charge and downloadable from the Nvidia website. Be sure to consult the Nvidia website for the various compatibility requirements of each version of DeepStream.

DeepStream can be installed via Jetpack and comes with several demos to get an individual started on coding for machine learning and computer vision applications. On a device that is set up, DeepStream can be installed via Ubuntu command “sudo apt-get”, if it was not installed via the SDK.

Nvidia DeepStream SDK is an important component of NVIDIA Metropolis, which has one of its applications collection vehicle taxes in Jakarta. In the past, it has also been found on Tesla cars. Many of the sample applications associated with this SDK are designed to run on self-driving cars.

Isaac SDK

The Isaac SDK is developed to support the control of mobile robots. It has a structure similar to that of ROS with support for visualization, simulation, and more. The Isaac SDK is fundamentally based on the Isaac Robot Engine with support for a variety of math-planning and visualization algorithms. Simulation is a key stage in robotics design and a critical component to prevent product failures. The SDK package also includes tutorials for several basic packages controlling existing robots, such as the Kaya.  

TensorRT

TensorRT comes with the ONNX (Open Neural network Exchange) parser with support for video-processing and natural language processing [15]. It provides good support for general AI and machine-learning applications. The following diagram indicates how TensorRT functions within a larger system. On the diagram, below it is indicated that Tensorflow takes in one of the common machine-learning libraries and outputs directly to the physical hardware of the embedded system.

Machine Learning Libraries

It must be emphasized, even with the application of GPGPU programming to other fields, the major application of GPU programming remains rooted in the area of machine learning and image processing. This means that an engineer working with a GPU should be familiar, if not fluent, with the common machine libraries used in industry. Tensorflow and PyTorch are among the most popular and widely-used.

Facebook’s PyTorch is an open-source, machine-learning library derived from the Torch, a repository of machine-learning libraries written in Lua. In simple terms, its two top-level functions are access to machine-learning algorithms and a NumPy based interface that supports the GPU-accelerated image processing. Broadly speaking, its API is divided into the following components: torchtext for natural language processing, torchaudio for spoken language processing, torchvision for image processing, and torch for general access to machine-learning algorithms.

Google Brain’s Tensorflow is written C++ with Python bindings. In Summer 2019, Google released Tensorflow2, a major overhaul of the previous version of Tensorflow. Though it has support for Java and Go, the most stable version of Tensorflow2 is the python version. Just as PyTorch implements the Torch interface, Tensorflow relies on the Keras machine-learning interface. Its API is more detailed but also harder to use as a beginner, with separate libraries for audio processing, data handling, debugging, and error checking, as well as a Keras API (tf.keras) for training models.

Both libraries can accomplish the same end-goal, but there are minor differences between the two that can dramatically influence project deadlines, depending on the needs of the individual engineer. PyTorch’s syntax is rooted in NumPy and is widely regarded as having better API documentation and support, but lacking a native data visualizer that Tensorflow provides, meaning that users will have to export data to an alternate visualizer to track the process of their programs. In addition, Tensorflow 2 is relatively new. Unlike other software updates, Tensorflow 2 dramatically overhauled the previous version of Tensorflow [22, 23]. While Tensorflow 2 promises advantages over the previous version, it is young and untested, meaning that programmers that run into issues have relatively fewer sources to consult compared to users of PyTorch.