NVIDIA RTX AI Toolkit is a suite that every developer with RTX AI PCs need
Well made for RTX AI PCs
2 min. read
Published on
Share this article
Improve this guide
Read our disclosure page to find out how can you help MSPoweruser sustain the editorial team Read more
Key notes
- NVIDIA’s RTX AI Toolkit delivers 4x faster, 3x smaller AI models.
- Developers customize models with QLoRa, optimize with TensorRT™, and deploy easily with AI Inference Manager SDK.
- Adobe, Blackmagic Design, and Topaz integrate RTX AI Toolkit to boost AI performance in creative apps.
NVIDIA arrived with a lot of exciting announcements recently. Besides announcing Project G-Assist, a new chatbot that helps you with in-game suggestions and PC settings while you’re playing, the tech giant also announced the NVIDIA RTX AI Toolkit. What is it?
Here’s a good piece of news for app developers. With NVIDIA RTX AI Toolkit, you will soon be able to build app-specific AI models. It’s a new suite that consists of tools and SDKs “for model customization, optimization, and deployment on RTX AI PCs.” There are about 100 million RTX PCs around, and the tech maker is not playing around.
“The AI ecosystem has built hundreds of thousands of open-source models for app developers to leverage, but most models are pretrained for general purposes and built to run in a data center,” the company says in the announcement.
You, as an app developer, can use open-source QLoRa tools to customize pre-trained models, followed by NVIDIA TensorRT for model quantization for faster performance and less RAM consumption.
Besides, you also get the new NVIDIA AI Inference Manager SDK that simplifies the deployment of AI models on PCs, with software partners like Adobe, Blackmagic Design, and Topaz integrating components of the toolkit into their creative apps.
According to NVIDIA, using the new Toolkit can also help reduce the app’s size by up to 3x, improve its performance by up to 4x, and easily deploy it within their apps.
Earlier this year, NVIDIA also launched Chat with RTX, a locally-run AI chatbot that’s perfect for tasks like analyzing documents, summarizing a YouTube video, and more. But, the app itself comes at quite a big size with 40GB and requirements like 8GB of VRAM.
