Highlights
- NVIDIA Chat with RTX works offline and can feed on your data and instructions for personalised chat support.
- It works on Windows RTX machines only for now.
- Find the Chat with RTX pros and cons below.
NVIDIA, the world’s leading AI chipmaker and also the 3rd most valuable US company has released a new chatbot called ‘Chat with RTX’. It is free, works offline on Windows RTX PC or workstation, and is a tech demo for now. However, the main sell is that you can feed your data to this chatbot and get answers to your queries based on that. Let’s see how it works, what are its system requirements and everything else you need to know.
What is NVIDIA Chat with RTX, how it works
- NVIDIA calls it your personalised AI chatbot as it will be connected to your content like notes, docs, videos, etc.
- It runs locally on a Windows machine. You can choose between LLMs downloaded on your PC like Mistral or Llama. The processing as well as the data in and out of the process stays on the machine.
- The chatbot’s UI consist of 2 sections. While on one side you select the LLM for processing, you can connect the personal data or link in the other part. You can upload files in formats like .txt, .doc/.docx, .pdf, and .xml.
- In the backend, the tool employs the NVIDIA RTX GPU on your device to do tasks like “retrieval-augmented generation (RAG), TensorRT-LLM, and RTX acceleration” to produce the result. Here are more details on what GPU you precisely need for this to work:
NVIDIA Chat with RTX system requirements
- GPU: The PC or workstation should be running on NVIDIA RTX 30-series or higher graphics processor.
- Memory: The GPU should have at least 8GB of VRAM.
- Software: The device must have Windows 10 and above. Ensure all the latest NVIDIA GPU drivers are installed on your system.
Also Read
NVIDIA Chat with RTX features and concerns
| Pros | Cons |
| The NVIDIA Chat with RTX processes everything locally, so you can consider it privacy-friendly. | NVIDIA admits the chatbot doesn’t retain context and so it fails in long conversations as you can’t ask follow-up queries. |
| It is better for distilling facts out of a document or a bunch of documents. | It could falter if you ask it to give you a summary of document(s). |
| It is free and offers a personalised experience. | Hardware requirements can be a barrier for some users. |
Since it is a tech demo, it would be rough around the edges for now. However, it appears to be a nice Edge AI application that could streamline work and learning for some folks.
