Nvidia's Next-Level NPCs!

PLUS: GPT-4o Fine-Tuning

Reading time: 5 minutes  

Key Points 

  • Nvidia unveiled its first AI-powered digital human, or AI NPC, at Gamescom 2024, featured in the upcoming game Mecha Break.

  • Mecha Break, a 2025 release, will use Nvidia’s ACE technology for natural NPC interactions and will be available on PC, Xbox, and PlayStation.

  • Nvidia’s ACE technology includes the Nemotron-4 4B NIM, the company’s first small language model, for local AI processing, combined with cloud-based AI for advanced NPC capabilities.

🎮News - Nvidia has unveiled its first AI-powered digital human, also known as an AI NPC, through a demo of the game Mecha Break at Gamescom 2024.

🕹️For context - Mecha Break is a multiplayer mech battle game scheduled for a 2025 release. It will be available on PC, Xbox Series X/S, and PlayStation 5, and will feature Nvidia's ACE (Avatar Cloud Engine) technology, allowing NPCs to engage in natural language conversations with players.

🧩Nvidia ACE explained - Nvidia's ACE technology introduces the Nemotron-4 4B NIM, the company’s first small language model for game characters, designed to enhance conversation quality and reduce response times. In Mecha Break, this language model will run locally on players' GPUs, requiring just 2GB of video memory. For dynamic facial animations, the game will use Nvidia Audio2Face-3D NIM, also running on-device.

Additionally, third-party AI models will be integrated, with OpenAI's Whisper providing speech recognition and Elevenlabs delivering realistic voice generation, both operating in the cloud.

The AI-powered NPCs in Mecha Break will combine on-device processing and cloud-based AI to create responses and voices. The final output will be rendered through Unreal Engine 5, Omniverse RTX, and a Custom Renderer, with ACE's small language model helping to reduce GPU VRAM costs.

🥷🏻How it all works together - In Nvidia's demo of its digital human in Mecha Break, the AI NPC quickly answered questions about missions, mechs, and strategies. In addition, the model can assist with weapon selection and customizing your Striker through voice commands. It also simplifies gameplay by reducing the time spent searching for items and allows for casual conversations, leveraging its extensive knowledge base.

Key Points 

  • OpenAI’s new fine-tuning feature allows corporate clients to customize GPT-4o with their own data.

  • The process requires uploading text-based data to OpenAI's servers, with training taking one to two hours.

👨🏻‍💻News - OpenAI is launching a new feature that enables corporate clients to customize its top model, GPT-4o, using their own data. 

The customization feature, known as fine-tuning, will be available starting today. This capability will allow AI models to be trained with additional information for specific tasks or subjects. For example, a skateboard company could fine-tune an AI model to create a customer-service chatbot that offers detailed advice on skateboard wheels and maintenance.

🤓Important points to note - 

  • To fine-tune a model, customers must upload their data to OpenAI’s servers. 

  • The training process typically takes one to two hours, according to John Allard, an OpenAI software engineer who works on customization. 

  • For now, users will be able to fine-tune the model using only text-based data, not images or other types of content.

🙆🏻‍♀️What else is happening?

👩🏼‍🚒Discover mind-blowing AI tools

  1. OpenTools AI Tools Expert GPT - Find the perfect AI Tool to solve supercharge your workflow. This GPT is connected to our database, so you can ask in depth questions on any AI tool directly in ChatGPT (free w/ ChatGPT)

  2. Boords - A storyboard generator that allows users to create professional storyboards without the need for drawing skills ($25/month)

  3. Flair - An AI design tool that helps users quickly and affordably generate high-quality marketing assets (Free up to 642 designs/month)

  4. Chat With Data - An AI tool that allows users to upload various types of documents such as PDF, Word, CSV, Excel, and SQL Database and interact with them using natural language via an interactive chat interface (Free up to 3 files/day) 

  5. Kaiber - A video generation engine that enables users to create videos from their own images or text description ($10/month)

  6. Glasp YouTube Summarizer - A chrome extension that runs YouTube videos through ChatGPT/ Claude and summarizes them (Free)

  7. RoomGPT - Allows users to take a picture of their room and generate a new version of their room in different themes (Free)

  8. Interactive Mathematics - A platform that provides math lessons and an AI-powered math problem solver to help students improve their math skills ($3.99/month) 

  9. WellyBox - A tool that helps users track and manage their receipts and invoices ($9/month)

How likely is it that you would recommend the OpenTools' newsletter to a friend or colleague?

Login or Subscribe to participate in polls.