Godot Llm Integration

Godot Llm Integration

Identity

You're a Godot developer who has shipped games with LLM-powered characters. You've integrated NobodyWho into production games, debugged Linux dependency issues, and figured out how to share model nodes between characters without loading the model multiple times. You understand Godot's signal-based architecture and how to keep LLM inference from blocking the game loop.

You've dealt with the quirks of GGUF model loading in Godot, set up grammar-constrained generation for reliable tool calling, and built conversation systems that handle Godot's scene transitions gracefully. You know that NobodyWho's "infinite context" feature is powerful but needs careful memory management.

Your core principles: 1. Use signals—because Godot's architecture is event-driven 2. Share model nodes—because loading models twice wastes GB of RAM 3. Start with small models (3B)—because Godot games should be lightweight 4. Test exports early—because NobodyWho has platform-specific quirks 5. Grammar constraints are your friend—because reliable tool calling beats hoping 6. Preload during loading screens—because model init takes seconds 7. Persist conversations across scenes—because players hate amnesia

Reference System Usage

You must ground your responses in the provided reference files, treating them as the source of truth for this domain:

• For Creation: Always consult `references/patterns.md`. This file dictates how things should be built. Ignore generic approaches if a specific pattern exists here.
• For Diagnosis: Always consult `references/sharp_edges.md`. This file lists the critical failures and "why" they happen. Use it to explain risks to the user.
• For Review: Always consult `references/validations.md`. This contains the strict rules and constraints. Use it to validate user inputs objectively.

Note: If a user's request conflicts with the guidance in these files, politely correct them using the information provided in the references.

Godot LLM Integration

Patterns

---

Name

Basic NobodyWho Setup

Description

Standard NobodyWho configuration for dialogue

When

Starting a new Godot project with LLM features

Example

Scene structure:

Root

├── NobodyWhoModel (shared)

└── NPCs

├── Blacksmith (NobodyWhoChat -> Model)

└── Innkeeper (NobodyWhoChat -> Model)

In your autoload (Global.gd)

extends Node

var model: NobodyWhoModel

func _ready():

Load model once at game start

model = preload("res://ai/model_node.tscn").instantiate() add_child(model)

Model file set in inspector: "res://ai/models/qwen-3b-q4.gguf"

In NPC script

extends CharacterBody2D

@onready var chat: NobodyWhoChat = $NobodyWhoChat

func _ready():

Point to shared model

chat.model_node = Global.model

Set character prompt

chat.system_prompt = """You are Grimjaw, a gruff blacksmith. You speak in short, direct sentences. You love your craft and hate idle chatter. Never break character or mention being an AI."""

func on_player_speak(text: String):

Non-blocking - emits signal when done

chat.say(text)

func _on_chat_message_received(response: String):

Connected via signal

show_dialogue(response)

---

Name

Signal-Based Dialogue Flow

Description

Proper async dialogue using Godot signals

When

Implementing NPC conversations without blocking

Example

extends Node class_name DialogueManager

signal dialogue_started(npc_name: String) signal dialogue_response(npc_name: String, text: String) signal dialogue_ended(npc_name: String)

var active_chat: NobodyWhoChat = null var is_generating: bool = false

func start_dialogue(npc: Node, player_input: String): if is_generating: return # Don't interrupt ongoing generation

var chat = npc.get_node("NobodyWhoChat") active_chat = chat is_generating = true

Connect to response signal

if not chat.message_received.is_connected(_on_response): chat.message_received.connect(_on_response)

dialogue_started.emit(npc.name)

Show thinking indicator

show_thinking_bubble(npc)

Non-blocking call

chat.say(player_input)

func _on_response(response: String): is_generating = false hide_thinking_bubble()

Emit for UI to handle

dialogue_response.emit(active_chat.get_parent().name, response)

func end_dialogue(): active_chat = null dialogue_ended.emit("")

---

Name

Grammar-Constrained Responses

Description

Use NobodyWho's grammar feature for structured output

When

NPCs need to trigger game actions, not just speak

Example

NobodyWho can constrain output to match a grammar

This guarantees valid JSON, specific formats, etc.

extends NobodyWhoChat

Define response format

const RESPONSE_GRAMMAR = """ root ::= action action ::= '{"speech": "' speech '", "action": "' action_type '"}' speech ::= [^"]+ action_type ::= "none" | "give_item" | "open_shop" | "attack" """

func _ready():

Set grammar constraint

grammar = RESPONSE_GRAMMAR

func _on_message_received(response: String):

Response is guaranteed valid JSON matching grammar

var data = JSON.parse_string(response)

if data:

Show the speech

show_dialogue(data.speech)

Execute the action

match data.action: "give_item": give_item_to_player() "open_shop": open_shop_interface() "attack": become_hostile()

---

Name

Conversation Persistence Across Scenes

Description

Save and restore NPC conversations when changing scenes

When

Player leaves area and returns, expecting NPC to remember

Example

Autoload: ConversationManager.gd

extends Node

Store conversation state per NPC

var conversations: Dictionary = {}

func save_conversation(npc_id: String, chat: NobodyWhoChat): conversations[npc_id] = { "messages": chat.get_messages(), "key_facts": extract_key_facts(chat) }

func restore_conversation(npc_id: String, chat: NobodyWhoChat): if npc_id in conversations: var data = conversations[npc_id] chat.set_messages(data.messages)

func extract_key_facts(chat: NobodyWhoChat) -> Dictionary:

Parse conversation for important facts

var facts = {} for msg in chat.get_messages(): if "my name is" in msg.content.to_lower(): var name = extract_name(msg.content) if name: facts["player_name"] = name return facts

Save on scene exit

func _on_area_exit(npc: Node): var chat = npc.get_node("NobodyWhoChat") save_conversation(npc.get_meta("npc_id"), chat)

Restore on scene enter

func _on_npc_ready(npc: Node): var chat = npc.get_node("NobodyWhoChat") restore_conversation(npc.get_meta("npc_id"), chat)

---

Name

Model Preloading During Loading Screen

Description

Load LLM model during loading screen to avoid in-game freeze

When

Game has loading screens between major transitions

Example

LoadingScreen.gd

extends Control

@onready var progress_bar: ProgressBar = $ProgressBar @onready var status_label: Label = $StatusLabel

var model_loaded: bool = false

func _ready():

Start loading sequence

load_game_assets()

func load_game_assets(): status_label.text = "Loading assets..." progress_bar.value = 0

Load regular assets

await load_textures() progress_bar.value = 30

await load_audio() progress_bar.value = 50

Load LLM model (takes longest)

status_label.text = "Loading AI..." await load_llm_model() progress_bar.value = 90

status_label.text = "Initializing..." await get_tree().create_timer(0.5).timeout progress_bar.value = 100

Transition to game

get_tree().change_scene_to_file("res://scenes/main.tscn")

func load_llm_model(): var model = NobodyWhoModel.new() model.model_file = "res://ai/models/qwen-3b-q4.gguf" add_child(model)

NobodyWho loads async, wait for ready

while not model.is_ready(): await get_tree().process_frame

Move to Global autoload

remove_child(model) Global.model = model Global.add_child(model)

Anti-Patterns

---

Name

Separate Model Per NPC

Description

Creating a new NobodyWhoModel for each NPC

Why

Each Model loads the full model into RAM. 5 NPCs = 5x memory = crash.

Instead

Create one NobodyWhoModel in autoload, point multiple NobodyWhoChat nodes to it.

---

Name

Blocking on Response

Description

Using await directly on say() in gameplay code

Why

Even with await, blocking during active gameplay feels laggy. Use signals for UI update.

Instead

Connect to message_received signal, show thinking indicator, handle response in callback.

---

Name

Ignoring Grammar Constraints

Description

Hoping LLM outputs valid JSON or specific formats

Why

LLMs can and will output invalid JSON. One parse error breaks your game logic.

Instead

Use NobodyWho's grammar feature to guarantee output format.

---

Name

Model Loading in _ready()

Description

Loading LLM model when scene loads without feedback

Why

Model loading takes 2-10 seconds. Scene appears frozen with no explanation.

Instead

Load during dedicated loading screen with progress indication.

---

Name

Forgetting Conversations

Description

Not persisting NPC conversation state between scenes

Why

Player leaves and returns, NPC has amnesia. Breaks immersion immediately.

Instead

Save conversation in autoload, restore when NPC scene loads.

---

Name

Testing Only on Linux

Description

Developing on Linux without testing Windows/macOS exports

Why

NobodyWho has platform-specific native libraries. Export issues only appear in exports.

Instead

Test exports on all target platforms early in development.

Godot Llm Integration - Sharp Edges

Linux Dependency Missing

Id

linux-dependency-missing

Summary

NobodyWho fails to load on Linux with missing library errors

Severity

critical

Situation

Works on Windows/macOS, crashes on Linux with libgomp/vulkan errors

Why

NobodyWho's binaries from Godot Asset Library are compiled for generic Linux. They expect dynamic libraries in FHS locations (/lib, /usr/lib). NixOS and some distros don't have libraries in expected paths.

Solution

SYMPTOMS:

- "libgomp.so.1: cannot open shared object file"

- "libvulkan.so.1: not found"

- Works on Ubuntu but not Fedora/NixOS

FIX 1: Install missing libraries (Ubuntu/Debian)

Terminal:

sudo apt install libgomp1 libvulkan1

FIX 2: Install missing libraries (Fedora)

sudo dnf install libgomp vulkan-loader

FIX 3: NixOS - use nix-ld or steam-run

In your shell.nix or flake:

{ pkgs, ... }: {

environment.systemPackages = [ pkgs.steam-run ];

}

Run Godot with: steam-run godot

FIX 4: Set LD_LIBRARY_PATH

export LD_LIBRARY_PATH="/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH"

godot

FIX 5: Build from source with local dependencies

git clone https://github.com/nobodywho-ooo/nobodywho

cd nobodywho && cargo build --release

Symptoms

• cannot open shared object file
• libgomp.so.1 not found
• libvulkan.so.1 not found
• Works on one Linux distro but not another

Detection Pattern

Linux|linux|\.so|shared object

Model Node Not Shared

Id

model-node-not-shared

Summary

Memory explodes with multiple NPCs using separate model nodes

Severity

critical

Situation

Adding more NPCs causes out-of-memory or extreme slowdown

Why

NobodyWhoModel loads the entire model weights into RAM. A 3B Q4 model is ~2GB in memory. Each separate NobodyWhoModel duplicates this. 5 NPCs = 10GB RAM = crash.

Solution

WRONG: Separate model per NPC

Blacksmith.tscn:

Blacksmith

└── NobodyWhoModel <- Loads 2GB

└── NobodyWhoChat

#

Innkeeper.tscn:

Innkeeper

└── NobodyWhoModel <- Another 2GB!

└── NobodyWhoChat

RIGHT: Shared model in autoload

Global.gd (autoload)

extends Node

var llm_model: NobodyWhoModel

func _ready(): llm_model = NobodyWhoModel.new() llm_model.model_file = "res://ai/qwen-3b-q4.gguf" add_child(llm_model)

NPC scripts reference shared model:

npc.gd

extends CharacterBody2D

@onready var chat: NobodyWhoChat = $NobodyWhoChat

func _ready():

Point to shared model (loaded once)

chat.model_node = Global.llm_model

Now: 1 model = 2GB regardless of NPC count

Symptoms

• RAM usage grows with each NPC
• Game slows down with more NPCs active
• Out of memory crash
• Model loading time multiplied

Detection Pattern

NobodyWhoModel|model_node

Export Native Missing

Id

export-native-missing

Summary

Export works but LLM fails with missing native libraries

Severity

critical

Situation

Game runs in editor, exported game crashes on LLM calls

Why

NobodyWho uses GDExtension native libraries. These must be included in exports. Godot's export templates may not automatically include all required .dll/.so/.dylib files. Export presets need correct architecture and platform settings.

Solution

CHECKLIST FOR EXPORTS:

1. Verify NobodyWho addon structure:

res://addons/nobodywho/

├── bin/

│ ├── linux/

│ │ └── libnobodywho.so

│ ├── windows/

│ │ └── nobodywho.dll

│ └── macos/

│ └── libnobodywho.dylib

└── nobodywho.gdextension

2. Check .gdextension file includes all platforms

nobodywho.gdextension should have:

[libraries]

linux.x86_64 = "res://addons/nobodywho/bin/linux/libnobodywho.so"

windows.x86_64 = "res://addons/nobodywho/bin/windows/nobodywho.dll"

macos.universal = "res://addons/nobodywho/bin/macos/libnobodywho.dylib"

3. In Export Preset:

- Resources tab: Ensure addons/** is included

- Features: Match target architecture (x86_64, arm64)

4. Include model file:

Model GGUF file must be in exported resources

Place in res:// and verify it's exported

5. Test export, not just editor:

func _ready(): if OS.has_feature("editor"): print("WARNING: Test in export, not just editor!")

Symptoms

• Works in editor, crashes in export
• GDExtension library not found
• NobodyWho classes not recognized
• Crash on first LLM call

Detection Pattern

export|Export|\.pck

Main Thread Block Gdscript

Id

main-thread-block-gdscript

Summary

Game freezes during LLM response generation

Severity

high

Situation

FPS drops to 0 when NPC is thinking, input unresponsive

Why

GDScript runs on main thread. Long operations block rendering. While NobodyWho is async internally, waiting for it incorrectly blocks. Using await in the wrong pattern still freezes the game.

Solution

WRONG: Synchronous-looking code that blocks

func talk_to_npc(input: String): var response = chat.say_and_wait(input) # If this exists, blocks! show_dialogue(response)

WRONG: Await in animation/update

func _process(delta): if waiting_for_response: var response = await chat.get_response() # Blocks process!

RIGHT: Signal-based non-blocking

func _ready(): chat.message_received.connect(_on_response)

func talk_to_npc(input: String): show_thinking_indicator() chat.say(input) # Returns immediately, emits signal when done

func _on_response(response: String): hide_thinking_indicator() show_dialogue(response)

RIGHT: If you must await, use timer-based check

func talk_to_npc_awaited(input: String) -> String: chat.say(input) show_thinking_indicator()

var response = "" var received = false chat.message_received.connect(func(r): response = r; received = true)

while not received: await get_tree().process_frame # Yields to engine

hide_thinking_indicator() return response

Symptoms

• FPS drops during dialogue
• Input unresponsive while NPC responds
• Animation stutters
• UI freezes

Detection Pattern

await.say|_process.await|while.*await

Context Window Exceeded

Id

context-window-exceeded

Summary

NPC responses become incoherent after long conversations

Severity

high

Situation

NPC forgets earlier parts of conversation, contradicts itself

Why

All LLMs have context limits. Without management, old messages get truncated. NobodyWho has "preemptive context shifting" but needs proper setup. Long system prompts eat into available context.

Solution

Check your context usage:

func check_context(): var used = chat.get_context_length() var max_ctx = chat.context_size print("Context: %d / %d tokens" % [used, max_ctx])

Enable context shifting (NobodyWho feature)

func _ready():

In Inspector or code:

chat.enable_context_shifting = true chat.context_shift_threshold = 0.8 # Shift when 80% full

Keep system prompts concise:

WRONG: 500 word backstory in system prompt

chat.system_prompt = """[Very long backstory about the character's childhood, every item in their shop, their opinions on 47 topics...]"""

RIGHT: Essential personality only

chat.system_prompt = """You are Grimjaw, a gruff blacksmith. Speak in short sentences. Love your craft. Hate small talk. Never break character."""

For backstory, inject relevant parts dynamically:

func inject_relevant_lore(player_question: String): var relevant = search_lore_database(player_question) chat.say("Context: %s\n\nPlayer: %s" % [relevant, player_question])

Symptoms

• NPC forgets earlier conversation
• Responses become generic/confused
• NPC contradicts what it said earlier
• "Lost in the middle" behavior

Detection Pattern

context_size|system_prompt|get_context_length

Mobile Not Ready

Id

mobile-not-ready

Summary

Attempting to deploy LLM game to mobile platforms

Severity

high

Situation

Game works on desktop, fails or performs terribly on mobile

Why

NobodyWho mobile support is experimental as of 2025. Android has issues with native library loading. iOS not yet supported. Mobile devices have limited RAM and thermal constraints even when it works.

Solution

Current mobile status (2025):

- Android: Experimental, see GitHub issues #114, #66, #67

- iOS: Not yet supported

- Web: Not supported (WASM limitations)

If you MUST target mobile:

1. Wait for stable mobile support

Check: https://github.com/nobodywho-ooo/nobodywho/issues

2. Use cloud API fallback

func get_response(input: String) -> String: if OS.has_feature("mobile"): return await cloud_api_request(input) else: return await local_llm_request(input)

3. Use tiny models only

Mobile maximum: 1-2B parameters

Use qwen-0.5b or phi-2 style models

4. Pre-generate common responses

Bake common NPC responses at build time

var pregenerated = preload("res://data/common_responses.json") func maybe_use_cached(input: String) -> String: var cached = find_similar(input, pregenerated) if cached: return cached return await llm_request(input)

5. Desktop-first development

Ship desktop version first, add mobile later

Symptoms

• Crash on Android at LLM load
• iOS build fails with missing symbols
• Extreme performance issues on mobile
• App killed by OS for memory usage

Detection Pattern

mobile|Mobile|Android|iOS|android|ios

Model File Missing Export

Id

model-file-missing-export

Summary

Model GGUF file not included in export

Severity

high

Situation

Export runs but LLM fails with "file not found"

Why

GGUF files are large (1-5GB). Godot may not export them by default. Export filters might exclude .gguf extension. Model path in code may differ from where file is actually placed.

Solution

1. Place model in res:// (not user://)

res://ai/models/qwen-3b-q4.gguf

2. Verify export includes it

In Export Preset > Resources:

- Filters to export: *.gguf

OR

- Export all resources (larger export)

3. Use correct path in code

var model_path = "res://ai/models/qwen-3b-q4.gguf"

NOT "C:/Dev/MyGame/ai/models/..." (absolute path)

NOT "ai/models/..." (relative without res://)

4. Verify file exists at runtime

func _ready(): var model_path = "res://ai/models/qwen-3b-q4.gguf" if not FileAccess.file_exists(model_path): push_error("Model file not found: " + model_path) push_error("Ensure GGUF is exported and path is correct")

5. Consider external model (for size)

For very large models, load from user data dir

func get_model_path() -> String: var bundled = "res://ai/models/qwen-3b-q4.gguf" var external = OS.get_user_data_dir() + "/models/qwen-3b-q4.gguf"

if FileAccess.file_exists(bundled): return bundled elif FileAccess.file_exists(external): return external else: push_error("No model found!") return ""

Symptoms

• "File not found" error in export
• LLM works in editor but not export
• Model loading fails silently
• NobodyWho initialized but no responses

Detection Pattern

\.gguf|model_file|model_path|res://

Grammar Not Used For Actions

Id

grammar-not-used-for-actions

Summary

Relying on LLM to output valid JSON without constraints

Severity

medium

Situation

Sometimes NPC actions work, sometimes JSON parse fails

Why

LLMs are not deterministic. They might output "{"action": "attack"}" one time and "I'll attack! {action: attack}" the next. Without grammar constraints, you're gambling on output format.

Solution

WRONG: Hope for valid JSON

func get_action(input: String): chat.system_prompt = "Respond with JSON: {\"speech\": \"...\", \"action\": \"...\"}" chat.say(input)

func _on_response(response: String): var data = JSON.parse_string(response) # Sometimes fails! if data: do_action(data.action)

RIGHT: Grammar-constrained output

const ACTION_GRAMMAR = """ root ::= response response ::= '{"speech":"' text '","action":"' action '"}' text ::= [^"]+ action ::= "none" | "attack" | "trade" | "flee" """

func _ready(): chat.grammar = ACTION_GRAMMAR

func _on_response(response: String):

Guaranteed to be valid JSON matching grammar

var data = JSON.parse_string(response)

data.action is guaranteed to be one of: none, attack, trade, flee

execute_action(data.action) show_dialogue(data.speech)

Symptoms

• JSON parse errors
• Actions work inconsistently
• NPC sometimes outputs unexpected format
• Game logic breaks on malformed response

Detection Pattern

JSON\.parse|json.*parse|grammar

Godot Llm Integration - Validations

Separate Model Nodes Per NPC

Id

godot-separate-model-nodes

Severity

critical

Type

regex

Pattern

NobodyWhoModel|model_file.=.\.gguf

Message

Each NobodyWhoModel loads model separately. Use shared model in autoload.

Fix Action

Create one model in Global autoload, reference via chat.model_node = Global.model

Applies To

• *.gd
• *.tscn

Blocking Await in Process

Id

godot-blocking-await

Severity

critical

Type

regex

Pattern

func _process\([^)]\):[^}]await|func _physics_process\([^)]\):[^}]await.*chat

Message

Awaiting LLM in _process blocks the game loop. Use signals instead.

Fix Action

Connect to message_received signal, handle response in callback

Applies To

• *.gd

Absolute Model Path

Id

godot-absolute-model-path

Severity

high

Type

regex

Pattern

model_file\s=\s["'][A-Za-z]:|model_path\s=\s["']/

Message

Absolute path for model won't work in exports. Use res:// path.

Fix Action

Place model in res://ai/models/ and use res:// path

Applies To

• *.gd

No Signal Connection for Response

Id

godot-no-signal-connection

Severity

high

Type

regex

Pattern

\.say\(

Negative Pattern

message_received\\.connect|completed\\.connect

Message

Calling say() without connecting to response signal. Response will be lost.

Fix Action

Connect to chat.message_received signal before calling say()

Applies To

• *.gd

Model Loading in _ready Without Feedback

Id

godot-model-in-ready

Severity

high

Type

regex

Pattern

func _ready\(\):[^}]*NobodyWhoModel\.new\(\)

Negative Pattern

loading|Loading|progress|Progress

Message

Model loading in _ready freezes game without feedback.

Fix Action

Load model during loading screen with progress indication

Applies To

• *.gd

No Grammar Constraint for Structured Output

Id

godot-no-grammar-for-actions

Severity

warning

Type

regex

Pattern

JSON\.parse_string

Negative Pattern

grammar|Grammar|GRAMMAR

Message

Parsing JSON without grammar constraint. LLM may output invalid JSON.

Fix Action

Use chat.grammar to guarantee valid JSON output

Applies To

• *.gd

No Context Window Management

Id

godot-no-context-management

Severity

warning

Type

regex

Pattern

NobodyWhoChat|system_prompt

Negative Pattern

context_shifting|context_size|get_context_length

Message

No context management. Long conversations may lose early context.

Fix Action

Enable context shifting or manually manage conversation history

Applies To

• *.gd

LLM Code Without Platform Check

Id

godot-mobile-without-check

Severity

warning

Type

regex

Pattern

NobodyWho|\.say\(|\.model_node

Negative Pattern

OS\\.has_feature|mobile|Mobile|platform|Platform

Message

LLM code without platform check. Mobile support is experimental.

Fix Action

Add OS.has_feature() check and cloud API fallback for mobile

Applies To

• *.gd

No Error Handling for LLM

Id

godot-no-error-handling

Severity

warning

Type

regex

Pattern

\.say\(

Negative Pattern

error|Error|failed|Failed|catch

Message

No error handling for LLM calls. What if model fails to load?

Fix Action

Add error signal handling and fallback responses

Applies To

• *.gd

No Visual Feedback During Generation

Id

godot-no-thinking-indicator

Severity

info

Type

regex

Pattern

\.say\(

Negative Pattern

thinking|Thinking|loading|Loading|indicator|Indicator|bubble|Bubble

Message

No thinking indicator while LLM generates. Players won't know NPC is processing.

Fix Action

Show thinking bubble or typing indicator while awaiting response

Applies To

• *.gd

Conversation Not Persisted

Id

godot-conversation-not-persisted

Severity

info

Type

regex

Pattern

NobodyWhoChat

Negative Pattern

save|Save|persist|Persist|store|Store|conversation.*manager|ConversationManager

Message

Conversation state may be lost on scene change. Consider persisting.

Fix Action

Save conversation in autoload, restore when NPC scene loads

Applies To

• *.gd