Unity MonoBehaviour base class for LLM client functionality. Handles both local and remote LLM connections, completion parameters, and provides tokenization, completion, and embedding capabilities. More...

Inheritance diagram for LLMUnity.LLMClient:

[legend]

Public Member Functions
virtual void	Awake ()
	Unity Awake method that validates configuration and assigns local LLM if needed.

virtual async void	Start ()
	Unity Start method that initializes the LLM client connection.

virtual bool	IsAutoAssignableLLM (LLM llmInstance)
	Determines if an LLM instance can be auto-assigned to this client. Override in derived classes to implement specific assignment logic.

virtual void	SetGrammar (string grammarString)
	Sets grammar constraints for structured output generation.

virtual void	LoadGrammar (string path)
	Loads grammar constraints from a file.

virtual async Task< List< int > >	Tokenize (string query, Action< List< int > > callback=null)
	Converts text into a list of token IDs.

virtual async Task< string >	Detokenize (List< int > tokens, Action< string > callback=null)
	Converts token IDs back to text.

virtual async Task< List< float > >	Embeddings (string query, Action< List< float > > callback=null)
	Generates embedding vectors for the input text.

virtual async Task< string >	Completion (string prompt, Action< string > callback=null, Action completionCallback=null, int id_slot=-1)
	Generates text completion.

void	CancelRequest (int id_slot)
	Cancels an active request in the specified slot.

Public Attributes
bool	advancedOptions = false
	Show/hide advanced options in the inspector.

int	numPredict = -1
	Maximum tokens to generate (-1 = unlimited)

bool	cachePrompt = true
	Cache processed prompts to speed up subsequent requests.

int	seed = 0
	Random seed for reproducible generation (0 = random)

float	temperature = 0.2f
	Sampling temperature (0.0 = deterministic, higher = more creative)

int	topK = 40
	Top-k sampling: limit to k most likely tokens (0 = disabled)

float	topP = 0.9f
	Top-p (nucleus) sampling: cumulative probability threshold (1.0 = disabled)

float	minP = 0.05f
	Minimum probability threshold for token selection.

float	repeatPenalty = 1.1f
	Penalty for repeated tokens (1.0 = no penalty)

float	presencePenalty = 0f
	Presence penalty: reduce likelihood of any repeated token (0.0 = disabled)

float	frequencyPenalty = 0f
	Frequency penalty: reduce likelihood based on token frequency (0.0 = disabled)

float	typicalP = 1f
	Locally typical sampling strength (1.0 = disabled)

int	repeatLastN = 64
	Number of recent tokens to consider for repetition penalty (0 = disabled, -1 = context size)

int	mirostat = 0
	Mirostat sampling mode (0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)

float	mirostatTau = 5f
	Mirostat target entropy (tau) - balance between coherence and diversity.

float	mirostatEta = 0.1f
	Mirostat learning rate (eta) - adaptation speed.

int	nProbs = 0
	Include top N token probabilities in response (0 = disabled)

bool	ignoreEos = false
	Ignore end-of-stream token and continue generating.

Properties
bool	remote `[get, set]`
	Whether this client uses a remote server connection.

LLM	llm `[get, set]`
	The local LLM instance (null if using remote)

string	APIKey `[get, set]`
	API key for remote server authentication.

string	host `[get, set]`
	Remote server hostname or IP address.

int	port `[get, set]`
	Remote server port number.

string	grammar `[get, set]`
	Current grammar constraints for output formatting.

Detailed Description

Unity MonoBehaviour base class for LLM client functionality. Handles both local and remote LLM connections, completion parameters, and provides tokenization, completion, and embedding capabilities.

Definition at line 20 of file LLMClient.cs.

Member Function Documentation

◆ Awake()

virtual void LLMUnity.LLMClient.Awake ( )

inlinevirtual

Unity Awake method that validates configuration and assigns local LLM if needed.

Reimplemented in LLMUnity.LLMAgent.

Definition at line 209 of file LLMClient.cs.

◆ CancelRequest()

void LLMUnity.LLMClient.CancelRequest ( int id_slot )

inline

Cancels an active request in the specified slot.

Parameters

id_slot Slot ID of the request to cancel

Definition at line 605 of file LLMClient.cs.

◆ Completion()

virtual async Task< string > LLMUnity.LLMClient.Completion	(	string	prompt,
		Action< string >	callback = null,
		Action	completionCallback = null,
		int	id_slot = -1 )

inlinevirtual

Generates text completion.

Parameters

prompt	Input prompt text
callback	Optional streaming callback for partial responses
completionCallback	Optional callback when completion finishes
id_slot	Slot ID for the request (-1 for auto-assignment)

Returns: Task that returns the generated completion text

Definition at line 579 of file LLMClient.cs.

◆ Detokenize()

virtual async Task< string > LLMUnity.LLMClient.Detokenize	(	List< int >	tokens,
		Action< string >	callback = null )

inlinevirtual

Converts token IDs back to text.

Parameters

tokens	Token IDs to decode
callback	Optional callback to receive the result

Returns: Decoded text

Definition at line 529 of file LLMClient.cs.

◆ Embeddings()

virtual async Task< List< float > > LLMUnity.LLMClient.Embeddings	(	string	query,
		Action< List< float > >	callback = null )

inlinevirtual

Generates embedding vectors for the input text.

Parameters

query	Text to embed
callback	Optional callback to receive the result

Returns: Embedding vector

Definition at line 548 of file LLMClient.cs.

◆ IsAutoAssignableLLM()

virtual bool LLMUnity.LLMClient.IsAutoAssignableLLM ( LLM llmInstance )

inlinevirtual

Determines if an LLM instance can be auto-assigned to this client. Override in derived classes to implement specific assignment logic.

Parameters

llmInstance LLM instance to evaluate

Returns: True if the LLM can be auto-assigned

Reimplemented in LLMUnity.LLMEmbedder.

Definition at line 351 of file LLMClient.cs.

◆ LoadGrammar()

virtual void LLMUnity.LLMClient.LoadGrammar ( string path )

inlinevirtual

Loads grammar constraints from a file.

Parameters

path	Path to grammar file

Definition at line 435 of file LLMClient.cs.

◆ SetGrammar()

virtual void LLMUnity.LLMClient.SetGrammar ( string grammarString )

inlinevirtual

Sets grammar constraints for structured output generation.

Parameters

grammarString Grammar in GBNF or JSON schema format

Definition at line 425 of file LLMClient.cs.

Here is the caller graph for this function:

◆ Start()

virtual async void LLMUnity.LLMClient.Start ( )

inlinevirtual

Unity Start method that initializes the LLM client connection.

Definition at line 223 of file LLMClient.cs.

◆ Tokenize()

virtual async Task< List< int > > LLMUnity.LLMClient.Tokenize	(	string	query,
		Action< List< int > >	callback = null )

inlinevirtual

Converts text into a list of token IDs.

Parameters

query	Text to tokenize
callback	Optional callback to receive the result

Returns: List of token IDs

Definition at line 510 of file LLMClient.cs.

Member Data Documentation

◆ advancedOptions

bool LLMUnity.LLMClient.advancedOptions = false

Show/hide advanced options in the inspector.

Definition at line 25 of file LLMClient.cs.

◆ cachePrompt

bool LLMUnity.LLMClient.cachePrompt = true

Cache processed prompts to speed up subsequent requests.

Definition at line 62 of file LLMClient.cs.

◆ frequencyPenalty

float LLMUnity.LLMClient.frequencyPenalty = 0f

Frequency penalty: reduce likelihood based on token frequency (0.0 = disabled)

Definition at line 94 of file LLMClient.cs.

◆ ignoreEos

bool LLMUnity.LLMClient.ignoreEos = false

Ignore end-of-stream token and continue generating.

Definition at line 122 of file LLMClient.cs.

◆ minP

float LLMUnity.LLMClient.minP = 0.05f

Minimum probability threshold for token selection.

Definition at line 82 of file LLMClient.cs.

◆ mirostat

int LLMUnity.LLMClient.mirostat = 0

Mirostat sampling mode (0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)

Definition at line 106 of file LLMClient.cs.

◆ mirostatEta

float LLMUnity.LLMClient.mirostatEta = 0.1f

Mirostat learning rate (eta) - adaptation speed.

Definition at line 114 of file LLMClient.cs.

◆ mirostatTau

float LLMUnity.LLMClient.mirostatTau = 5f

Mirostat target entropy (tau) - balance between coherence and diversity.

Definition at line 110 of file LLMClient.cs.

◆ nProbs

int LLMUnity.LLMClient.nProbs = 0

Include top N token probabilities in response (0 = disabled)

Definition at line 118 of file LLMClient.cs.

◆ numPredict

int LLMUnity.LLMClient.numPredict = -1

Maximum tokens to generate (-1 = unlimited)

Definition at line 58 of file LLMClient.cs.

◆ presencePenalty

float LLMUnity.LLMClient.presencePenalty = 0f

Presence penalty: reduce likelihood of any repeated token (0.0 = disabled)

Definition at line 90 of file LLMClient.cs.

◆ repeatLastN

int LLMUnity.LLMClient.repeatLastN = 64

Number of recent tokens to consider for repetition penalty (0 = disabled, -1 = context size)

Definition at line 102 of file LLMClient.cs.

◆ repeatPenalty

float LLMUnity.LLMClient.repeatPenalty = 1.1f

Penalty for repeated tokens (1.0 = no penalty)

Definition at line 86 of file LLMClient.cs.

◆ seed

int LLMUnity.LLMClient.seed = 0

Random seed for reproducible generation (0 = random)

Definition at line 66 of file LLMClient.cs.

◆ temperature

float LLMUnity.LLMClient.temperature = 0.2f

Sampling temperature (0.0 = deterministic, higher = more creative)

Definition at line 70 of file LLMClient.cs.

◆ topK

int LLMUnity.LLMClient.topK = 40

Top-k sampling: limit to k most likely tokens (0 = disabled)

Definition at line 74 of file LLMClient.cs.

◆ topP

float LLMUnity.LLMClient.topP = 0.9f

Top-p (nucleus) sampling: cumulative probability threshold (1.0 = disabled)

Definition at line 78 of file LLMClient.cs.

◆ typicalP

float LLMUnity.LLMClient.typicalP = 1f

Locally typical sampling strength (1.0 = disabled)

Definition at line 98 of file LLMClient.cs.

Property Documentation

◆ APIKey

string LLMUnity.LLMClient.APIKey

getset

API key for remote server authentication.

Definition at line 148 of file LLMClient.cs.

◆ grammar

string LLMUnity.LLMClient.grammar

getset

Current grammar constraints for output formatting.

Definition at line 190 of file LLMClient.cs.

◆ host

string LLMUnity.LLMClient.host

getset

Remote server hostname or IP address.

Definition at line 162 of file LLMClient.cs.

◆ llm

LLM LLMUnity.LLMClient.llm

getset

The local LLM instance (null if using remote)

Definition at line 141 of file LLMClient.cs.

◆ port

int LLMUnity.LLMClient.port

getset

Remote server port number.

Definition at line 176 of file LLMClient.cs.

◆ remote

bool LLMUnity.LLMClient.remote

getset

Whether this client uses a remote server connection.

Definition at line 127 of file LLMClient.cs.

The documentation for this class was generated from the following file:

Runtime/LLMClient.cs

Public Member Functions

Public Attributes

Properties

Detailed Description

Member Function Documentation

◆ Awake()

◆ CancelRequest()

◆ Completion()

◆ Detokenize()

◆ Embeddings()

◆ IsAutoAssignableLLM()

◆ LoadGrammar()

◆ SetGrammar()

◆ Start()

◆ Tokenize()

Member Data Documentation

◆ advancedOptions

◆ cachePrompt

◆ frequencyPenalty

◆ ignoreEos

◆ minP

◆ mirostat

◆ mirostatEta

◆ mirostatTau

◆ nProbs

◆ numPredict

◆ presencePenalty

◆ repeatLastN

◆ repeatPenalty

◆ seed

◆ temperature

◆ topK

◆ topP

◆ typicalP

Property Documentation

◆ APIKey

◆ grammar

◆ host

◆ llm

◆ port

◆ remote