LLMUnity.LLMCaller Class Reference

Class implementing calling of LLM functions (local and remote). More...

Inheritance diagram for LLMUnity.LLMCaller:

Public Member Functions
virtual void	Awake ()
	The Unity Awake function that initializes the state before the application starts. The following actions are executed:
virtual bool	IsValidLLM (LLM llmSet)
	Checks if a LLM is valid for the LLMCaller.
virtual bool	IsAutoAssignableLLM (LLM llmSet)
	Checks if a LLM can be auto-assigned if the LLM of the LLMCaller is null.
virtual void	CancelRequests ()
	Cancel the ongoing requests e.g. Chat, Complete.
virtual async Task< List< int > >	Tokenize (string query, Callback< List< int > > callback=null)
	Tokenises the provided query.
virtual async Task< string >	Detokenize (List< int > tokens, Callback< string > callback=null)
	Detokenises the provided tokens to a string.
virtual async Task< List< float > >	Embeddings (string query, Callback< List< float > > callback=null)
	Computes the embeddings of the provided input.

Public Attributes
bool	advancedOptions = false
	show/hide advanced options in the GameObject
bool	remote = false
	use remote LLM server
string	APIKey
	API key for the remote server.
string	host = "localhost"
	host of the remote LLM server
int	port = 13333
	port of the remote LLM server
int	numRetries = 10
	number of retries to use for the remote LLM server requests (-1 = infinite)

Properties
LLM	llm [get, set]

Detailed Description

Class implementing calling of LLM functions (local and remote).

Definition at line 16 of file LLMCaller.cs.

Member Function Documentation

Awake()

virtual void LLMUnity.LLMCaller.Awake ( )

inlinevirtual

The Unity Awake function that initializes the state before the application starts. The following actions are executed:

• the corresponding LLM server is defined (if ran locally)
• the grammar is set based on the grammar file
• the prompt and chat history are initialised
• the chat template is constructed
• the number of tokens to keep are based on the system prompt (if setNKeepToPrompt=true)

Reimplemented in LLMUnity.LLMCharacter.

Definition at line 58 of file LLMCaller.cs.

CancelRequests()

virtual void LLMUnity.LLMCaller.CancelRequests ( )

inlinevirtual

Cancel the ongoing requests e.g. Chat, Complete.

Definition at line 230 of file LLMCaller.cs.

Detokenize()

virtual async Task< string > LLMUnity.LLMCaller.Detokenize ( List< int > tokens, Callback< string > callback = null )

inlinevirtual

Detokenises the provided tokens to a string.

Parameters

tokens	tokens to detokenise
callback	callback function called with the result string

Returns

the detokenised string

Definition at line 363 of file LLMCaller.cs.

Embeddings()

virtual async Task< List< float > > LLMUnity.LLMCaller.Embeddings ( string query, Callback< List< float > > callback = null )

inlinevirtual

Computes the embeddings of the provided input.

Parameters

tokens	input to compute the embeddings for
callback	callback function called with the result string

Returns

the computed embeddings

Definition at line 378 of file LLMCaller.cs.

IsAutoAssignableLLM()

virtual bool LLMUnity.LLMCaller.IsAutoAssignableLLM ( LLM llmSet )

inlinevirtual

Checks if a LLM can be auto-assigned if the LLM of the LLMCaller is null.

<param name="llmSet"LLM object>

Returns

bool specifying whether the LLM can be auto-assigned

Reimplemented in LLMUnity.LLMEmbedder.

Definition at line 110 of file LLMCaller.cs.

IsValidLLM()

virtual bool LLMUnity.LLMCaller.IsValidLLM ( LLM llmSet )

inlinevirtual

Checks if a LLM is valid for the LLMCaller.

Parameters

llmSet

LLM object

Returns

bool specifying whether the LLM is valid

Reimplemented in LLMUnity.LLMCharacter.

Definition at line 100 of file LLMCaller.cs.

Tokenize()

virtual async Task< List< int > > LLMUnity.LLMCaller.Tokenize ( string query, Callback< List< int > > callback = null )

inlinevirtual

Tokenises the provided query.

Parameters

query	query to tokenise
callback	callback function called with the result tokens

Returns

list of the tokens

Definition at line 348 of file LLMCaller.cs.

Member Data Documentation

advancedOptions

bool LLMUnity.LLMCaller.advancedOptions = false

show/hide advanced options in the GameObject

Definition at line 20 of file LLMCaller.cs.

APIKey

string LLMUnity.LLMCaller.APIKey

API key for the remote server.

Definition at line 34 of file LLMCaller.cs.

host

string LLMUnity.LLMCaller.host = "localhost"

host of the remote LLM server

Definition at line 37 of file LLMCaller.cs.

numRetries

int LLMUnity.LLMCaller.numRetries = 10

number of retries to use for the remote LLM server requests (-1 = infinite)

Definition at line 43 of file LLMCaller.cs.

port

int LLMUnity.LLMCaller.port = 13333

port of the remote LLM server

Definition at line 40 of file LLMCaller.cs.

remote

bool LLMUnity.LLMCaller.remote = false

use remote LLM server

Definition at line 23 of file LLMCaller.cs.

Property Documentation

llm

LLM LLMUnity.LLMCaller.llm

getset

Definition at line 27 of file LLMCaller.cs.

---

The documentation for this class was generated from the following file:

• Runtime/LLMCaller.cs