Class implementing helper functions for setup and process management. More...

Static Public Member Functions
static void	AddErrorCallBack (Callback< string > callback)
	Add callback function to call for error logs.

static void	RemoveErrorCallBack (Callback< string > callback)
	Remove callback function added for error logs.

static void	ClearErrorCallBacks ()
	Remove all callback function added for error logs.

static int	GetMaxFreqKHz (int cpuId)

static bool	IsSmtCpu (int cpuId)

static int	AndroidGetNumBigCores ()
	Calculates the number of big cores in Android similarly to ncnn (https://github.com/Tencent/ncnn)

static int	AndroidGetNumBigCoresCapacity ()
	Calculates the number of big cores in Android similarly to Unity (https://docs.unity3d.com/2022.3/Documentation/Manual/android-thread-configuration.html)

Static Public Attributes
static string	Version = "v2.3.0"
	LLM for Unity version.

static string	LlamaLibVersion = "v1.1.12"
	LlamaLib version.

static string	LlamaLibReleaseURL = $"https://github.com/undreamai/LlamaLib/releases/download/{LlamaLibVersion}"
	LlamaLib release url.

static string	LlamaLibURL = $"{LlamaLibReleaseURL}/undreamai-{LlamaLibVersion}-llamacpp.zip"
	LlamaLib url.

static string	LlamaLibExtensionURL = $"{LlamaLibReleaseURL}/undreamai-{LlamaLibVersion}-llamacpp-full.zip"
	LlamaLib extension url.

static string	libraryPath = GetAssetPath(Path.GetFileName(LlamaLibURL).Replace(".zip", ""))
	LlamaLib path.

static string	LLMUnityStore = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "LLMUnity")
	LLMnity store path.

static string	modelDownloadPath = Path.Combine(LLMUnityStore, "models")
	Model download path.

static string	LLMManagerPath = GetAssetPath("LLMManager.json")
	Path of file with build information for runtime.

static readonly Dictionary< string,(string, string, string)[]>	modelOptions
	Default models for download.

Detailed Description

Class implementing helper functions for setup and process management.

Definition at line 99 of file LLMUnitySetup.cs.

Member Function Documentation

◆ AddErrorCallBack()

static void LLMUnity.LLMUnitySetup.AddErrorCallBack ( Callback< string > callback )

inlinestatic

Add callback function to call for error logs.

Definition at line 469 of file LLMUnitySetup.cs.

◆ AndroidGetNumBigCores()

static int LLMUnity.LLMUnitySetup.AndroidGetNumBigCores ( )

inlinestatic

Calculates the number of big cores in Android similarly to ncnn (https://github.com/Tencent/ncnn)

Returns

Definition at line 550 of file LLMUnitySetup.cs.

◆ AndroidGetNumBigCoresCapacity()

static int LLMUnity.LLMUnitySetup.AndroidGetNumBigCoresCapacity ( )

inlinestatic

Calculates the number of big cores in Android similarly to Unity (https://docs.unity3d.com/2022.3/Documentation/Manual/android-thread-configuration.html)

Returns

Definition at line 604 of file LLMUnitySetup.cs.

◆ ClearErrorCallBacks()

static void LLMUnity.LLMUnitySetup.ClearErrorCallBacks ( )

inlinestatic

Remove all callback function added for error logs.

Definition at line 481 of file LLMUnitySetup.cs.

◆ GetMaxFreqKHz()

static int LLMUnity.LLMUnitySetup.GetMaxFreqKHz ( int cpuId )

inlinestatic

Definition at line 486 of file LLMUnitySetup.cs.

◆ IsSmtCpu()

static bool LLMUnity.LLMUnitySetup.IsSmtCpu ( int cpuId )

inlinestatic

Definition at line 520 of file LLMUnitySetup.cs.

◆ RemoveErrorCallBack()

static void LLMUnity.LLMUnitySetup.RemoveErrorCallBack ( Callback< string > callback )

inlinestatic

Remove callback function added for error logs.

Definition at line 475 of file LLMUnitySetup.cs.

Member Data Documentation

◆ libraryPath

string LLMUnity.LLMUnitySetup.libraryPath = GetAssetPath(Path.GetFileName(LlamaLibURL).Replace(".zip", ""))

static

LlamaLib path.

Definition at line 113 of file LLMUnitySetup.cs.

◆ LlamaLibExtensionURL

string LLMUnity.LLMUnitySetup.LlamaLibExtensionURL = $"{LlamaLibReleaseURL}/undreamai-{LlamaLibVersion}-llamacpp-full.zip"

static

LlamaLib extension url.

Definition at line 111 of file LLMUnitySetup.cs.

◆ LlamaLibReleaseURL

string LLMUnity.LLMUnitySetup.LlamaLibReleaseURL = $"https://github.com/undreamai/LlamaLib/releases/download/{LlamaLibVersion}"

static

LlamaLib release url.

Definition at line 107 of file LLMUnitySetup.cs.

◆ LlamaLibURL

string LLMUnity.LLMUnitySetup.LlamaLibURL = $"{LlamaLibReleaseURL}/undreamai-{LlamaLibVersion}-llamacpp.zip"

static

LlamaLib url.

Definition at line 109 of file LLMUnitySetup.cs.

◆ LlamaLibVersion

string LLMUnity.LLMUnitySetup.LlamaLibVersion = "v1.1.12"

static

LlamaLib version.

Definition at line 105 of file LLMUnitySetup.cs.

◆ LLMManagerPath

string LLMUnity.LLMUnitySetup.LLMManagerPath = GetAssetPath("LLMManager.json")

static

Path of file with build information for runtime.

Definition at line 119 of file LLMUnitySetup.cs.

◆ LLMUnityStore

string LLMUnity.LLMUnitySetup.LLMUnityStore = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "LLMUnity")

static

LLMnity store path.

Definition at line 115 of file LLMUnitySetup.cs.

◆ modelDownloadPath

string LLMUnity.LLMUnitySetup.modelDownloadPath = Path.Combine(LLMUnityStore, "models")

static

Model download path.

Definition at line 117 of file LLMUnitySetup.cs.

◆ modelOptions

readonly Dictionary<string, (string, string, string)[]> LLMUnity.LLMUnitySetup.modelOptions

static

Initial value:

= new Dictionary<string, (string, string, string)[]>()
        {
            {"Medium models", new(string, string, string)[]
             {
                 ("Llama 3.1 8B", "https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf?download=true", "https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/main/LICENSE"),
                 ("Mistral 7B Instruct v0.2", "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf?download=true", null),
                 ("Gemma 2 9B it", "https://huggingface.co/bartowski/gemma-2-9b-it-GGUF/resolve/main/gemma-2-9b-it-Q4_K_M.gguf?download=true", "https://ai.google.dev/gemma/terms"),
                 ("OpenHermes 2.5 7B", "https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF/resolve/main/openhermes-2.5-mistral-7b.Q4_K_M.gguf?download=true", null),
             }},
            {"Small models", new(string, string, string)[]
             {
                 ("Llama 3.2 3B", "https://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-3b-instruct-q4_k_m.gguf", null),
                 ("Phi 3.5 4B", "https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF/resolve/main/Phi-3.5-mini-instruct-Q4_K_M.gguf", null),
             }},
            {"Tiny models", new(string, string, string)[]
             {
                 ("Llama 3.2 1B", "https://huggingface.co/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-1b-instruct-q4_k_m.gguf", null),
                 ("Qwen 2 0.5B", "https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/resolve/main/qwen2-0_5b-instruct-q4_k_m.gguf?download=true", null),
             }},
            {"RAG models", new(string, string, string)[]
             {
                 ("All MiniLM L12 v2", "https://huggingface.co/leliuga/all-MiniLM-L12-v2-GGUF/resolve/main/all-MiniLM-L12-v2.Q4_K_M.gguf", null),
                 ("BGE large en v1.5", "https://huggingface.co/CompendiumLabs/bge-large-en-v1.5-gguf/resolve/main/bge-large-en-v1.5-q4_k_m.gguf", null),
                 ("BGE base en v1.5", "https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/main/bge-base-en-v1.5-q4_k_m.gguf", null),
                 ("BGE small en v1.5", "https://huggingface.co/CompendiumLabs/bge-small-en-v1.5-gguf/resolve/main/bge-small-en-v1.5-q4_k_m.gguf", null),
             }},
        }

Default models for download.

Definition at line 122 of file LLMUnitySetup.cs.

◆ Version

string LLMUnity.LLMUnitySetup.Version = "v2.3.0"

static

LLM for Unity version.

Definition at line 103 of file LLMUnitySetup.cs.

The documentation for this class was generated from the following file:

Runtime/LLMUnitySetup.cs

Static Public Member Functions

Static Public Attributes

Detailed Description

Member Function Documentation

◆ AddErrorCallBack()

◆ AndroidGetNumBigCores()

◆ AndroidGetNumBigCoresCapacity()

◆ ClearErrorCallBacks()

◆ GetMaxFreqKHz()

◆ IsSmtCpu()

◆ RemoveErrorCallBack()

Member Data Documentation

◆ libraryPath

◆ LlamaLibExtensionURL

◆ LlamaLibReleaseURL

◆ LlamaLibURL

◆ LlamaLibVersion

◆ LLMManagerPath

◆ LLMUnityStore

◆ modelDownloadPath

◆ modelOptions

◆ Version