Class implementing helper functions for setup and process management. More...

Static Public Member Functions
static void	AddErrorCallBack (Callback< string > callback)
	Add callback function to call for error logs.

static void	RemoveErrorCallBack (Callback< string > callback)
	Remove callback function added for error logs.

static void	ClearErrorCallBacks ()
	Remove all callback function added for error logs.

static int	GetMaxFreqKHz (int cpuId)

static bool	IsSmtCpu (int cpuId)

static int	AndroidGetNumBigCores ()
	Calculates the number of big cores in Android similarly to ncnn (https://github.com/Tencent/ncnn)

static int	AndroidGetNumBigCoresCapacity ()
	Calculates the number of big cores in Android similarly to Unity (https://docs.unity3d.com/2022.3/Documentation/Manual/android-thread-configuration.html)

Static Public Attributes
static string	Version = "v2.4.1"
	LLM for Unity version.

static string	LlamaLibVersion = "v1.2.1"
	LlamaLib version.

static string	LlamaLibReleaseURL = $"https://github.com/undreamai/LlamaLib/releases/download/{LlamaLibVersion}"
	LlamaLib release url.

static string	libraryName = GetLibraryName(LlamaLibVersion)
	LlamaLib name.

static string	libraryPath = GetAssetPath(libraryName)
	LlamaLib path.

static string	LlamaLibURL = $"{LlamaLibReleaseURL}/{libraryName}.zip"
	LlamaLib url.

static string	LlamaLibExtensionURL = $"{LlamaLibReleaseURL}/{libraryName}-full.zip"
	LlamaLib extension url.

static string	LLMUnityStore = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "LLMUnity")
	LLMnity store path.

static string	modelDownloadPath = Path.Combine(LLMUnityStore, "models")
	Model download path.

static string	LLMManagerPath = GetAssetPath("LLMManager.json")
	Path of file with build information for runtime.

static readonly Dictionary< string,(string, string, string)[]>	modelOptions
	Default models for download.

Detailed Description

Class implementing helper functions for setup and process management.

Definition at line 100 of file LLMUnitySetup.cs.

Member Function Documentation

◆ AddErrorCallBack()

static void LLMUnity.LLMUnitySetup.AddErrorCallBack ( Callback< string > callback )

inlinestatic

Add callback function to call for error logs.

Definition at line 500 of file LLMUnitySetup.cs.

◆ AndroidGetNumBigCores()

static int LLMUnity.LLMUnitySetup.AndroidGetNumBigCores ( )

inlinestatic

Calculates the number of big cores in Android similarly to ncnn (https://github.com/Tencent/ncnn)

Returns

Definition at line 581 of file LLMUnitySetup.cs.

◆ AndroidGetNumBigCoresCapacity()

static int LLMUnity.LLMUnitySetup.AndroidGetNumBigCoresCapacity ( )

inlinestatic

Calculates the number of big cores in Android similarly to Unity (https://docs.unity3d.com/2022.3/Documentation/Manual/android-thread-configuration.html)

Returns

Definition at line 635 of file LLMUnitySetup.cs.

◆ ClearErrorCallBacks()

static void LLMUnity.LLMUnitySetup.ClearErrorCallBacks ( )

inlinestatic

Remove all callback function added for error logs.

Definition at line 512 of file LLMUnitySetup.cs.

◆ GetMaxFreqKHz()

static int LLMUnity.LLMUnitySetup.GetMaxFreqKHz ( int cpuId )

inlinestatic

Definition at line 517 of file LLMUnitySetup.cs.

◆ IsSmtCpu()

static bool LLMUnity.LLMUnitySetup.IsSmtCpu ( int cpuId )

inlinestatic

Definition at line 551 of file LLMUnitySetup.cs.

◆ RemoveErrorCallBack()

static void LLMUnity.LLMUnitySetup.RemoveErrorCallBack ( Callback< string > callback )

inlinestatic

Remove callback function added for error logs.

Definition at line 506 of file LLMUnitySetup.cs.

Member Data Documentation

◆ libraryName

string LLMUnity.LLMUnitySetup.libraryName = GetLibraryName(LlamaLibVersion)

static

LlamaLib name.

Definition at line 110 of file LLMUnitySetup.cs.

◆ libraryPath

string LLMUnity.LLMUnitySetup.libraryPath = GetAssetPath(libraryName)

static

LlamaLib path.

Definition at line 112 of file LLMUnitySetup.cs.

◆ LlamaLibExtensionURL

string LLMUnity.LLMUnitySetup.LlamaLibExtensionURL = $"{LlamaLibReleaseURL}/{libraryName}-full.zip"

static

LlamaLib extension url.

Definition at line 116 of file LLMUnitySetup.cs.

◆ LlamaLibReleaseURL

string LLMUnity.LLMUnitySetup.LlamaLibReleaseURL = $"https://github.com/undreamai/LlamaLib/releases/download/{LlamaLibVersion}"

static

LlamaLib release url.

Definition at line 108 of file LLMUnitySetup.cs.

◆ LlamaLibURL

string LLMUnity.LLMUnitySetup.LlamaLibURL = $"{LlamaLibReleaseURL}/{libraryName}.zip"

static

LlamaLib url.

Definition at line 114 of file LLMUnitySetup.cs.

◆ LlamaLibVersion

string LLMUnity.LLMUnitySetup.LlamaLibVersion = "v1.2.1"

static

LlamaLib version.

Definition at line 106 of file LLMUnitySetup.cs.

◆ LLMManagerPath

string LLMUnity.LLMUnitySetup.LLMManagerPath = GetAssetPath("LLMManager.json")

static

Path of file with build information for runtime.

Definition at line 122 of file LLMUnitySetup.cs.

◆ LLMUnityStore

string LLMUnity.LLMUnitySetup.LLMUnityStore = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "LLMUnity")

static

LLMnity store path.

Definition at line 118 of file LLMUnitySetup.cs.

◆ modelDownloadPath

string LLMUnity.LLMUnitySetup.modelDownloadPath = Path.Combine(LLMUnityStore, "models")

static

Model download path.

Definition at line 120 of file LLMUnitySetup.cs.

◆ modelOptions

readonly Dictionary<string, (string, string, string)[]> LLMUnity.LLMUnitySetup.modelOptions

static

Initial value:

= new Dictionary<string, (string, string, string)[]>()
        {
            {"Medium models", new(string, string, string)[]
             {
                 ("Llama 3.1 8B", "https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf?download=true", "https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/main/LICENSE"),
                 ("Mistral 7B Instruct v0.2", "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/resolve/main/mistral-7b-instruct-v0.2.Q4_K_M.gguf?download=true", null),
                 ("Gemma 2 9B it", "https://huggingface.co/bartowski/gemma-2-9b-it-GGUF/resolve/main/gemma-2-9b-it-Q4_K_M.gguf?download=true", "https://ai.google.dev/gemma/terms"),
                 ("OpenHermes 2.5 7B", "https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF/resolve/main/openhermes-2.5-mistral-7b.Q4_K_M.gguf?download=true", null),
             }},
            {"Small models", new(string, string, string)[]
             {
                 ("Llama 3.2 3B", "https://huggingface.co/hugging-quants/Llama-3.2-3B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-3b-instruct-q4_k_m.gguf", null),
                 ("Phi 3.5 4B", "https://huggingface.co/bartowski/Phi-3.5-mini-instruct-GGUF/resolve/main/Phi-3.5-mini-instruct-Q4_K_M.gguf", null),
             }},
            {"Tiny models", new(string, string, string)[]
             {
                 ("Llama 3.2 1B", "https://huggingface.co/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-1b-instruct-q4_k_m.gguf", null),
                 ("Qwen 2 0.5B", "https://huggingface.co/Qwen/Qwen2-0.5B-Instruct-GGUF/resolve/main/qwen2-0_5b-instruct-q4_k_m.gguf?download=true", null),
             }},
            {"RAG models", new(string, string, string)[]
             {
                 ("All MiniLM L12 v2", "https://huggingface.co/leliuga/all-MiniLM-L12-v2-GGUF/resolve/main/all-MiniLM-L12-v2.Q4_K_M.gguf", null),
                 ("BGE large en v1.5", "https://huggingface.co/CompendiumLabs/bge-large-en-v1.5-gguf/resolve/main/bge-large-en-v1.5-q4_k_m.gguf", null),
                 ("BGE base en v1.5", "https://huggingface.co/CompendiumLabs/bge-base-en-v1.5-gguf/resolve/main/bge-base-en-v1.5-q4_k_m.gguf", null),
                 ("BGE small en v1.5", "https://huggingface.co/CompendiumLabs/bge-small-en-v1.5-gguf/resolve/main/bge-small-en-v1.5-q4_k_m.gguf", null),
             }},
        }

Default models for download.

Definition at line 125 of file LLMUnitySetup.cs.

◆ Version

string LLMUnity.LLMUnitySetup.Version = "v2.4.1"

static

LLM for Unity version.

Definition at line 104 of file LLMUnitySetup.cs.

The documentation for this class was generated from the following file:

Runtime/LLMUnitySetup.cs

Static Public Member Functions

Static Public Attributes

Detailed Description

Member Function Documentation

◆ AddErrorCallBack()

◆ AndroidGetNumBigCores()

◆ AndroidGetNumBigCoresCapacity()

◆ ClearErrorCallBacks()

◆ GetMaxFreqKHz()

◆ IsSmtCpu()

◆ RemoveErrorCallBack()

Member Data Documentation

◆ libraryName

◆ libraryPath

◆ LlamaLibExtensionURL

◆ LlamaLibReleaseURL

◆ LlamaLibURL

◆ LlamaLibVersion

◆ LLMManagerPath

◆ LLMUnityStore

◆ modelDownloadPath

◆ modelOptions

◆ Version