nobodywho-ooo/nobodywho?utm_source=trendshift.io: Trending on GitHub
On-device AI for any device. NobodyWho is an inference engine that lets you run LLMs locally and efficiently.
Features
Run locally, offline — no API keys needed or hidden fees
Run any chat LLM — Gemma, Qwen, Mistral and more
Fast, type-safe tool calling — automatically generates structured grammars from your function signatures, no schema writing needed
Multimodal input — provide image and audio information to your LLM
Text-to-speech — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends
Speech-to-text — transcribe audio into text with Whisper
Voice Activity Detection — know when to stop listening and start transcribing with Silero
Model downloading — load models directly from Hugging Face or any URL
Under the Hood
flowchart TD
FL["Flutter"]:::lang
PY["Python"]:::lang
GO["Godot"]:::lang
K["Kotlin"]:::lang
S["Swift"]:::lang
RN["React Native"]:::lang
FRB["flutter_rust_bridge"]:::glue
P3["PyO3"]:::glue
GX["gdext"]:::glue
U["UniFFI"]:::glue
FL --> FRB
PY --> P3
GO --> GX
K --> U
S --> U
RN --> U
CORE["NobodyWho · Rust<br/>chat · templates · grammars · sampling · context shifting"]:::core
FRB --> CORE
P3 --> CORE
GX --> CORE
U --> CORE
CORE --> LCPP["llama.cpp<br/>text · vision · embeddings · reranking"]:::engine
CORE --> ORT["ONNX Runtime<br/>speech-to-text · text-to-speech · VAD"]:::engine
LCPP --> HW["Hardware<br/>Vulkan · Metal · CUDA · GPU · CPU"]:::hw
ORT --> HW
classDef lang fill:#e8eefc,stroke:#5b7bd5,color:#11204a
classDef glue fill:#f3f0fb,stroke:#8b7bd5,color:#2a1f4a
classDef core fill:#eaf6ee,stroke:#4fa46a,color:#0f3b1f
classDef engine fill:#fdf0e3,stroke:#d58f3b,color:#4a2d0b
classDef hw fill:#f0eeea,stroke:#a09a90,color:#3a352e
Loading
GPU-accelerated inference via Vulkan or Metal — runs fast on any OS
Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format
Powered by the wonderful llama.cpp
You can test our inference engine on iOS, Android, Vision Pro and Apple Watch.
Platforms
Binding Install Runs on Documentation
Kotlin Maven Central Desktop, Android docs.nobodywho.ooo/kotlin
Swift SPM macOS, iOS, visionOS, watchOS docs.nobodywho.ooo/swift
React Native / Expo npm Desktop, Android, iOS docs.nobodywho.ooo/react-native
Flutter pub.dev Desktop, Android, iOS docs.nobodywho.ooo/flutter
Python PyPI Desktop docs.nobodywho.ooo/python
Godot AssetLib Desktop, Android docs.nobodywho.ooo/godot
Desktop means Linux, macOS and Windows throughout. Three gaps worth knowing before you start:
Godot has no iOS export. Use the Flutter, React Native or Swift binding on iOS.
Windows ARM64 is not supported yet.
There is no web export. It is tracked in issue #111.
⭐ Useful to you? Star the repo, it's the easiest way to say thanks.
Requirements
Inference uses the GPU where available and CPU where not. The real constraint is memory.
Desktop
Hardware — any 64-bit Linux, macOS or Windows machine. Windows is x86_64 only, there are no ARM64 Windows builds. macOS accelerates through Metal out of the box, Linux and Windows need a GPU driver with Vulkan support and fall back to CPU without one.
Memory — roughly 1.5× the model file in free RAM, or 2× on a machine that's already busy. 8 GB is a comfortable floor for models up to ~2 GB; 16 GB or more above that.
Discrete GPUs — NobodyWho offloads as many layers as fit in free VRAM and runs the rest on the CPU, so a model larger than your VRAM still works, just slower. If barely any of it fits, it skips the GPU and stays on CPU. It uses one GPU, the card with the most free memory.
Mobile
iOS — iPhone 11 or newer, 4 GB RAM or more.
Android — Snapdragon 855 / Adreno 640 / 6 GB RAM or better.
Rule of thumb — the device needs roughly twice the model file size in available RAM. iOS reserves around 2 GB, Android 2 to 4 GB depending on vendor. Models under 1 GB run smoothly on any phone.
Models
Any model in the GGUF format works. Pass a hf:owner/repo:QUANT reference, an HTTPS URL, or a local path anywhere a model path is expected; remote models are downloaded and cached on first use. Pass "auto" to fit one to available memory.
Start with Qwen3 0.6B: at ~330 MB it is small enough for any phone and good enough to tell whether the integration works.
Quick Start
Python pip install nobodywho
from nobodywho import Chat
chat = Chat('hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M') response = chat.ask('What is the capital of Denmark?') print(response.completed()) # The capital of Denmark is Copenhagen.
Python documentation · PyPI
Kotlin
// Android
implementation("ai.nobodywho:nobodywho-android:
// Desktop JVM (Linux, macOS, Windows)
implementation("ai.nobodywho:nobodywho:
Use the version from the Maven Central badge above.
import ai.nobodywho.Chat
val chat = Chat.fromPath( modelPath = "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M" )
val response = chat.ask("What is the capital of Denmark?").completed() println(response) // The capital of Denmark is Copenhagen.
Kotlin documentation
Swift Add via Swift Package Manager:
https://github.com/nobodywho-ooo/nobodywho-swift.git
import NobodyWho
let chat = try await Chat.fromPath( modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M" )
let response = try await chat.ask("What is the capital of Denmark?").completed() print(response) // The capital of Denmark is Copenhagen.
Swift documentation · GitHub · starter app
React Native / Expo
React Native
npm install react-native-nobodywho
Expo
npx expo install react-native-nobodywho
import { Chat } from "react-native-nobodywho";
const chat = await Chat.fromPath({ modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M", });
const msg = await chat.ask("What is the capital of Denmark?").completed(); console.log(msg); // The capital of Denmark is Copenhagen.
RN / Expo documentation · NPM · RN starter app · Expo starter app
Flutter flutter pub add nobodywho
import 'package:nobodywho/nobodywho.dart' as nobodywho;
void main() async { await nobodywho.NobodyWho.init();
final chat = await nobodywho.Chat.fromPath( modelPath: 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M', );
final msg = await chat.ask('What is the capital of Denmark?').completed(); print(msg); // The capital of Denmark is Copenhagen. }
Flutter documentation · pub.dev · starter app
Godot Install NobodyWho from inside the editor:
In Godot 4.5+, open the AssetLib tab and search for NobodyWho.
Download and import it, making sure Ignore asset root is ticked in the import dialogue.
Reload the project.
You can also grab a specific version from the releases page and import the zip the same way.
Godot documentation
Local Server NobodyWho provides an experimental local server that implements the OpenAI Chat Completions API.
Start the server :
uvx --from 'git+https://github.com/nobodywho-ooo/nobodywho.git#subdirectory=nobodywho/server' nobodywho-server --model hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M --name qwen
It listens on http://127.0.0.1:8888 and serves /v1/models and /v1/chat/completions. See the docs for more info.
Documentation
The documentation has everything you might want to know: https://docs.nobodywho.ooo/
Working with a coding agent? Point it at llms.txt or llms-full.txt, or install the NobodyWho skill so your agent can look up the current APIs and documentation:
npx skills add https://github.com/nobodywho-ooo/nobodywho --skill nobodywho
Community
Discord & Matrix — ask us anything
Issues and Discussions — bugs and feature requests
CONTRIBUTING.md — set up the repo and send a PR
CHANGELOG.md · SECURITY.md · CODE_OF_CONDUCT.md
⭐ Star the repo if NobodyWho is useful to you. It is how people find us.
License
NobodyWho is licensed under the EUPL-1.2. You may use it in proprietary and commercial projects, free of charge. There has been some confusion about this, so to be precise:
Linking two programs or linking an existing software with your own work does not – at least under European law – produce a derivative or extend the coverage of the linked software licence to your own work. [1]
If you distribute modified versions of the code in this repo, you must open source those changes.
Feel free to make proprietary projects using NobodyWho, but don't make a proprietary fork of NobodyWho.
Source: https://github.com/nobodywho-ooo/nobodywho?utm_source=trendshift.io
