Local AI Guide
FgoGotran Local Guide
Run your own GGUF model on a Windows PC and let FgoGotran on your phone call the local translation service over your trusted LAN.
Purpose and limits
FgoGotran Local is a local-runtime control tool. After you confirm once, it can prepare Python and llama.cpp automatically; GGUF models are always managed by you.
Translation runs locally
Once the model is ready, FGO dialogue is handled by llama-server and the GGUF model on your PC — no cloud translation API key needed.
Large runtime files stay out of Git
The optional Python and llama.cpp downloads are written only to the ignored user_data folder; you can also configure everything manually. The tool never changes drivers, firewalls or routers, and never downloads GGUF or TTS assets.
The first automatic setup needs a network connection and asks for confirmation first; the Python publisher signature and the llama.cpp size and SHA-256 checks must pass. Get GGUF models from trusted publishers yourself — the local endpoint still uses its own API key.
How it connects
The control panel and the phone's translation endpoint are two different addresses.
PC IP:18080http://127.0.0.1:18081It only ever listens on the local machine. Your phone cannot use or open this address.
Quick start
Do these in order the first time, and do not skip “Ready” or the connection test.
Start and check Python
Double-click Start-FgoGotranLocal.cmd. The launcher reuses a 64-bit Python 3.11–3.13; if none is available, you can confirm a signed, SHA-256-verified private Python 3.13.15 download.
The private Python lives in user_data and does not modify your system PATH or file associations.
Prepare a complete llama.cpp
With no existing configuration, you can confirm an automatic download of the official Windows x64 build, verified by size and SHA-256; NVIDIA uses a CUDA build matched to your driver, otherwise a CPU build is used.
You can also press n to skip and choose manually; an existing non-empty path is never overwritten.
Prepare a GGUF translation model
Download an Instruction or Chat GGUF from a trusted publisher that understands Japanese and reliably outputs Chinese, then select the model folder and file in the control panel.
The launcher never downloads, updates, replaces or deletes GGUF models.
Start the control panel
Once the environment is ready, the control panel opens in your browser; keep the command window open while you use it. The complete current profile starts the model automatically.
The control panel defaults to http://127.0.0.1:18081 and is only reachable from this PC.
Fill in the model settings
A successful automatic setup writes the llama-server.exe path; you always set the GGUF model folder yourself, then scan and choose a model.
Save and start the service
Confirm “Trusted LAN + this PC” and the Model ID. If auto-start fails or was turned off, start the service manually on the overview page and wait until the status becomes “Ready”.
Runtime parameters are not hot-reloaded; if the model is already running, restart the service to apply new settings.
Confirm the connection test
Startup checks OpenAI Chat Completions compatibility automatically. You can also tap “Start test” under “Connection test” to re-check.
A pass only means the connection, authentication, format and non-empty output are fine — it does not guarantee translation quality for every FGO line.
Connect the Android app
On the overview page, tap “Show Endpoint” and “Show API Key”, then enter both full values and the visible Model ID into FgoGotran's “Custom / Local AI”, and tap “Test API” and “Apply API settings”.
Choose llama.cpp and GGUF
The runtime, GPU driver, model architecture and chat template must be compatible with each other.
llama.cpp runtime package
- Automatic setup picks an official Windows x64 CUDA build matching the version reported by nvidia-smi.
- If no NVIDIA/CUDA combination matches, it falls back to the official CPU x64 build.
- The main package and the CUDA DLL package from the same release are verified and unpacked together.
- Updates are unpacked into a new folder; the old version is kept until testing passes.
GGUF model
- Choose an Instruction or Chat model; do not use an unaligned Base model.
- Make sure it understands Japanese and reliably outputs the Chinese you selected.
- Make sure llama.cpp supports the model architecture and chat template.
- Download from a trusted publisher and verify the checksum when one is provided.
| Available VRAM | Suggested start | Notes |
|---|---|---|
| 6–8 GB VRAM | 7B/8B Q4 | Good for a first load and speed test |
| 10–12 GB VRAM | 8B–14B Q4 | Real usage still depends on architecture and context |
| 16 GB VRAM | ~14B Q4 | You can also try higher quantization on smaller models |
Model architecture, context size, batch size and other GPU programs all affect real VRAM usage.
Control panel
The page opens at 127.0.0.1:18081 by default and has four tabs.
Overview
Start, restart or stop llama-server, and check the runtime status and current profile summary. Endpoint and API key are hidden by default and can be revealed temporarily.
Model settings
Manage up to 12 profiles, choose llama.cpp and GGUF, and set the port, network scope and inference parameters.
Connection test
Re-check the API key, Model ID, request format and non-empty Chinese output. A compatibility test also runs automatically when the service starts.
System
Check Python, ports, GPU, memory diagnostics and the runtime log; LAN addresses and local paths are hidden by default and can be revealed temporarily.
Keep these defaults for your first test
| Parameter | Default | Guidance |
|---|---|---|
Context Size | 8192 | Reduce this first when VRAM is short; too small can limit longer prompts. |
GPU Layers | 999 | Offload as many layers as possible to the GPU; set to 0 when no GPU is available. |
Batch Size | 512 | Reduce gradually only if memory or VRAM pressure is still too high. |
UBatch Size | 256 | Must be less than or equal to Batch Size. |
CPU Threads | 0 (auto) | Let llama.cpp choose automatically the first time. |
Flash Attention | On | Usually saves resources; try Auto or Off if you hit incompatibility or startup errors. |
Force-disable model thinking | Off | Follows the model by default. Enable only when the model docs explicitly support it and testing calls for it. |
Prompt Cache / Metrics / Slots | On | Keep the defaults for your first test so caching, performance checks and request status work. |
Connect an Android phone
Tap “Show Endpoint” on the overview page and copy the current address instead of assembling the IP and port from memory.
Your phone must not use localhost, 127.0.0.1, 0.0.0.0, the PC hostname or the router address.
- Put the phone and the PC on the same trusted Wi-Fi or LAN.
- Set the current Windows network to “Private network”.
- In the current profile choose “Trusted LAN + this PC”, save, and restart the service.
- Wait until the status becomes “Ready”, tap “Show Endpoint” on the overview page, and open the /health address using the same PC IP and port.
- In FgoGotran's API settings, choose “Custom / Local AI”.
- Tap “Show Endpoint” and “Show API Key” on the overview page, fill in both full values plus the model name, then test and apply the settings.
http://<PC LAN IP>:18080/v1/chat/completionshttp://<PC LAN IP>:18080/healthRuntime status reference
Only “Ready” means the phone can reliably start translation requests right now.
STOPPEDStoppedThe service has not started yet, or has stopped cleanly.
STARTINGStartingChecking the configuration and creating the llama-server process.
LOADINGLoading modelllama-server is running and loading the GGUF and runtime buffers into memory or VRAM.
VERIFYINGChecking compatibilityThe health check passed; now verifying that Chat Completions returns usable text.
RECOVERINGApplying compatibility fallbackForce-disable thinking is incompatible with the current model; restarting once with the model's default behavior.
READYReadyThe model passed the health and compatibility checks, so the phone can connect.
BUSYTranslatingllama-server is handling a request; short queues are normal.
ERRORErrorService startup, model loading or the compatibility check failed; check the status details and runtime log.
Tuning order
Change one category at a time, save, restart, then judge the result.
Close other GPU programs first
Browsers, game recorders and other AI tools may hold VRAM. Free them, then restart the model.
Reduce Context Size first
This is the first adjustment when VRAM is short or loading fails. Change one setting at a time and restart to observe.
Switch to a smaller model or quantization
If it still will not load, choose a smaller or lower-quantization GGUF; that is usually more reliable than blindly changing several low-level parameters.
Then reduce Batch and UBatch
Reduce Batch Size and UBatch Size gradually, keeping UBatch no larger than Batch.
Finally check runtime compatibility
Check whether the llama.cpp build, graphics driver, DLLs, model architecture, chat template and Flash Attention are compatible.
To judge translation quality, test several FGO lines in a row — names, terminology, omitted subjects and longer sentences; do not rely on the built-in compatibility test alone.
Troubleshooting
Check by symptom first, then read the last few lines under “System → Runtime log”.
Python not found
Install 64-bit Python 3.11–3.13. With several Python installs, point FGO_LOCAL_PYTHON at the python.exe you want.
The control panel will not open
Check that the launch command window is still running, then open http://127.0.0.1:18081. If the port is taken, change the control port with FGO_LOCAL_CONTROL_PORT.
llama-server path is invalid
Select the actual file named llama-server.exe, use an absolute path, and keep all DLLs from the same release package.
GGUF not found when scanning
Check that the file ends in .gguf, the model folder is an absolute path, and the model is inside that folder or its subfolders, then scan again.
It exits while loading, or keeps loading
Check “System → Runtime log”. Common causes are insufficient VRAM or RAM, missing DLLs, a runtime/driver mismatch, or an unsupported model architecture.
Compatibility test failed
Make sure the Model ID matches the llama-server alias, then check errors such as authentication, chat template, thinking-only output or empty output. The test times out after 45 seconds.
The phone cannot connect
Confirm the phone and PC are on the same trusted network, the profile uses “Trusted LAN + this PC”, the service is ready, and check the Windows private-network firewall and any VPN.
Nothing changed after saving settings
The model, Model ID, port, network and inference parameters are runtime settings. Restart llama-server after saving.
Security, backup and updates
LAN HTTP is not encrypted in transit, so it is only suitable for a trusted home or private network.
Network security
Allow only private networks in the firewall. Do not use public Wi-Fi, do not disable the whole firewall, and never map port 18080 or 18081 to the internet.
API Key
Do not share the full key. If you suspect a leak, stop the service first, then replace the key and update every phone; the old key stops working immediately.
Configuration backup
Back up user_data before updating or resetting. This folder contains the API key and must not be uploaded, published or sent along with troubleshooting screenshots.
Safe updates
Stop the service first, then unpack the new llama.cpp into a new folder. Remove the old version only after updating the path and passing load and translation tests.
Final checklist
Confirm everything before running FGO translation for long sessions.
- Using 64-bit Python 3.11, 3.12 or 3.13
- llama.cpp comes from a trusted source and is fully extracted; automatic downloads passed the size and SHA-256 checks
- Required DLLs remain next to llama-server.exe
- The GGUF is an Instruction or Chat model you obtained from a trusted publisher that supports Japanese input and Chinese output
- “Trusted LAN + this PC” is selected for the phone connection
- Windows Firewall allows private networks only
- The runtime status has become “Ready”
- The startup compatibility test or a manual connection test has passed
- The phone's API address comes from the overview page and includes /v1/chat/completions
- The model name on the phone exactly matches the Model ID on the PC
- The phone uses the complete new API Key
- Port 18080 or 18081 is not forwarded to the internet