From ec3681d6fc07eed03d558931c41178ac549b3d99 Mon Sep 17 00:00:00 2001 From: TrueCharts Bot Date: Thu, 11 Jun 2026 04:12:55 +0200 Subject: [PATCH] =?UTF-8?q?feat(local-ai):=20update=20image=20docker.io/lo?= =?UTF-8?q?calai/localai=20v4.3.6=20=E2=86=92=20v4.4.0=20(#49004)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker.io/localai/localai](https://redirect.github.com/mudler/LocalAI) | minor | `d62ab7b` β†’ `78a86bf` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/18710) for more information. Add the preset `:preserveSemverRanges` to your config if you don't want to pin your dependencies. --- ### Release Notes
mudler/LocalAI (docker.io/localai/localai) ### [`v4.4.0`](https://redirect.github.com/mudler/LocalAI/releases/tag/v4.4.0) [Compare Source](https://redirect.github.com/mudler/LocalAI/compare/v4.3.6...v4.4.0) ### πŸŽ‰ LocalAI 4.4.0 Release! πŸš€




LocalAI 4.4.0 is out! This is a big, **multimodal-and-distributed** release. Two brand-new audio backends land - **parakeet.cpp** (NVIDIA NeMo Parakeet ASR) and **CrispASR** (a multi-architecture ASR **and** TTS engine) - alongside native **object detection + segmentation** (`rfdetr-cpp`), **video understanding** in `llama-cpp`, and **LTX-2 video generation** in `stablediffusion-ggml`. Distributed mode grows up: **prefix-cache-aware routing** is on by default, and file transfers become **resumable**. There's a new **intelligent middleware** layer for request routing, PII filtering and cloud-model proxying, a **security hardening** pass that closes a credential-leak class across every outbound HTTP client, an interactive **`local-ai chat`** CLI, **RAG source citations** for agents, and a long run of reasoning / tool-call streaming fixes. *** #### πŸ“Œ TL;DR | Area | Summary | | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | πŸŽ™οΈ **Two new ASR backends** | `parakeet-cpp` (NeMo FastConformer TDT/CTC/RNNT, streaming, word/segment timestamps) and `crispasr` (many ASR architectures **+ TTS** in one binary). | | 🧭 **Intelligent Middleware** | Capability-based model **routing**, **PII** detection/redaction, **cloud-model proxies** + a MITM proxy for subscription-auth Claude Code / Codex. | | πŸ›°οΈ **Distributed v4** | Prefix-cache-aware routing (on by default), **NATS JWT auth + TLS/mTLS**, worker registration-token enforcement, resumable HTTP file transfers, boot-time model prefetch, ds4 layer-split inference. | | πŸŽ₯ **Video, both ways** | Video **input** (understanding) in `llama-cpp` via mtmd, and video **generation** via **LTX-2** in `stablediffusion-ggml`. | | πŸ‘οΈ **Detection + Segmentation** | New native `rfdetr-cpp` backend (RF-DETR), 32 prebuilt GGUFs, bbox + per-detection PNG masks. | | πŸ” **Outbound HTTP hardening** | `pkg/httpclient` refuses cross-host credential-leaking redirects across every outbound client (GHSA-3mj3-57v2-4636). | | πŸ—£οΈ **TTS per-request control** | `instructions` + a generic `params` map plumbed end to end (Qwen3-TTS VoiceDesign / CustomVoice, Chatterbox). | | πŸ’» **`local-ai chat`** | Interactive terminal chat against a running server, with `/models`, `/model`, `/clear`. | | πŸ“š **RAG citations** | Agent answers now append a clickable `Sources:` block from the Knowledge Base. | | 🧠 **Models** | Gemma 4 QAT family + QAT-matched **MTP** speculative-decoding bundles, Ideogram4, LTX-2.3 22B GGUFs. | *** #### πŸš€ New Features & Major Enhancements ##### πŸŽ™οΈ Audio Gets Serious: Two New ASR Backends This release doubles down on speech-to-text with two independent, cgo-less Go backends (purego, `CGO_ENABLED=0`), each shipping a full CI matrix, gallery importer and docs. **`parakeet-cpp` - NVIDIA NeMo Parakeet ([#​10084](https://redirect.github.com/mudler/LocalAI/issues/10084)).** Wraps [parakeet.cpp](https://redirect.github.com/mudler/parakeet.cpp), a C++/ggml port of NeMo Parakeet (FastConformer TDT/CTC/RNNT/hybrid) that matches the upstream PyTorch models on CPU. Text transcription, OpenAI-compatible **word timestamps**, and **cache-aware streaming** (16 kHz PCM chunks, ``/`` utterance boundaries). GGUFs for all 10 Parakeet models Γ— 5 quants ship in [`mudler/parakeet-cpp-gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf). Follow-ups in this cycle made it production-grade: - **Dynamic batching ([#​10112](https://redirect.github.com/mudler/LocalAI/issues/10112))** - concurrent transcription requests are batched for throughput. - **Real, NeMo-faithful segment timestamps ([#​10207](https://redirect.github.com/mudler/LocalAI/issues/10207))** - words are grouped into segments exactly like NeMo's `get_segment_offsets` (sentence-punctuation boundaries by default, opt-in `segment_gap_threshold` silence splitting in encoder frames). Streaming `FinalResult` segments now carry `start`/`end` when the library exposes the ABI v4 JSON entry points. - **`nemotron-3.5-asr` multilingual streaming ([#​10199](https://redirect.github.com/mudler/LocalAI/issues/10199))** + per-request language selection. **`crispasr` - many architectures + TTS in one backend ([#​10099](https://redirect.github.com/mudler/LocalAI/issues/10099)).** Wraps [CrispASR](https://redirect.github.com/CrispStrobe/CrispASR) (a whisper.cpp/ggml fork, MIT) through its session C-ABI. One backend serves **ASR or TTS** depending on the loaded model, with the architecture auto-detected from the GGUF (or forced via `backend:`). The gallery gains **36 `-crispasr` entries (32 ASR + 4 TTS)**: - **ASR** (e2e-verified across Whisper / Parakeet / Moonshine): parakeet, canary, cohere, qwen3, voxtral, granite, fastconformer-ctc, wav2vec2, hubert, data2vec, glm-asr, kyutai-stt, firered-asr, moonshine, mimo-asr, and more. - **TTS** (all four e2e-verified to valid 24 kHz mono WAV): **vibevoice**, **chatterbox**, **qwen3-tts CustomVoice**, **orpheus** - via `backend:` / `codec:` / `speaker:` / `voice:` model options. *** ##### 🧭 Intelligent Middleware: Routing, PII Filtering & Cloud Proxies A new middleware layer ([#​9802](https://redirect.github.com/mudler/LocalAI/issues/9802)) analyzes, routes, filters and transforms chat requests before they hit a model. - **Capability-based routing.** Requests are classified (e.g. via an ArchRouter-style model) and scored across the capabilities they may require, then routed to the smallest model that satisfies them - easy requests go to small specialized models, hard or uncertain ones to larger general-purpose models. Classified embeddings are reused via cosine similarity so similar requests skip re-classification. - **PII filtering.** Private information is detected per-pattern and can be **redacted, rerouted, or blocked**, with a streaming PII filter that preserves a buffered-emit invariant on `/v1/chat/completions`, Anthropic `/v1/messages`, and `/v1/completions`. A per-model PII pattern editor lives in the model config UI. - **Cloud model proxies + MITM.** Cloud models and a MITM proxy can take part in routing/filtering - send easy requests to local models and hard ones to the cloud, and use **Claude Code / Codex subscriptions (OAuth)** through the PII filter via the MITM proxy (subject to provider ToS). Emits `proxy_connect` + `proxy_traffic` audit events and restores its listener from `runtime_settings.json` on restart. Usage stats are recorded end to end and surfaced in REST, the UI, and MCP. Outbound clients used by this path were also the trigger for the security pass below. *** ##### πŸ›°οΈ Distributed Mode v4 Distributed mode keeps maturing across routing, security and resilience. **Prefix-cache-aware routing, on by default ([#​10071](https://redirect.github.com/mudler/LocalAI/issues/10071)).** Routing now biases toward the replica that already holds the relevant KV/prefix cache, as a **load-guarded hint that never routes worse than today's round-robin**. A generic prefix tree (`pkg/radixtree`) maps cumulative prompt-prefix hashes to nodes; `core/services/nodes/prefixcache` turns the rendered prompt into a deterministic xxhash chain and makes a filter-then-score decision (narrow to load-eligible replicas, then prefer the longest-prefix match), feeding a `preferredNodeID` into the existing atomic `SELECT ... FOR UPDATE` pick. Observations sync across frontends over NATS. Round-robin is the floor; disable with `--distributed-prefix-cache=false`. **NATS JWT auth + TLS/mTLS ([#​10159](https://redirect.github.com/mudler/LocalAI/issues/10159)).** Previously anyone with access to the NATS port could publish backend-install messages or agent jobs (an SSRF / accidental-exposure risk). This adds JWT authentication and TLS/mTLS options, with workers acquiring and auto-refreshing their NATS credentials. Complemented by **worker file-transfer registration-token enforcement ([#​10183](https://redirect.github.com/mudler/LocalAI/issues/10183))**. **Resumable file transfers ([#​10109](https://redirect.github.com/mudler/LocalAI/issues/10109)).** Large model GGUFs over flaky/throttled links no longer restart from byte 0. The worker's `PUT /v1/files/` honors `Content-Range` (308/416 resume semantics, `X-Content-SHA256` binding, final-hash verification) and the master-side stager HEAD-probes for the last accepted offset and resumes, switching to an outer time budget (`LOCALAI_FILE_TRANSFER_BUDGET`, default 1h) with exponential backoff. **ds4 layer-split distributed inference ([#​10098](https://redirect.github.com/mudler/LocalAI/issues/10098)).** Manual layer-split inference for the ds4 backend: a **coordinator** owns layers `0:K` and listens; **workers** dial in and own higher ranges, each loading only its slice of the GGUF (a new dependency-free `ds4-worker` binary, driven via `local-ai worker ds4-distributed`). Fully back-compatible when `ds4_role` is absent. **Operational glue.** Boot-time gallery prefetch via `LOCALAI_PREFETCH_MODELS` ([#​10108](https://redirect.github.com/mudler/LocalAI/issues/10108)); a gated `X-LocalAI-Node` response header for attribution ([#​9976](https://redirect.github.com/mudler/LocalAI/issues/9976)); plus fixes: self-heal stale "model not loaded" routing ([#​10181](https://redirect.github.com/mudler/LocalAI/issues/10181)), stage directory-based models to remote nodes ([#​10175](https://redirect.github.com/mudler/LocalAI/issues/10175)), in-flight tracking for non-LLM methods - VAD, diarize, voice ([#​10238](https://redirect.github.com/mudler/LocalAI/issues/10238)), reconciler survives frontend restarts ([#​9981](https://redirect.github.com/mudler/LocalAI/issues/9981)), cross-replica OpCache sync ([#​9983](https://redirect.github.com/mudler/LocalAI/issues/9983)), and the reinstall/upgrade UI no longer sticks on "reinstalling" ([#​10214](https://redirect.github.com/mudler/LocalAI/issues/10214)). *** ##### πŸŽ₯ Video, Both Directions **Video input / understanding in `llama-cpp` ([#​10216](https://redirect.github.com/mudler/LocalAI/issues/10216)).** Video-capable multimodal models (e.g. SmolVLM2-Video) can now be sent a video in a chat request, mirroring the existing image and audio paths. Tracks the upstream mtmd video landing ([ggml-org/llama.cpp#24269](https://redirect.github.com/ggml-org/llama.cpp/issues/24269)); `grpc-server.cpp` forwards `request->videos()` into the mtmd `files` vector on both the template and non-template paths, and the React chat UI accepts `video/*`, renders an inline `
--- ### Configuration πŸ“… **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. β™» **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. πŸ”• **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/renovatebot/renovate). --- charts/stable/local-ai/Chart.yaml | 4 ++-- charts/stable/local-ai/values.yaml | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/charts/stable/local-ai/Chart.yaml b/charts/stable/local-ai/Chart.yaml index e71622a9243..a62899a90d2 100644 --- a/charts/stable/local-ai/Chart.yaml +++ b/charts/stable/local-ai/Chart.yaml @@ -9,7 +9,7 @@ annotations: trueforge.org/min_helm_version: "3.14" trueforge.org/train: stable apiVersion: v2 -appVersion: 4.3.6 +appVersion: 4.4.0 dependencies: - name: common version: 29.3.4 @@ -36,5 +36,5 @@ sources: - https://github.com/trueforge-org/truecharts/tree/master/charts/stable/local-ai - https://hub.docker.com/r/localai/localai type: application -version: 18.6.4 +version: 18.7.0 diff --git a/charts/stable/local-ai/values.yaml b/charts/stable/local-ai/values.yaml index 2e49c6365aa..fb0bb653bfe 100644 --- a/charts/stable/local-ai/values.yaml +++ b/charts/stable/local-ai/values.yaml @@ -2,7 +2,7 @@ image: repository: docker.io/localai/localai pullPolicy: IfNotPresent - tag: v4.3.6@sha256:d62ab7bb1c578febf2a4ea2566c294d5f334bdd822e48a896ee10cca950ec177 + tag: v4.4.0@sha256:78a86bf170f400c36a7739644197c24416fc3832ed8f7bab0abf1a817d57b1c3 ffmpegImage: repository: docker.io/localai/localai pullPolicy: IfNotPresent