TrueCharts Bot
b9eecf1708
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker.io/localai/localai](https://redirect.github.com/mudler/LocalAI) | minor | `6fb3668` → `8c6f107` | --- > [!WARNING] > Some dependencies could not be looked up. Check the [Dependency Dashboard](../issues/18710) for more information. Add the preset `:preserveSemverRanges` to your config if you don't want to pin your dependencies. --- ### Release Notes <details> <summary>mudler/LocalAI (docker.io/localai/localai)</summary> ### [`v4.6.0`](https://redirect.github.com/mudler/LocalAI/releases/tag/v4.6.0) [Compare Source](https://redirect.github.com/mudler/LocalAI/compare/v4.5.6...v4.6.0) ### 🎉 LocalAI 4.6.0 Release! 🚀 <h1 align="center"> <br> <img height="300" src="https://raw.githubusercontent.com/mudler/LocalAI/refs/heads/master/core/http/static/logo.png"> <br> <br> </h1> LocalAI 4.6.0 is out! This is a reliability-focused release: AMD ROCm backends now run on-GPU at full speed, distributed model loads no longer wedge when a worker dies, and realtime sessions warm up predictably. It also brings conversation forking to the built-in chat UI, a Prometheus counter for PII/audit events, and an SSRF fix for the model gallery. **Highlights:** - 🔴 **AMD ROCm runs correctly** - ggml audio backends offload to the GPU, hipBLASLt kernel-tuning data is bundled (no more slow generic kernels), `rocm-vllm` installs the right wheel, and the ASIC ID table is found. - 🎙️ **Predictable realtime** - sessions eagerly warm the whole pipeline (VAD, ASR, LLM, TTS) up front, so the first turn no longer pays per-model cold-start stalls, plus a new `POST /backend/load` API and "Load into memory" UI button. - 🌿 **Forking chat** - retry *any* assistant answer, branch a new chat from any point, duplicate, or copy the whole conversation, directly in the built-in UI. - 🛡️ **Distributed hardening** - a dead worker can no longer pin the model-load advisory lock (the \~15-minute wedge is gone), and orphaned backend workers self-terminate instead of holding VRAM. - 📊 **PII/audit metrics** - PII detections/masks/blocks are exported as a Prometheus counter, so you can alert when the filter stops firing. - 🔒 **Gallery SSRF fix** - `POST /models/apply` config-URL fetches are validated against private/loopback/metadata addresses. Plus idempotent backend installs, tool-calling and reasoning fixes across the vLLM and Python/MLX backends, cloud-proxy compatibility with the newest reasoning models, and the usual set of dependency updates. *** #### 📌 TL;DR | Area | Summary | | --------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 🔴 **AMD ROCm reliability** | ggml audio backends now compile with `-DGGML_HIP=ON` and link HIP (real GPU offload); hipBLASLt `TensileLibrary` data bundled + `HIPBLASLT_TENSILE_LIBPATH` exported; `rocm-vllm` installs from the AMD wheel index on Python 3.12; `amdgpu.ids` symlinked so the ASIC table is found. | | 🎙️ **Realtime warm-up + load API** | Sessions block-warm the full pipeline at start (errors surface up front); new `POST /backend/load` / `POST /v1/backend/load`, a "Load into memory" UI action, and a `load_model` MCP tool. Opt out per pipeline with `disable_warmup: true`. | | 🌿 **Forking chat** | Regenerate any assistant answer (not just the last), branch a new chat from any turn, duplicate a chat, or copy it as Markdown - all client-side in the React UI. | | 🛡️ **Process & distributed lifecycle** | A dead worker no longer pins the per-model PostgreSQL advisory lock (bounded load ceiling + context-scoped `lock_timeout`); backend workers self-terminate on parent death (`LOCALAI_BACKEND_PARENT_WATCH`); the watchdog stops logging optional `Free()` as an error. | | ⚙️ **Idempotent backend installs** | `POST /backends/apply` and the `LOCALAI_EXTERNAL_BACKENDS` boot loop no longer re-pull an already-installed backend unless `force: true`. | | 📊 **PII/audit Prometheus counter** | `localai_pii_events_total{kind,origin,action,direction}` on `/metrics`, complementing the `/api/pii/events` ring buffer. | | 🔒 **Gallery SSRF hardening** | Gallery config URL fetches run through `ValidateExternalURL`, blocking private, loopback, link-local, and cloud-metadata addresses. | | 🧩 **Tool-calling & reasoning fixes** | Non-streaming vLLM tool calls restored; MLX/Python backends decode tool-call arguments for chat templates and split closing-only `</think>` reasoning blocks. | *** #### 🚀 New Features & Major Enhancements ##### 🔴 AMD ROCm backends run correctly on-GPU Four coupled fixes make ROCm/hipBLAS backends actually run on AMD hardware, and at full speed, instead of silently falling back to CPU or slow generic kernels: - **GPU offload for ggml audio backends** ([#​10667](https://redirect.github.com/mudler/LocalAI/issues/10667)): `rocm-qwen3-tts-cpp`, `rocm-omnivoice-cpp`, `acestep-cpp`, and `vibevoice-cpp` were building CPU-only because their Makefiles passed the no-op `-DGGML_HIPBLAS=ON` (upstream ggml only understands `-DGGML_HIP=ON`) and the CMake link loop omitted `hip`. They now use the same hipblas recipe as llama-cpp and link the HIP backend. - **hipBLASLt kernel-tuning data** ([#​10660](https://redirect.github.com/mudler/LocalAI/issues/10660), [#​10672](https://redirect.github.com/mudler/LocalAI/issues/10672)): the packager bundled rocBLAS data but not the parallel hipBLASLt `TensileLibrary_lazy_gfx*.dat` files, so every arch silently used slow kernels and logged `Cannot read "TensileLibrary_lazy_gfx*.dat"`. The data is now bundled and `HIPBLASLT_TENSILE_LIBPATH` is exported by the `llama-cpp` and `turboquant` `run.sh`. - **`rocm-vllm` installs the right wheel** ([#​10642](https://redirect.github.com/mudler/LocalAI/issues/10642), [#​10651](https://redirect.github.com/mudler/LocalAI/issues/10651)): the backend was pulling the CUDA-only PyPI `vllm` (fatal `ModuleNotFoundError: No module named 'vllm'` on AMD). It now pins CPython 3.12 and installs vLLM from the ROCm wheel index (`https://wheels.vllm.ai/rocm/`). - **ASIC ID table found** ([#​10624](https://redirect.github.com/mudler/LocalAI/issues/10624), [#​10627](https://redirect.github.com/mudler/LocalAI/issues/10627)): the compute-only hipblas image lacks `/opt/amdgpu/share/libdrm/amdgpu.ids`, so every model load warned. Ubuntu's `libdrm-common` copy is now symlinked into place. > 🔗 PRs: [#​10667](https://redirect.github.com/mudler/LocalAI/issues/10667), [#​10672](https://redirect.github.com/mudler/LocalAI/issues/10672), [#​10651](https://redirect.github.com/mudler/LocalAI/issues/10651), [#​10627](https://redirect.github.com/mudler/LocalAI/issues/10627) ##### 🎙️ Realtime: eager pipeline warm-up + a load-into-memory API Realtime voice sessions now eagerly and blockingly warm the entire pipeline (VAD, transcription, LLM, TTS, sound detection, voice recognition) at session start instead of lazy-loading each sub-model on first use. The first turn no longer pays per-model cold-start stalls, and model-load errors surface up front at session start (as `model_load_error`) rather than mid-stream. Pipeline sub-models load concurrently, so a session warms in the time of its slowest stage, not the sum, and a failed stage names every broken model in a joined error. This also adds a LocalAI-native `POST /backend/load` (and `/v1/backend/load`), the inverse of `/backend/shutdown`, exposed as a "Load into memory" UI action and a `load_model` MCP admin tool, so admins can pre-warm any model (including full pipelines) on demand. The `--load-to-memory` startup flag now routes through the same engine. Opt out per pipeline with `disable_warmup: true`. > 🔗 PRs: [#​10662](https://redirect.github.com/mudler/LocalAI/issues/10662) ##### 🌿 Forking chat in the built-in UI The React chat UI gains conversation-management tools: regenerate *any* assistant answer (not just the last), branch a new chat from any answer, duplicate a chat into an independent copy, or copy the whole conversation to the clipboard as Markdown. Retrying a mid-conversation answer correctly truncates the conversation before re-asking, both in the DOM and in the request payload (this also fixes a latent stale-closure bug where a mid-conversation retry sent the downstream turns back to the model). All client-side, no backend changes. > 🔗 PRs: [#​10654](https://redirect.github.com/mudler/LocalAI/issues/10654) ##### 🛡️ Sturdier process and distributed lifecycle - **Dead-worker advisory-lock wedge** ([#​10600](https://redirect.github.com/mudler/LocalAI/issues/10600)): a distributed worker going mid-load could pin a per-model PostgreSQL advisory lock and fail every subsequent request to that model with `55P03` for \~15 minutes. The detached load context is now bounded by a model-load ceiling, the install wait honors cancellation via `singleflight.DoChan`, and `lock_timeout` is scoped to the caller's context budget instead of a deployment-global GUC. - **Parent-death safety net** ([#​10639](https://redirect.github.com/mudler/LocalAI/issues/10639)): if LocalAI is `SIGKILL`ed before teardown, spawned backend workers used to get reparented to init and linger, holding VRAM and their port. Each backend now polls its parent PID and self-terminates on reparenting. Configurable via `LOCALAI_BACKEND_PARENT_WATCH` (default on, auto-off on Windows) and `LOCALAI_BACKEND_PARENT_WATCH_INTERVAL` (default `2s`). C++ coverage is llama-cpp for now; Python covers all backends. - **Quieter watchdog** ([#​10602](https://redirect.github.com/mudler/LocalAI/issues/10602), [#​10607](https://redirect.github.com/mudler/LocalAI/issues/10607)): the optional `Free()` RPC returns gRPC `Unimplemented` for many backends and the federation proxy, so the watchdog no longer logs a misleading `Error freeing GPU resources` on eviction. A new `grpcerrors.IsUnimplemented` helper distinguishes it from genuine failures. - **Idempotent backend installs** ([#​10643](https://redirect.github.com/mudler/LocalAI/issues/10643)): `POST /backends/apply` and the `LOCALAI_EXTERNAL_BACKENDS` boot loop no longer re-download and re-extract an already-installed backend on every apply/boot. Pass `"force": true` (the UI's install button still does, doubling as "Reinstall"). > 🔗 PRs: [#​10600](https://redirect.github.com/mudler/LocalAI/issues/10600), [#​10639](https://redirect.github.com/mudler/LocalAI/issues/10639), [#​10607](https://redirect.github.com/mudler/LocalAI/issues/10607), [#​10643](https://redirect.github.com/mudler/LocalAI/issues/10643) ##### 📊 PII/audit events as a Prometheus counter The PII middleware / MITM audit pipeline now emits a single monotonic counter, `localai_pii_events_total{kind, origin, action, direction}`, on `/metrics`, instrumented at the `EventStore.Record` choke point. Labels are cardinality-bounded (no pattern or user IDs). This complements the capacity-bound `/api/pii/events` ring buffer and, crucially, makes silent filter failure alertable: `rate()` on the counter detects that the PII filter stopped firing after a deploy. > 🔗 PRs: [#​10641](https://redirect.github.com/mudler/LocalAI/issues/10641) ##### 🔒 Gallery SSRF hardening `POST /models/apply` with an empty `id` fetches the supplied `url` directly; in a default Docker setup (no API key) any reachable client could probe internal services or cloud-metadata (`169.254.169.254`) and exfiltrate a slice via the job error. Gallery config fetches now run through the existing `ValidateExternalURL` guard (the same one protecting the CORS proxy and media downloads), blocking private, loopback, link-local, unspecified, and metadata addresses. Only plain `http(s)://` is validated; `huggingface://`, `github:`, `oci://`, `ollama://`, and `file://` are untouched. > 🔗 PRs: [#​10673](https://redirect.github.com/mudler/LocalAI/issues/10673) *** #### 🐛 Bug Fixes (recap) - `fix(vllm)`: restore non-streaming tool-call extraction that regressed after [#​10351](https://redirect.github.com/mudler/LocalAI/issues/10351) (a capability flag was mistaken for run state) - [#​10638](https://redirect.github.com/mudler/LocalAI/issues/10638) - `fix(python-backends)`: decode tool-call `arguments` for chat templates (unbreaks MLX/Qwen3.5 agent loops) and split reasoning when a model emits only a closing `</think>` - [#​10658](https://redirect.github.com/mudler/LocalAI/issues/10658) - `fix(cloud-proxy)`: drop `temperature`/`top_p` and send `max_completion_tokens` so routing to the newest reasoning models (Claude Opus 4.x, GPT-5.x) stops 400ing - [#​10640](https://redirect.github.com/mudler/LocalAI/issues/10640) - `fix(config)`: revert defaulting `swa_full:true` for sliding-window-attention models (restores the memory-light reduced KV cache; still available as an explicit per-model opt-in) - [#​10674](https://redirect.github.com/mudler/LocalAI/issues/10674) - `fix(kokoros)`: implement the `AudioTranscriptionLive` trait stub so the backend compiles against the updated proto - [#​10612](https://redirect.github.com/mudler/LocalAI/issues/10612) - `fix(launcher)`: keep the desktop launcher's data/config under `~/.localai` instead of the GUI's working directory - [#​10610](https://redirect.github.com/mudler/LocalAI/issues/10610), [#​10613](https://redirect.github.com/mudler/LocalAI/issues/10613) *** #### 👒 Dependencies Submodule and backend bumps this cycle: - `ggml-org/llama.cpp` x4 - `ikawrakow/ik_llama.cpp` x4 - `CrispStrobe/CrispASR` x4 - `leejet/stable-diffusion.cpp` x3 - `vllm-metal` (darwin) x3 - `ggml-org/whisper.cpp` x2 - `mudler/parakeet.cpp` x1 - `localai-org/privacy-filter.cpp` x1 - `vllm-project/vllm` cu130 wheel to `0.24.0` Plus new gallery models added via the gallery agent ([#​10663](https://redirect.github.com/mudler/LocalAI/issues/10663), [#​10644](https://redirect.github.com/mudler/LocalAI/issues/10644)). *** #### 📖 Documentation - Docs version bump for the release - [#​10614](https://redirect.github.com/mudler/LocalAI/issues/10614) *** #### 🙌 New Contributors - [@​alaningtrump](https://redirect.github.com/alaningtrump) made their first contribution in [#​10657](https://redirect.github.com/mudler/LocalAI/issues/10657) *** **Full Changelog**: <https://github.com/mudler/LocalAI/compare/v4.5.6...v4.6.0> </details> --- ### Configuration 📅 **Schedule**: (UTC) - Branch creation - At any time (no schedule defined) - Automerge - At any time (no schedule defined) 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://redirect.github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMzAuMSIsInVwZGF0ZWRJblZlciI6IjQzLjEzMC4xIiwidGFyZ2V0QnJhbmNoIjoibWFzdGVyIiwibGFiZWxzIjpbImFwcC9sb2NhbC1haSIsImF1dG9tZXJnZSIsInJlbm92YXRlL2NvbnRhaW5lciIsInR5cGUvbWlub3IiXX0=-->
title
| title |
|---|
| README |
General Info
For more information about this Chart, please check the docs on the TrueCharts website
This chart is not maintained by the upstream project and any issues with the chart should be raised here
Installation
Helm-Chart installation
To install TrueCharts Helm charts using Helm, you can use our OCI Repository.
helm install mychart oci://oci.trueforge.org/truecharts/local-ai
For more information on how to install TrueCharts Helm charts, checkout the instructions on the website
Chart Specific Guides and information
All our charts have dedicated documentation pages. The documentation for this chart can be found here: https://truecharts.org/charts/stable/local-ai
Configuration Options
To view the chart specific options, please view Values.yaml included in the chart. The most recent version of which, is available here: https://github.com/trueforge-org/truecharts/blob/master/charts/stable/local-ai/values.yaml
All our Charts use a shared "common" library chart that contains most of the templating and options. For the complete overview of all available options, please checkout the documentation for them on the common docs on our website
For information about the common chart and all defaults included with it, please review its values.yaml file available here: https://github.com/trueforge-org/truecharts/blob/master/charts/library/common/values.yaml
Support
Sponsor TrueCharts
TrueCharts can only exist due to the incredible effort of our staff. Please consider making a donation or contributing back to the project any way you can!
All Rights Reserved - The TrueCharts Project