Show HN: QVAC SDK，一款用于构建本地 AI 应用的通用 JavaScript SDK
5 分•作者: qvac•4 个月前
大家好，今天我们发布 QVAC SDK [0]，这是一个通用的 JavaScript/TypeScript SDK，用于在桌面和移动端构建本地 AI 应用程序。<p>该项目完全开源，采用 Apache 2.0 许可证。我们的目标是让开发者更容易构建有用的本地优先 AI 应用，而无需拼凑大量不同的引擎、运行时和特定于平台的集成。在底层，SDK 基于 QVAC Fabric [1] 构建，这是我们的跨平台推理和微调引擎。<p>QVAC SDK 使用 Bare [2]，这是一个轻量级的跨平台 JavaScript 运行时，是 Pear 生态系统 [3] 的一部分。它几乎可以在任何地方用作 worker，并内置了 Node、Bun 和 React Native (Hermes) 的工具。<p>目前它支持以下功能：<p><pre><code> - 跨桌面、移动端和服务器的本地推理
- 支持 LLM、OCR、翻译、转录、
文本转语音和视觉模型
- 通过 Holepunch 堆栈 [4] 进行点对点模型分发，
类似于 BitTorrent，任何人都可以成为种子节点
- 基于插件的架构，因此可以轻松添加新的引擎和模型类型
- 完全点对点委托推理
</code></pre>
我们还投入了大量精力进行文档编写 [5]。这些文档的结构设计得易于人类和 AI 编码工具阅读，因此实际上，您通常可以很快地借助您最喜欢的编码助手取得很大进展。<p>我们知道还有一些工作需要改进：<p><pre><code> - 目前，由于 Bare 插件的打包效率还不够高，因此捆绑包的大小比我们想要的要大
- 插件工作流程可以更简单
- 树摇动已经成为可能，但目前它仍然需要一个 CLI 步骤，我们希望使其更自动化，并更好地集成到构建过程中
</code></pre>
这次发布仅仅是个开始。我们希望帮助人们更大规模地构建本地 AI。非常感谢您的任何反馈！完整愿景可在官方网站 [6] 上获取。<p>参考资料：<p>[0] SDK: <a href="http://qvac.tether.io/dev/sdk" rel="nofollow">http://qvac.tether.io/dev/sdk</a><p>[1] QVAC Fabric: <a href="https://github.com/tetherto/qvac-fabric-llm.cpp" rel="nofollow">https://github.com/tetherto/qvac-fabric-llm.cpp</a><p>[2] Bare: <a href="https://bare.pears.com" rel="nofollow">https://bare.pears.com</a><p>[3] Pear Runtime: <a href="https://pears.com" rel="nofollow">https://pears.com</a><p>[4] Holepunch: <a href="https://holepunch.to" rel="nofollow">https://holepunch.to</a><p>[5] 文档: <a href="https://docs.qvac.tether.io" rel="nofollow">https://docs.qvac.tether.io</a><p>[6] 网站: <a href="https://qvac.tether.io" rel="nofollow">https://qvac.tether.io</a>
查看原文
Hi folks, today we're launching QVAC SDK [0], a universal JavaScript/TypeScript SDK for building local AI applications across desktop and mobile.<p>The project is fully open source under the Apache 2.0 license. Our goal is to make it easier for developers to build useful local-first AI apps without having to stitch together a lot of different engines, runtimes, and platform-specific integrations. Under the hood, the SDK is built on top of QVAC Fabric [1], our cross-platform inference and fine-tuning engine.<p>QVAC SDK uses Bare [2], a lightweight cross-platform JavaScript runtime that is part of the Pear ecosystem [3]. It can be used as a worker pretty much anywhere, with built-in tooling for Node, Bun and React Native (Hermes).<p>A few things it supports today:<p><pre><code> - Local inference across desktop, mobile and servers
- Support for LLMs, OCR, translation, transcription,
text-to-speech, and vision models
- Peer-to-peer model distribution over the Holepunch stack [4],
in a way that is similar to BitTorrent, where anyone can become a seeder
- Plugin-based architecture, so new engines and model types can be added easily
- Fully peer-to-peer delegated inference
</code></pre>
We also put a lot of effort into documentation [5]. The docs are structured to be readable by both humans and AI coding tools, so in practice you can often get pretty far with your favorite coding assistant very quickly.<p>A few things we know still need work:<p><pre><code> - Bundle sizes are larger than we want right now because the current packaging of Bare add-ons is not as efficient as it should be yet
- Plugin workflow can be simpler
- Tree-shaking is already possible, but at the moment it still requires a CLI step, and we'd like to make that more automatic and better integrated into the build process
</code></pre>
This launch is only the beginning. We want to help people build local AI at a much larger scale. Any feedback is truly appreciated! Full vision is available on the official website [6].<p>References:<p>[0] SDK: <a href="http://qvac.tether.io/dev/sdk" rel="nofollow">http://qvac.tether.io/dev/sdk</a><p>[1] QVAC Fabric: <a href="https://github.com/tetherto/qvac-fabric-llm.cpp" rel="nofollow">https://github.com/tetherto/qvac-fabric-llm.cpp</a><p>[2] Bare: <a href="https://bare.pears.com" rel="nofollow">https://bare.pears.com</a><p>[3] Pear Runtime: <a href="https://pears.com" rel="nofollow">https://pears.com</a><p>[4] Holepunch: <a href="https://holepunch.to" rel="nofollow">https://holepunch.to</a><p>[5] Docs: <a href="https://docs.qvac.tether.io" rel="nofollow">https://docs.qvac.tether.io</a><p>[6] Website: <a href="https://qvac.tether.io" rel="nofollow">https://qvac.tether.io</a>