沙漠蚂蚁实验室:在设备端运行的本地化、高速模型
Desert Ant Labs: local, fast models that run on device

原始链接: https://desertant.com/blog/introducing-desert-ant-labs/

Desert Ant Labs 正式上线,提供一套专为端侧运行设计的 AI 模型,旨在速度、成本和隐私保护方面超越云端方案。Desert Ant 不走通用大模型路线,而是专注于小型、任务导向型模型,使开发者能够让手机和笔记本电脑本地运行智能功能,从而消除 Token 成本并降低延迟。 该平台首发 18 款模型(涵盖音频增强、转录和个人敏感信息去标识化等功能),并提供适用于 Swift、Kotlin 和 JavaScript 的统一 SDK。这些模型经过效率优化,即使在五年前的硬件上也能流畅运行。Desert Ant 倡导“主权”AI,确保用户数据留存在设备端,为云端 API 提供了一种安全的替代方案。 通过利用消费级硬件中未被充分挖掘的庞大算力,该公司旨在摆脱行业对昂贵数据中心的依赖。Desert Ant 正在改变 AI 开发范式:开发者无需再为每次 API 调用付费,而是可以将“小巧的大脑”直接嵌入产品中,在每个交互点实现实时、零成本的智能。对于每月活跃设备数在 10 万台以内的应用,这些模型均可免费使用。

Desert Ant Labs 因其一系列本地端 AI 模型在 Hacker News 上引发了热议。这家初创公司提供了一套 SDK,允许开发者在移动设备和桌面硬件上直接运行轻量且高效的模型,例如语音转录、语音增强和文本审核。这些模型在每月 10 万台活跃设备以内可免费使用。 开发者社区对此反应不一。许多用户赞赏其对离线隐私、速度的重视,以及利用闲置消费级硬件而非依赖云端算力的做法。然而,其商业模式引发了批评:持怀疑态度的人认为,对静态的本地软件采取 SaaS 式的订阅或使用费类似于“寻租”。也有人维护这种做法,认为这是专业中间件的标准许可方式,能让初创公司在获得成功后再逐步分摊成本。 技术反馈指出,虽然这一概念很有前景,但目前对 Apple CoreML 的依赖限制了其普及性。批评者还注意到,部分模型是现有开源项目的优化版本,这引发了对其独特价值主张的质疑。总体而言,尽管社区对“本地优先”的 AI 愿景充满热情,但在货币化和平台策略上仍存在分歧。
相关文章

原文

Today we're launching Desert Ant Labs, a European frontier AI lab building opinionated on-device intelligence. We believe the best path to efficient intelligence starts on-device.

We're building small, specialized models for audio, vision, and text – each model answers in milliseconds, and costs nothing to run, so you can put intelligence in every product interaction, without being limited by token cost or inference speed. Small enough to run on a five-year-old phone, fast enough to use on every frame or keystroke, and better than the API call you're already paying for.

The first 18 models are live today (12 stable and six in beta), accessible via one SDK for Swift, Kotlin, and JavaScript. One model per task, each built to be the fastest way to complete that task on a device:

  • Voz: transcribe 10 minutes of audio in two seconds on an iPhone – 4.7x faster than Whisper – with a start and end time on every word.
  • Clear: a 9MB model that can turn a five-minute laptop recording into studio quality audio in one second.
  • Redact: mask names, addresses, and card numbers, in real time, in 27 languages, so they never reach your servers.
  • Tongue: identify 84 languages from three words, with a 2MB model.
Language ID accuracy, three words in

Tongue · 2MB 0.933

293MB detector 0.887

Tongue names the language from three words, scoring 0.933 at 2MB against 0.887 for a 293MB detector.

And that's just to name a few. You can find full specs and benchmarks for the other fourteen, on desertant.com/models and Hugging Face. Every model is free up to 100k monthly active devices. No tokens, no logins.

Personal data caught, by system

Redact · 12MB 88.8

GLiNER-PII · 2.3GB 91.1

Rampart · 14.7MB 61.4

OpenAI filter · 3GB 60.2

Redact catches 88.8% of the personal data in a text, close to the 2.3GB GLiNER-PII, from a 12MB model.

We're building this in Europe, where "on-device" is the sovereign default. The data never leaves your customer's hands, the feature never depends on someone else's cloud, and what's never been uploaded can never be compelled.

How we got here

For five years we've been building our video app, Detail, with an on-device first approach. But when we introduced features like Auto Edit to create short clips, or audio enhancement for podcasts, we had to fall back to cloud APIs. And as the popularity of Detail grew, so did our infrastructure bills.

Every few months I'd hunt for useful on-device models. I'd surf Hugging Face for a model that could find filler words or clean up a recording. And, every June, we'd get great new tools to build with but the industry wasn't moving fast enough. The foundation was there: the chips, Core ML, the research. What was missing was everything between that foundation and actually implementing a feature in your app: a model you could drop in and ship with a few lines of code.

So, we trained the models ourselves. It turns out training a model is a product design challenge, and product is what we know. We designed models and local inference that beat cloud services on speed, quality, and cost, and outperform other local and cloud models on the task itself, at a fraction of their size.

We replaced Dolby for better, faster audio enhancement with Clear, and made our on-device transcriptions 5x faster with Voz. We also replaced Claude Sonnet with Clips, our 284MB model that turns a 10-minute video into a dozen clips in 5 seconds – 10x faster and using 470x less energy than Sonnet, with the same quality.

Clear audio enhancement speed, 5 minutes of audio

iPhone 16 Pro 302x

MacBook Pro (M5) 345x

Clear enhances, masters, and re-encodes a clip on the device, best of three, from a 9MB model. 302x realtime on a phone.
Transcription speed, 30 minutes of audio

Voz 319x

Apple SpeechAnalyzer 78x

Whisper large-v3-turbo 50x

Realtime factor over 30 continuous minutes on an M3 Ultra. Voz reaches 298x on an iPhone 17 Pro.

Detail 6, which will launch with iOS 27, replaces all of our cloud APIs with our own models, running entirely on the device.

We've all spent the past few years building with LLMs as if they were just another API. And, amid the hype around generalist frontier brains, we almost forgot they're not the only option.

Every developer I talk to has a wishlist of on-device models they'd build if cost wasn't a factor, or a feature they're bleeding tokens on that they'd happily swap for a local model. A call that runs the same way a hundred thousand times a day: cleaning a recording, tagging a photo, pulling a date out of a sentence, catching a name before the text hits your servers. None of these needs a frontier model.

NVIDIA's own researchers pulled apart three agent systems and estimated that 40 to 70% of their calls to a large model could go to a small, specialized one instead.

The compute is already paid for

The industry will spend about $450 billion on data centers this year. Meanwhile, the world ships more than a billion phones, tablets, and laptops with increasingly capable chips, perfectly suited to these kinds of tasks. There's more compute available in people's hands than in every AI data center on earth.

We have an unfair advantage with free inference. No per-call cost, so a feature runs on every message instead of the ones you can afford to check. No round-trip, and your customer's data never leaves the device. When inference costs nothing, the way we build products changes entirely.

Little brains in every product

To build with local models, the developer experience has to get a lot better. You need models you can use commercially, that beat the alternatives on your task in speed and quality, that you can drop into your app with a few lines of code, and are easy to discover.

Think of the first hundred models as the cerebellum, the little brain. The little brain handles the always-on work – balance, timing, the skills you never think about, so the rest of the brain is free to think. That's what we're building first: fast, specialized models for the work that runs all day, on the device, for free.

Then comes the cortex, the layer that decides which model answers. A small local model first, a bigger one when the job requires it, and the cloud only when the work has to leave the device. As open research advances and device silicon becomes more capable, the local models grow, and we'll train larger ones ourselves. Frontier intelligence, built from the small end up.

Cloud labs ship neutral models because per-token pricing needs a neutral model. Every Desert Ant model ships with a default we choose, and the levers you need to change that default. We optimize the model and the runtime together: on an iPhone, Clear and Voz run on the Neural Engine, and in the browser, Clear's same weights run through WebAssembly.

The SDK

Ready to get started? You can implement Desert Ant models in your app with our native Swift, Kotlin, and JavaScript SDK, available on GitHub.

Our docs are written for developers and agents and you can try the models on your Mac with the CLI, or in your browser on Hugging Face.

Building something cool with our models, or want to build them with us? Get in touch.

联系我们 contact @ memedata.com