Start an on-device AI terminal app from the hello-pear-qvac-tui template

This is the on-device AI counterpart to Start a terminal app from the hello-pear-bare template. It starts from the same shape—a Bare CLI that updates itself over the air through a worker—and adds a second worker that runs a language model on the user's own machine, plus a bare-tui terminal interface. There is no API key and no cloud round-trip.

holepunchto/hello-pear-qvac-tui is Holepunch's official boilerplate for this shape. The plumbing—CLI, workers, and IPC—is finished; the chat interface on top is only a demo, meant to be replaced with your own app.

Take this path when you want a terminal app that runs a model locally and updates over the air. If you already have an app built from hello-pear-bare and want to add on-device inference to it, follow the how-to Build a local AI chat TUI with QVAC instead. For a terminal app with no AI, start from hello-pear-bare.

Clone and run

1. Clone the repository

You need npm (via Node.js) and Bare on your PATH. Install Bare with npm i -g bare-runtime, then clone the repository and install its dependencies:

git clone https://github.com/holepunchto/hello-pear-qvac-tui
cd hello-pear-qvac-tui
npm install

The upstream README says the template runs on macOS, Linux, and Windows, each for arm64 and x64, and that QVAC ships native prebuilds for all of them and uses GPU acceleration (Vulkan or Metal) where it exists.

2. Run the app

npm start runs bare bin.mjs --no-updates: the app starts with over-the-air updates off. The first run downloads about 1 GB of model weights and caches them, according to the upstream README, so it takes a while; later runs start in seconds.

npm start

Ask a question and the answer streams back. These keys work in the interface:

3. Pick a model and a context window

Two flags matter, and bin.mjs defines them next to the usual --storage, --no-updates, and --version, plus --verbose, which logs engine and native addon detail to stderr:

const cmd = command(
  appName,
  summary(pkg.description),
  flag('--version|-v', 'Print the current version'),
  flag('--storage <dir>', 'custom storage directory'),
  flag('--model <name>', 'QVAC model constant to load'),
  flag('--ctx <tokens>', 'context window in tokens (default 8192)'),
  flag('--no-updates', 'disable OTA updates for this run'),
  flag('--verbose', 'log engine and native addon detail to stderr')
)

--model takes any completion model constant exported by @qvac/inference, since the template registers only the llamacpp-completion plugin, and nothing else changes. --ctx sets the context window in tokens, the budget that the conversation, the model's reasoning, and the answer share. The defaults live in package.json:

  "qvac": {
    "model": "LLAMA_3_2_1B_INST_Q4_0",
    "ctxSize": 8192
  }

The upstream README warns that the QVAC addon's own default context is only 1024 tokens, small enough that a couple of turns leave no room to answer, which is why the template sets 8192. See its list of models that work out of the box:

npm start -- --model QWEN3_1_7B_INST_Q4 --ctx 16384

Map the template

The repository also ships a CLAUDE.md that explains how to extend the template and the traps it has already hit. Point your coding agent at it before asking for changes.

How it's wired

There are two workers and one pattern. Each runs in its own Bare thread through PearRuntime.run and is spoken to over a FramedStream. The upstream README explains that loading a model file blocks its thread for seconds, so inference gets a worker of its own instead of freezing the interface. bin.mjs constructs the two clients side by side. The update client exists only when updates are on and upgrade is a real link, so the default npm start runs the inference worker alone:

// Two workers, same pattern: OTA updates and inference each run in their own
// Bare thread and talk over a FramedStream.
const app = updating
  ? new App({
      dir,
      app: isDev ? null : os.execPath(),
      updates,
      version: pkg.version,
      upgrade: pkg.upgrade,
      name: isWindows ? appName + '.exe' : appName
    })
  : null

const inference = new Inference({ model, ctxSize, verbose })

lib/inference.js is deliberately the same shape as app.js: a ready-resource that spawns its worker and turns the frames it receives into events. Learn one and you know the other:

  constructor({ model, ctxSize, gracePeriod, verbose } = {}) {
    super()

    this.model = model || 'LLAMA_3_2_1B_INST_Q4_0'
    this.ctxSize = ctxSize || 8192
    this.gracePeriod = gracePeriod ?? 5000
    this.verbose = verbose === true
    this.loaded = false
    this.percentage = 0

    this.IPC = null
    this.pipe = null

    this._seq = 0

    // Resolves when the worker confirms it has unloaded, or when the grace
    // period runs out — a wedged native unload must not block quitting.
    this._onclosed = null
    this._closed = new Promise((resolve) => {
      this._onclosed = resolve
    })
  }

  _open() {
    this.IPC = PearRuntime.run(require.resolve('../workers/qvac.js'), [
      this.model,
      String(this.ctxSize),
      this.verbose ? '1' : '0'
    ])
    this.pipe = new FramedStream(this.IPC)

    this.pipe.on('data', (data) => this._onmessage(data))
    this.pipe.on('error', (err) => this.emit('error', err))
    this.IPC.on('error', (err) => this.emit('error', err))
    this.IPC.on('exit', (code) => {
      if (code === 0 || this.closing !== null || this.closed) return
      this.emit('error', new Error(`Inference worker exited with code ${code}`))
    })
  }

An answer takes this path: the interface asks lib/inference.js, which sends an ask frame to the worker; the worker streams delta frames back; bin.mjs turns each one into a message for the interface; and a final end frame settles it. The load-bearing rule is that ui/app.js never imports the QVAC SDK, which is why the tests run headless in a fraction of a second. The how-to Build a local AI chat TUI with QVAC walks through every layer and the full frame protocol.

Updates

Updates are off until you give the template a real key. With the placeholder upgrade link still in package.json, bin.mjs skips the updater instead of letting it throw inside the worker, and says so in the transcript when you start with --updates:

// The updater needs a real Pear key. Until you publish and paste yours into
// package.json's "upgrade", running it would throw inside the worker thread —
// and pear-runtime turns a worker throw into a hard process exit, which no
// try/catch here could catch. So skip it and say so in the transcript.
const configured = !pkg.upgrade.includes('<')
const updating = updates !== false && configured

Create a link with pear touch, set upgrade in package.json, and start with updates on:

npm start -- --updates

Unlike hello-pear-bare, this template never swaps the build out from under a conversation. app.js stages a release but does not apply it, and a banner in the interface asks first. ctrl+r applies the staged update, and esc dismisses the banner while the model isn't answering. bin.mjs hands the banner its apply callback:

const ui = new UI({
  inference,
  model,
  version: pkg.version,
  // The banner is the confirm gate: app.js stages the update but never applies
  // it, so nothing swaps out from under a conversation until the user says so.
  onApplyUpdate: app ? () => app.applyUpdate() : null
})

A staged release is not downloaded on sight. The updater waits a random delay of up to an hour so a released fleet doesn't hit the seeder at once, and the interface tells you when one is scheduled. Restart the app to claim it sooner. For the model behind these updates, see Pear OTA.

Build a standalone binary

npm run make builds a standalone executable for your host into out/make, named hello-pear-qvac (plus .exe on Windows). HOST=linux-arm64 npm run make targets another supported platform, and macOS and Windows signing read the same environment variables as the hello-pear-bare builder. The per-platform make:<platform>-<arch> scripts in package.json build unsigned binaries into out/<platform>-<arch>. The upstream README says the model weights are downloaded on the first run, so plan for that wait and download in a built app too.

Release it the same way as any Pear app: stage, provision, and seed with Deploy your application, then install over the air with pear install. See Installing applications for how installs work.

Customize for your brand

Before you ship, set your identity in package.json:

• name, productName, description, author, and license. The template ships hello-pear-qvac as its name, and that name is also written into scripts/make.js, each make:* script, and .github/workflows/build.yaml. Change it in all of them.
• the upgrade pear:// link (see Updates).

Pick the name carefully, because two tools read it differently. bin.mjs hands the updater productName || name exactly as written, plus .exe on Windows, and the updater looks for that file name in your release at by-arch/<host>/app/<name>. bare-build turns anything other than letters and digits in the --name it is given into a hyphen, adds a hyphen before a capital that follows a lowercase letter or digit, and lowercases the result, so My Chat, MyChat, and my_chat all build my-chat. Use lowercase letters, digits, and hyphens everywhere, and keep every copy of the name identical. If they differ, the updater can't find the binary and reports update not found.

Remix it

Chat is only the demo. Most ideas need no new dependency—just a different prompt and output shape in ask() in workers/qvac.js—and the how-to's Remix it section lists a few. The upstream README's Remix it section goes further: swapping the QVAC plugin for another modality, using just the SDK, and deleting the updater or the interface when you don't need them.

Troubleshooting

• Stuck at 0% on the first run. The model is downloading, about 1 GB according to the upstream README. The percentage moves as blocks arrive.
• Answers stop mid-sentence, and get worse each turn. The context window is full. Raise --ctx.
• ggml_vulkan: … lines appear over the interface. llama.cpp writes to the error stream below any JavaScript logger, and bin.mjs repaints the screen. It is harmless.
• The app hangs on exit. The model must be unloaded first. That is the worker's close handshake, and quitting gives it a short grace period before giving up.
• An update downloaded but nothing happened. The confirm gate is waiting: press ctrl+r, then relaunch to run the new build.
• Out of memory, or only a few tokens per second. Pick a smaller model or quantization with --model.

Where to go next

• Start from a template—all the boilerplates, side by side.
• Build a local AI chat TUI with QVAC—the how-to behind this template: what it adds to hello-pear-bare and how a question flows end to end.
• Start a terminal app from the hello-pear-bare template—the template this one extends, with the update worker explained in full.
• Workers—why peer-to-peer and inference code run in their own threads.
• Pear OTA—the pear-runtime API behind the updater.
• Deploy your application—the full-control manual path for staging, provisioning, and release lines.
• Seeding with pear seed—keep your pear:// link online so peers can fetch releases.