Login
Cleave
Login

Cleave teaches one small computer to handle voice: to listen to a person, understand a request, and answer out loud. That computer is BeagleY-AI, a credit-card-sized board from BeagleBoard.org with a four-core 64-bit ARM processor and 4 GB of memory.

Cleave is one part of a larger effort, which this page calls the program: a set of projects at BeagleBoard.org that keep how a computer works readable, from the chip to the apps (the Big Picture lists them all). Cleave is an attempt to bring the program's pieces together on one board. It is also the first real system booted into BQ2, the program's small Linux system that rebuilds itself from source.

What it's for

Voice is a very hard area for writing software. It needs sound hardware, drivers (the code that talks to a particular chip), speech recognition, language models and speech synthesis, all working together on a small machine. Each of those comes with its own languages and build tools. AI agents, programs that write code and run commands on a person's behalf, now make it possible to take on that much at once.

Cleave's job is to find out which pieces voice needs, how they fit together on this board, and how to keep each one readable and rebuildable from source, as BQ2 does for itself.

Cleave is also where a local model is meant to run: a language model on the board itself, not on someone else's server. It also ties to gp-crystal, another project in the program: a way to edit code by voice while seeing it on a screen. No one is working on gp-crystal yet.

What it's made of

Glint, the program's readable operating system, uses cleave-dtc and cleave-kconfig too.

How it's shaped

One script, tools/build-beagley-ai-bsp.sh (BSP: board support package, the boot chain and kernel for one board), fetches every source at a pinned version, checks it, and then builds inside the BQ2 seed, in this order:

  1. LLVM (BQ2's compiler) with ARM support added, then cleave's tools;
  2. the secure firmware and both stages of U-Boot;
  3. the kernel, its modules, and the Wi-Fi software and firmware;
  4. the SD card image, which holds the BQ2 seed, cleave's files from image/beagley-ai/, and everything built above.

The board boots that card into cleave-init. Wi-Fi network names and passwords come from the boot partition, written there by bb-imager-rs, BeagleBoard.org's tool for writing SD cards, so none are built into the image. During bring-up, the first boots of a new board, its serial console is shared through the Fossil server. tools/serial-relay.sh runs on a computer wired to the board's serial port. It copies everything the board prints into one file on the server. It also reads commands from a second file there and sends them to the board, as if typed. So an agent with no access to the board can still read its console and give it commands.

The voice parts come next, on top of this: docs/beagley-ai-plan.md gives the phases.

What success looks like

The Status page says where the work stands today.

  1. Built in BQ2 alone. The boot chain, kernel and SD card image build inside the BQ2 seed, with no Python, Perl, flex or bison. Done: this build run built them all on aarch64-1.
  2. Boots. BeagleY-AI boots BQ2 into cleave-init from that build. Done: seen in the board's console log, after the change (a check-in, in Fossil's words) that set up its devices.
  3. On the network. The board joins Wi-Fi from the settings bb-imager-rs writes, and fetches over HTTPS. Partly: Wi-Fi worked with the fixes applied by hand. The image with them built in is still waiting for a boot test. HTTPS waits on a fix in BQ2.
  4. Serves Fossil. The board runs a Fossil server for this repository and answers a forum post itself. Not yet.
  5. Sound. Audio plays and records through the JK Embedded Audio Board, a small sound board that plugs onto BeagleY-AI. Not yet: the kernel build turns on its drivers, but the board's devicetree overlay (the addition that tells the kernel the sound board is there) and the audio programs aren't written.
  6. Local speech loop. A person speaks a request and hears an answer. Speech recognition (whisper.cpp, a C++ port of OpenAI's Whisper model), a small local language model (run by llama.cpp, a C++ program for running such models) and speech synthesis all run on the board's processor. Anything bigger is passed to a remote agent. Not yet.
  7. Using the AI accelerator. BeagleY-AI's processor also has TI's C7x signal processors, with a unit that multiplies matrices, built for running models. Mesa, the open-source graphics library, includes Teflon, which runs models on accelerators like these. A Teflon back end for C7x is being written elsewhere, and cleave will use it, not write its own (ticket). Not yet.
  8. Voice-driven development. A spoken request becomes an OCX post, an agent changes the code, and the board rebuilds itself, using gp-crystal's way of editing by voice. Not yet.

Later, the same voice work should reach BeagleBadge, BeagleBoard.org's badge board, playing through a car's Bluetooth speakers. It should also help with voice commands in Stone, the program's iPhone app.

The virtues behind it

Like every project in this program, cleave follows the four virtues of OCX:

Building

On any machine with Crystal (shards is Crystal's build tool):

shards build      # cleave's tools, in bin/
crystal spec      # their tests

The whole BeagleY-AI build needs a 64-bit ARM Linux computer with git, curl, tar and bubblewrap (a tool that runs a program in a sealed-off view of the system, here the BQ2 seed). job fetches the sources and then builds; WORK is any empty directory:

tools/build-beagley-ai-bsp.sh job WORK   # results in WORK/out

Working on cleave

Cleave is kept in Fossil at https://ollama.openbeagle.org/cleave. The maintainer and AI agents work in its forum, and each build sent to aarch64-1 reports back in the thread that asked for it.