Earshot gives AI coding agents local audio analysis, review, and sound generation on CPU. It surfaces results through a CLI, Python API, and MCP server, using CLAP and an AudioSet tagger to describe, match, rank, and locate sounds. Version 0.3 added a 201-sound SFX bank; version 0.2.0 introduced forge v2 and the fit workflow.
What Earshot gives a coding agent on a local CPU
Describe returns calibrated tags, alternative readings, and measurements. Match compares a file to a text query and returns a verdict (strong, plausible, weak, or no match) with cosine similarity and a z-score. Rank sorts candidate files by how well they fit a description, and find extracts the best moments from a long recording as time-trimmed segments.
Timeline produces a listening transcript with labeled events, sections, loudness, beat grid, and sync points. Sheet renders the same information as an annotated PNG. Critique scores a video or audio file against a profile of professional launch films and returns timestamped fixes.
Forge generates sounds through physical synthesis: modal resonators, noise sources, reverb, and microphone modeling. It scores each candidate against description matches, realness checks, closeness to reference recordings, and model-free measurements. The agent reads the critic's words, inspects spectrograms next to references, rewrites the recipe, and runs another iteration. Fit translates a recording into a Python module with readable parameters; the module renders new takes, variations, and morphs.
Agents access the CLI by spawning earshot commands, import earshot.api in Python, or connect to the MCP server where models stay loaded.
Set up Earshot on CPU-only Linux
Python 3.10 or later and ffmpeg/ffprobe on PATH are required. Check that ffmpeg -version and ffprobe -version both print their versions before continuing. Clone the repository, create and activate a virtual environment, install PyTorch for CPU, then install earshot with the optional MCP, LLM, and development dependencies.
git clone https://github.com/Leonxlnx/earshot && cd earshot
python -m venv .venv
source .venv/bin/activate
python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install -e ".[mcp,llm,dev]"
earshot doctor --load
The first use of a model downloads CLAP (laion/larger_clap_general) and the AudioSet tagger (MIT/ast-finetuned-audioset-10-10-0.4593) into .cache/hf inside the repository or the directory named by EARSHOT_HOME. The two models total approximately 1.1 GB at pinned revisions with safetensors weights. After the initial download, earshot loads them from the cache and does not contact Hugging Face Hub again; a damaged cache triggers an error naming the folder, not a silent re-download.
Set EARSHOT_OFFLINE=1, HF_HUB_OFFLINE=1, or TRANSFORMERS_OFFLINE=1 to prevent any download attempt and return an error if models are missing from the cache. Use earshot doctor --fetch to download both models before the first workflow run.
Review audio with describe, match, rank, and locate commands
Describe a file to get its sound characteristics in words, with calibrated z-scores and alternative readings:
earshot describe enter.ogg
The output reports duration, transient shape, the most likely descriptions with z-scores, peak level, and brightness. Each description includes a z-score calibrated against a reference vocabulary. Match a file against a query and optional alternative descriptions:
earshot match key.ogg "a deep thocky mechanical keyboard press" \
--alts "a clicky blue switch keyboard" "a mouse click"
The result includes a verdict, z-score, and rank among vocabulary descriptions. Find the best moments in a long recording:
earshot find keys.ogg "a single mechanical key press" --top 4
Each match returns a time window (start and end seconds), stroke duration, and z-score. Rank multiple files by how well they match a query:
earshot rank "a fast crisp air whoosh" path/to/*.wav
Every command prints a plain-language summary by default. Add --json for the full result or --out f.json to save it. Commands that use models (describe, match, rank, find, timeline, sheet, critique, forge, fit) load CLAP and the AudioSet tagger before answering, taking approximately 20 to 40 seconds per CLI call; measure, sfx, and kit load no models.
Create a WAV file with forge and inspect its output
List available forge v2 recipes and reference sets:
earshot forge --list
Run a forge search with a recipe, local reference recordings you have rights to use, a budget for candidate renders, and an output directory. Forge v2 covers 34 motion-design categories including keypresses, whooshes, and impacts:
earshot forge keypress --refs path/to/real/keypresses/ \
--budget 144 --variants 4 --dir made/keypress
The output directory holds the best diverse sounds as WAV files, a report.md summarizing the search and each pick in the critic's words, compare.png showing spectrograms and envelopes next to the nearest reference, audition.wav with made and reference sounds in alternating pairs, and report.json with parameters and scores for every candidate. Picks are re-rendered on new random seeds (default three) and must differ in both parameters and sound. Inspect the WAVs, read report.md for the critic's description of each pick, and view compare.png to see how the spectrograms align with references.
Use a named reference set from the project's built-in collection if you've laid out licensed recordings as described in earshot/forge/refsets.py:
earshot forge whoosh --refset whoosh_fast --budget 96 --dir made/whoosh
Fit (introduced in 0.2.0) translates a recording into a Python module that re-renders it with variations. Use recordings you have rights to use; the generated recipe remains derivative of the source and must carry the source's license:
earshot fit click.wav -o made/click_fit.py --report
The module includes its own render function. Run it directly with the environment's Python to generate new takes:
python made/click_fit.py 3 take3.wav
The SFX bank (added in 0.3) renders 201 sounds by ID with no models or network:
earshot sfx list --kind whoosh
earshot sfx render whoosh_fast/crisp -o w.wav --dur 0.8 --apex 0.32
earshot sfx check w.wav
Knobs adjust duration, apex timing, and pitch without resampling. The check command audits for clicks, cut tails, clipping, DC, comb ring, re-rising tails, and other artifacts using model-free measurements.
Judge whether the workflow fits
For more than a few calls, use the MCP server to keep models loaded across requests or write one Python script that imports from earshot.api. Forge examples using --refs require local reference recordings you have rights to use. Without references, the critic runs in designed mode, reporting realness but not requiring it. Reference sets in earshot/forge/refsets.py are split by source: a forge side the search may use and a bench side the fixed benchmark hears, but the search does not.
The search optimizes earshot's own models, and those models can be satisfied without the sound improving. After a search, made sounds pass every model check, including held-out checks, so model checks no longer distinguish made from real; the 1/3-octave tonal balance measurement still can. Judge a made sound by the model-free measurements, spectrograms next to references, robust re-renders on new seeds, and a human blind test, not by the model scores alone. The project author's blind test for version 1 recipes used one listener and 15 trials, with 8 correct identifications out of 15.
FAQ
Can I use forge without local reference recordings?
Yes. Without --refs or --refset, the critic runs in designed mode, reporting realness but not requiring it. The search still scores candidates on description match, realness, and model-free measurements, but does not compare them to real recordings or measure tonal balance against references.
Does earshot send audio files to a server?
The project documents local processing for all commands except earshot llm FILE --yes-upload, which requires your own Gemini API key and explicit consent. The supplied evidence does not establish the network behavior of every command.
Run describe on a sample file
Activate the virtual environment and run earshot describe on a short audio file to confirm the models load and the analysis prints calibrated tags and alternative readings.
Verification status: Supported by official documentation only under the conditions below; not reproduced in a test environment by IGPROD. Checked 2026-10-20.
Conditions:
- Python 3.10+ and ffmpeg / ffprobe on PATH are required for the documented setup.
- The first model use downloads approximately 1.1 GB; offline use requires the models to be cached.
- Forge searches using reference sets require local reference recordings; without them, the critic runs in designed mode.
Related guides
- Talorys: a personal AI agent in your Cloudflare account: an agent that runs in your own account.
Sources:
