Whisper on device

I haven't written any Electron apps before, at least until ChatGPT increased my boldness this month.

I've been working on an on-device transcription app that uses Whisper to record what I'm saying all day and store it locally.

While recording myself all day sounds wild, it's been helpful in a bunch of situations:

• By recording my mastermind conversation, I was able to ask ChatGPT for my action items
• I can record myself rambling, copy all the text, and ask ChatGPT to help summarize my thoughts
• In person meeting notes without Zoom recordings (yes, people actually meet in person)

I haven't come up with a real name for the project yet (that's when I normally enlist the help of my brilliant wife), so for now, we'll just call it "Scribe".

Scribe tech stack

Scribe is an Electron app that uses Nextron — think Electron meets Next.js static export mode.

We're using Whisper, but not the repository I linked above.

Instead, we're using Whisper.cpp and the Node Addon, which is a high performance port of the original lib.

Since I'm recording myself all the time, I don't want data leaving my device. That's led to me using SQLite locally to store the output of the transcripts.

Did you know SQLite had full-text-search support? Because I didn't (but ChatGPT did).

Yeah, I know, it's ugly — but it works.

While the search is already pretty useful, I want to use embeddings to add contextual search so that I can search by intent, not just exact strings.

p.s. — Ellie is my dog and yes, she did know it was 5:03 PM, AKA dinner time.

Roadmap

Up next, on device LLM meeting summaries powered by llama.cpp. Any guesses who writes that project? This Georgi Gerganov guy ships.

legal reminders: I'm not a lawyer. You might need to ask for consent to record based on where you're living and who you are talking to. Again, I'm not a lawyer.