On a Thursday morning in September I started with a single web page in Claude. Eight minutes later I typed my first real request, typo and all: "need jarvis voice".
Five days later I had an Iron Man style console that talks back. It keeps every conversation as a thread on a board, reads the real machine it runs on, installs as an app on a phone, and is live on the web for anyone to try.
This is the honest account of how it got there. Not the highlight reel, the ledger. I went back through every session log and the full git history, so the numbers here are counted, not remembered.
What JARVIS is
You tap the glowing core and speak, or type. JARVIS answers, hears and speaks through one AI service you connect, Gemini or ChatGPT. Everything you ask goes into a thread on the board: talk, research with its sources, pictures, videos that play where they are. Ask in the thread you have open, or start a new one. Every figure on screen is measured, not simulated: real processor load, real network, real weather where you are.
It runs in the browser with nothing to install. There is also a PC version with a small server of its own, which can read what a browser can't. Either way, your keys stay on your device.
How it started, day by day
Thursday: ambition, then the first "no". The first hour was all wishes: a voice, a model from the internet, a radar. Within 75 minutes I had dropped the idea of running AI models on my own machine and uninstalled Ollama. It was too slow and too heavy. The services would do the thinking.
Then came the principle that shaped everything after. The radar on screen was fake, and I said so: it "is just mimicking". From then on, every number JARVIS shows had to be real.
By lunchtime the shape appeared: a glowing core, with conversations as small floating windows around it, "like in a movie". By evening I had hit my Claude usage limit twice.
Friday: a product, not a demo. JARVIS moved from the middle of the screen to the bottom, where he became the main button: tap to talk. Much of the day went into the phone layout. I made a few changes with OpenAI's Codex and had Claude review them. At 21:37 I created the GitHub repository. The first commit landed nine minutes later, a day and a half into the project.
Just before midnight came the question that set the architecture: why not talk to OpenAI or Gemini directly? The web version became serverless. The page talks to the AI service straight from your browser, and your key never leaves it.
Saturday: the busiest day, and the biggest rethink. I sent 209 messages. By 1:29 in the morning I had moved from Claude Pro to Claude Max. By lunchtime we had a new rule for the whole design: the connection is the core. One connected service answers, hears and speaks.
Out went OpenRouter, Claude as a chat option, and the two speech models we had been running ourselves, Kokoro for the voice and Moonshine for hearing. In a browser they were five to ten times slower than on a PC. I also wrote down the standard I wanted the code held to: nicely written, no hacks, an architecture anyone can follow.
At one point that day I had $1.90 of OpenAI credit left. Gemini's free tier ran out in the middle of a test.
Sunday: conversation first, and the moment it got out of hand. A test suite that drives a real browser arrived at lunchtime. In the afternoon I dictated an idea by voice: the AI model, not the app's own rules, should decide where each reply belongs. Talk stays in the conversation, and research gets a thread.
Then, at 21:48, the most useful message of the week. I asked why there were two notification boxes on screen doing the same job, and admitted "this project is getting out of hand". I was losing track of how it worked.
Claude re-read the entire repository and wrote a six-step plan to simplify it. I replied "ok do all six". The six steps were committed between 22:47 and 00:22. Each was type-checked, unit-tested, run end to end in a real browser, and checked on desktop and phone before the next began.
Monday: fast, then focused. Answers had been slow, and now we found out why. Every question went out with the web search tool attached and the model's thinking left at its default. So we offered search only when a question wants the world, and asked the model to think briefly. A greeting went from 7.8 seconds to 2.8. A simple fact went from 5.6 to 1.8. A news search went from 44.7 seconds to 7.2.
Then came a slider, so each person can choose how long the model thinks. Next, an experiment: letting JARVIS find and open files on the PC. Halfway through, I realised it was a different product. I deleted the branch.
The afternoon undid Sunday's big idea. Testing found two bugs with the same cause: "open a new thread and find X" left the new thread empty and answered elsewhere, and a YouTube video landed in the caption under JARVIS, which fades after a few seconds. Both came from the model choosing where each reply belonged. It guessed well most of the time, and a guess that is right most of the time is a bug generator. So the app decides now: every answer goes into the thread you have open, or a new one. The same afternoon, the app's own command parser went the same way. It had been scanning every sentence for command words, and "silence of the lambs, who directed it" muted him. A command now counts only when it is the whole of what you said.
The obstacles
- AI in a browser is slow. Voice and hearing models that feel instant on a PC crawl in a web page. The answer was to stop running models ourselves.
- There is no "sign in with ChatGPT" for apps like this. People have to bring an API key, and most have never seen one. So the first-run guide explains where to get a key and what it costs, and Gemini's free tier is the default way in.
- Free tiers run out exactly when you test. Gemini's allowance for voice ran out mid-session more than once. That is why the console falls back to the device's own voice and says so, instead of going quiet.
- Browsers are not the same. Brave has no built-in dictation. Microphone and sound permissions behave differently everywhere. A phone draws its own bars over your page.
- The model has limits too. I hit Claude's usage limit three times. Ten times the conversation outgrew the model's memory and had to be summarised, and the work carried on from those notes.
- Services don't always mean what they list. One OpenAI account offered a model the API then refused for every single question.
What we got wrong
The QA log has 127 numbered fixes. A sample, because they are more instructive than the successes:
- The instrument panels always covered the threads, because they lived on two separate layers.
- "Tidy up" piled every window into the top-left corner.
- A thread was named ".", the leftover of a sentence the parser misread.
- JARVIS said "Put all 1 thread away, sir. They're…"
- Gemini's key travelled in the request address, where servers write it to logs. It moved to a header.
- Closing a thread put another in front, so the next question landed in the wrong conversation.
- After a change to how windows glide, dropping one window onto another quietly stopped making a group, and no test noticed. A check in a real browser found it.
- One message appeared in two notification boxes at once. That was the "two pills".
- "Show me the weather in Paris" opened the weather panel and threw the question away. "Delete all the duplicates in a list in Python" offered to delete the whole board.
Most were found the same way. I used it, took a screenshot and said what looked wrong. I sent 202 screenshots in five days.
What went right
- A rule about truth. "Truthful over friendly": a count means what the screen shows, and a confirmation reports what actually happened. It keeps catching real bugs. On the last day JARVIS said "Opened" for a file that Windows had not opened. The wording turned out to be hiding a real failure.
- Real data only. No fake readings, even when a fake would have looked better.
- One codebase, two versions. The same code runs as a web page and on a PC. A fix lands in both.
- Tests that use a real browser. There are 122 unit tests and 23 end-to-end scenarios. Each scenario starts from a clean console, and all of them run on every push.
- Deleting. The git history shows 40,281 lines added and 16,611 removed. Some of the best days were mostly removals.
Everything we rethought
- Day one: local AI models through Ollama, dropped by mid-morning. A "drive mode" nobody needed.
- Day two: a proxy server we did not need, once the web version could talk to the services directly.
- Day three: OpenRouter, Claude as a chat option, and our own voice and hearing models.
- Day four: hand-written rules for what the user meant. The model decides now, and the console's own commands shrank to a small, fixed set. The OpenAI software library went too, replaced by plain web requests.
- Day five: PC control, which became its own future product. And the day-four idea itself: the model no longer decides where a reply goes, and the app no longer reads commands out of the middle of sentences. One rule each, decided by the app.
An early "code mode", where JARVIS handed a request to Claude Code, got as far as a plan and a trial run. Each of these felt like a setback at the time. None of them was.
Who did what
I sent 476 messages. A typical one was 14 words. I sent 202 screenshots, and I stopped Claude mid-task 17 times to change course.
Claude replied 4,295 times and made 6,354 tool calls: 2,394 commands, 1,620 file edits and 1,464 actions in a browser. That is about nine replies and thirteen actions for every message I wrote.
Five days, counted from the session logs and git history: what I put in, and what the models did with it.
Alt text · 14 characters
By the numbers
My part was deciding. What JARVIS is, and what it is not. What looks wrong. When something is getting too complicated. When an idea belongs to a different product. That meant testing on my own devices, and saying no, often.
The models' part was everything that turns a decision into working software: the code, the tests, the documentation, the QA log, the commits, and checking all of it in a real browser.
The bottleneck was never typing code. It was knowing what I wanted, seeing what was wrong, and saying no.
Opus 5 and Fable 5.1
Two Claude models did almost all of the work.
Claude Opus 5 carried most of days one to three: the first build, the web version and the move to one connected service. It came back on day five for the PC experiment and the afternoon's rethink. It gave 2,654 replies, 3.03 million output tokens and 3,761 tool calls.
Claude Fable 5.1 joined on day two and carried most of day four and the morning of day five: conversation first, the six-step simplification, faster answers and the thinking slider. It gave 1,579 replies, 2.30 million output tokens and 2,533 tool calls.
The logs show a difference in working style. Fable wrote more per reply, 1,454 tokens against 1,142, and made more tool calls per reply, 1.60 against 1.42. Fable also leaned on the command line: 1,261 commands to 535 direct file edits, often writing one small script that changed many files at once. Opus worked file by file, with 1,132 commands to 1,081 edits.
Treat this as a record, not a benchmark. The two models worked on different days and on different jobs. A few small runs from Claude Sonnet 5 came from that early code-mode trial.
What it cost
I started on Claude Pro. It lasted about five hours on the first day: I hit its usage limit at 14:21, and again at 19:21. By the third day I had moved to Claude Max, and even that ran out once, late on the fourth night. A build like this runs for hours at a time: reading the whole project, running commands, driving a browser. That is what the subscription pays for.
The app itself cost almost nothing to run. Gemini's free tier and a few dollars of OpenAI credit covered testing, and GitHub Pages hosts the web version for free.
What I would tell someone starting
- Your job moves from typing to deciding. Most of my messages were one sentence. The hard part was knowing which sentence.
- Say no early. The best decisions of the week were removals, and the first came 75 minutes in.
- Write your principles down. "Truthful over friendly" and "no hacks" settled dozens of small arguments before they started.
- Use it like a user. Screenshots of real screens found what tests missed. Then turn each find into a test.
- Schedule the simplification. Complexity arrives quietly. When you notice two things doing one job, stop and re-read everything.
- Don't let anything guess. A rule that is right most of the time produces bugs that look random. When a bug seems random, look for the guess behind it and replace it with one rule.
- Know which product you are building. A good idea can still belong somewhere else.
- Measure the machine too. The logs told me things my memory had wrong.
In one paragraph
Five days, 39 hours of active work, one person and two AI models produced a working product, 132 commits and 127 fixes. The models wrote almost every line. The product, though, came from a long run of small human decisions: what to keep, what to drop, what looked wrong, and when to stop and simplify. Building with AI didn't remove the hard part of making software. It removed everything around it, and left the hard part standing in plain view.
Try it at jarvis.edgarasneverdauskas.com. The code is open source under AGPL-3.0 at github.com/Evirtual/jarvis. Built with Claude Code, Claude Opus 5 and Claude Fable 5.1.