San Francisco
I’m Arvin Bhangu, and I’m building Roam, an autonomous vehicle company: hardware, autonomy and manufacturing in one system.
Before Roam I built the first uncensored large language model, trained from scratch, tokenizer through serving stack. It reached 1.2 million people in 30 hours. Before that, a mortgage company underwriting against bitcoin and private company stock, and a decade of shipping software and hardware starting at 14.
SentencePiece byte-fallback BPE trained on the corpus, 50,304 pieces, so nothing arrives out of vocabulary.
24TB of text and code, cut by about 87% through deduplication and quality filtering, leaving 800 billion training tokens.
56 blocks at d_model 8,192, and multi-query attention: 64 query heads reading one key head and one value head.
512 A100s across 64 nodes. Megatron-LM and DeepSpeed ZeRO-1, tensor parallel 8, data parallel 64, BF16 with FP32 master weights.
Instruction following, without collapsing back into the refusal behaviour it was built to avoid.
Perplexity and bits-per-byte on held-out text, then reasoning, coding and hallucination rate against the models I had running.
Multi-query KV caching, dynamic batching, INT8 weight-only quantization, streaming generation.
Autoscaling GPU capacity while 1.2 million people arrived in the 30 hours after the live Fox News segment.
Who I back
Why I build
I like working on problems that are hard, useful, and worth spending years on.
I don't care much whether the answer is software, AI, or a machine. I care whether it works, whether people need it, and whether it can become important.
I want to build things that didn't exist before.
Seven documents, published in full.
Almost everything protecting money and identity today rests on two assumptions: that the mathematics holds, and that whoever is attacking you is a person. A large enough quantum computer ends the first. A capable enough model ends the second.
I wrote these before either assumption was uncomfortable to question. What breaks in authentication and settlement when both go, and what can be built that does not require betting neither arrives. They are still the questions I work on.
Read all sevenSoftware I shipped between 2009 and 2021 reached more than 25 million people, before any of the current work.
I trained the model from the tokenizer up and built the stack that served it.
I set what the model would refuse, because the training data and the post-training were mine.
I built the lending and settlement stack that let people buy property against bitcoin or private company stock without selling either.
Banks had no way to underwrite volatile or untraded collateral, so a borrower could be genuinely wealthy and still fail a mortgage application.
I hold a private pilot licence and have flown more than a hundred hours. I have spent more time in airports than almost anyone who does not work in one, and I have watched the same ground operations repeat in ninety-one countries, in every language, at the same pace they ran thirty years ago.
I build things that fly too, on a much smaller scale: drones and radio-controlled aircraft, which means control loops, telemetry, radio links and the failure modes you only meet by putting something in the air yourself.
Not until I understand how it works. Ninety-one countries so far, including two months crossing Colombia by motorcycle, eighteen days living with a tribe in the Peruvian Amazon, and four months in China building hardware.
Hardware I built
Lenses, image sensors and the capture path.
Application-specific silicon and the thermal and power design around it.
Power delivery for devices expected to run unattended.
Generation and storage, sized against real load.
Control loops, telemetry and radio links.
Local storage and GPU boxes built to train and serve models without renting them.
