Home

How small can a language model be while still doing something useful? I wanted to find out, and had some spare time over the holidays.

Z80-μLM is a character-level language model with 2-bit quantized weights ({-2,-1,0,+1}) that runs on a Z80 with 64KB RAM. The entire thing: inference, weights, chat UI, it all fits in a 40KB .COM file that you can run in a CP/M emulator and hopefully even real hardware!

It won't write your emails, but it can be trained to play a stripped down version of 20 Questions, and is sometimes able to maintain the illusion of having simple but terse conversations with a distinct personality.

--

The extreme constraints nerd-sniped me and forced interesting trade-offs: trigram hashing (typo-tolerant, loses word order), 16-bit integer math, and some careful massaging of the training data meant I could keep the examples 'interesting'.

The key was quantization-aware training that accurately models the inference code limitations. The training loop runs both float and integer-quantized forward passes in parallel, scoring the model on how well its knowledge survives quantization. The weights are progressively pushed toward the 2-bit grid using straight-through estimators, with overflow penalties matching the Z80's 16-bit accumulator limits. By the end of training, the model has already adapted to its constraints, so no post-hoc quantization collapse.

Eventually I ended up spending a few dollars on Claude API to generate 20 questions data (see examples/guess/GUESS.COM), I hope Anthropic won't send me a C&D for distilling their model against the ToS ;P

But anyway, happy code-golf season everybody :)

The article discusses the lessons learned from fighting internet censorship in Iran and Russia, and outlines strategies for staying ahead of censors in 2025. It emphasizes the importance of developing adaptable, decentralized technologies to circumvent censorship and empower users to access the open internet.

The article explores the process of creating a virtual bookshelf using the Claude language, a new AI-powered programming tool. It discusses the challenges and techniques involved in building an interactive and visually appealing bookshelf application.

The article discusses the growing use of remote exams and AI-powered tools in the accounting industry, with the Association of Chartered Certified Accountants (ACCA) addressing concerns about AI-assisted cheating and ensuring the integrity of professional qualifications.

The article explores the development of a panorama visualization tool called St-Panorama, which combines spherical images and street-level data to create an immersive virtual environment. The tool aims to enhance urban planning and analysis by providing a comprehensive view of city landscapes.

libgodc is a Go library that provides a simple and efficient way to interact with the Oracle Database Connection (ODC) library. It allows Go developers to access and manage Oracle databases directly from their Go applications.

The article discusses the upcoming launch of the digital euro, a new form of digital currency that will be issued and controlled by the European Central Bank. It explains the potential benefits and uses of the digital euro, as well as the challenges and implications of its introduction for individuals and businesses.

The article explores the potential benefits and drawbacks of introducing a four-day workweek, including improved work-life balance, increased productivity, and concerns about potential job losses and economic impact.

China conducted extensive live-fire military drills around Taiwan in response to a visit by a U.S. congressional delegation, raising tensions in the region and prompting concerns about the potential for conflict.

The article discusses Elon Musk's unsuccessful attempt to reduce government spending, which instead led to a 6% increase. It suggests that Musk's plans to 'slash' government spending through his involvement with Dogecoin failed to materialize as intended.

The article discusses the author's experience of switching to eSIM in 2025 and their subsequent regret. It highlights the challenges and limitations faced with eSIM technology, as well as the lack of flexibility and control compared to traditional SIM cards.

The article explores how market design principles can be applied to improve food distribution and aid programs to better serve the needs of the poor. It discusses how insights from economic theory and practical implementations can help address challenges in delivering food assistance effectively and equitably.

This article discusses a technique for quickly restoring a large number of files (over 1 million) from a backup, focusing on using parallel processing and optimized file transfer methods to improve the restoration speed.

The article discusses the decline of night train services in Europe and the challenges faced by companies trying to revive this mode of transportation. It explores the economic, operational, and regulatory factors that have contributed to the demise of night trains and the efforts to reinvigorate this service.

The article discusses the rising prices of DDR5 RAM modules, which are essential for the latest computer hardware. It highlights the impact of these price increases on the consumer electronics market and the efforts of companies like Framework to provide more affordable options for users.

Hacker News has been my homepage and my inspiration for many years. While I’m still learning the ropes of building a public website, I created DreyX.com out of a simple necessity: I wanted a better way to track AI news without all the fluff. Literally a tool built by a curious reader, for curious readers. Thoughts? Suggestions?

The article discusses a proposal by political scientist David Runciman to give children as young as six the right to vote, as part of a broader plan for post-pandemic reconstruction. Runciman argues that this would help address the imbalance in political representation and empower younger generations to shape the future.

The article discusses the retirement of the MetroCard system in New York City, as the Metropolitan Transportation Authority (MTA) transitions to a contactless payment system. This change aims to modernize the city's subway and bus fare collection, providing a more convenient and efficient payment experience for commuters.

This article examines how cities are increasingly using their procurement power to influence corporate practices, such as pushing tech companies to improve privacy and security policies. It highlights the growing trend of cities leveraging their buying power to drive positive change across various industries.

Korean Air, a major South Korean airline, suffered a data breach that exposed the personal information of thousands of its employees. The breach affected employee data such as names, dates of birth, and contact details.