What’s worked in Computer Science: 1999 v. 2015
In 1999, Butler Lampson gave a talk about the past and future of “computer systems research”. Here are his opinions from 1999 on “what worked”. Yes...
Read moreNews
Short Story on AI: A Cognitive Discontinuity.
The idea of writing a collection of short stories has been on my mind for a while. This post is my first ever half-serious attempt at...
Read moreInfinite disk
Hardware performance “obviously” affects software performance and affects how software is optimized. For example, the fact that caches are multiple orders of magnitude faster than RAM...
Read moreWhat a Deep Neural Network thinks about your #selfie
Convolutional Neural Networks are great: they recognize things, places and people in your personal photos, signs, people and lights in self-driving cars, crops, forests and traffic...
Read moreWhy Intel added cache partitioning
Typical server utilization is between 10% and 50%. Google has demonstrated 90% utilization without impacting latency SLAs. Xkcd estimated that Google owns 2 million machines. If...
Read moreNeural Networks, Types, and Functional Programming
Deep learning, despite its remarkable successes, is a young field – perhaps ten years old. While models called artificial neural networks have been studied for decades,...
Read moreVisual Information Theory
I love the feeling of having a new way to think about the world. I especially love when there’s some vague idea that gets formalized into...
Read moreNeural Networks, Types, and Functional Programming
Deep learning, despite its remarkable successes, is a young field – perhaps ten years old. While models called artificial neural networks have been studied for decades,...
Read moreSteve Yegge’s prediction record
I try to avoid making predictions1. It’s a no-win proposition: if you’re right, hindsight bias makes it look like you’re pointing out the obvious. And most...
Read moreCalculus on Computational Graphs: Backpropagation
Backpropagation is the key algorithm that makes training deep models computationally tractable. For modern neural networks, it can make training with gradient descent as much as...
Read moreUnderstanding LSTM Networks
Humans don’t start their thinking from scratch every second. As you read this essay, you understand each word based on your understanding of previous words. You...
Read more