Calculus on Computational Graphs: Backpropagation
Backpropagation is the key algorithm that makes training deep models computationally tractable. For modern neural networks, it can make training with gradient descent as much as...
Read moreNews
Understanding LSTM Networks
Humans don’t start their thinking from scratch every second. As you read this essay, you understand each word based on your understanding of previous words. You...
Read moreReading postmortems
I love reading postmortems. They’re educational, but unlike most educational docs, they tell an entertaining story. I’ve spent a decent chunk of time reading postmortems at...
Read moreSlashdot and Sourceforge
If you’ve followed any tech news aggregator in the past week (the week of the 24th of May, 2015), you’ve probably seen the story about how...
Read moreThe googlebot monopoly
TIL that Bell Labs and a whole lot of other websites block archive.org, not to mention most search engines. Turns out I have a broken website...
Read moreA defense of boring languages
Boring languages are underrated. Many appear to be rated quite highly, at least if you look at market share. But even so, they’re underrated. Despite the...
Read moreAdvantages of monorepos
Here’s a conversation I keep having: Someone: Did you hear that Facebook/Google uses a giant monorepo? WTF! Me: Yeah! It’s really convenient, don’t you think? Someone:...
Read moreWe used to build steel mills near cheap power. Now that’s where we build datacenters
Why are people so concerned with hardware power consumption nowadays? Some common answers to this question are that power is critically important for phones, tablets, and...
Read moreReading citations is easier than most people think
It’s really common to see claims that some meme is backed by “studies” or “science”. But when I look at the actual studies, it usually turns...
Read moreGiven that we spend little effort on testing, how should we test software?
I’ve been reading a lot about software testing, lately. Coming from a hardware background (CPUs and hardware accelerators), it’s interesting how different software testing is. Bugs...
Read moreWhat happens when you load a URL?
I’ve been hearing this question a lot lately, and when I do, it reminds me how much I don’t know. Here are some questions this question...
Read more