Deno 1.20 Release Notes

Fine-Tune a Semantic Segmentation Model with a Custom Dataset

Post Content

Image search with 🤗 datasets

Post Content

Accelerate BERT inference with Hugging Face Transformers and AWS Inferentia

Post Content

Deep Neural Nets: 33 years ago and 33 years from now

The Yann LeCun et al. (1989) paper Backpropagation Applied to Handwritten Zip Code Recognition is I believe of some historical significance because it is, to my...

Why is it so hard to buy things that work well?

There’s a cocktail party version of the efficient markets hypothesis I frequently hear that’s basically, “markets enforce efficiency, so it’s not possible that a company can...

Guiding Text Generation with Constrained Beam Search in 🤗 Transformers

Post Content

How to use Google Analytics in Deno Deploy

An example of how to use server-side Google Analytics in Deno

The 2030 Self-Driving Car Bet

It’s my honor to announce that John Carmack and I have initiated a friendly bet of $10,000* to the 501(c)(3) charity of the winner’s choice: By...

BERT 101 – State Of The Art NLP Model Explained

Post Content

How to publish Deno modules to NPM

dnt and oak – making code work across Deno and Node.js

Misidentifying talent

[Click to collapse / expand section on sports] Here are some notes from talent scouts: Recruit A: … will be a real specimen with chance to...

Learning with not Enough Data Part 2: Active Learning

This is part 2 of what to do when facing a limited amount of labeled data for supervised learning tasks. This time we will get some...

Deno 1.19 Release Notes

Deno 1.19 improves our web streams integration, introduces a `deno vendor` subcommand, and adds many other new features.

Fine-Tune ViT for Image Classification with 🤗 Transformers

Post Content

Static Files on Deno Deploy

Deno Deploy runs Deno programs at the edge, now with support for static files.

Andrew Ng: Unbiggen AI

Andrew Ng has serious street cred in artificial intelligence. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late...

How AI Will Change Chip Design

The end of Moore’s Law is looming. Engineers and designers can do only so much to miniaturize transistors and pack as many of them as possible...

A list of every web API in Deno

Have you ever wondered how web compatible Deno is? In this blog post I explain every single web API available in Deno.

Atomically Thin Materials Significantly Shrink Qubits

Quantum computing is a devilishly complex technology, with many technical hurdles impacting its development. Of these challenges two critical issues stand out: miniaturization and qubit quality....

Cocktail party ideas

You don’t have to be at a party to see this phenomenon in action, but there’s a curious thing I regularly see at parties in social...

A decade of major cache incidents at Twitter

This was co-authored with Yao Yue This is a collection of information on severe (SEV-0 or SEV-1, the most severe incident classifications) incidents at Twitter that...

Getting Started with Sentiment Analysis using Python

Post Content

Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers

Post Content

Updated Public Financial Information

Post Content

Making the web better. With blocks!

You’ve probably seen web editors based on the idea of blocks. I’m typing this in WordPress, which has a little + button that brings up a...

Deno in 2021

Retrospect of what happened in 2021 and what’s coming in 2022

Supercharged Searching on the 🤗 Hub

Post Content

Welcome Stable-baselines3 to the Hugging Face Hub 🤗

Post Content

Deno 1.18 Release Notes

Deno 1.18 marks the finalization of the Web Cryptography API, and adds, stabilizes, and improves many features.

​How to Upgrade Debian 10 Buster to Debian 11 Bullseye

This article shows how to upgrade your Debian 10 Buster system to Debian 11, Bullseye via command line.

Case Study: Millisecond Latency using Hugging Face Infinity and modern CPUs

Post Content

Boosting Wav2Vec2 with n-grams in 🤗 Transformers

Post Content

Deploy GPT-J 6B for inference using Hugging Face Transformers and Amazon SageMaker

Post Content

January 4th incident update

Post-mortem report for the 40 minute service outage on on 4th January 2022.

Updating The Single Most Influential Book of the BASIC Era

In a way, these two books are responsible for my entire professional career. With early computers, you didn’t boot up to a fancy schmancy desktop, or...

Active Learning with AutoNLP and Prodigy

Post Content

Gradio is joining Hugging Face!

Post Content

The container throttling problem

This is an excerpt from an internal document David Mackey and I co-authored in April 2019. The document is excerpted since much of the original doc...

Deno 1.17 Release Notes

Deno 1.17 adds JSON imports, import completions in the REPL, and much more.

Perceiver IO: a scalable, fully-attentional model that works on any modality

Post Content

Deno joins TC39

Deno has joined the JavaScript standards committee to improve the language for everyone.

Some thoughts on writing

I see a lot of essays framed as writing advice which are actually thinly veiled descriptions of how someone writes that basically say “you should write...

Breaking Out of the Box

Training CodeParrot 🦜 from Scratch

Post Content

Some latency measurement pitfalls

This is a pseudo-transcript (actual words modified to be more readable than a 100% faithful transcription) of a short lightning talk I did at Twitter a...

Learning with not Enough Data Part 1: Semi-Supervised Learning

When facing a limited amount of labeled data for supervised learning tasks, four approaches are commonly discussed.

Introducing Snowball Fight ☃️, our first ML-Agents environment

Post Content

Web Streams at the Edge

Deno Deploy provides web standard streaming infrastructure to easily whip up real-time applications.

Getting Started with Hugging Face Transformers for IPUs with Optimum

Post Content

Introducing the Data Measurements Tool: an Interactive Tool for Looking at Datasets

Post Content

Major errors on this blog (and their corrections)

Here’s a list of errors on this blog that I think were fairly serious. While what I think of as serious is, of course, subjective, I...

Accelerating PyTorch distributed fine-tuning with Intel technologies

Post Content

Slack Introduces New Platform With Help From Deno

Today Slack has announced their next generation development platform. Slack chose Deno for its “secure by default” principles, its web standard APIs, and its first-class TypeScript...

Individuals matter

One of the most common mistakes I see people make when looking at data is incorrectly using an overly simplified model. A specific variant of this...

Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers

Post Content

Connecting to Postgres from the edge

Many serverless at edge products can not connect to Postgres because they don’t support TCP. Deno Deploy can.

Deno 1.16 Release Notes

Deno 1.16 adds support for new JSX transforms, reading files from disk using fetch, and much more.

Culture matters

Three major tools that companies have to influence behavior are incentives, process, and culture. People often mean different things when talking about these, so I’ll provide...

Scaling up BERT-like model Inference on modern CPU – Part 2

Post Content

Deno Deploy Beta 3

Deno Deploy is a multi-tenant distributed JavaScript VM running in 28 data centers across the world.

How to Install Odoo 15 on Ubuntu 20.04

This article explains how to install and deploy Odoo 15 inside a Python virtual environment on Ubuntu 20.04. Odoo is a popular open-source suite of business...

Large Language Models: A New Moore’s Law?

Post Content

Course Launch Community Event

Post Content

Train a Sentence Embedding Model with 1B Training Pairs

Post Content

How to Sell UX Research with Two Simple Questions

Willingness to look stupid

People frequently1 think that I’m very stupid. I don’t find this surprising, since I don’t mind if other people think I’m stupid, which means that I...

The Age of Machine Learning As Code Has Arrived

Post Content

What to learn

It’s common to see people advocate for learning skills that they have or using processes that they use. For example, Steve Yegge has a set of...

Some reasons to work on productivity and velocity

A common topic of discussion among my close friends is where the bottlenecks are in our productivity and how we can execute more quickly. This is...

Fine tuning CLIP with Remote Sensing (Satellite) images and captions

Post Content

Deno 1.15 Release Notes

Deno 1.15 improves Web Crypto support, adds deno uninstall subcommand, stabilizes URLPattern and more.

Showcase Your Projects in Spaces using Gradio

Post Content

Hosting your Models and Datasets on Hugging Face Spaces using Streamlit

Post Content

What is an SSL certificate? / SSL certificate: A Complete Insight

SSL certificate is a digital certificate issued by Certificate Authorities to the websites, which ensures all the information exchanged between the user’s web browser and the...

The value of in-house expertise

An alternate title for this post might be, “Twitter has a kernel team!?”. At this point, I’ve heard that surprised exclamation enough that I’ve lost count...

How to Train Really Large Models on Many GPUs?

[Updated on 2022-03-13: add expert choice routing.] [Updated on 2022-06-10]: Greg and I wrote a shorted and upgraded version of this post, published on OpenAI Blog:...

Summer at Hugging Face

Post Content

September 23 incident update

Post-mortem report for the 35 minute service outage on on September 23, 2021.

A Content Model Is Not a Design System

IPv4 vs IPv6

The Internet Protocol (IP) is a set of rules for that specifies addressing and routing the data between computers.

Deno 1.14 Release Notes

Deno 1.14 improves Web Crypto support, adds customization options to deno fmt and deno lint, introduces URLPattern, and more.

Introducing Optimum: The Optimization Toolkit for Transformers at Scale

Post Content

Hugging Face and Graphcore partner for IPU-optimized Transformers

Post Content

A Gentle Introduction to Graph Neural Networks

What components are needed for building learning algorithms that leverage the structure and properties of graphs?

Understanding Convolutions on Graphs

Understanding the building blocks and design choices of graph neural networks.

Deno Deploy Beta 2

Deno Deploy is a multi-tenant distributed JavaScript VM running in 25 data centers across the world.

Measurement, benchmarking, and data analysis are underrated

A question I get asked with some frequency is: why bother measuring X, why not build something instead? More bluntly, in a recent conversation with a...

Design for Safety, An Excerpt

Deno on MDN

Web compatibility data for Deno is now available on MDN.

How to Fix the WordPress White Screen of Death

Are you worried about the WordPress White Screen of Death? Then here are some practical ways to solve such errors and best practices to avoid WSOD.

Deno 1.13 Release Notes

Deno 1.13 stabilizes the native HTTP server, and introduces FFI, a streaming WebSocket API, the new self.structuredClone() function and more.

What is Localhost

Localhost is a hostname that refers to the computer system on which the calling program is running.

Sustainable Web Design, An Excerpt

How to Reverse a String in Python

This article goes through several different ways to reverse a string in Python.

Voice Content and Usability

July 27 incident update

Post-mortem report for the 6 minute service outage on July 27, 2021.

Designing for the Unexpected

Deep Learning over the Internet: Training Language Models Collaboratively

Post Content

Deno 1.12 Release Notes

The Deno 1.12 release adds server side WebSockets, support for WASM threads, new web crypto APIs and more.

Welcome spaCy to the Hugging Face Hub

Post Content

What are Diffusion Models?

[Updated on 2021-09-19: Highly recommend this blog post on score-based generative modeling by Yang Song (author of several key papers in the references)]. [Updated on 2022-08-27:...

Deploy Hugging Face models easily with Amazon SageMaker

Post Content

Distill Hiatus

After five years, Distill will be taking a break.

Linux ifconfig Command

ifconfig is a network management tool, used to configure and view the status of the network interfaces in Linux

Sentence Transformers in the Hugging Face Hub

Post Content

Deno Deploy Beta 1

Deno Deploy is a multi-tenant distributed JavaScript VM running in 25 data centers across the world.

A from-scratch tour of Bitcoin in Python

I find blockchain fascinating because it extends open source software development to open source + state. This seems to be a genuine/exciting innovation in computing paradigms;...

How to Install Plex Media Server on Ubuntu 20.04

This article explains how to install Plex Media Server on Ubuntu 20.04.

Deno 1.11 Release Notes

The Deno 1.11 release adds new web APIs, resolves long standing issues, and improves editor experience.

What is a 503 Service Unavailable Error

This article explains what a 503 error means, why you’re getting an HTTP 503 code, and how to troubleshoot these errors.

Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API

Post Content

Kinda a big announcement

The other day I was talking to a young developer working on a code base with tons of COM code, and I told him that even...

Contrastive Representation Learning

The goal of contrastive representation learning is to learn such an embedding space in which similar sample pairs stay close to each other while dissimilar ones...

Using & Mixing Hugging Face Models with Gradio 2.0

Post Content

How to Exclude in Grep

In this article, we’re going to show you how to exclude one or multiple words, patterns or directories when searching with grep.

Deno 1.10 Release Notes

Deno 1.10 improves the built-in test runner, adds support for Web Storage API, structured clone, and more.

How to Install Gitea on Ubuntu 20.04

Gitea is is a fast and easy to use self-hosted git server written in Go. This article explains how to install and configure Gitea on Ubuntu...

Adversarial Reprogramming of Neural Cellular Automata

Reprogramming Neural CA to exhibit novel behaviour, using adversarial attacks.

Understanding Linux File Permissions

In Linux, file permissions, attributes, and ownership control the access level that the system processes and users have to files.

Scaling-up BERT Inference on CPU (Part 1)

Post Content

Introducing 🤗 Accelerate

Post Content

Mailbag #2

Last month, I emailed readers announcing an upcoming mailbag post—the WBW post version of an AMA. 1,500 questions poured in—remarkably interesting, creative questions on a wide...

Deno 1.9 Release Notes

Deno 1.9 adds a new native HTTP/2 web server, a vastly improved op infrastructure, import completions in the LSP, an interactive permission prompt, blob url support.

How to Install and Configure an NFS Server on Ubuntu 20.04

This article explains how to set up an NFSv4 Server on Ubuntu 20.04. We’ll also show you how to mount an NFS file system on the...

Weight Banding

Weights in the final layer of common visual models appear as horizontal bands. We investigate how and why.

Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker

Post Content

Jakub Konka Hired Full Time

Post Content

Branch Specialization

When a neural network layer is divided into multiple branches, neurons self-organize into coherent groupings.

Understanding BigBird’s Block Sparse Attention

Post Content

Announcing the Deno Company

We have raised money to enable the millions of web programmers out there to maximally leverage their craft.

Short Story on AI: Forward Pass

The inspiration for this short story came to me while reading Kevin Lacker’s Giving GPT-3 a Turing Test. It is probably worth it (though not required)...

How to Install and Configure VNC on Ubuntu 20.04

This article explains how to install and configure a VNC server on Ubuntu 20.04. We’ll also show you how to securely connect to the VNC server...

The Partnership: Amazon SageMaker and Hugging Face

Post Content

Reducing Toxicity in Language Models

Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet. Pretrained language...

How to Set Up WireGuard VPN on Debian 10

This article explains how to install and configure WireGuard on Debian 10 that will act as a VPN server.

My Journey to a serverless transformers pipeline on Google Cloud

Post Content

How to Install Asterisk on Ubuntu 20.04

This tutorial explains how to install Asterisk on Ubuntu 20.04. Asterisk is a popular open-source PBX platform.

Fine-Tune Wav2Vec2 for English ASR in Hugging Face with 🤗 Transformers

Post Content

Install and Configure Fail2ban on Debian 10

This article explains how to install and configure Fail2ban on Debian 10. Fail2ban is a tool that helps protect your Linux machine from brute-force and other...

Hugging Face Reads, Feb. 2021 – Long-range Transformers

Post Content

Multimodal Neurons in Artificial Neural Networks

We report the existence of multimodal neurons in artificial neural networks, similar to those found in the human brain.

March 2 incident update

Post-mortem report for the 98 minute service outage on March 2, 2021.

Deno 1.8 Release Notes

Deno 1.8 adds experimental support for the WebGPU API, adds support for importing private modules, stabilizes import maps, revamps coverage tooling, adds ICU to provide internationalization...

How to Setup FTP Server with VSFTPD on Ubuntu 20.04

This article describes how to install and configure an FTP server on Ubuntu 20.04 that you use to share files between your devices.

Simple considerations for simple people building fancy neural networks

Post Content

Python For Loop

The for loop in Python iterates over the items of a sequence and repeatedly executes a block of statements.

How to Install VMware Workstation Player on Ubuntu 20.04

This article describes how to install VMware Workstation Player on Ubuntu 20.04.

Self-Organising Textures

Neural Cellular Automata learn to generate textures, exhibiting surprising properties.

Retrieval Augmented Generation with Huggingface Transformers and Ray

Post Content

Hugging Face on PyTorch / XLA TPUs

Post Content

How to Install and Use Wine on Ubuntu 20.04

This article describes how to install Wine on Ubuntu 20.04. Wine is an open-source compatibility layer that allows you to run Windows applications on Unix-like operating...

Visualizing Weights

We present techniques for visualizing, contextualizing, and understanding neural network weights.

How to Install Microsoft Edge Browser on Ubuntu 20.04

This article explains how to install the Microsoft Edge Browser web browser on Ubuntu 20.04.

Curve Circuits

Reverse engineering the curve detection algorithm from InceptionV1 and reimplementing it from scratch.

How to Install Python 3.9 on Debian 10

In this article, we’ll show you how to install Python 3.9 on Debian 10.

High-Low Frequency Detectors

A family of early-vision neurons reacting to directional transitions from high to low spatial frequency.

Bash wait Command

wait is a command that waits for the given jobs to complete and returns the exit status of the waited for command.

Faster TensorFlow models in Hugging Face Transformers

Post Content

Deno 1.7 Release Notes

Deno 1.7, one of our largest releases to date, adds cross compilation and 60% smaller binaries for deno compile, a DNS resolver API, support for data...

Fit More and Train Faster With ZeRO via DeepSpeed and FairScale

Post Content

Python Tuples

This article will walk you through the basics of Python tuples. Tuples are similar to lists, with the main difference being that the lists are mutable...

How we sped up transformer inference 100x for 🤗 API customers

Post Content

Deno in 2020

Retrospect of what happened in Deno in 2020

Website Redesigned with Community-Sourced Translations

Post Content

Find and Replace in Vim / Vi

Find and replace is a powerful feature of Vim, which allows you to make changes to your text quickly.

Bash: Write to File

This article explains how to write text to a file in Bash, using the redirection operators and tee command.

Controllable Neural Text Generation

[Updated on 2021-02-01: Updated to version 2.0 with several work added and many typos fixed.] [Updated on 2021-05-26: Add P-tuning and Prompt Tuning in the “prompt...

Bash read Command

read is a bash built-in command that reads a line from the standard input (or from the file descriptor) and split into words.

Against essential and accidental complexity

In the classic 1986 essay, No Silver Bullet, Fred Brooks argued that there is, in some sense, not that much that can be done to improve...

Listing Linux Services with Systemctl

This article explains how to use the systemctl command to list Linux services and check their status.

How to Use the nmap Command

Nmap is a powerful network scanning tool for security audits and penetration testing.

An overview of end-to-end entity resolution for big data

An overview of end-to-end entity resolution for big data, Christophides et al., ACM Computing Surveys, Dec. 2020, Article No. 127 The ACM Computing Surveys are always a great...

Zig 0.7.1 Released: 69 Bugs Fixed

Post Content

Chattr Command in Linux (File Attributes)

This article explains how to use the chattr command to change file attributes on Linux file systems.

Naturally Occurring Equivariance in Neural Networks

Neural networks naturally learn many transformed copies of the same feature, connected by symmetric weights.

Deno 1.6 Release Notes

Deno 1.6 adds the ability to build your Deno projects into fully standalone, self-contained executables with `deno compile`. The release also introduces a built-in LSP for...

Bias in word embeddings

Bias in word embeddings, Papakyriakopoulos et al., FAT*’20 There are no (stochastic) parrots in this paper, but it does examine bias in word embeddings, and how...

Zig at FOSDEM 2021: CfP is Open!

Post Content

How to Install PHP 8 on Ubuntu 20.04

This article will show you how to install PHP 8 on Ubuntu 20.04 and integrate it with Nginx and Apache.

Basename Command in Linux

basename is a command-line utility that strips directory and suffix from given file names.

Seeing is believing: a client-centric specification of database isolation

Seeing is believing: a client-centric specification of database isolation, Crooks et al., PODC’17. Last week we looked at Elle, which detects isolation anomalies by setting things up...

How to Convert Integer into String in Python

In Python, we can convert integers and other data types to strings using the built-in str() function.

Elle: inferring isolation anomalies from experimental observations

Elle: inferring isolation anomalies from experimental observations, Kingsbury & Alvaro, VLDB’20 Is there anything more terrifying, and at the same time more useful, to a database...

How to Install Flask on Ubuntu 20.04

In this article we’ll discuss how to install Flask on Ubuntu 20.04 inside a Python virtual environment.

Understanding RL Vision

With diverse environments, we can analyze, diagnose and edit deep reinforcement learning models using attribution.

who Command in Linux

who is a command-line utility that displays a list of currently logged in users. It can also show the current run level, time of the last...

Achieving 100Gbps intrusion prevention on a single server

Achieving 100 Gbps intrusion prevention on a single server, Zhao et al., OSDI’20 Papers-we-love is hosting a mini-event this Wednesday (18th) where I’ll be leading a panel discussion...

How to Install Python 3.9 on Ubuntu 20.04

In this article, we’ll show you two ways to install Python 3.9 on Ubuntu 20.04.

How to Install Nvidia Drivers on Ubuntu 20.04

This article describes how to install the proprietary Nvidia drivers on Ubuntu 20.04.

Virtual consensus in Delos

Virtual consensus in Delos, Balakrishnan et al. (Facebook, Inc.), OSDI’2020 Before we dive into this paper, if you click on the link above and then download...

Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models

Post Content

How to Undo Last Git Commit

In Git you can undo changes using the git reset command followed by the commit identifier.

Porting fairseq wmt19 translation system to transformers

Post Content

Install Odoo 14 on CentOS 8

This article explains how to install and deploy Odoo 14 inside a Python virtual environment on CentOS 8.

Helios: hyperscale indexing for the cloud & edge (part II)

Helios: hyperscale indexing for the cloud & edge, Potharaju et al., PVLDB’20 Last time out we looked at the motivations for a new reference blueprint for large-scale data...

Hyperparameter Search with Transformers and Ray Tune

Post Content

Rmmod Command in Linux

In this article, we’ll talk about how to use the rmmod command to remove modules from the Linux Kernel.

How to Build an Open-Domain Question Answering System?

[Updated on 2020-11-12: add an example on closed-book factual QA using OpenAI API (beta). A model that can answer any question with regard to factual knowledge...