projects

TorchDynamo

I started the TorchDynamo project, which is now part of PyTorch 2.0. TorchDynamo is a Python-level JIT compiler designed to make unmodified PyTorch programs faster. TorchDynamo hooks into the frame evaluation API in CPython (PEP 523) to dynamically modify Python bytecode right before it is executed. It rewrites Python bytecode in order to extract sequences of PyTorch operations into an FX Graph which is then just-in-time compiled with a customizable backend. It creates this FX Graph through bytecode analysis and is designed to mix Python execution with compiled backends to get the best of both worlds: usability and performance.

TorchInductor

I started the TorchInductor project, which is now part of PyTorch 2.0. TorchInductor is a new compiler backend for PyTorch. TorchInductor took inspiration from how PyTorch users were writing high performance custom kernels: increasingly using the Triton language. We also wanted a compiler backend that used similar abstractions to PyTorch eager, and was general purpose enough to support the wide breadth of features in PyTorch. TorchInductor uses a pythonic define-by-run loop level IR to automatically map PyTorch models into generated Triton code on GPUs and C++/OpenMP on CPUs. TorchInductor’s core loop level IR contains only ~50 operators, and it is implemented in Python, making it easily hackable and extensible.

GoDaddy Domain Appraisals

While at GoDaddy, I created a system that uses neural networks to predict the resale price of a domain name in the aftermarket. GoDaddy Domain Appraisals (GoValue) is available to millions of GoDaddy customers and provides estimated values to help both buyers and sellers more effectively price domain names. GoValue is 1.25x better at predicting past domain name sale prices than human experts. You can try it yourself, learn how it works, or listen to me talk about it on a podcast.

Google Native Client

While interning at Google in 2010, I added support for sandboxing self-modifying code to Native Client (part of Google Chrome) to support dynamic language runtimes and JIT compilers. Native Client is a sandbox for running untrusted machine code in the web browser. We published this work in PLDI.

OpenTuner

OpenTuner is an open-source framework building domain-specific multi-objective program autotuners. OpenTuner supports customizable configuration representations, an extensible technique representation to allow for domain-specific techniques, and an easy to use interface for communicating with the tuned program. I was the primary author and original creator of this project.

PetaBricks

PetaBricks is a language and compiler where algorithmic choices are exposed explicitly to create programs that define a search space of possible algorithms. The PetaBricks compiler then uses empirical autotuning to search over these algorithms for an optimal version. I was the primary author and original creator of this project.

Kendo

Kendo is a library that allows multithreaded programs, that would normally produce non-deterministic output, to execute deterministically by enforcing a dynamically computed and efficient ordering of lock acquisitions. The primary author of this work is Marek Olszewski.

DMTCP

DMTCP (Distributed Multi-Threaded Checkpointing) is a tool to transparently checkpoint and restart the state of a distributed cluster computation that communicates through MPI or sockets. It works on unmodified binaries at the user level. I was the first author on the original DMTCP paper and designed and implemented the distributed component that converted a single-process checkpointer into a distributed checkpointer. I have moved on from this project, but it is continued by many wonderful researchers, and there are now 100+ refereed publications using or extending DMTCP.

Algorithmic Trading on LendingClub.com

Starting in 2012 I also did algorithmic trading on the secondary market of the peer-to-peer lending platform LendingClub. I used neural networks to predict the likelihood of default of specific loans, and then both buy and sell notes based on those predictions. LendingClub has since shut down, but I enjoyed this side project.