Build Docker image for research projects with Jeremy Jordan

15 views
Skip to first unread message

Chang Lee

unread,
Mar 28, 2020, 7:47:18 AM3/28/20
to Penny University
Attendees
  • Chang Lee
  • Jeremy Jordan
Background

I have been working on a computer vision project at work in the past few months. One of the packages our team is investigating is detectron2
This is extremely annoying for me to as I have a Windows laptop and

  • Many dependencies does not have Windows support
  • I want to use CUDA but Windows Subsystem Linux (WSL) does not support CUDA 
  • Our company Macs GPUs are locked
  • No access to Cloud due to sensitive data legal issues
So I wanted to build a Docker image to use for both local development and production servers. I know Jeremy uses Docker for ML local development on image classification problems so I reached out to him.

Dockerfile for research projects

tl;dr Use the researcher's Dockerfile!

As a Docker noob, I thought I had to develop the Dockerfile from scratch. Instead, after talking with Jeremy for 10 minutes, he mentioned that the detectron2 team published their Dockerfile in the repo, so I could have built from there. 

Oh my gawd 🤣. This simple solution I didn't think of saved me at least 20 hours of tweaking with OS and CUDA!!

He says it's been a recent trend that researchers include the Dockerfile.

Jeremy's set up 

We also talked about how he incorporates Docker into his workflow. He uses VSCode to manage the containers with both the Docker and Remote Development extentions.

To manage Python dependencies, he uses 

  • requirements.txt
  • requirements-dev.txt 

to record the top-level dependencies. 

I have been using `pipenv` but am considering move to Jeremy's set up for research projects like this because the installations often include binaries and other messy stuff. 

Good resources 

I didn't know that Nvidia publish Dockerfiles that has CUDA for people to use. This is a huge time saver.
Reply all
Reply to author
Forward
0 new messages