Running Docker Image On Nersc

1 minute read

Published:

Running Docker image on NERSC

Shifter is developed at NERSC and is used to create image (e.g., Docker image) to run on NERSC due to the reserved root permission. This documentation shows the steps to use Shifter to download and run the image. And this docomentation shows how to run image in Jupyter.

Download Docker image

Here we use an example image ees16/tinerator:latest.

shifterimg -v pull docker:ees16/tinerator:latest

To see a list of images,

shifterimg images

Run image from the command line

To run the image from Cori command line,

shifter --image=docker:ees16/tinerator:latest -e HOME=$HOME /bin/bash

Run image on Jupyter

doc

You do not need to create a new kernel for this. The new kernel name inside ``~/.local/share/jupyter/kernels` will be used.

In this case, it is called shifter-jupyter.

Create kernel spec file

  • create kernel spec file kernel.json and put it under ~/.local/share/jupyter/kernels/shifter-jupyter/kernel.json
  • Put the following in the kernel.json file
{
 "argv": [
  "shifter",
  "-e",
  "HOME=/global/homes/p/pshuai",
  "--volume=/global/homes/p/pshuai/docker_files:/home/jovyan/work",
  "--image=ees16/tinerator:latest",
  "/opt/conda/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "shifter-jupyter",
 "language": "python"
}

Mount volume

Before mounting any local volume to the container, do the following.

setfacl -m u:nobody:x ~/ 
setfacl -m u:nobody:x ~/docker_files/ 

In this case, ~/docker_files is open to anyone in the system.

launch Jupyter

Use Docker on Mac

install Docker

First download Docker Desktop for Mac from here. Install the package on your mac.

Download Docker image

Here we use ees16/tinerator as an example

docker pull ees16/tinerator

See all available images

docker images

View the content of the image

docker run -it ees16/tinerator sh

Lauch the image on Jupyter

docker run -v $PWD:/home/jovyan/work -p 8888:8888 ees16/tinerator:latest jupyter lab

Note to kill the processes which are using or listening to port 8888, do the following

lsof -i TCP:8888
kill <PID>

If the notebook does not open, try to paste one of the links (similar) to the browser (i.e.,http://127.0.0.1:8888/?token=44dbc7fd4da2598a8797ff0657721b74589e9444315f5802 ).

http://(a65d926e15ba or 127.0.0.1):8888/?token=44dbc7fd4da2598a8797ff0657721b74589e9444315f5802