### Running minerl within a Container

This blog post outlines the steps required to run the minerl environment, which is used for the The MineRL Benchmark for Agents that Solve Almost-Lifelike Tasks (MineRL BASALT) competition.

There are several benefits to running this way:

• The minerl environment and OpenAI VPT models are tricky to set up
• The containerised version has all the correct versions and paths set up in 1 command
• This is particularly helpful for Windows users, who seem to experience most pain
• Submission to the competition requires a container version
• Developing in the right environment ensures that this is practically seamless
• Getting familiar with containers is a useful side-effect

### Outline

These are the steps that this involves, and leaves the final machine pretty quick to start/stop:

• Create / use a Host machine
• Ideally this machine will have a GPU
• I have published a blog post about how to do this in the cloud
• So that your whole GPU-based set-up runs for 22c per hour
• If you already have a decent desktop machine, this is a NOOP
• Enable the host to run podman (which is almost identical to docker) including the Nvidia pass-through into containers that we need
• I have published a blog post about how to do this for an Ubuntu (Linux) machine, so if that’s your preference, then the instructions are right there
• If you’re Windows-based, then I’d be glad to point to point to good docker / podman and Nvidia resources to link too…
• Build the minerl containers (these are one-line commands) to set up the environment nicely.
• This step is the focus of this blog post

Below, I’ll refer to podman (since it’s more Open Source oriented), but it’s likely that docker would work just as well.

### Ensure podman has the right registries

Run the following to check that the docker.io registry is enabled (since that’s where the aicrowd docker definition files are stored) :

Other useful commands for podman:

### Create the container for minerl-basalt-base

Our set-up is going to be based on the Dockerfile found here, which is within the basalt-2022-behavioural-cloning-baseline repo (i.e. it’s an official thing, that’s known to work with the rest of the minerl / aicrowd infrastructure):

The podman build command about includes the tag minerl-basalt-base so that we can consistently refer to the same basic container, which we can then build on.

Fortunately, once the base container is built, the various steps are cached, and if we add on to the end of the file (or run subsequent steps), the unchanged precursor steps are just retrieved from the local cache, and so it’s much quicker.

### Run the container for minerl-basalt-base

Run the following to see that the container ‘boots’ extremely fast, and is ready to run, including

Once we exit, though, the container evaporates : There’s nothing left - any changes are gone!

### Building a container for development : minerl-basalt-dev

These are just suggestions, that illustrate how we can now build on to of minerl-basalt-base

We’re going to add 3 new files into the basalt-2022-behavioural-cloning-baseline directory:

Contents of Dockerfile-dev (this is very similar in style to the original Dockerfile, except that it builds on minerl-basalt-base - so is quick to iterate with):

Contents of apt-dev.txt (this includes additional packages, over-and-above those already installed) :

Contents of environment-dev.txt (this includes additional dependencies, over-and-above those already installed) :

Finally, build the new development container with :

The simplest way to run it is (though we’re going to get more fancy soon) :

### Using port forwarding to the host

The following container run command also makes (say) jupyter-lab available in your browser:

Note that if your host is not your desktop machine (i.e. you are doing a ssh into your host), then if you also portforward your host to your local machine (e.g.: ssh -L 8585:localhost:8585) then the whole chain from container to your desktop localhost should work.

### Using host folder mounting

In order to have some persistent state stored by the container, we’ll mount host directories so that they’re available inside the container for read/write. Care has to be taken if we’re using rootless volumes :

### Complete commandline for minerl-basalt-dev

These ideas can be combined (i.e. port forwarding and directory sharing) :

### Terminate the GCP VM when done…

Using the scripted commands that we have (if you followed the previous blog post) in our ~/.bashrc :

### End

All done!

Shout-out to Google for helping this process by generously providing Cloud Credits to help get this all implemented!