Learn how to install and configure Docker Engine in a Vagrant VM, plus connect it to your local Docker CLI and your preferred IDE.

Whether or not you are planning to deploy the software you're developing to containers, using Docker to run them, along with the services Docker depends on, in your local developer machine can be extremely helpful to test integration scenarios and avoid problems when the software is deployed to a remote environment.

This is something that's easy to do using Docker Desktop, but what if you just need the Docker Engine and the Docker CLI, and maybe a plug in to your favorite IDE? Can we do this without Docker Desktop?

In this article, I will show you how to install and configure Docker Engine in a Vagrant VM, as well as how to connect it to your local Docker CLI and to two of the most popular IDEs we have today: IntelliJ and VS Code.

Note: I will assume you are using a Mac operating system with ZSH as the default shell, but these steps can be easily modified to work on other operating systems.

Setting Up Your Computer

The first thing you have to do is to make sure Docker Desktop and Docker CLI are not installed on your Mac, since later you will have to install the Docker CLI on its own. If you need to uninstall these, follow the steps outlined here.

You will also need Homebrew to proceed with the steps below. You can see how to install it here, or simply run the command below:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Now you need Vagrant. Vagrant is a tool that helps us to install and manage virtual machines, and it is in one of these virtual machines, a Ubuntu one, that our Docker Engine will be running.

First, install Docker CLI, VirtualBox, and Vagrant with the commands below.

brew install virtualbox

brew install vagrant

brew install docker

brew install docker-compose

This last command will install only Docker CLI, not the Docker Engine or Docker Desktop, so if you try to use Docker now, you will receive the following error message:

docker run hello-world

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.

See 'docker run --help'.
Setting Up Vagrant

Now that you have everything that you need on your Mac, it's time to install what goes into Vagrant. First, let's create a VM to host Docker Engine. An important note here is that it will create a Vagrantfile in the current folder, so it might be a good idea to move to your home folder or create a particular folder to host it.

cd ~

vagrant init ubuntu/focal64

The Vagrantfile in the current folder will contain the VM settings. You will probably need to configure some port forwardings in this file, as shown in the example below:

config.vm.network "forwarded_port", guest: 3000, host: 3000

Now ssh into the VM and install the Docker Engine.

vagrant up

vagrant ssh

curl -fsSL https://get.docker.com -o get-docker.sh

sudo sh get-docker.sh

sudo apt install docker-compose

You will also need to create a user group for Docker and add the Vagrant user.

sudo addgroup docker

sudo adduser vagrant docker

Now exit the VM and reload.

exit

vagrant reload

Docker Engine and Docker CLI should be working inside the VM now. Let's test it.

vagrant ssh

docker run hello-world

Now let's go back to the host and configure it to connect to the Docker Engine that is running inside the VM.

exit

Add the Vagrant ssh key to the host known keys and configure Docker CLI to use the Vagrant VM.

export DOCKER_HOST=ssh://vagrant@127.0.0.1:2222

ssh-add --apple-use-keychain ~/.vagrant/machines/default/virtualbox/private_key

If you are running an old version of MacOS, chances are that you should replace the param --apple-use-keychain by the param -K.

Now run the commands below to make the two previous commands run when you start a new terminal. Adjust it to use the -K param if necessary.

echo -n "\nexport DOCKER_HOST=ssh://vagrant@127.0.0.1:2222\n" >> ~/.zshrc

echo -n "\nssh-add --apple-use-keychain ~/.vagrant/machines/default/virtualbox/private_key\n" >> ~/.zshrc

And reload your .zshrc to make sure it is working:

source ~/.zshrc

Now you should be ready to use Docker CLI in the host:

docker run hello-world

In case you have an error in the output of this command, with the message below, try running the command specified in the message:

 ssh -l vagrant -p 2222 -- 127.0.0.1 docker system dial-stdio

and it should fix the problem.

docker: error during connect:

Post "http://docker.example.com/v1.24/containers/create": command [ssh -l vagrant -p 2222 -- 127.0.0.1 docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Host key verification failed.

The last step is to test Docker Compose. Go to some project that has a properly configured docker-compose.yml file and spin that up:

docker-compose up

If you have a problem with credentials when running docker-compose, you need to make sure the credsStore is not set in the file ~/.docker/config.json. If it is, remove it. Just delete the json entry, save the file, and try again.

Testing Docker Compose

The last step is to test Docker Compose.

First, you need to make sure the credsStore is not set in the file ~/.docker/config.json. If it is, remove it. Just delete the json entry.

Now Docker Compose should be working fine. Go to some project that has a properly configured docker-compose.yml file and spin that up:

docker-compose up
Configuring IntelliJ

Docker CLI is ready, so now it's time to configure the IDEs.

Let's start by configuring IntelliJ to connect to Docker.

In the Docker settings, add a Docker configuration using SSH with the Host 127.0.0.1, port 2222, and OpenSSH config as authentication type:

Now let's configure IntelliJ to use Docker Compose via SSH.

In the run configuration of the Docker Compose file, set the DOCKER_HOST env var:

Configuring VS Code

If you use VS Code, it is even simpler. With the Docker CLI properly configured, Visual Studio should connect to Docker without problems and the Docker Explorer should show images and containers; you can run Docker commands as usual.

But you can also configure VS Code to work from the Vagrant VM, using Remote Development:

Install Remote SSH extension.

Click on the button "Open a Remote Window," in the lower lefthand corner, select "Connect to Host," and then select "VagrantVM." You are now connected to your remote Vagrant VM. Go to the Terminal and type 'docker ps' to test it.

That's it! You can now use Vagrant to host your Docker backend.

 

References

GitHub Gist, Set up a Vagrant VM for docker-based development

Linux Handbook, How to Set Up Remote Access to Docker Daemon [Detailed Guide]


Author

Rafael Romão

Rafael Romão is a Software Engineer at Avenue Code. He has studied software development since 1998 and has worked as a software developer since 2002. His main areas of interest are software architecture and design, as well as the development of highly productive teams.


You Need to Know Big O Notation: Here's Why

READ MORE

DApps: The New Generation of Applications

READ MORE

How to Save Time and Effort Using The Test Pyramid

READ MORE