Learn how to install and configure Docker Engine in a Vagrant VM, plus connect it to your local Docker CLI and your preferred IDE.
Whether or not you are planning to deploy the software you're developing to containers, using Docker to run them, along with the services Docker depends on, in your local developer machine can be extremely helpful to test integration scenarios and avoid problems when the software is deployed to a remote environment.
This is something that's easy to do using Docker Desktop, but what if you just need the Docker Engine and the Docker CLI, and maybe a plug in to your favorite IDE? Can we do this without Docker Desktop?
In this article, I will show you how to install and configure Docker Engine in a Vagrant VM, as well as how to connect it to your local Docker CLI and to two of the most popular IDEs we have today: IntelliJ and VS Code.
Note: I will assume you are using a Mac operating system with ZSH as the default shell, but these steps can be easily modified to work on other operating systems.
Setting Up Your Computer
The first thing you have to do is to make sure Docker Desktop and Docker CLI are not installed on your Mac, since later you will have to install the Docker CLI on its own. If you need to uninstall these, follow the steps outlined here.
You will also need Homebrew to proceed with the steps below. You can see how to install it here, or simply run the command below:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
Now you need Vagrant. Vagrant is a tool that helps us to install and manage virtual machines, and it is in one of these virtual machines, a Ubuntu one, that our Docker Engine will be running.
First, install Docker CLI, VirtualBox, and Vagrant with the commands below.
brew install virtualbox
brew install vagrant
brew install docker
brew install docker-compose
This last command will install only Docker CLI, not the Docker Engine or Docker Desktop, so if you try to use Docker now, you will receive the following error message:
docker run hello-world
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
Setting Up Vagrant
Now that you have everything that you need on your Mac, it's time to install what goes into Vagrant. First, let's create a VM to host Docker Engine. An important note here is that it will create a Vagrantfile in the current folder, so it might be a good idea to move to your home folder or create a particular folder to host it.
cd ~
vagrant init ubuntu/focal64
The Vagrantfile in the current folder will contain the VM settings. You will probably need to configure some port forwardings in this file, as shown in the example below:
config.vm.network "forwarded_port", guest: 3000, host: 3000
Now ssh into the VM and install the Docker Engine.
vagrant up
vagrant ssh
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo apt install docker-compose
You will also need to create a user group for Docker and add the Vagrant user.
sudo addgroup docker
sudo adduser vagrant docker
Now exit the VM and reload.
exit
vagrant reload
Docker Engine and Docker CLI should be working inside the VM now. Let's test it.
vagrant ssh
docker run hello-world
Now let's go back to the host and configure it to connect to the Docker Engine that is running inside the VM.
exit
Add the Vagrant ssh key to the host known keys and configure Docker CLI to use the Vagrant VM.
export DOCKER_HOST=ssh://vagrant@127.0.0.1:2222
ssh-add --apple-use-keychain ~/.vagrant/machines/default/virtualbox/private_key
If you are running an old version of MacOS, chances are that you should replace the param --apple-use-keychain by the param -K.
Now run the commands below to make the two previous commands run when you start a new terminal. Adjust it to use the -K param if necessary.
echo -n "\nexport DOCKER_HOST=ssh://vagrant@127.0.0.1:2222\n" >> ~/.zshrc
echo -n "\nssh-add --apple-use-keychain ~/.vagrant/machines/default/virtualbox/private_key\n" >> ~/.zshrc
And reload your .zshrc to make sure it is working:
source ~/.zshrc
Now you should be ready to use Docker CLI in the host:
docker run hello-world
In case you have an error in the output of this command, with the message below, try running the command specified in the message:
ssh -l vagrant -p 2222 -- 127.0.0.1 docker system dial-stdio
and it should fix the problem.
docker: error during connect:
Post "http://docker.example.com/v1.24/containers/create": command [ssh -l vagrant -p 2222 -- 127.0.0.1 docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Host key verification failed.
The last step is to test Docker Compose. Go to some project that has a properly configured docker-compose.yml file and spin that up:
docker-compose up
If you have a problem with credentials when running docker-compose, you need to make sure the credsStore is not set in the file ~/.docker/config.json. If it is, remove it. Just delete the json entry, save the file, and try again.
Testing Docker Compose
The last step is to test Docker Compose.
First, you need to make sure the credsStore is not set in the file ~/.docker/config.json. If it is, remove it. Just delete the json entry.
Now Docker Compose should be working fine. Go to some project that has a properly configured docker-compose.yml file and spin that up:
docker-compose up
Configuring IntelliJ
Docker CLI is ready, so now it's time to configure the IDEs.
Let's start by configuring IntelliJ to connect to Docker.
In the Docker settings, add a Docker configuration using SSH with the Host 127.0.0.1, port 2222, and OpenSSH config as authentication type:
Now let's configure IntelliJ to use Docker Compose via SSH.
In the run configuration of the Docker Compose file, set the DOCKER_HOST env var:
Configuring VS Code
If you use VS Code, it is even simpler. With the Docker CLI properly configured, Visual Studio should connect to Docker without problems and the Docker Explorer should show images and containers; you can run Docker commands as usual.
But you can also configure VS Code to work from the Vagrant VM, using Remote Development:
Install Remote SSH extension.
Click on the button "Open a Remote Window," in the lower lefthand corner, select "Connect to Host," and then select "VagrantVM." You are now connected to your remote Vagrant VM. Go to the Terminal and type 'docker ps' to test it.
That's it! You can now use Vagrant to host your Docker backend.
References
GitHub Gist, Set up a Vagrant VM for docker-based development
Linux Handbook, How to Set Up Remote Access to Docker Daemon [Detailed Guide]
