Google recently announced that the ChromeDriver now has support for headless mode, but what exactly is a headless browser? When and why should we use it? How does it differ from the standard web browsers?

This article will demonstrate the use of the new headless mode on Chrome and introduce you to one of the fundamental practices of modern software development: automated tests in a  Continuous Integration environment. We will walk through a quick tutorial on how to use the new ChromeDriver in headless mode as well as how to integrate it with Jenkins in order to run the tests.

Introduction to Headless Browsers

Headless browsers are browsers like any other but without a graphical user interface (GUI). The GUI allows us humans to visualize an aesthetic representation of the page source code (usually in HTML, CSS, and Javascript). Headless browsers fetch the page the same way standard browsers do, and can render the page (screenshot), and save it as an image file if requested.

Although headless browsers don’t render the page for users, they are not faster than standard browsers. The main advantages of using them are that these browsers are more lightweight and therefore require less computational resources, such as memory and processing. They are ideal to use in environments that don't have a GUI: for example, a Continuous Integration server.

Setup ChromeDriver

First of all, we will need to download a version of ChromeDriver that supports the headless mode. In this article I am going to use version 2.3, which you can download for your operating system here.

Building a Small Test with Selenium

Now, we will create a small application to run an automated test with Selenium using the ChromeDriver's headless mode. I will start a project using Maven and list the dependencies and plugins we will need to build a basic script to automate a simple test.

Here’s my pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>ac</groupId>
  <artifactId>snippets</artifactId>
  <version>1.0-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>snippets</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <junit.version>4.11</junit.version>
    <selenium.version>3.4.0</selenium.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>${junit.version}</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.seleniumhq.selenium</groupId>
      <artifactId>selenium-java</artifactId>
      <version>${selenium.version}</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-simple</artifactId>
      <version>1.7.25</version>
    </dependency>
  </dependencies>

  <build>
    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.20</version>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>
</project>

And here is our main script:

 1  package ac;
 2 
 3  import..
 15
 16 
 17 public class HeadlessTest {
 18 Logger log = LoggerFactory.getLogger(HeadlessTest.class);
 19 
 20 @Test
 21 public void HeadlessChromeDriverTest() throws IOException {
 22 
 23 System.setProperty("webdriver.chrome.driver", "src/test/resources/drivers/chromedriver");
 24 ChromeOptions chromeOptions = new ChromeOptions();
 25 chromeOptions.addArguments("--headless");
 26 
 27 WebDriver driver = new ChromeDriver(chromeOptions);
 28 driver.navigate().to("https://www.avenuecode.com/");
 29 
 30 String pageTitle = driver.getTitle();
 31 log.info("Page opened: {}", pageTitle);
 32 
 33 File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
 34 //copying the file into /screenshots directory
 35 FileUtils.copyFile(scrFile, new File("output/screenshots/homepage.png"));
 36 
 37 Assert.assertTrue(pageTitle.contains("Trusted Advisors for E-Commerce | Avenue Code"));
 38 log.info("Quitting driver");
 39 driver.quit();
 40 }
 41 }

At line 23, we set the location of our driver (the one you downloaded for your operating system in section 2).

At line 24, we instantiate the ChromeOptions object which contains all the options for your driver. You can view it as a sort of “configuration” parameter for Chrome. The most important detail to notice is at line 25, where we tell the driver to use the headless mode by passing the argument “--headless”.

Line 27 instantiates the ChromeDriver receiving your options object and fetching the Avenue Code website at line 28.

What happens from line 30 to 37 is quite simple: we test whether the website title matches our String in line 38.

One of the most important sections are lines 33 through 35. At line 33, we create a new file named “srcFile” and reference it to the screenshot taken of the page by the driver. Then we copy this file to the directory specified at line 35. So every time we need to take a screenshot from a page, we have to repeat the process of lines 33 and 35. For cleaner and more concise code, you can create a method to repeat this process and receive the file name as a parameter.

To build and run the test from the terminal we can type:

mvn clean verify

This will execute all your tests inside the test source directory (src/test/java). This works thanks to the “mvn-surefire-plugin” we added into our pom.xml.

And here are the test results:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running ac.HeadlessTest
Starting ChromeDriver 2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5) on port 10205
Only local connections are allowed.
Jul 05, 2017 7:24:39 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: OSS
[main] INFO ac.HeadlessTest - Page opened: Trusted Advisors for E-Commerce | Avenue Code
[main] INFO ac.HeadlessTest - Quitting driver
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.768 s - in ac.HeadlessTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.224 s
[INFO] Finished at: 2017-07-05T19:24:44-03:00
[INFO] Final Memory: 18M/183M
[INFO] ------------------------------------------------------------------------

 And we can check the screenshot taken by the driver at the “screenshot” directory of the project.

Now, it’s important to have this project in a source control repository if you want to integrate it with Jenkins in the next section.

A Quick Introduction to Continuous Integration

Continuous Integration (CI) is a term we are hearing more and more often and is a requirement for most modern software development. Here’s a definition of CI by John Ferguson Smart:

“Continuous Integration, in its simplest form, involves a tool that monitors your version control system for changes. Whenever a change is detected, this tool automatically compiles and tests your application. If something goes wrong, the tool immediately notifies the developers so that they can fix the issue immediately.”

This process makes software development more reliable and less prone to bugs by providing fast feedback. We also usually run regression and integration tests, check code quality, and provide metrics for every build.

But we have to keep in mind that Continuous Integration is more like a practice than certain tools: the entire team has to learn how to think the “Continuous Integration way”, and needs to understand all the pipeline steps from development through deployment to keep the CI server reliable.

Setting up Jenkins

Now that you know what a CI server should provide, it is time to create a simple Job that will run our automated tests from the first part of the tutorial.

Jenkins is a Java application, so you will need a Java Development Kit (JDK) installation of at least version 5.0. To check whether you have an installation of Java, you can type the following command in your terminal:

$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

You can install Jenkins by accessing the website and downloading the installer for your operational system. Once installed, it will start an instance of Jenkins in your machine and you will be able to access it by opening your browser and typing the following URL:

http://localhost:8080

Another way to run Jenkins is to download the Jenkins binary file as a Generic Java Package. This file is distributed as a WAR file so you can use it to run independently of your operational system and without an installation process. After downloading the WAR file, go to the directory that contains the “jenkins.war” and type the following command from your terminal to start Jenkins at the port 8080:

$ java -jar jenkins.war

If this is the first time you start Jenkins, it will ask you for a password. You can find the password on Windows where it is stored at “C:/Program Files (x86) /Jenkins/secrets/initialAdminPassword”. On my Linux, it was stored at “/.jenkins/secrets/initialAdminPassword”. It is important to note that this will be the default password for your “admin” user unless you change it afterwards.

Depending on the version of Jenkins you installed, it will suggest that you download and install a few plugins when you open it for the first time, and you can either install the recommended plugins or do it later as we will in the next section. Here is what my Jenkins homepage looks like:

That’s it! If you see the Jenkins homepage, that means you are almost ready to create your first job.

Creating Your First Job

In order to get the best of Jenkins, we will need a few plugins to incorporate into our jobs. To install plugins you must first go to the “Manage Jenkins” section on the left side panel and then click on “Manage Plugins”. This page is where you can install new plugins by clicking on the “Available” tab and then on the “Installed” tab to see the plugins you already have.

For this tutorial we will need the following plugins:

1. Git plugin
2. JDK Parameter plugin

We will also need to implement some configurations in order for the plugins to work correctly. In order to configure the plugins, we go to “Manage Jenkins” and then to “Global Tool Configuration”. You should see the “JDK” and “Git” sections in this page.

Let’s first set our JDK parameters. After clicking on “Add JDK”, the section will expand and display a few fields to which you can assign any name you want. For this example, I named them “JDK 1.8". Now for the JDK itself, you have two options: to provide the PATH of your JDK installation in your machine or let Jenkins automatically download it for you. If you choose the automatic option, all you need to do is provide the version you want Jenkins to use. Here is what my JDK configuration looks like:

 Configuring Git is almost the same process. You can either specify a name and the path to your executable Git or choose to let Jenkins automatically download it. Here is what my Git configuration looks like:

With everything set, we can now create our job to run the automated test using our headless driver.

On the Jenkins homepage, click on “New Item” on the left sidebar. Then, choose a name for your job and select the “Freestyle project”. On the next page we will configure our job.

Let’s start by configuring the “Source Code Management” section. Since we are using Git for version control we can select the “Git” option. Next, we can add the repository URL into the “Repository URL” field and specify the branch you want to clone from. In my case, I used the master branch. Here is my source code configuration:

In the “Build” section, we tell Jenkins how it should build our project after cloning the repository. Here, we should put the same command we used to run our tests locally in the first part of this article. If you are on a Mac or a Linux you can add the “Execute shell” for your build step. If you are on Windows, use the “Execute Windows batch command”. Now, type “mvn clean test” into the command field and save your job. Here is my Build section configuration:

With everything set, we can now run the job! Go to the Jenkins dashboard and you will see your newly created job. Now you just need to click on the small clock with the green arrow on the right side to start the build. The first time might take a few minutes because it will download the code from the repository as well as all the needed dependencies.

Here are the results on the console output of Jenkins (which should be similar to the output you had running locally in your machine). Note that I omitted most of the parts of the log and left the important lines where Jenkins clones the repository as well as builds and runs the project :

Cloning the remote Git repository
Cloning repository https://dematsum@bitbucket.org/dematsum/ac_snippets.git
..
Fetching upstream changes from https://dematsum@bitbucket.org/dematsum/ac_snippets.git
 > git --version # timeout=10
 > git fetch --tags --progress https://dematsum@bitbucket.org/dematsum/ac_snippets.git +refs/heads/*:refs/remotes/origin/*
..
Checking out Revision b82c92ebee900bf44748522a227f2a703c0d84ba (refs/remotes/origin/master)
..
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running ac.HeadlessTest
Starting ChromeDriver 2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5) on port 32365
Only local connections are allowed.
[main] INFO ac.HeadlessTest - Page opened: Trusted Advisors for E-Commerce | Avenue Code
[main] INFO ac.HeadlessTest - Quitting driver
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.628 s - in ac.HeadlessTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.771 s
[INFO] Finished at: 2017-07-05T19:28:13-03:00
[INFO] Final Memory: 21M/198M
[INFO] ------------------------------------------------------------------------
Finished: SUCCESS

If there are any changes to the code, the job will retrieve them from the repository every time you run it. You can also navigate from the Jenkins UI (localhost:8080) through the project folder “workspace” and look for the folder “screenshots” where we saved the screenshot taken by the Chrome webdriver.

It’s important to note that Jenkins doesn’t work only for Java. You can use it in any programming language you want. All you have to do is set the right shell command you use to run your project, and it should work without any problem.

Troubleshooting 

1. Check if you can run the test locally on your machine. If there is a failure while running directly from your terminal, it is very likely that it will fail trying to run on Jenkins as well.
2. If you are using different machines with different operating systems both locally and for hosting Jenkins (for example Mac OS locally and Jenkins on a Linux), you must use the appropriate ChromeDriver for each operating system.
3. If your test works locally, try executing the processes Jenkins would do manually: clone a clean copy of the project from your repository and try executing the "mvn clean test" command. You may find out that the problem is related to the repository, or that you didn't push all your local code, etc.

 References

Bidelman, Eric. (2017, April). Getting Started with Headless Chrome. Retrieved from https://developers.google.com/web/updates/2017/04/headless-chrome

Sangaline, Evan. (2017, April 14). Running Selenium With The New Headless Chrome. Retrieved from https://intoli.com/blog/running-selenium-with-headless-chrome

Smart, J. Ferguson. Jenkins: The Definitive Guide. Sebastopol: O'Reilly Media, 2011.