Skip to main content

Comparing PDFs with the Applitools ImageTester CLI

The Applitools PDF Tool allows you to easily run visual UI tests on a collection of image files, by placing them inside a directory (also works with nested directory structure). It runs as a standalone jar file and can be invoked as a process by any programming language.

Steps for running tests

  1. Download the latest ImageTester.jar file from here: https://github.com/applitools/ImageTester.
Note

If you don't have your Applitools account yet, please sign up first and get your Applitools API Key that will be used next to execute the tests.

You can get the API Key by clicking on the Person Icon | "My API Key"

  1. Add your API Key as an environment variable (optional, but recommended).

  2. Create a folder which will contain the pdf under test (let's say /under_test).

  3. Copy new version of the pdf into the /under_test folder

  4. Run the ImageTester.jar jar file as shown below:

java -jar ImageTester.jar -k $APPLITOOLS_API_KEY -f <PATH>/under_test/
Note

The $APPLITOOLS_API_KEY is the environment variable name which contains the API key. You can instead just use -k <PASTE YOUR KEY>

  1. This will create your baseline for the test.

  2. Copy the second version of the pdf file into the under_test folder.

  3. Run the ImageTester jar file (same as in #5)

  4. This will now create a checkpoint and compare against the baseline that was set in #5

  5. Navigate to your dashboard to see the test results.

Advanced options

The tool build in java and requires minimal set of parameters the minimal command will look as follow:

java -jar ImageTester.jar -k [api-key] -f [path-to-pdf-file]

* If not set, the Jar will assume that the folder under test is the execution folder

  • Required parameters:
    • -k [api-key] - Applitools api key
  • Optional parameters and flags:
    • -f [path] - A path to target folder or file
    • -a [app-name] - Set the application name under directoryTest; default = ImageTester
    • -p [http://proxy{,user,pass}] - Set proxy and optional username + password
    • -s [server] - Set Applitools server url
    • -ml [match-level] - Set the comparison level, one from Strict/Content/Layout; Default = Strict
    • -br [branch] - Set the branch
    • -pb [parent-branch] - Set the parent branch
    • -bn [baseline] - Set custom baseline name
    • -vs [WidthxHeight] - Set the viewport size identifier
    • -lf [log-file] - Set log fine name to enable logging
    • -os [osname] - Set custom os
    • -ap [browser name] - Set browser or equivalent hosting application name
    • -th [number] - Specify max. concurrent workers (Threads). default= 3
    • -fn [testName] - Force all test names to be specific name. This will force all tests to be matched with a single baseline.
    • -fb [batchName] - Set unified flat batch to contain all the discovered tests regardless their hierarchy.
      • To add batch id to the flat batch use the following syntax:-fb BATCH_NAME<>BATCH_ID
    • -sq [sequenceName] - Set batch sequenceName for unified insights in applitools' dashboard.
    • -ms [{width x height}] - Match the size of the images to a specific width and/or height ie. 1000x- adjust by width, x600-adjust by height, 1000x600- fit to the exact size (note, if both provided, may loose proportions)
    • -ic [{header,footer,left,right}] - Set pixels to cut from each side (one or more) in the format [header,footer,left,right], including missing notations ie: -ic ,,10,4
    • -nc - Send batch notification on complete.
    • -as - Set automatic save on failures
    • -st - Split steps to individual tests
    • -id - Ignore displacement of shifting elements.
    • -pn - Prompt new tests, new tests will not be saved automatically, the user will have to review and save manually.
    • -dv - Disable SSL certificate validation. !!!Unsecured!!!
    • -lo - Use legacy files order to comply with baselines that were created with versions below 2.0
    • -ac [Level:GuidelineVer] - Set accessibility validation and optionally it's arguments split by semicolumns ':' default: "AA:WCAG_2_0", available options: [AA|AAA:WCAG_2_0|WCAG_2_1]
    • -dcb - ImageTester will not automatically close batch(es) when test is complete.
    • -mp - ImageTester will read from and execute tests based on the batch mapper CSV.
    • -ir - Ignore regions will be applied to all pages. Example: -ir "300,300,500,100|500,500,200,200" will create ignored regions at
      • x:300, y:300, with width:500, height:100 and
      • x:500, y:500, with width:200, height:200
    • -cr - Apply content regions to all pages. Example: -cr "300,300,500,100|500,500,200,200" will create content regions at
      • x:300, y:300, with width:500, height:100 and
      • x:500, y:500, with width:200, height:200
    • -lr - Layout regions will be applied to all pages. Example: -ir "300,300,500,100|500,500,200,200" will create layout regions at
      • x:300, y:300, with width:500, height:100 and
      • x:500, y:500, with width:200, height:200

For Documents (PDFs) only

  • -di [dpi] - Set the quality of the conversion on PDF files
  • -sp [pages] - Comma separated page numbers\range to include in PDF testing (ie: 1,2,5,7,10-15); Default all included
  • -pp [password] - The password if the PDF files protected
  • -pn - Preserve original directory test names when specifying pages

Using The Batch Mapper

The Batch Mapper is a feature that allows you to specify tests from a CSV instead of supplying a path or file in the traditional way.

A common use-case is the desire to run tests on a PDF that has changed it's composition. Consider a PDF that initially contains 4 pages (Page 1, Page 2, Page 3, Page 4). Now imagine that this PDF changes over time, and 2 new pages are added to the beginning of the PDF. It can be difficult to test these pages given the current composition of the ImageTester.

For this use case, it can be beneficial to use the Batch Mapper feature.

To use it, supply a path to a batch mapper configuration file (a '|' delimited CSV).

Batch Mapper parameters

  • filePath- The location of files (PDF/Image) being tested
  • testName - The name of the test as it will appear on the Applitools dashboard (Optional)
  • app - The app name of the test as it will appear on the Applitools dashboard (Optional)
  • os - The operating system to be tested on (Optional)
  • browser - The browser to be tested on (Optional)
  • viewport - The viewport for the test -vs flag (Optional)
  • matchsize - The size to adjust the image , -ms flag (Optional)
  • pages- Pages to be tested for this PDF. (ie: 1,2,5,7,10-15) (Optional)
  • matchLevel- The MatchLevel for the test (Optional)

Additional Notes

To have each of your tests appear in the same batch, set the APPLITOOLS_BATCH_ID environment variable in your CLI environment.

Sample CSV:

filePath|testName|app|os|browser|viewport|matchsize|pages|matchLevel
docs/a.pdf|Test1|AppA|Linux|Chrome|1024x748||1|Strict
docs/a.pdf|Test2|AppA||||x748|1-3|Layout
docs/b.pdf|Test3|AppB|||||2-5|

Enterprise features in combination with Eyes Utilities

By placing the Eyes-Utilities jar into the same folder as the ImageTester Jar, new enterprise api features made possible by providing an enterprise read-key.

java -jar ImageTester.jar -k [api-key] -vk [view-key][options]

  • Required parameters:
    • -k [api-key] - Applitools api key
    • -vk [view-key] - Applitools enterprise view-key
  • Selective flags - Required one or more
    • -gd - Get diff images of the failed steps
    • -gi - Get images of the failed steps
    • -gg - Get animated gifs of the failed steps
  • Optional parameters and flags:
    • -of [path] - Specify custom output path or path-template

CI/CD integration

Once setting the required environment variables, the ImageTester is able to pick them up and use to sync with the results with other tools that out there for CI/CD integration.

The parameters:

  • JOB_NAME - The name of the job as it was set in CI/CD
  • APPLITOOLS_BATCH_ID - Job id, a unique identifier of the current job.

Analyze your test results

Congratulations! You've successfully run your first visual UI test with Applitools Eyes! A detailed report is ready for your inspection at the Applitools Eyes test manager. Watch this 5 minute video to get acquainted with the test manager and to learn the basics of baseline maintenance.

Login to Applitools and analyze the results.

Having Trouble?

Don't suffer in silence! Let us help you. Please reach out to us to get your project working.

Taking the next steps with Applitools

Congratulations on completing this quickstart! There's still so much to learn about visual testing with Applitools, but you're off to a great start.

Resources for next steps:

  1. 🤖 Learning how visual testing works
  2. ↔️ Setting match levels for visual checkpoints
  3. 💥 Troubleshooting common issues
  4. 🐞 Reporting bugs
  5. 🗺 Detailed overview of visual testing with Applitools

You can also: