Using GitHub Copilot to Automate Tests

Advanced Topics — August 9, 2021

Code completion is nothing new. Tools like IntelliSense have allowed developers to become more productive by attempting to automatically complete the name of a function or statement they’re in the middle of writing, but the tools available only have a certain level of actual “intelligence” available.

With GitHub’s limited release of Copilot, they’re taking advantage of the powers of AI to boost developer productivity by not only trying to complete a function’s name, but by trying to suggest the entire function itself!

What is GitHub Copilot?

GitHub Copilot is a new tool from GitHub that provides realtime code suggestions when working inside of VS Code as an extension.

Code suggestion from GitHub Copilot
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-page-query

It’s powered by OpenAI, trained on billions of lines of public code, courtesy of projects hosted on GitHub itself, giving it the ability to have a wide variety of authors and languages to base those suggestions off of.

How does GitHub Copilot work?

Copilot will typically give suggestions as you type in any scenario, but the two compelling examples that GitHub provides is being able to create a function based off of a comment or based off of the name of the function.

For instance, if I were to write:

// returns a sorted array of objects based on date

Copilot will get started with a new function, then once accepted, continue to write that function.

Copilot suggesting a new function
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-function-by-comment

Alternatively, I can supply the function name like:

function sortArrayByKey

Where Copilot could then write that entire function for me!

Copliot suggesting code for a function
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-function-by-name

If I don’t like the first answer it gives, I also have the option to look through alternatives, including pulling up all the answers Copilot thing could be helpful in this context.

GitHub Copilot synthesizing multiple results
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-synthesized-function-results-by-name

What other kinds of things can Copliot do?

There aren’t really any hard limitations beyond the basic gist of the tool being a way to suggest code. That just means you’ll be able to get a suggestion for finishing a function, but not necessarily for scaffolding a project with new files and directories.

While it can suggest really anything like an entire React component, it seems to really excel at utility functions that have a clear goal such as custom functions to sort an array like above or filtering data. Building components can be pretty specific and not include dependencies, where those functions often only require a specific input and output.

Copilot can even read context from the active file. If you’re trying to sort or filter data based on what exists in that file, it will try to use that context in its suggestion.

Context-aware suggestions from Copilot
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-filter-data

Can Copilot write tests?

Writing functions is one thing, but can Copliot produce real, meaningful tests? Well, it depends.

Copilot test suggestions for a utility function
https://gist.github.com/colbyfayock/eb523d3359de66c7899c0b14330fc5b3#file-unit-test

Context is an important factor with tests. If you’re writing a unit test with Jest to harden business logic or a Cypress test to walk through your UI, that information is typically scattered in other files or not even available until it renders in a browser. Copilot can write a test based on similar situations, but it wouldn’t “just work”.

That said, when using testing frameworks like Applitools, you can get visual testing coverage with code that doesn’t need to be super specific to your application, only requiring it to load in the browser, where Applitools steps in and does the heavy lifting.

https://twitter.com/colbyfayock/status/1410066304585850886

Part of Copilot’s wide net of training data includes a variety of Applitools examples, meaning we can easily write our new Eyes checks right in our existing testing frameworks.

Is Copilot the future of development?

The release of Copilot and tools like it bring a new era in what the software we’re using can do for us in day-to-day development. By leveraging AI and machine learning, we’re able to train computers to do the hard lifting and mundane tasks while we focus on solving the real problems, not how to fight with our text editor.

For a lot of developers, getting this information is a typical Google search with a look at a Stack Overflow answer. Even if only using it for common functions, we’re saving time and mental resources not having to look those things up ourselves.

But Copilot is still just another tool in our belt. It shows us what’s possible, but like all other tools, it’s up to the developer to ultimately take that knowledge and use it to its potential.

Are you ready?

Get started Schedule a demo