Using GitHub Copilot to Automate Tests

Code completion is nothing new. Tools like IntelliSense have allowed developers to become more productive by attempting to automatically complete the name of a function or statement they’re in the middle of writing, but the tools available only have a certain level of actual “intelligence” available.

With GitHub’s limited release of Copilot, they’re taking advantage of the powers of AI to boost developer productivity by not only trying to complete a function’s name but by trying to suggest the entire function itself!

What is GitHub Copilot?

GitHub Copilot is a new tool from GitHub that provides real-time code suggestions when working inside of VS Code as an extension.

It’s powered by OpenAI, trained on billions of lines of public code, courtesy of projects hosted on GitHub itself, giving it the ability to have a wide variety of authors and languages to base those suggestions on.

How does it work?

Copilot will typically give suggestions as you type in any scenario, but the two compelling examples that GitHub provides are being able to create a function based on a comment or based on the name of the function.

For instance, if I were to write:

// returns a sorted array of objects based on date

Copilot will get started with a new function, then once accepted, continue to write that function.

Alternatively, I can supply the function name like:

function sortArrayByKey

Where Copilot could then write that entire function for me!

If I don’t like the first answer it gives, I also have the option to look through alternatives, including pulling up all the answers Copilot thing could be helpful in this context.

What other kinds of things can it do?

There aren’t really any hard limitations beyond the basic gist of the tool being a way to suggest code. That just means you’ll be able to get a suggestion for finishing a function, but not necessarily for scaffolding a project with new files and directories.

While it can suggest really anything like an entire React component, it seems to really excel at utility functions that have a clear goal such as custom functions to sort an array like above or filtering data. Building components can be pretty specific and not include dependencies, where those functions often only require a specific input and output.

Copilot can even read the context from the active file. If you’re trying to sort or filter data based on what exists in that file, it will try to use that context in its suggestion.

Can it write tests?

Writing functions is one thing, but can Copliot produce real, meaningful tests? Well, it depends.

Context is an important factor in tests. If you’re writing a unit test with Jest to harden business logic or a Cypress test to walk through your UI, that information is typically scattered in other files or not even available until it renders in a browser. Copilot can write a test based on similar situations, but it wouldn’t “just work”.

That said, when using testing frameworks like Applitools, you can get visual testing coverage with code that doesn’t need to be super specific to your application, only requiring it to load in the browser, where Applitools steps in and does the heavy lifting.

Part of Copilot’s wide net of training data includes a variety of Applitools examples, meaning we can easily write our new Eyes checks right in our existing testing frameworks.

Is Copilot the future of development?

The release of Copilot and tools like it brings a new era in what the software we’re using can do for us in day-to-day development. By leveraging AI and machine learning, we’re able to train computers to do the heavy lifting and mundane tasks while we focus on solving the real problems, not how to fight with our text editor.

For a lot of developers, getting this information is a typical Google search with a look at a Stack Overflow answer. Even if only using it for common functions, we’re saving time and mental resources not having to look those things up ourselves.

But Copilot is still just another tool in our belt. It shows us what’s possible, but like all other tools, it’s ultimately up to the developer to take that knowledge and use it to its potential.