Automating with GitLab Duo, Part 1: Generating tests

Automated testing is time-consuming and can feel like it’s not moving a project forward. However, as many developers have likely experienced, automated testing provides an overall positive return on investment. In building a custom module (we'll call it gitlab-helper for this article), this was particularly true.

Our initial development focused on migrating tried and used functionality from existing scripts to a new module whose sole purpose was to serve as a baseline for future functionality. Although existing scripts lacked automated testing, their consistent usage was strong anecdotal evidence the functionality worked as expected.

Our objective was to deliver a more mature solution to this problem, so automated testing became a necessity. This introduced the challenge of building efficiently, while balancing the time to test and ensure a robust product; and with a total of three team members, this was no small bottleneck. Therefore, the team decided to take advantage of GitLab Duo, our suite of AI capabilities, for test generation, improving speed and quality of the delivered product.

In this three-part series on automating with GitLab Duo, we will cover:

1. How we used GitLab Duo to generate tests for our code
2. How we worked interactively with GitLab Duo for more complex situations
3. The results we were able to achieve (Spoiler: 1 developer + GitLab Duo = 84% coverage in 2 days)

Using GitLab Duo to generate tests for code

While functionality is available across tools, this article will cover using GitLab Duo in VS Code, with the GitLab Workflow extension for VS Code to generate tests. Links to other GitLab Duo options are available in the references below.

Install and enable GitLab Duo

As a prerequisite to using GitLab Duo, we ensured we had a GitLab Duo-enabled account. If you don't have GitLab Duo, you can sign up for a free 60-day trial.

To use GitLab Duo Chat in VS Code, we followed the instructions for installation. Then, we were able to see the GitLab Duo Chat extension on the sidebar and open the Chat window.

Generate tests with Chat

gitlab-helper is a custom module built for standardizing interaction with the GitLab API across the team's work and extends other library functionalities to simplify development and scripting work. Once a method or feature was migrated to gitlab-helper and appeared to be implemented appropriately, the process to generate tests for it was simple:

• Select the method, class, or entire file in the IDE.
• Right-click on the selected code.
• Under GitLab Duo Chat, select Generate tests.

Within a few seconds, tests were generated and presented in the GitLab Duo Chat window. These tests can be reviewed and or added to the codebase, via copy/paste, into existing or new test files. As is the case with most natural language processing generations today, particularly around context, some of the initial tests created by GitLab Duo failed, thus requiring finetuning (for instance, when dealing with nested dependencies).

Pro tip: GitLab Duo does not auto-create files to add generated tests to. We found it was helpful to create new test files and add a # Tests Generated by Duo comment at the top of them and suffix them with _duo.py to indicate where the tests came from.

GitLab Duo provided a great starting point for building out gitlab-helper’s automated testing and greatly improved test writing efficiency and code coverage, speeding up the development process substantially. Alongside GitLab Duo, numerous iterations of valuable tests were introduced into the gitlab-helper module with human oversight.

Read the next installment in this series where we share what we learned while using GitLab Duo for generating automated tests and working interactively with AI for more complex situations.

References

There’s more than one way to use GitLab Duo to generate tests, check out the other options below:

• The GitLab UI
• The GitLab Web IDE (VS Code in the cloud)
• VS Code, with the GitLab Workflow extension for VS Code
• JetBrains IDEs, with the GitLab Duo Plugin for JetBrains
• Visual Studio for Windows, with the GitLab Extension for Visual Studio