REST API testing with Katalon

Katalon Background

Recently, I had the opportunity of figuring out how to test a REST API using Katalon. Katalon is a free, QA-friendly testing studio that can handle both API and user interface test automation. Basically once the tests are written, executing the tests is as simple as clicking a button from within the app, or another step in a CI/CD pipeline via console mode execution. In an environment of having QA people manually running tests, Katalon is a good first step to test automation.

REST API Testing Background

In breaking down a project to have a split front end (user interface) and back end (API), it is helpful to have tests around both the back end as well as front end. Typically, a REST API contains basic CRUD methods (Create, Read, Update, and Delete), all of which should be tested and verified as working correctly. Basing an integration test flow on the order of the acronym allows to you test each method by interacting with your own test data, and not changing any existing data. Specifically, an integration test flow can look like this:

1. Create a new resource – verify response values match what was passed in to create
2. Read the resource – verify response values match values in create
3. Update the resource – verify response values match what was passed in to update
4. Read the resource – verify that update actually persisted the change
5. Delete the resource – verify success response code
6. Read the resource – verify that 404 not found is returned

Project Structure

The best way to explain the project structure is with this graphic taken from Katalon docs:

The Tests Explorer is the side panel, while Test Cases, Object Repository, and Test Suites are all top level folders displayed in this panel. Actual test cases live in the Test Cases folder, web service call definitions (“Web service end-point” in the graphic) live in the object repository, and test suites live in the Test Suites folder. Other points of interest in the Tests Explorer are the Profiles folder and the Reports folder.

Global Variables

Global variables are defined by “profile”, and can be used to exchange data between different objects and tests. You select the profile to the right of the run button. If you are doing API tests and plan to leverage endpoints in multiple environments, you can have a profile for each environment and specify the base URLs in a global variable.

Local Variables

Local variables are used within individual objects. They can be set to global variables, which is how we can utilize things like base URLs in our requests. To use a local variable, you use the “${ }” syntax. For example, if you have a “url” local variable”, you can specify “${url}/api/users” as the request URL.

Verification Script

Verification scripts are custom scripts that are optionally run after the response is received. Within these verification scripts, we can set global variables. It’s important to note that the global variable has to be defined in the profile or the request will fail. In the demo project, we will set the global object id to the newly created object’s id.

Test Case

The test cases can vary in scope greatly. You can potentially have a test case with every request and response covered by multiple verify keywords accounting for every element in every response, or you can break it down and have separate test cases for each step in the previously mentioned integration test flow. To verify information, you set the output of the request to be a variable that will be the response, then verify the response variable data. Verify Element Property Value is what is used in the demo project for verifying parts of the JSON based response from the API

NOTE: if you only select “Send Request”, the verification script associated with the request object will not run. Since we set the global variable in the verification script of the Create request object, we must perform a Send Request And Verify, or the global variable won’t be set.

Test Suite

Test suites are a collection of test cases that can run in sequence. In defining an test suite, you can execute each test case one after the other, which is leveraged in the demo project as multiple steps. After each execution of a test suite, a report is generated in the Reports folder. These are labelled by when they are run, and put in a folder based on the test suite.

Demo Project

To speed up getting started with a Katalon project, I have created a two sample repositories on GitHub, one for the actual Katalon project, and one for a sample .NET core with a Demo API endpoint that the Katalon project points to. Executing the demo Katalon project is just a matter of pulling down both repositories, starting an instance of the .NET core API, and then running the Katalon Users Integration Suite.

Katalon mock API: https://github.com/mvalenta/katalon-mock-api

Katalon demo project: https://github.com/mvalenta/katalon-api-demo

Test cases in the demo project can be broken down further to have a test on each resource property in the response (eg, a test case for first name and a test case for last name). It will help with the reports that are generated, as the overall number of test cases will be higher. So, for example, if one property doesn’t get updated correctly but the rest do, your percent of failed tests doesn’t drop greatly. In the demo project, this would mean that 1/6 of test cases would fail (~16% fail rate) vs 1/14 test cases would fail (~7% fail rate), where 6 is the number of integration steps and 14 is the total number of verify keywords in all of the integration steps.

Conclusion

While my experience with Katalon as a test automation tool is limited, it already feels useful. My thoughts are that the tests can take some time to set up initially, but that is alleviated by the fact that you can just switch to the “Script view” of the test cases and copy paste the similar steps. Additionally, the increased time to set up the project is offset by having the repeatable integration tests. Being able to do large scale regression tests in less that 5 minutes is a serious plus. Lastly, I believe the ultimate best value comes at a later stage in being able to integrate a console mode execution of these tests into a CI/CD pipeline that runs automatically and frequently. What’s better than having a user click a single button to do some complex work? Make it so they don’t have to click any button to have complex work done.

Hopefully this post will give you a solid foundation in testing REST APIs with Katalon. Despite this post being pretty extensive, I absolutely don’t cover everything, and recommend going to the official katalon docs site for more information.