A Beginners Guide to Integration of Postman with Jenkins

In recent years, API testing has grown in significance as a component of software development. With the rise of micro-services and the need for seamless integration between different systems, APIs play a major role in ensuring the stability and reliability of applications. One of the most widely used tools for API testing is Postman. Postman is an indispensable tool for any API testing workflow because it offers a comprehensive set of features for testing, documenting, and sharing APIs. If you want to start testing your APIs but don't know where to begin, Start Here. These best practices walks you through everything you need to know.

Introduction to Postman

Postman is an indispensable tool for any API testing workflow because it offers a comprehensive set of features for testing, documenting, and sharing APIs. If you want to start testing your APIs but don't know where to begin, Start Here. These best practices walks you through everything you need to know.

Prerequisites:

1. Should already have idea about Postman
2. Should have already installed Jenkins (https://www.jenkins.io/doc/book/installing/)

Postman Scripts

In Postman, a pre-request script is a JavaScript code that runs before a request is sent. It allows you to manipulate the request or the environment variables used in the request. Similarly test script is a JavaScript code which runs after the request has been sent and is often used to test assertions. Here is a portion of some of the example requests we have used to create a collection. We also used the public API's from https://reqres.in.

Retrieve User

HTTP Method: GET
Endpoint URL: https://reqres.in/ api/users/2

Test Snippets Code:

//to check status code  
pm.test("Status code is 200", function () { 
    pm.response.to.have.status(200); 
}); 

// to check code name contains a string 
pm.test("Status code name has string", function () { 
    pm.response.to.have.status("OK"); 
}); 

// to check Response time is less than 1000ms  
pm.test("Response time is less than 1000ms", function () { 
    pm.expect(pm.response.responseTime).to.be.below(1000); 

}); 

// Content-type header check 
pm.test("Content-Type is present", function () { 
   pm.response.to.have.header("Content-Type"); 
}); 

There are much more vast & amazing ways to manipulate and fully utilize Postman Test Scripts, but for the purposes of this blog we assume that using simple test snippets are more than enough to demonstrate the task at hand. You can follow various other Postman articles and blogs to further enhance your skills and be fluent with Postman Scripts

Introduction to CI/CD

This is an Introduction to CI Integration with Postman. The main purpose of this is to help beginners as well intermediate users to understand how we can integrate Postman into the CI/CD Pipeline specifically with Jenkins. By the end of this we can assure you of understanding basics about the things listed below:

1. Understand the basics of Newman
2. Installing Newman
3. Installing Custom Newman Reporters
4. Running a Postman Collection in Newman to generate Reports
5. Basics about Jenkins
6. Additional Setup in Jenkins & Running Builds
7. Configuring Email Plugin in Jenkins
8. Sending Notification to Mattermost via Script

Understanding the basics of Newman

Newman is a command-line Collection Runner for Postman. It is similar to a Postman Runner but like the name suggests it runs on a terminal. The main advantage of Newman is to make sure that we can directly run the Postman Collection through command line without all the fuss of loading and handling the GUI. Simply put with the help Newman we can run all the scripts, workflows, environments that we have painstakingly written and designed in Postman in a much more robust clean and faster manner.

Installing Newman

To get started we must have NodeJS installed along with Node Package Manager (npm)
[Open the Command Line, here I am using Linux (Manjaro) for screenshots].

Then, we simply use the command to install newman globally:

​sudo npm install -g newman

Installing Custom Newman Reporters
This is required to export out a user friendly output instead of the default CLI output. Lets use newman-reporter-htmlextra for this purpose. In order to install the htmlextra we will be using the following command in the terminal:

​sudo npm install -g newman-reporter-htmlextra​

Installing Custom Newman Reporters

This is required to export out a user friendly output instead of the default CLI output. Lets use newman-reporter-htmlextra for this purpose. In order to install the htmlextra we will be using the following command in the terminal:

sudo npm install -g newman-reporter-htmlextra

Running a Postman Collection in Newman to generate Reports

And just like that we have everything setup properly. Now lets check if Newman is working properly by exporting a report by running a collection. For the purpose of testing we will be using a exported Postman Collection created from https://reqres.in/

Here, we have the Collection name: Test_Collection.postman_collection.json

Now by simply using the command below we can run the Postman Collection through the Terminal:

$ newman run Postman/newman/Test_Collection.postman_collection.json --reporters htmlextra,cli,json --reporter-json-export Postman/newman/Report.json --reporter-htmlextra-export Postman/newman/Report.html

This command simply states the location where the collection that is to be run exists, which reporters to use, and where to export the final Output. Here have also used the previously installed Newman Reporter to generate a HTML Report which can be considered as more user friendly.

Basics about Jenkins

Jenkins is a fork of Hudson and is a continuous build system for software projects. It is a self-contained, open-source automation server which can be used to automate all sorts of tasks. Due to its open-source nature it is very popular and since it is self contained & it runs on local machine. Unlike some other alternatives such as Github Actions, Travis, Gitlab, etc. it is completely free. Also did we mention that it supports plugins?

For the purposes of this article, we will focus on running our Postman Collection with Test Scripts in Jenkins. Also this article presumes that Jenkins is already installed with the necessary plugins and credentials already configured. If you require detailed articles on the installation of Jenkins you can visit the official documentation or check out our related articles.

Additional Steps in Jenkins & Running Builds

Assuming that we have the stable version of Jenkins up and running let us further configure a few things before getting started with a build. First of all lets configure NodeJS:

• Go to your Jenkins server & sign in.
• Go to Manage Jenkins > Manage Plugins and install the NodeJS plugin.

• Go to Manage Jenkins > Global Tool Configuration and under NodeJS, select Add NodeJS.
• Enter a name for the Node.js installation.
• In Global npm packages to install, enter: newman newman-reporter-htmlextra to install newman and newman-reporter package. Note - To install multiple packages enter the name of node package separated by space

• Select Save.

Now we can start creating a Project and start Running builds. Lets create a Project from the Dashboard. Click on New Item & give a name to the Project. Also select Free Style project for now and Click on OK. Now scroll down and select "Provide Node & npm bin/ folder to PATH". Now we are ready to build.

Simply go to the created project and navigate to Configure from the sidebar. Now scroll down to the Build Steps and click on "Add build Step" and select "Execute Shell". Now Command text box should appear where we can enter the previous CLI Command to run on every build. For simplicity I have used this command.

newman run collection/TestingPostmanIntegration_Collection.postman_collection.json --reporters htmlextra,cli,json --reporter-json-export report/TestingPostmanIntegration_Report.json --reporter-htmlextra-export report/TestingPostmanIntegration_Report.html

Configuring Email Plugin in Jenkins

After we have a build configured it is very simple to configure and send emails for every scheduled builds. This can be achieved via the built in email notification but we will use the Extended Email Notification plugin which can be installed via the plugin installation steps mentioned above.
For this we just need to configure the Global setting for Jenkins as listed below:

1. Navigate to Manage Jenkins
2. Click on System
3. Scroll to the bottom until you find Extended Email Notification
4. Enter the SMTP Server and the respective SMTP Port Number. Make sure to tick TLS or SSL accordingly or consult with the IT department to get the server details
5. Add Jenkins Credentials (username & password) which will be used to send emails to the receipt list. This has to be set only once so make sure to use correct credentials with proper security measures
6. Configure Global Triggers (during which cases will the email be sent)
7. Save the configuration

Now that the Global Email is set, we simply need to just add this to the Post Build Step.

1. Go to the Project in which Email Notification needs to be setup
2. Go to Configure
3. Navigate to Post-build Actions
4. Add previously created Editable Email Notification
5. Configure the Recipient List, Content Type, Attachment, Subject Body etc.
6. Under advanced Setting also set the Trigger to always to send an email regardless of the build status
7. Click on Save

Now we have setup the email notification to the pipeline. We can simply Run the build again which should show that Email has been sent to the emails listed in the recipient list.

Sending Notification to Mattermost via Script

Now that we know the basics about Jenkins and how to install new plugins, configure emails etc. Let us try something a bit more complex. We can use a script to combine everything we know into a neat package and and run the script instead of the lengthy command line. The script has can be found in a GitHub repository https://github.com/rajeet/Postman-Mattermost-Reporter.git & we can clone it. Now lets use the following list of commands in the Shell instead. We have used our specific Mattermost channel to receive the summary report & we have also used Postman API Collection to pull the collection.

git clone https://github.com/rajeet/Postman-Mattermost-Reporter.git ./
npm install package.json
echo 'MATTERMOST_URL=""' >> .env
echo 'API_COLLECTION_URL=""' >> .env
npm run generate_report

And in this manner we can view the test report on our emails after the build has been completed. Furthermore by integrating a script we can simplify as well as integrate reports in to Mattermost. In Conclusion, it is very much possible to integrate Postman/Newman into CI/CD platform such as Jenkins to improve the testing process. And this is just scratching the surface. With such a versatile & open-source tool such as Jenkins, the possibilities are endless & the only requirements are endless curiosity & drive to find answers to those curious questions.

Used References

https://github.com/postmanlabs/newman
https://npm.io/package/newman-reporter-htmlextra
https://learning.postman.com/docs/introduction/overview/
https://www.jenkins.io/doc/book/getting-started/