What is continuous integration?
“In software engineering, continuous integration is the practice of merging all developers’ working copies to a shared mainline several times a day.”
In simple terms, continuous integration means having code changes, environment changes or any computer-based process be self-sufficient.
So, the main question is, how does that affect us here at Invotra? Within the company, we have a process that currently requires manual interaction in order to function correctly. This is how we use our in-house tools to talk to Rainforest and vice versa. Currently, from a resource perspective, it will take a member of the team up to an hour to complete one iteration of this process, and then a further amount of time to test each part.
This means that in order to run this process effectively, it requires a huge amount of time from one or more team members.
What is the process?
Using Rainforest and the test writing engine, we want to harness the ability to test out internal phases at greater speed and, in turn, reduce the overall cost. Here’s a breakdown of how each part of the automation solution will function.
Stage 1 – Collection
Jira
The first stage of this process is to collect all of our jira tasks that are ready for the internal stage of testing. This is done via the use of Jira’s REST APIs and a GET request. This GET request will return results in a JSON format based on the JQL I request.
Now this has ran and the required parts from the JSON have been filtered, I will store the task name and it’s current status for later.
Rainforest
As I said earlier we will be using Rainforest’s test writing engine going forward, the test writing engine allows us to input the steps we would use for our testing, to then be written in the Rainforest acceptable format for their testers.
I now need to request the Rainforest test written task’s unique ids. This is done via the use of Rainforest’s CLI (command line integration). Using “Rainforest download” I download every task currently on our Rainforest account.
This would be handled using a bash script to run the console command.
Stage 2 – Comparing
Now that both the data from Jira and Rainforest has been collected, I need to check that tasks returned have had tests written. Due to our naming standards, we can quickly check if any of the Jira task titles match up to the returned Rainforest tasks. There are multiple ways this can be done but for simple terms, an -iname (task title) is used on the Rainforest directory.
If nothing is returned, the task is not present in Rainforest (not yet written). If something is returned we store that unique id for later. Again this will use a console command contained within a bash script
Stage 3 – Construction
Now that we have a list of tasks with their matching Rainforest test written tasks, we have to check that they have a pull request against them (A pull request is code provided by the developer with the required task changes).
This is done by checking if the specific task in Github has the request against it. If the task passes all three of these stages we would construct a topic branch with the pull request changes.
Topic branch – A mock, in-house website with the code changes placed against it. This would be done by using the APIs built into our Jenkins box to build the topic branch.
Stage 4 – Testing
Once the topic branch has been built, we would request the site URL and update the specific Rainforest task with that testing URL. This could be handled in a multitude of ways but for this, SED is used.
SED is a simple UNIX command that allows me to input the URL into the Rainforest task file without opening it. Following this, the Rainforest task is kicked off for testing again using the CLI Rainforest provided. Once testing has concluded a simple XML file will be returned stating if their users passed or failed the task.
Again, each part of this would be using a bash script containing the required console commands / CLI commands.
Stage 5 – Clearing
Now that we have received the file containing the results and used a simple check for the pass or fail status, we update the Jira task to reflect this.
This will be done using a POST request, this time to Jira. Updating the status of the task showing Passed/Failed and adding the XML to the comment of the task.
This is so that, if required, the dev can look over what caused it to fail, or, as a sanity check our team can review a sample of tasks.
The final stage would use a script to check the returned results and then trigger a coded API post request.
This entire process would be contained within a Jenkins build, meaning that this could continuously occur in the background with little human interaction – minus the test writing and checking of failed tasks.
This means that the overall time for this process will be reduced from an average of 1 hour to 10mins per task. Achieving our end goal of reducing time and overall cost at this stage.
Conclusion
So, how do you apply this logic to your daily business or workdays? I was always told that if you have to repeat something more than twice in a computer environment, find a way to automate it. For instance, if you have to run a series of terminal commands each day, write a script to run them for you. Or, in the case of tools you might use day-to-day, see if they have APIs present that you can use requests for, allowing you to automate a common process. A final note is to understand that constant integration or automation may take a while to set up but in the long run, will save a huge amount of time.
