REST service health check on automatic deployment - rest

Question

I'm using the AWS CodeDeploy platform for automatic deployment of my REST services. The deployment script got a lot of steps that copy/configure/do other staff. If any of the steps fails - the entire deployment fails for this server and I got a clear notification about it. So, the last step I need is a health check - a validation that configuration was appropriate and all is up and running.
Of cause, I can make a couple curl POSTs, parse their results and use some extracted values within more curl POSTs to get some sanity coverage, but all this parsing sounds like a wheel invention.
Is there any convinient testing framework/tool that can be easily "packed" and invoked in scripts without installing a huge testing siutes on each of my production servers?

Answer 1

Given that you're doing REST you probably can rely on the status codes instead of parsing the body. If you get a code that's not in 2xx, then something is wrong.
If you want a more elaborate check you could add a special endpoint that does some DB queries and maybe sends some harmless queries to its integrations.
And the most complicated option would be to implement a smart post-deployment steps that follow some workflow procedure. You'd need either to use an elaborate bash-scripting, or use more advanced programming languages and frameworks (like RestAssured in Java or RestClient in Groovy).
Don't forget to introduce a loop with some timeout that does a health check since your first request may be sent too early while the app is still being deployed.
Here is an example of simple bash-script that checks the status and the version of the app:
#!/usr/bin/env bash
# Helps to define whether application deployment was successful by checking
# connection to HTTP resource. If the page is loaded and the response is 200
# or 201, then the script finishes successfully. In case connection refused
# is or Gateway Timeout (503) the script is trying to connect again within
# timeout period. Otherwise script finishes with fail.
# Needs required parameter url to application and optional parameters timeout
# (by default equals to 180) and artifact version. If artifact version
# parameter is given and the response is 200 or 201, then script also checks
# that # deployed version (gets from $url/version) equals to the passed
# version. If not, the script finishes with fail. Example of usage in bash
# script:
# sh post_deployment_test.sh http://blah.com/version 100 1.0.102-20160404.101644-5
# result=$?
#
# If $result value equals to 0, then connection is successfully established,
# otherwise, it is not established.
url=$1
timeout=$2
version=$3
if [ -z "$timeout" ]; then
timeout=180
fi
counter=0
delay=3
while [ $counter -le $timeout ]; do
command="curl -L -s -o /dev/null -w %{http_code} $url"
echo "Executing: $command"
status_code=$($command)
curl_code=$?
# Curl error code CURLE_COULDNT_CONNECT (7) means fail to connect to host or proxy.
# It occurs, in particular, in case when connection refused.
if [ $curl_code -ne 0 ] && [ $curl_code -ne 7 ]; then
echo "Connection is not established"
exit 1
fi
if [ $curl_code = 7 ] || [ $status_code = 503 ]; then
echo "Connection has not been established yet, because connection refused or service unavailable. Trying to connect again"
sleep $delay
let counter=$counter+$delay
continue
elif [ $status_code = 200 ] || [ $status_code = 201 ]; then
if [ -z "$version" ]; then
echo "Connection is successfully established"
exit 0
else
grep_result=`curl -L -s $url | grep $version`
if [ -z "$grep_result" ]; then
echo `curl -L -s $url`
echo "Deployed version doesn't equal to expected"
exit 1
else
echo "Connection is successfully established"
exit 0
fi
fi
else
echo "Connection is not established"
exit 1
fi
done
echo "Connection is not established"
exit 1

Answer 2

I've found something nice I was looking for: jasmine-node as a test runtime + frisby.js as a validation script tool.
It's both really portable (I just run npm install during the deployment) and really convenient in terms of scripting, e.g.(official example from frisby):
var frisby = require('frisby');
.get('https://api.twitter.com/1/statuses/user_timeline.json?screen_name=brightbit')
.expectStatus(200)
.expectHeaderContains('content-type', 'application/json')
.expectJSON('0', {
place: function(val) { expect(val).toMatchOrBeNull("Oklahoma City, OK"); }, // Custom matcher callback
user: {
verified: false,
location: "Oklahoma City, OK",
url: "http://brightb.it"
}
})
.expectJSONTypes('0', {
id_str: String,
retweeted: Boolean,
in_reply_to_screen_name: function(val) { expect(val).toBeTypeOrNull(String); }, // Custom matcher callback
user: {
verified: Boolean,
location: String,
url: String
}
})
.toss();