how to write e2e test automation for application containing kafka, postgres, and rest api docker containers - rest

Question

I have an app which is setup by docker-compose. The app contains docker containers for kafka, postgres, rest api endpoints.
One test case is to post data to endpoints. In the data, there is a field called callback URL. the app will parse the data and send the data to the callback URL.
I am curious whether there is any test framework for similar test cases. and how to verify the callback URL is hit with data?

Answer 1

Docker compose support has been added to endly. In the pipeline workflow for the app (app.yaml), you can add a "deploy" task and start the docker services by invoking docker-compose up.
Once test task is completed and your callback url is invoked, in your validation task, you can check to see if it was invoked with the expected data. For this you can utilize endly's recording feature and replay it to validate the callback request.
Below is an example of an ETL application app.yaml using docker-compose with endly to start the docker services. Hope it helps.
tasks: $tasks
defaults:
app: $app
version: $version
sdk: $sdk
useRegistry: false
pipeline:
build:
name: Build GBQ ETL
description: Using a endly shared workflow to build
workflow: app/docker/build
origin:
URL: ./../../
credentials: localhost
buildPath: /tmp/go/src/etl/app
secrets:
github: git
commands:
- apt-get -y install git
- export GOPATH=/tmp/go
- export GIT_TERMINAL_PROMPT=1
- cd $buildPath
- go get -u .
- $output:/Username/? ${github.username}
- $output:/Password/? ${github.password}
- export CGO_ENABLED=0
- go build -o $app
- chmod +x $app
download:
/$buildPath/${app}: $releasePath
/$buildPath/startup.sh: $releasePath
/$buildPath/docker-entrypoint.sh: $releasePath
/$buildPath/VERSION: $releasePath
/$buildPath/docker-compose.yaml: $releasePath
deploy:
start:
action: docker:composeUp
target: $target
source:
URL: ${releasePath}docker-compose.yaml

Answer 2

In your question below, where is Kafka involved? Both sound like HTTP calls.
1)Post data to endpoint
2)Against send data to the callback URL
One test case is to post data to endpoints. In the data, there is a field called callback URL. the app will parse the data and send the data to the callback URL.
Assuming the callback URL is an HTTP endpoint(e.g. REST or SOAP) with POST/PUT api, then it's better to expose a GET endpoint on the same resource. In that case, when callback POST/PUT is invoked, the server side state/data changes and next, use the GET api to verify the data is correct. The output of the GET API is the Kafka data which was sent to the callback URL(this assumes your 1st post message was to a kafka topic).

Answer 3

You can achieve this using traditional JUnit way using bit of code or via declarative way where you can completely bypass coding.
The example has dockerized Kafka containers to bring up locally and run the tests
This section Kafka with REST APIs explains automated way of testing combination of REST api testing with Kafka data streams .
e.g.
---
scenarioName: Kafka and REST api validation example
steps:
- name: produce_to_kafka
url: kafka-topic:people-address
operation: PRODUCE
request:
recordType: JSON
records:
- key: id-lon-123
value:
id: id-lon-123
postCode: UK-BA9
verify:
status: Ok
recordMetadata: "$NOT.NULL"
- name: verify_updated_address
url: "/api/v1/addresses/${$.produce_to_kafka.request.records[0].value.id}"
operation: GET
request:
headers:
X-GOVT-API-KEY: top-key-only-known-to-secu-cleared
verify:
status: 200
value:
id: "${$.produce_to_kafka.request.records[0].value.id}"
postCode: "${$.produce_to_kafka.request.records[0].value.postcode}"

Answer 4

Idaithalam is a low code Test automation Framework, developed using Java and Cucumber. It leverages Behavior Driven Development (BDD). Tester can create test cases/scripts in simple Excel with API Spec. Excel is a simplified way to create Json based test scripts in Idaithalam. Test cases can be created quickly and tested in minutes.
As a tester, you need to create Excel and pass it to Idaithalam Framework.
First, generate the Json based test scripts(Virtualan Collection) from Excel. During test execution, this test script collection can be directly utilized.
Then it generates Feature files from the Virtualan Collection and its executed.
Lastly, It generates test report in BDD/Cucumber style.
This provide complete testing support for REST APIs, GraphQL, RDBMS DB and Kafka Event messages
Refer following link for more information to set up and execute.
https://tutorials.virtualan.io/#/Excel
How to create test scripts using excel