I am using mitmdump -dd > outfile to parse content,which gives me the complete request and response "headers and its body content"(which also removes the junk part of the traffic i.e no certificate's and no compressed data).
But this is making my file really large.How can I just get only the request part of the traffic....
Any advice or link how can this be done??
Thanks
The way i handled not exact but similar case is using -s flag of mitmdump and get the flow.request.content in my script and log it to some log file that will be pretty neat and clean
Here below code might be of some help ( save as t.py and run mitmdump -s t.py )
from mitmproxy import http
import time,re
import logging
def response(flow: http.HTTPFlow) -> None:
flow.response.headers["newheader"] = "foo"
def request(flow: http.HTTPFlow) -> None:
print(flow.request.content)
request_content = flow.request.content
# here u get the request content and then log it and use it
Related
I am trying to run performance test on scenario tagged as perf from the below feature file-
#tag1 #tag2 #tag3
**background:**
user login
#tag4 #perf
**scenario1:**
#tag4
**scenario2:**
Below is my .scala file setup-
class PerfTest extends Simulation {
val protocol = karateProtocol()
val getTags = scenario("Name goes here").exec(karateFeature("classpath:filepath"))
setUp(
getTags.inject(
atOnceUsers(1)
).protocols(protocol)
)
I have tried passing the tags from command line and as well as passing the tag as argument in exec method in scala setup.
Terminal command-
mvn clean test-compile gatling:test "-Dkarate.env={env}" "-Dkarate.options= --tags #perf"
.scala update:- I have also tried passing the tag as an argument in the karate execute.
val getTags = scenario("Name goes here").exec(karateFeature("classpath:filepath", "#perf"))
Both scenarios are being executed with either approach. Any pointers how i can force only the test with tag perf to run?
I wanted to share the finding here. I realized it is working fine when i am passing the tag info in .scala file.
My scenario with perf tag was a combination of GET and POST call as i needed some data from GET call to pass in POST call. That's why i was seeing both calls when running performance test.
I did not find any reference in karate gatling documentation for passing tags in terminal execution command. So i am assuming that might not be a valid case.
I would like to create a separate log file for each test method. And i would like to do this in the conftest.py file and pass the logfile instance to the test method. This way, whenever i log something in a test method it would log to a separate log file and will be very easy to analyse.
I tried the following.
Inside conftest.py file i added this:
logs_dir = pkg_resources.resource_filename("test_results", "logs")
def pytest_runtest_setup(item):
test_method_name = item.name
testpath = item.parent.name.strip('.py')
path = '%s/%s' % (logs_dir, testpath)
if not os.path.exists(path):
os.makedirs(path)
log = logger.make_logger(test_method_name, path) # Make logger takes care of creating the logfile and returns the python logging object.
The problem here is that pytest_runtest_setup does not have the ability to return anything to the test method. Atleast, i am not aware of it.
So, i thought of creating a fixture method inside the conftest.py file with scope="function" and call this fixture from the test methods. But, the fixture method does not know about the the Pytest.Item object. In case of pytest_runtest_setup method, it receives the item parameter and using that we are able to find out the test method name and test method path.
Please help!
I found this solution by researching further upon webh's answer. I tried to use pytest-logger but their file structure is very rigid and it was not really useful for me. I found this code working without any plugin. It is based on set_log_path, which is an experimental feature.
Pytest 6.1.1 and Python 3.8.4
# conftest.py
# Required modules
import pytest
from pathlib import Path
# Configure logging
#pytest.hookimpl(hookwrapper=True,tryfirst=True)
def pytest_runtest_setup(item):
config=item.config
logging_plugin=config.pluginmanager.get_plugin("logging-plugin")
filename=Path('pytest-logs', item._request.node.name+".log")
logging_plugin.set_log_path(str(filename))
yield
Notice that the use of Path can be substituted by os.path.join. Moreover, different tests can be set up in different folders and keep a record of all tests done historically by using a timestamp on the filename. One could use the following filename for example:
# conftest.py
# Required modules
import pytest
import datetime
from pathlib import Path
# Configure logging
#pytest.hookimpl(hookwrapper=True,tryfirst=True)
def pytest_runtest_setup(item):
...
filename=Path(
'pytest-logs',
item._request.node.name,
f"{datetime.datetime.now().strftime('%Y%m%dT%H%M%S')}.log"
)
...
Additionally, if one would like to modify the log format, one can change it in pytest configuration file as described in the documentation.
# pytest.ini
[pytest]
log_file_level = INFO
log_file_format = %(name)s [%(levelname)s]: %(message)
My first stackoverflow answer!
I found the answer i was looking for.
I was able to achieve it using the function scoped fixture like this:
#pytest.fixture(scope="function")
def log(request):
test_path = request.node.parent.name.strip(".py")
test_name = request.node.name
node_id = request.node.nodeid
log_file_path = '%s/%s' % (logs_dir, test_path)
if not os.path.exists(log_file_path):
os.makedirs(log_file_path)
logger_obj = logger.make_logger(test_name, log_file_path, node_id)
yield logger_obj
handlers = logger_obj.handlers
for handler in handlers:
handler.close()
logger_obj.removeHandler(handler)
In newer pytest version this can be achieved with set_log_path.
#pytest.fixture
def manage_logs(request, autouse=True):
"""Set log file name same as test name"""
request.config.pluginmanager.get_plugin("logging-plugin")\
.set_log_path(os.path.join('log', request.node.name + '.log'))
I am trying to use Python to program watson_developer_cloud and I am getting an API key error. My code is really simple:
import json
from watson_developer_cloud import VisualRecognitionV3 as VisualRecognition
visual_recognition = VisualRecognition('2016-05-20', api_key='<myAPIkey>')
test_url = 'https://www.ibm.com/ibm/ginni/images/ginni_bio_780x981_v4_03162016.jpg'
print(json.dumps(visual_recognition.classify(images_url=test_url)))
But for some reason when I run it, I get a SSL error:
requests.exceptions.SSLError: hostname 'gateway-a.watsonplatform.net' doesn't match either of 'access.alchemyapi.com', 'www.access.alchemyapi.com'
I know the API key (omitted above) works because when I use CURL or NodeJS it works just fine.
I am really stumped. Could someone please help?
Figured it out. It turns out you require python>=2.7.8 because of server name identification (SNI). My version of python is 2.7.3, and it does not support that and therefore I am getting the error seen above.
So this is why CURL and other attempts to use the API key work, whereas my python was generating a SSLError. Happy upgrading!
Quick Hack:
Change the file - watson_developer_cloud_service.py:243 as below:
diff watson_developer_cloud_service.py watson_developer_cloud/watson_developer_cloud_service.py
243c243
< params=params, data=data, files=files, verify=False, **kwargs)
---
> params=params, data=data, files=files, **kwargs)
after that -
rm -f /usr/lib/python2.7/site-packages/watson_developer_cloud-0.14.0-py2.7.egg
followed by ..
python setup.py install
I'm using soapui with groovy script step
I want to print the full url of my REST request.
I tried using:
myFile.append( testRunner.testCase.testSteps["My Test Name"].getProperty( "requestUri" ));
and I got null.
You will not be able to see the request info from a test step groovy script. However, the groovy script assertion has access to that information.
You can use this to easily retrieve the full endpoint:
def endpoint = messageExchange.getEndpoint()
The below is working fine for me. you can use the same code just need to change your step name.
Note: Make sure your same Test step should have run prior to below code.else you will get the error
[Cannot invoke method getURL() on null object], see error log for details.
Working Code:
def tr=testRunner.testCase.getTestStepByName("TriggerRequestTransactionsReportsService_V)
def String endPointUrlSave= tr.getHttpRequest().getResponse().getURL();
log.info "Your EndpointUrl is : " + endPointUrlSave;
I would like to change the following batch script to Scala (just for fun), however, the script must keep running and listen for changes to the *.mkd files. If any file is changed, then the script should re-generate the affected doc. File IO has always been my Achilles heel...
#!/bin/sh
for file in *.mkd
do
pandoc --number-sections $file -o "${file%%.*}.pdf"
done
Any ideas around a good approach to this will be appreciated.
The following code, taken from my answer on: Watch for project files also can watch a directory and execute a specific command:
#!/usr/bin/env scala
import java.nio.file._
import scala.collection.JavaConversions._
import scala.sys.process._
val file = Paths.get(args(0))
val cmd = args(1)
val watcher = FileSystems.getDefault.newWatchService
file.register(
watcher,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_DELETE
)
def exec = cmd run true
#scala.annotation.tailrec
def watch(proc: Process): Unit = {
val key = watcher.take
val events = key.pollEvents
val newProc =
if (!events.isEmpty) {
proc.destroy()
exec
} else proc
if (key.reset) watch(newProc)
else println("aborted")
}
watch(exec)
Usage:
watchr.scala markdownFolder/ "echo \"Something changed!\""
Extensions have to be made to the script to inject file names into the command. As of now this snippet should just be regarded as a building block for the actual answer.
Modifying the script to incorporate the *.mkd wildcards would be non-trivial as you'd have to manually search for the files and register a watch on all of them. Re-using the script above and placing all files in a directory has the added advantage of picking up new files when they are created.
As you can see it gets pretty big and messy pretty quick just relying on Scala & Java APIs, you would be better of relying on alternative libraries or just sticking to bash while using INotify.