How to in "def request()" return the response directly - mitmproxy

like title, I wanna in "def request()" to process data, and return a response directly;
I don't wanna flow through the target server;
this way is feasible? thanks!!!

Here's an example on how to do that:
"""Send a reply from the proxy without sending any data to the remote server."""
from mitmproxy import http
def request(flow: http.HTTPFlow) -> None:
if flow.request.pretty_url == "http://example.com/path":
flow.response = http.Response.make(
200, # (optional) status code
b"Hello World", # (optional) content
{"Content-Type": "text/html"} # (optional) headers
)
Source: https://github.com/mitmproxy/mitmproxy/blob/main/examples/addons/http-reply-from-proxy.py

Related

How to check for proper format in my API response

Currently running tests for my REST API which:
takes an endpoint from the user
using that endpoint, grabs info from a server
sends it to another server to be translated
then proceeds to jsonify the data.
I've written a series of automated tests running and I cannot get one to pass - the test that actually identifies the content of the response. I've tried including several variations of what the test is expecting but I feel it's the actual implementation that's the issue. Here's the expected API response from the client request:
{ "name": "random_character", "description": "Translated description of requested character is output here" }
Here is the testing class inside my test_main.py:
class Test_functions(unittest.TestCase):
# checking if response of 200 is returned
def test_healthcheck_PokeAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/")
status_code = response.status_code
self.assertEqual(status_code, 200)
# the status code should be a redirect i.e. 308; so I made a separate test for this
def test_healthcheck_ShakesprAPI(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.status_code, 308)
def test_response_content(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertEqual(response.content_type,
'application/json') <<<< this test is failing
def test_trans_shakespeare_response(self):
manualtest = app.test_client(self)
response = manualtest.get("/pokemon/charizard")
self.assertFalse(b"doth" in response.data)
Traceback:
AssertionError: 'text/html; charset=utf-8' != 'application/json' - text/html; charset=utf-8 + application/json
Any help would be greatly appreciated

alamofire request gets text/html response, while curl and postman get json response

=================================
Note: Using #Larme's trick to print out debugDescription of the request, and comparing with my working curl request, I was able to figure out the dumb bugs I made. 1. In the server request handler, I return a serializerError when something unrecognized, pretty confusing. 2. I made a stupid mistake in my request from swift, putting "GET_RECIPES" instead of "GET_RECIPE".
================================
I have a http service implemented with django rest framework, when I send requests via swift/alamofire, it cannot get the correct json response. However, requests sent via curl and postman get the correct json response.
So I am confused what where is the issue, the django service side or swift request side?
I have tried using .responseString instead of .responseJSON in swift to print out the resposne, but still the data is not in the response, basically the error occurs when the request reaches the server side.
From django server side, the error reads "TypeError: Object of type 'property' is not JSON serializable". OK it seems the issue is from django side...
But from curl and postman, I can get the json response without an issue, with the response header containing "Content-Type": "application/json", and for the django side everything is also OK. Then does this mean the django server can handle json response and it should be the issue of the swift request??
Code in swift,
let parameters: [String: Any] = [
"type": "GET_RECIPE",
"details": ["ingredients" : ["egg", "bread"]]
]
let headers = ["Content-Type": "application/json"]
Alamofire.request(url, mothod: .post, parameters: parameters,
headers: headers, encoding: JSONEncoding.default)
.responseJSON {response in
if let data = response.result.value {
print(data)
}
}
Code of the request handler
class RecipesSerilizer(serializers.ModelSerializer):
class Meta:
model = Recipes
fields = ('id', 'url', 'author', 'category', 'title', 'description',
'instructions', 'tip', 'raw', 'score')
def get_recipes_given_ingredients(data):
logger.info('Get recipes for {}'.format(data.get('details')))
details = data.get('details')
ingredients = details.get('ingredients')
logger.info('GET_RECIPE for ingredients {}'.format(ingredients))
recipes = queries.get_recipe_recommendation_given_ingredients(ingredients)
serializer = RecipesSerilizer(recipes, many=True)
return Response(serializer.data)
Trace stack from the server side:
Internal Server Error: /get-recipes/
Traceback (most recent call last):
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\django\core\handlers\base.py", line 145, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\django\core\handlers\base.py", line 143, in _get_response
response = response.render()
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\django\template\response.py", line 106, in render
self.content = self.rendered_content
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\response.py", line 72, in rendered_content
ret = renderer.render(self.data, accepted_media_type, context)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\renderers.py", line 733, in render
context = self.get_context(data, accepted_media_type, renderer_context)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\renderers.py", line 688, in get_context
'content': self.get_content(renderer, data, accepted_media_type, renderer_context),
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\renderers.py", line 424, in get_content
content = renderer.render(data, accepted_media_type, renderer_context)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\renderers.py", line 107, in render
allow_nan=not self.strict, separators=separators
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\utils\json.py", line 28, in dumps
return json.dumps(*args, **kwargs)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\json\__init__.py", line 238, in dumps
**kw).encode(obj)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\json\encoder.py", line 201, in encode
chunks = list(chunks)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\json\encoder.py", line 437, in _iterencode
o = _default(o)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\site-packages\rest_framework\utils\encoders.py", line 68, in default
return super(JSONEncoder, self).default(obj)
File "C:\Users\Yuanjun\Anaconda2\envs\online_bid\lib\json\encoder.py", line 180, in default
o.__class__.__name__)
TypeError: Object of type 'property' is not JSON serializable
[14/May/2019 08:29:32] "POST /get-recipes/ HTTP/1.1" 500 124585
i think your problem is that you are trying to send a post to a get request.
try changing your alamofire request as follows:
let parameters: [String: Any] = [
"type": "GET_RECIPE",
"details": ["ingredients" : ["egg", "bread"]]
]
let headers = ["Content-Type": "application/json"]
Alamofire.request(url, mothod: .get, parameters: parameters,
headers: headers, encoding: JSONEncoding.default)
.responseJSON {response in
if let data = response.result.value {
print(data)
}
}
Probably the server crashes while handling your request or cannot find the given URL(because of the trailing slash).
text/html is usually returned if the server has crashed while running in DEBUG mode. This is how it shows the crash reason in a pretty way with the stack trace.
It is really hard to tell what happened in your case. It would be great if you provided the stack trace of the error.

Groovy script for Jenkins: execute HTTP request without 3rd party libraries

I need to create a Groovy post build script in Jenkins and I need to make a request without using any 3rd party libraries as those can't be referenced from Jenkins.
I tried something like this:
def connection = new URL( "https://query.yahooapis.com/v1/public/yql?q=" +
URLEncoder.encode(
"select wind from weather.forecast where woeid in " + "(select woeid from geo.places(1) where text='chicago, il')",
'UTF-8' ) )
.openConnection() as HttpURLConnection
// set some headers
connection.setRequestProperty( 'User-Agent', 'groovy-2.4.4' )
connection.setRequestProperty( 'Accept', 'application/json' )
// get the response code - automatically sends the request
println connection.responseCode + ": " + connection.inputStream.text
but I also need to pass a JSON in the POST request and I'm not sure how I can do that. Any suggestion appreciated.
Executing POST request is pretty similar to a GET one, for example:
import groovy.json.JsonSlurper
// POST example
try {
def body = '{"id": 120}'
def http = new URL("http://localhost:8080/your/target/url").openConnection() as HttpURLConnection
http.setRequestMethod('POST')
http.setDoOutput(true)
http.setRequestProperty("Accept", 'application/json')
http.setRequestProperty("Content-Type", 'application/json')
http.outputStream.write(body.getBytes("UTF-8"))
http.connect()
def response = [:]
if (http.responseCode == 200) {
response = new JsonSlurper().parseText(http.inputStream.getText('UTF-8'))
} else {
response = new JsonSlurper().parseText(http.errorStream.getText('UTF-8'))
}
println "response: ${response}"
} catch (Exception e) {
// handle exception, e.g. Host unreachable, timeout etc.
}
There are two main differences comparing to GET request example:
You have to set HTTP method to POST
http.setRequestMethod('POST')
You write your POST body to outputStream:
http.outputStream.write(body.getBytes("UTF-8"))
where body might be a JSON represented as string:
def body = '{"id": 120}'
Eventually it's good practice to check what HTTP status code returned: in case of e.g. HTTP 200 OK you will get your response from inputStream while in case of any error like 404, 500 etc. you will get your error response body from errorStream.

What's going wrong when I try to create a review comment through Github's v3 API?

I'm trying to create a review commit through Github's v3 API and am not succeeding. Consider this repository. There's a single pull request and for the purposes of this question let's say I want to leave a 'changes requested' review on that PR. Here's the code I've got:
#!/usr/bin/env python3
import requests
import json
TOKEN='YOUR_TOKEN_HERE'
REPO = "blt/experiment-repo"
PR_NUM = 1
COMMIT_SHA_1 = "4160bee478c3c985eaaa35f161cc922fe20b354a"
COMMIT_SHA_2 = "df9d13a2e35f9b6c228e1f30ea30585ed85af26a"
def main():
pr_comment_headers = {
'user-agent': 'benedikt/0.0.1',
'Authorization': 'token %s' % TOKEN,
# Accept header per
# https://developer.github.com/changes/2016-12-16-review-requests-api/
'Accept': 'application/vnd.github.black-cat-preview+json',
}
msg = "BLEEP BLOOP I AM A ROBOT"
payload = { 'commit_id': COMMIT_SHA_2,
'body': msg,
'event': "REQUEST_CHANGES" }
# Per https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review
review_url = "https://api.github.com/repos/%s/pulls/%s/reviews" % (REPO, PR_NUM)
res = requests.post(review_url, headers = pr_comment_headers,
json = json.dumps(payload))
print(res)
print(res.text)
if __name__ == '__main__':
main()
I've marked in code comments where I've discovered the API endpoints to hit and with what payloads. Excepting, I must have goofed somewhere because when I run the above program I receive:
<Response [422]>
{"message":"Validation Failed","errors":["Variable commitOID of type GitObjectID was provided invalid value","Variable event of type PullRequestReviewEvent was provided invalid value"],"documentation_url":"https://developer.github.com/v3/pulls/reviews/#create-a-pull-request-review"}
I've verified that the commit SHAs are the exact ones that Github shows and REQUEST_CHANGES is the string in the documentation.
What am I missing?
I think you need to let requests encode the request body instead of encoding it yourself with json.dumps(), something like this: requests.post(..., json=payload)

How to get http request header info from the server side with spray RestAPI

I am new to Scala and Spray. I have written a simple REST API according to the instructions given in this blog post.
http://www.smartjava.org/content/first-steps-rest-spray-and-scala
And all are working as expected.
Now I want to modify the program to print the HTTP headers like Encoding, Language, remote-address, etc.. I would like to print all the header information (purpose is to log these information)
But I could not find a proper documentation or examples. Could anyone please help me to get this done.
If you need to extract a specific header:
optionalHeaderValueByName("Encoding") { encodingHeader =>
println(encodingHeader)
complete("hello")
}
alternatively you can access the raw request object and directly extractive the headers. Here's a custom directive that logs all the headers:
def logHeaders(): Directive0 = extract(_.request.headers).map(println)
Usage
logHeaders() {
complete("hello")
}
Here's how I got it working.
Directive:
def logHeaders(innerRoute: Route): (RequestContext => Unit) = extract(_.request.headers) { headers =>
headers.foreach(h => logger.info("header: {} = {}", h.name, h.value))
innerRoute
}
Usage:
logHeaders() {
complete("hello")
}