How to discover undocumented mimdump inline script parameters? - mitmproxy

Question

I am trying to parse different elements of the request and response header with inline scripting and mitmdump. Some features are undocumented. I will post the lessons learned in reply to this question.

Answer 1

Why not use the official documentation?
http://mitmproxy.org/doc/scripting/inlinescripts.html
The canonical API documentation is the code, which you can browse locally or in our GitHub repo. You can view the API documentation using pydoc (which is installed with Python by default), like this:
pydoc libmproxy.protocol.http.HTTPRequest
gives better output.

Answer 2

The use of dir() in an inline script shows all the varables that you can use for parsing.
def response(context, flow):
print dir(flow)
print dir(flow.request)
for cookie in flow.response.headers["Set-Cookie"]:
print "%s:\t%s" % (flow.request.host, cookie)
Results for dir(flow)
['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__getattribute__', '__hash__',
'__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__', '__weakref__', '_backup', '_stateobject_attributes',
'_stateobject_long_attributes', 'accept_intercept', 'backup', 'client_conn', 'copy', 'error', 'from_state',
'get_state', 'id', 'intercept', 'intercepting', 'kill', 'live', 'load_state', 'match', 'modified', 'replace',
'reply', 'request', 'response', 'revert', 'server_conn', 'type']
results for dir(flow.request)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__',
'__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_assemble_first_line', '_assemble_head',
'_assemble_headers', '_stateobject_attributes', '_stateobject_long_attributes', 'anticache', 'anticomp',
'assemble', 'constrain_encoding', 'content', 'copy', 'decode', 'encode', 'form_in', 'form_out', 'from_state',
'from_stream', 'get_cookies', 'get_decoded_content', 'get_form_urlencoded', 'get_path_components',
'get_query', 'get_state', 'headers', 'host', 'httpversion', 'is_replay', 'load_state', 'method', 'path',
'port', 'pretty_host', 'pretty_url', 'replace', 'scheme', 'set_form_urlencoded', 'set_path_components',
'set_query', 'size', 'stickyauth', 'stickycookie', 'timestamp_end', 'timestamp_start', 'update_host_header',
'url']
Results for dir(flow.response)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__',
'__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__',
'__str__', '__subclasshook__', '__weakref__', '_assemble_first_line', '_assemble_head', '_assemble_headers',
'_refresh_cookie', '_stateobject_attributes', '_stateobject_long_attributes', 'assemble', 'code', 'content',
'copy', 'decode', 'encode', 'from_state', 'from_stream', 'get_cookies', 'get_decoded_content', 'get_state',
'headers', 'httpversion', 'is_replay', 'load_state', 'msg', 'refresh', 'replace', 'size', 'stream',
'timestamp_end', 'timestamp_start']