I'm doing some work with mitmproxy and was wondering how can I get the headers and data that is passed via GET/POST commands. For example I see the following outputted from mitmproxy:
IPADDRESS GET http://www.example.com/wp-content/uploads/2015/01/9418ASD23801.jpg
But I would want to see all of the data that is being passed included user agents, headers, etc. The same goes for post examples.
For new version of mitmproxy --flow-detail 2 can be used, example with docker:
docker run --rm -it -p 8080:8080 mitmproxy/mitmproxy mitmdump --flow-detail 2
Related
I am trying to send an email using curl on Windows 10 through an MS Exchange Server. The email is sent to another user in the same Active Directory domain, and the user who is sending the email is already authenticated under Windows.
Given the above, I can successfully send an email using telnet as follows:
or PowerShell as follows:
Given that the above two commands work, I am looking for the equivalent curl command. I have looked at the following SO answer:
Using curl to send email
and tried the following command (and many other combinations of options), but with no success.
curl -v --url "webmail.jXs.com:25" --mail-from "r#jXs.com" --mail-rcpt "s#jXs.com" --upload-file mail.txt
I get the following output:
where the mail.txt file is:
So, what is wrong with my curl command? Thanks.
Apparently when using curl, we have to specify the smtp communication protocol in the url (which is not necessary when using telnet or powershell). So, adding smtp:// to the url made the curl command work.
curl -v --url "smtp://webmail.jXs.com:25" --mail-from "r#jXs.com" --mail-rcpt "s#jXs.com" --upload-file mail.txt
I am developing a dashboard that connects to Splunk via REST API and displays data on various charts/graphs etc. In order to get the data I have to make a POST request via curl (node.js). Everything is working great. However when I try to make a Post request with a dbxquery, it fails and returns 'fatal dbxquery unknown command.' I was wondering if anyone had encountered this before.
curl -H 'Authorization: Basic auth token' -k https://devfg.com:8089/services/search/jobs -d search=" | dbxquery query=\"SELECT count(*) FROM db.table\" connection=\"connection\"" -d output_mode=json
Are the permissions for the dbxquery command set to be executable from any app? Check under app permissions to see if the command is globally exported.
Alternatively, you may need to escape the *, so \*.
Otherwise, you should be able to run the dbxquery via a curl command.
I'm following a https://app.pluralsight.com/library/courses/docker-web-development/table-of-contents which uses the older microsoft/aspnetcore-build image but I'm running core 2.1 so I'm using microsoft/dotnet:2.1-sdk instead.
The command I'm running is:
docker run -it -p 8080:5001 -v ${pwd}:/app -w "/app"
microsoft/dotnet:2.1-sdk
and then once inside the TTY I do a dotnet run which gives me the following output:
Using launch settings from /app/Properties/launchSettings.json...
info:
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[0]
User profile is available. Using '/root/.aspnet/DataProtection-Keys'
as key repository; keys will not be encrypted at rest.
info:
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[58]
Creating key {5445e854-c1d9-4261-82f4-0fc3a7543e0a} with creation date
2018-12-14 10:41:13Z, activation date 2018-12-14 10:41:13Z, and
expiration date 2019-03-14 10:41:13Z.
warn:
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
No XML encryptor configured. Key
{5445e854-c1d9-4261-82f4-0fc3a7543e0a} may be persisted to storage in
unencrypted form.
info:
Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[39]
Writing data to file
'/root/.aspnet/DataProtection-Keys/key-5445e854-c1d9-4261-82f4-0fc3a7543e0a.xml'.
warn: Microsoft.AspNetCore.Server.Kestrel[0]
Unable to bind to https://localhost:5001 on the IPv6 loopback
interface: 'Cannot assign requested address'.
warn: Microsoft.AspNetCore.Server.Kestrel[0]
Unable to bind to http://localhost:5000 on the IPv6 loopback
interface: 'Cannot assign requested address'.
Hosting environment: Development
Content root path: /app
Now listening on: https://localhost:5001
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.
Then, when I open browser on my host and navigate to http://localhost:8080 I get a "This page isn't working" "localhost didn't send any data" " ERR_EMPTY_RESPONSE"
I've tried a couple different port combinations too with the same result.
Can anyone spot where I went wrong? Or have any ideas / suggestions?
Not sure if this question still relevant for you, but I also encountered this issue and left my solution here for others. I used PowerShell with the next docker command (almost the same as your command, just used internal port 90 instead of 5000 and used --rm switch which will automatically remove the container when it exits):
docker run --rm -it -p 8080:90 -v ${pwd}:/app -w "/app" microsoft/dotnet /bin/bash
And after that, I got the interactive bash shell, and when typing dotnet run I got the same output as you and cannot reach my site in the container via localhost:8080.
I resolved it by using UseUrls method or --urls command-line argument. They (UseUrls method or --urls command-line argument) indicates the IP addresses or host addresses with ports and protocols that the server should listen on for requests. Below descriptions of solutions which worked for me:
Edit CreateWebHostBuildermethod in Program.cs like below:
public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
WebHost.CreateDefaultBuilder(args)
.UseUrls("http://+:90") //for your case you should use 5000 instead of 90
.UseStartup<Startup>();
You can specify several ports if needed using the next syntax .UseUrls("http://+:90;http://+:5000")
With this approach, you just typed dotnet run in bash shell and then your container will be reachable with localhost:8080.
But with the previous approach you alter the default behavior of your source code, which you can forget and then maybe should debug and fix in the future. So I prefer 2nd approach without changing the source code. After typing docker command and getting an interactive bash shell instead of just dotnet run type it with --urls argument like below (in your case use port 5000 instead of 90):
dotnet run --urls="http://+:90"
In the documentation there is also a 3rd approach where you can use ASPNETCORE_URLS environment variable, but this approach didn't work for me. I used the next command (with -e switch):
docker run --rm -it -p 8080:90 -v ${pwd}:/app -w "/app" -e "ASPNETCORE_URLS=http://+:90" microsoft/dotnet /bin/bash
If you type printenv in bash you will see that ASPNETCORE_URLS environment variable was passed to the container, but for some reason dotnet run is ignoring it.
We can use the command docker images to list the Docker images we have on local host.
Now I want to get the same information from a remote server by sending an HTTP GET request in Firefox or Chrome. Does Docker provide some REST API to do this?
I did a lot of search. For example:
Examples using the Docker Engine SDKs and Docker API
It provides a way something like this:
curl --unix-socket /var/run/docker.sock http:/v1.24/containers/json
I know a little about Unix sockets, and I don't think this is what I want. The URL (http:/v1.24/containers/json) is so weird and don't even have a server name in it. I don't think it can work on a remote server. (It does work on a local server.)
Is there any official documentation that Docker provides on this topic?
You need to expose the Docker daemon on a port.
You can configure the Docker daemon to listen to multiple sockets at the same time using multiple -H options:
listen using the default Unix socket, and on two specific IP addresses on this host.
$ sudo dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106 -H tcp://10.10.10.2
The Docker client will honor the DOCKER_HOST environment variable to set the -H flag for the client. Use one of the following commands:
https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-socket-option
You need to do this by creating a systemd dropin:
mkdir -p /etc/systemd/system/docker.service.d/
cat > /etc/systemd/system/docker.service.d/10_docker.conf <<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// -H tcp://0.0.0.0:2376
EOF
Then reload and restart Docker:
systemctl daemon-reload
systemctl restart docker
Note: this way you would be exposing your host and you shouldn't do it this way in production. Please read more about this on the link I shared earlier.
I am following the instructions at: http://apiaxle.com/docs/statistics-and-analytics-in-apiaxle/ . Unfortunately currently (May 17, 2014) apiAxle is redirecting me to the endPointserver and I am not getting statist
menelaos:~$ curl 'http://localhost:3000/v/api/test/stats?
granularity=hour&format_timestamp=ISO'
Response:
{"meta":{"version":1,"status_code":404},"results":{"error":
{"type":"ApiUnknown","message":"No api specified (via subdomain)"}}}
I also tried using the subdomain but that didn't work either:
menelaos:~$ curl 'http://test.api.localhost:3000/v/api/test/stats?granularity=hour&format_timestamp=ISO'
Typically you run multiple instances of apiaxle-proxy (which provides access to your endpoints), and a single instances of apiaxle-api (which provides access to statistics, key creation, and other API management functionality).
For example, you might be running the proxy like this:
apiaxle-proxy -f 1 -p 3000 -q
To run the API, you would run something like this:
apiaxle-api -f 1 -p 5000 -q
Note that the API needs to run on a separate port. Also note that it shouldn't be accessible to the outside world as it doesn't have any authentication.
Using the above example, your curl command would look like this:
curl -H 'content-type: application/json' \
-X GET \
'http://localhost:5000/v1/api/test/stats' \
-d '{"granularity":"hour","format_timestamp":"ISO"}'
Note that the parameters need to be sent as JSON.