How to save Charles session to a file from command line? - charles-proxy

I wonder how to save charles session to a file.
Consider following script :
open -ga Charles --args -headless -config charles.xml results.chls
#...some interactions here
pgrep -f Charles | xargs kill
I'm expecting to see something in the results.chls but file is empty....

I figured it out myself. Seems like the only way is enable web control access for Charles and use http like this:
curl --silent -x localhost:8888 http://control.charles/session/export-har -o "${EXPORT_FILE}" > /dev/null

Related

what is the valid http request that can be used within a prestop hook?

According to the below documentation, the line "HTTP - Executes an HTTP request against a specific endpoint on the Container."
https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#hook-handler-implementations
Using preStop hook, I tried to curl to run the following script but it returns nothing. Is the prestop hook limited to use the Http request within the container i.e, localhost?
echo "test curl" > /proc/1/fd/1
echo $(curl -s /dev/null http://google.com) > /proc/1/fd/1
echo $(curl -s -o /dev/null -w "%{http_code}" http://google.com) > /proc/1/fd/1
No, as I know you are not limited to use preStop's httpGet only withing the container. Your cointainer should just have access yo requested url, etc. So in your case you should have access to google.
May I know what exactly you wanna to achieve? Are you trying to redirect curl output to proc with PID:1 ?
Your command perfectly works in containers(that has curl itself), when I specify redirect to STDOUT, I mean /proc/self/fd/1
kubectl exec -ti curl -- bash
root#curl:/# echo $(curl -s -o /dev/null -w "%{http_code}" http://google.com) > /proc/self/fd/1
301
Btw, you can use exec instead of httpGet in preStop, where you can combine echo and curl
Yaml will be similar to
lifecycle:
preStop:
exec:
command: ["curl", "-XPOST", "-s", "http://google.com" > "/proc/1/fd/1"]
Please play with command and adjust for your needs. I havent tested it, wrote on flight

How do I send a command to a remote system via ssh with concourse

I have the need to start a java rest server with concourse that lives on an Ubuntu 18.04 machine. The version of concourse my company uses is 5.5.11. The server code is written in Java, so a simple java -jar <uber.jar> suffices from the command line (see below). In production, I will not have this simple luxury, hence my question.
I have an scp command working that copies the .jar from concourse to the target Ubuntu machine:
scp -i /tmp/key.p8 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ./${NEW_DIR}/${ARTIFACT_NAME}.${ARTIFACT_FILE_TYPE} ${SRV_ACCOUNT_USER}#${JAVA_VM_HOST}:/var/www
Note that my private key is passed with -i and I can confirm that is working.
I followed this other SO Q&A that seemed to be promising: Getting ssh to execute a command in the background on target machine
, but after trying a few permutations of the suggested solution and other answers, I still don't have my rest service kicked off.
I've tried a few permutations of this line in my concourse script:
ssh -f -i /tmp/pvt_key1.p8 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ${SRV_ACCOUNT_USER}#${JAVA_VM_HOST} "bash -c 'nohup java -jar /var/www/${ARTIFACT_NAME}.${ARTIFACT_FILE_TYPE} -c \"/opt/testcerts/clientkeystore\" -w \"password\" > /dev/null 2>&1 &'"
I've tried with and without the -f and -t switches in ssh, with and without the file stream redirection, with and without nohup and the Linux background ('&') command and various ways to escape the quotes.
At the bash prompt, this line successfully starts my server. The two switches are needed to point to the certificate and provide the password:
java -jar rest-service.jar -c "/opt/certificates/clientkeystore" -w "password"
I really think this is possible to do in Concourse, but I'm stuck at this point.
After a lot of trial an error, it seems I needed to do this:
ssh -f -i /tmp/pvt_key1.p8 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ${SRV_ACCOUNT_USER}#${JAVA_VM_HOST} "bash -c 'sudo java -jar /var/www/${ARTIFACT_NAME}.${ARTIFACT_FILE_TYPE} -c \"/path/to/my/certificate\" -w \"password\" > /var/www/log.txt 2>&1 &'"
The key was I was missing the 'sudo' portion of the command. Using nohup as opposed to putting in a Linux bash background indicator ('&') seems to give me an error in the pipeline. This works for me, but others are welcome to post responses with better answers or methods that might be a better practice.

What command should I use with curl in terminal if I want to download an image from a website and put it in a certain folder?

I have seen
curl -o project/folder/image.png -OL example.com/image.png
Would this command work?
Edit: I figured it out

How to set up cron using curl command?

After apache rebuilt my cron jobs stopped working.
I used the following command:
wget -O - -q -t 1 http://example.com/cgi-bin/loki/autobonus.pl
Now my DC support suggests me to change the wget method to curl. What would be the correct value in this case?
-O - is equivalent to curl's default behavior, so that's easy.
-q is curl's -s (or --silent)
--retry N will substitute for wget's -t N
All in all:
curl -s --retry 1 http://example.com/cgi-bin/loki/autobonus.pl
try run change with the full path of wget
/usr/bin/wget -O - -q -t 1 http://example.com/cgi-bin/loki/autobonus.pl
you can find the full path with:
which wget
and more, check if you can reach the destination domain with ping or other methods:
ping example.com
Update:
based on the comments, seems to be caused by the line in /etc/hosts:
127.0.0.1 example.com #change example.com to the real domain
It seems that you have restricted options in terms that on the server where the cron should run you have the domain pinned to 127.0.0.1 but the virtual host configuration does not work with that.
What you can do is to let wget connect by IP but send the Host header so that the virtual host matching would work:
wget -O - -q -t 1 --header 'Host: example.com' http://xx.xx.35.162/cgi-bin/loki/autobonus.pl
Update
Also probably you don't need to run this over the web server, so why not just run:
perl /path/to/your/script/autobonus.pl

Default user agent in Wget

I want to know what default user agent is passed if I use wget from command line without specifying explicit user agent.
I have some code which cahnges output based on user agent .
wget http://www.google.com -O test.html
"wget -d" will show the request made to the server.
$ wget -d http://www.google.com -O/dev/null 2>&1 |grep ^User-Agent
User-Agent: Wget/1.13.4 (linux-gnu)
User-Agent: Wget/1.13.4 (linux-gnu)
User-Agent: Wget/1.13.4 (linux-gnu)
At your shell prompt, do:
> man wget
scroll down to -U agent-string, which states:
"Wget normally identifies as Wget/version, version being the current version number of Wget".
So do:
> wget --version
which will give you the version, and thus your user-agent.
Incidently, you may find that some sites block wget, so depending on what you're doing you may need to change this.
On my Fedora 13 system, it shows Wget/1.12 (linux-gnu)
Run wget and sniff the communication.
You can also check the web server's log, usually it containts the user agent of the connecting clients.
This is what i got off the latest wget for windows: Wget/1.11.4
You could verify this with a network protocol analyzer such as Wireshark.
With Wireshark you can inspect the headers and every other detail of the whole protocol stack involved.
Wireshark is both free and open source: http://www.wireshark.org/