I'd like to use socket library to connect to a server (not necessary a webserver) through an http proxy. Is it possible?
For example (with requests library):
import requests
r = requests.get("http://www.google.com", #but not necessary to http port.
proxies={"http": "http://ipproxy:portproxy"})
-UPDATE-
import urllib.request
pr = "ipproxy:portproxy"
while True:
try:
proxy = urllib.request.ProxyHandler({'http': pr})
opener = urllib.request.build_opener(proxy)
urllib.request.install_opener(opener)
url = "ftp://ip" # or ssh:// or some other port
data = None
headers = {}
req = urllib.request.Request(url, data, headers)
print ("Request sent")
except:
print ("An error occurred")
It is possible for most types of connections (HTTP and FTP etc., possibly HTTPS, although this is a bit more tricky) using the urllib module with a ProxyHandler object.
Related
I configured an HTTPS website on AWS, which allows visiting from a white list of IPs.
My local machine runs with a VPN connection, which is in the white list.
I could visit the website from web browser or by the java.net.http package with the below code:
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("https://mywebsite/route"))
.GET() // GET is default
.build();
HttpResponse<Void> response = client.send(request,
HttpResponse.BodyHandlers.discarding());
But if I replaced the code with a Vertx implementation from io.vertx.ext.web.client package, I got a 403 forbidden response from the same website.
WebClientOptions options = new WebClientOptions().setTryUseCompression(true).setTrustAll(true);
HttpRequest<Buffer> request = WebClient.create(vertx, options)
.getAbs("https://mywebsite/route")
.ssl(true).putHeaders(headers);
request.send(asyncResult -> {
if (asyncResult.succeeded()) {
HttpResponse response = asyncResult.result();
}
});
Does anyone have an idea why the Vertx implementation is rejected?
Finally got the root cause. I started a local server that accepts the testing request and forwards it to the server on AWS. The testing client sent the request to localhost and thus "Host=localhost:8080/..." is in the request header. In the Vert.X implementation, a new header entry "Host=localhost:443/..." is wrongly put into the request headers. I haven't debug the Vert.X implementation so I have no idea why it behaviors as this. But then the AWS firewall rejected the request with a rule that a request could not come from localhost.
I've used zeep against a SOAP-service, and it works perfectly. However, when this service is placed behind Gravitee API Manager, I'm unable to get it to work any longer, just get a 404 response.
Here's my code :
from zeep import Client, Settings
import base64,sys, logging, traceback
from requests import Session
from zeep.transports import Transport
import requests
from lxml import etree
wsdl= 'https://link_to_service_on_gravitee'
session = Session()
session.verify = False
session.headers['Api-Key']= 'xxxxx'
transport = Transport(session=session)
settings = Settings(raw_response=True, strict=False, xml_huge_tree=True)
client = Client(wsdl, transport=transport, settings=settings)
data = []
data.append(
{'ServerProcessId': 'GL07',
'OrderNumber': 1}
)
cdata = []
cdata.append(
{'Username': 'xxx',
'Client': 'yyy',
'Password': 'zzz'}
)
node = client.create_message(client.service, 'GetResult',input=data, credentials=cdata)
print('*** SOAP Message')
print(etree.tostring(node))
print('*** End SOAP Message')
response = requests.Response()
try:
response = client.service.GetResult(input=data, credentials=cdata)
print(response)
except Exception as e:
print(response.headers)
logging.error(traceback.format_exc())
When I test the message generated by Python (etree.tostring(node)) in SOAPUI, it works correctly. Also, if I alter the api-key, I get an error about authentication problems, so Gravitee seems to accept the key from my code.
But the response I get, with correct api-key, is always [404].
Got it to work when I downloaded the wsdl to a local file, but don't want to do this for every wsdl.
Any ideas ?
There are multiple reasons for getting a 404 from Gravitee:
* Did you create an API
* Did you create a simple plan for this API
* Did you deploy the API to the gateway.
Once all those steps are done, you should be able to consume your API.
Hope it helps,
Regards,
So I am very new to networking and the Socket module in Python. So I watched some Youtube tutorials and found one on how to write the code for a simple server. My problem is right when the server receives data from the client, the server close() and loses connection to the client right when it receives the data. I want the server to automatically lose connection to the client but not "shutdown" or close(). I want to set it (if its possible) so that while the server is running in my Python Shell, if I want to close() the connection I use hot keys like for example "Control+E"? Here is my code so far:
#!/usr/bin/python
import socket
import sys
# Create a TCP/IP socket to listen on
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Prevent from "adress already in use" upon server restart
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Bind the socket to port 8081 on all interfaces
server_address = ('localhost',8081)
print ('starting up on %s port %s')%(server_address)
server.bind(server_address)
#Listen for connections
server.listen(5)
#Wait for one incoming connection
connection, client_address = server.accept()
print 'connection from', connection.getpeername()
# Let's recieve something
data = connection.recv(4096)
if data:
print "Recived ", repr(data)
#send the data back nicely formatted
data = data.rstrip()
connection.send("%s\n%s\n%s\n"%('-'*80, data.center(80),'-'*80))
# lose the connection from our side (the Server side)
connection.shutdown(socket.SHUT_RD | socket.SHUT_WR)
connection.close()
print 'Connection closed'
# And stop listening
server.close()
==================================================================================
Here is the code I am using (on the server side):
#!/usr/bin/python
import socket, sys
import select
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Let's set the socket option reuse to 1, so that our server terminates quicker
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("localhost", 8081))
srv.listen(5)
while True:
print "Waiting for a client to connect"
(client, c_address) = srv.accept() #blocking wait for client
print "Client connected"
# Client has connected, add him to a list which we can poll for data
client_list = [client]
while close_socket_condition == 0:
ready_to_read, ready_to_write, in_error = select.select(client_list, [], [] , 1) #timeout 1 second
for s in ready_to_read: #Check if there is any socket that has data ready for us
data = client.recv(1024) # blocks until some data is read
if data:
client.send("echo:" + data)
client.close()
close_socket_condition = 1
And here is the error it is giving me when I try to send a string to the server:
data = s.recv(1024)
File "C:\Python27\lib\socket.py", line 170, in _dummy
raise error(EBADF, 'Bad file descriptor')
error: [Errno 9] Bad file descriptor
Here is example on a non-blocking socket read with similar structure as yours.
The server will establish a socket in localhost, and wait for a client to connect. After that it will start polling the socket for data, and also keep checking the exit condition close_socket_condition. Handling ctrl-e or other exit events will be left as an exercise :)
First we start socket, very much the same way as you:
#!/usr/bin/python
import socket, sys
import select
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Let's set the socket option reuse to 1, so that our server terminates quicker
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("localhost", 8081))
srv.listen(5)
Then we declare our external exit condition close_socket_condition, and start eternal while loop that will always welcome new clients:
close_socket_condition = 0
while True:
print "Waiting for a client to connect"
(client, c_address) = srv.accept() #blocking wait for client
print "Client connected"
Now a client has connected, and we should start our service loop:
# Client has connected, add him to a list which we can poll for data
client_list = [client]
while close_socket_condition == 0:
Inside the service loop we will keep polling his socket for data and if nothing has arrived, we check for exit condition:
ready_to_read, ready_to_write, in_error = select.select(client_list, [], [] , 1) #timeout 1 second
for s in ready_to_read: #Check if there is any socket that has data ready for us
data = client.recv(1024) # blocks until some data is read
if data:
client.send("echo:" + data)
client.close()
close_socket_condition = 1
This code is simplified example, but the server will keep accepting new clients, and always reuse the connection. It does not handle client side terminations etc.
Hope it helps
I'm making a server and client in Python 3.3 using the socket module. My server code is working fine, but this client code is returning an error. Here's the code:
import socket
import sys
import os
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server_address = ('192.168.1.5', 4242)
sock.bind(server_address)
while True:
command = sock.recv(1024)
try:
os.system(command)
sock.send("yes")
except:
sock.send("no")
And here's the error:
Error in line: command = sock.recv(1024)
OSError: [WinError 10057] A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied
What on earth is going on?
It looks like you're confused about when you actually connect to the server, so just remember that you always bind to a local port first. Therefore, this line:
server_address = ('192.168.1.5', 4242)
Should actually read:
server_address = ('', 4242)
Then, before your infinite loop, put in the following line of code:
sock.connect(('192.168.1.5', 4242))
And you should be good to go. Good luck!
EDIT: I suppose I should be more careful with the term "always." In this case, you want to bind to a local socket first.
You didn't accept any requests, and you can only recv and/or send on the accepted socket in order to communicate with client.
Does your server only need one client to be connected? If so, try this solution:
Try adding the following before the while loop
sock.listen(1) # 1 Pending connections at most
client=sock.accept() # Accept a connection request
In the while loop, you need to change all sock to client because server socket cannot
be either written or read (all it does is listening at 192.168.1.5:4242).
I'm trying to make a http get, using Lua Socket:
local client = socket.connect('warm-harbor-2019.herokuapp.com',80)
if client then
client:send("GET /get_tweets HTTP/1.0\r\n\r\n")
s, status, partial = client:receive(1024)
end
end
I expect s to be a tweet, since the get that I'm making returns one.
But I'm getting:
http/1.1 404 object not found
Here is a runnable version of your code example (that exhibit the problem you described):
local socket = require "socket"
local client = socket.connect('warm-harbor-2019.herokuapp.com',80)
if client then
client:send("GET /get_tweets HTTP/1.0\r\n\r\n")
local s, status, partial = client:receive(1024)
print(s)
end
If you read the error page returned, you can see that its title is Heroku | No such app.
The reason for that is that the Heroku router only works when a Host header is provided. The easiest way to do it is to use the actual HTTP module of LuaSocket instead of TCP directly:
local http = require "socket.http"
local s, status, headers = http.request("http://warm-harbor-2019.herokuapp.com/get_tweets")
print(s)
If you cannot use socket.http you can pass the Host header manually:
local socket = require "socket"
local client = socket.connect('warm-harbor-2019.herokuapp.com',80)
client:send("GET /get_tweets HTTP/1.0\r\nHost: warm-harbor-2019.herokuapp.com\r\n\r\n")
local s, status, partial = client:receive(1024)
print(s, status, partial)
With my version of LuaSocket, s will be nil, status will be "closed" and partial will contain the full HTTP response (with headers etc).