There are two uses for SO_REUSEADDR:
binding two servers on the same address (for server performance)
binding a client then a server (for example for hole punching)
It seems that the second one doesn't work on linux (I tested on RedHat and Chromium OS) although it works on macOS.
I made this little code:
import socket
conn = socket.create_connection(("google.fr", 80))
if len(conn.getsockname()) == 2:
family = socket.AF_INET
else:
family = socket.AF_INET6
s = socket.socket(family)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s.bind(conn.getsockname())
This code works on macOS but fails with OSError: [Errno 98] Address already in use otherwise.
Is there any way to make it work? If not, where does this behavior come from?
Related
I have used Eclipse, Jython & Pydev for a long time. Upgrading as and when new releases arrived. All has worked very well until recently when the Pydev code completion started marking common items such as dir or print as 'undefined variable'. But the program ran correctly.
The error log showed:
'The python client still hasn't connected back to the eclipse java vm (will retry..)'
'Attempt: 2 of 5 failed, trying again...(socket connected still null)'
and more attempts to 5 out of 5
'Error connecting to python process(most likely cause for failure is firewall blocking...misconfigured network)'
Also, attempting to create a live jython console CTRL+ALT+ENTER gives the following error:
'Create interactive Console' has encountered a problem
Error initializing console.
Unexpected error connecting to console.
Failed to receive suitable Hello response from pydevconsole. Last msg received: Console already exited with value: 1 while waiting for an answer.
I have spent a lot of time looking for answers here and elsewhere that have included suggestions to check:
Mixed 32/64 bit installs; Firewall problems; IPV4 preference; localhost entries; path issues and others: all I've checked out with no success so far.
Software is Windows 10, Eclipse 4.21.0, Pydev 9.1.0.2021, Java JDK 11.0.13, Jython 2.7.2
I should be most grateful for any further help on this problem.
Many thanks
Well, you get that message because PyDev does spawn a shell and communicate with it for collecting code-completion results and the same happens for the interactive console.
Now, in both cases it seems like the socket communication is being prevented in your use case (as the messages states, the usual culprit is some firewall -- or possibly an anti-virus -- or some network misconfiguration).
Unfortunately, it's really hard for me to diagnose this as it's pretty much machine-dependent (all I can say is that I checked things here and Jython is working well, so, the issue lies in some misconfiguration on your machine).
I actually have plans to stop requiring that socket communication (for getting the completions from a shell) and using stdin/stdout (https://www.brainwy.com/tracker/PyDev/1183), but this still isn't done.
As Jython does run for you, you can try to create a simple server to verify a connection works.
i.e.: Create an echo_server.py with:
import socket
if __name__ == '__main__':
HOST = '127.0.0.1'
PORT = 65432
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print('Connected by', addr)
data = conn.recv(1024)
conn.sendall(data)
print('Echo server finished')
and a client.py with:
import socket
if __name__ == '__main__':
HOST = '127.0.0.1'
PORT = 65432
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(b'Hello, world')
data = s.recv(1024)
print('Received %r' % (data,))
Then run the server part and then the client part and check if that does work properly for you.
I'm trying to set up a secure socket server on esp32 with micropython. I used/tried the latest bulid (esp32-idf3-20200117-v1.12-68-g3032ae115.bin) with a self-signed certificate.
I saw a lot of memory leak related problem with ssl.wrap_socket() on esp32/esp8266 but what I got is different:
mbedtls_ssl_handshake error: -4310
Traceback (most recent call last):
File "boot.py", line 100, in <module>
OSError: [Errno 5] EIO
and the connection fails of course. I try to connect from my laptop. The exact same code of client side works if I start a secure socket server on the laptop itself (127.0.0.1) so I suppose that the client side is OK and the problem is on the server side.
I could not find any solution for this problem yet. I tried certificate and key both in 'der' and 'pem' format the result is the same.
The toy server code I tried:
import esp
esp.osdebug(None)
import gc
gc.collect()
import usocket as socket
import ssl
import machine
import network
KEY_PATH = 'server_key.der'
CERT_PATH = 'server_cert.der'
ssid= "AP_local"
pw="passwd"
ap = network.WLAN(network.AP_IF) # create access-point interface
ap.config(essid=ssid, password=pw, authmode=4) # set the ESSID of the access point
ap.ifconfig(('192.168.3.1', '255.25.255.0', '192.168.3.1', '8.8.8.8'))
ap.active(True)
with open(KEY_PATH, 'rb') as f:
key = f.read()
print(key)
print(" ")
with open(CERT_PATH, 'rb') as f:
cert = f.read()
#s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ad = ap.ifconfig()
addr = socket.getaddrinfo(ad[0], 8443)[0][-1]
s.bind(addr)
s.listen(5)
import gc
gc.collect()
while True:
print("ssl connection started")
cl, addr = s.accept()
scl = ssl.wrap_socket(cl, server_side=True, cert=cert, key=key)
print(gc.mem_free())
l = 0
while True:
req = scl.read(1024)
print(req)
if not req or b'\r\n' in req:
break
response = '\r\n'.join(['HTTP/1.1 200 OK',
'Content-Type: text/plain',
'OK',
'Connection: close', '\r\n'])
scl.write(response.encode("utf-8"))
scl.close()
I hope someone could help me with this, thanks!
OK, after spending a few days on searching and reading and testing I got a solution and a possible reason for this issue. I hope it will help others a little.
I want to put forward that I didn't dig into the source code of mbedtls so the reasoning is somewhat phenomenological.
First, I tried to connect from a python program with default settings which failed as shown above. Then I tried with the openssl cli, which also failed.
I tried with the example given at https://github.com/micropython/micropython/blob/master/examples/network/http_server_ssl.py and failed again but with different error codes.
Finally I found a very useful page:
https://tls.mbed.org/api/ssl_8h.html#a55ed67b6e414f9b381ff536d9ea6b9c0
which helped to understand where the problem occures.
Now depending on requirements different errors occured from -7900,-7780, -7380, -7d00.
It turned out that although in the documentation the cipher suit being used is automatically agreed during handshake but there is some bug in it or the implementation of some of them in micropython is different.
I didn't tested all the available ciphers on my laptop but e.g.: AES_256_GCM_SHA384
cipher works.
For me it is enough now.
I'm debugging a 3rd-party network application and trying to figure out why it reports errors when calling setsockopt with IP_ADD_MEMBERSHIP to set up a multicast group. The application is in C++, but I've written an MWE in python that replicates the same syscalls:
import socket
import struct
ETH0_IP = "192.168.88.85"
ETH0_1_IP = "192.168.88.254"
MULTICAST_IP = "224.0.0.7"
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
ip = socket.inet_aton(ETH0_IP)
s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, ip)
group = struct.pack("4s4s", socket.inet_aton(MULTICAST_IP), ip)
s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, group)
# s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s2 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
ip2 = socket.inet_aton(ETH0_1_IP)
s2.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, ip2)
group2 = struct.pack("4s4s", socket.inet_aton(MULTICAST_IP), ip2)
# the second group is added to the first socket so that we can only bind to one socket and read data from it
s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, group2)
At the second IP_ADD_MEMBERSHIP call I get error OSError: [Errno 98] Address already in use.
I found out this only happens when ETH0_1_IP is a subinterface of ETH_0_IP. And I'm not sure if this is expected. If it is, is there a way to actually detect this situation and discard subinterfaces of already bound interfaces? Further, would my multicast socket receive data sent to the subinterface if registration for it fails with the above error?
For the sake of completeness:
$ cat /etc/network/interfaces
auto lo
iface lo inet loopback
iface lo inet6 loopback
auto eth0:1
iface eth0:1 inet static
address 192.168.88.254
netmask 255.255.240.0
Linux is tracking your alias interface as the same interface and so rejecting the attempt to re-use the interface.
In a bit more detail, I have run your code successfully on CentOS 7 using two separate physical interfaces with no changes. If I then change the code to use an alias on the same physical address, it fails with the same error that you see.
Digging a little further, I see that if I dump the interface indeces (using SIOCGIFINDEX) for the physical adaptor and the alias, they do indeed have the same index.
If you want to use Python to check this for yourself, have a quick look at https://gist.github.com/firaxis/0e538c8e5f81eaa55748acc5e679a36e for some code (missing imports of ctypes and socket) and then try something like this:
print(Interface(name="eth0").index)
print(Interface(name="eth0:1").index)
I'm trying to communicate with an AD9913 evaluation board (Analog Devices), which has a Cypress FX-2 USB controller. I wrote a fully functional Python library which works flawlessly on Windows 7 (using a driver generated through libusb-win32-1.2.6.0's inf-wizard.exe), but now I try to run the same software from a Raspberry Pi 3 with Raspbian Jessie and cannot seem to write or read.
The following minimal example works on Windows 7:
import os
os.environ['PYUSB_DEBUG'] = 'debug'
import usb.core
print "usb.core file:", usb.core.__file__
# get device
dev = usb.core.find(idVendor=0x0456, idProduct=0xEE25)
if dev is None:
raise ValueError("Device not found")
dev.set_configuration()
# get the firmware version
print "Write: ", dev.write(0x01, [0x00])
print "Read: ", dev.read(0x81, 7)
The ouput here is:
usb.core file: C:\Python27\lib\site-packages\usb\core.pyc
Write: 1
Read: array('B', [1, 0])
However, on the Raspberry several problems occur. First, the product ID is 0xEE24 (which I found out through dmesg), and second, the code example above gives
usb.core file: /usr/local/lib/python2.7/dist-packages/usb/core.pyc
Write:
Traceback (most recent call last):
File "usbtesting_minimal_linux.py", line 29, in <module>
print "Write: ", dev.write(0x01, [0x00])
File "/usr/local/lib/python2.7/dist-packages/usb/core.py", line 948, in write
self.__get_timeout(timeout)
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 824, in bulk_write
timeout)
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 920, in __write
_check(retval)
File "/usr/local/lib/python2.7/dist-packages/usb/backend/libusb1.py", line 595, in _check
raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 5] Input/Output Error
I'm executing the script by calling sudo python minimal_example.py, otherwise (as expected) I get usb.core.USBError: [Errno 13] Access denied (insufficient permissions). (For the testing described above, I removed the udev rules I added to allow access to my regular user account, and rebooted and replugged the device afterwards.)
I also noticed that on Windows, Configuration 1 (which is the only configuration) as revealed by adding
cfg = dev.get_active_configuration()
to aforementioned example script, has only one Interface 0, where as on Linux Interface 0 has no endpoints, but there are additional interfaces Interface 0, 1, Interface 0, 2 and Interface 0,3, the first of which is identical to the sole interface shown by Windows.
Detaching kernel drivers as described in this post -- which given my current understanding as a USB novice should not be necessary here -- does not resolve the problem.
I suspect that the issue stems from a difference between the Windows/libusb and Linux implementation/backend of the USB communication (already hinted at by the fact that the productID shown is different for the very same piece of hardware attached), which might require some changes to the way I'm writing to the USB endpoint.
I'm happy to provide more information if needed and welcome all hints and suggestions.
Most likely, this error caused by the firmware is not uploaded. Usually speaking, FX-2 USB controller does not save the firmware.
Although I still cannot communicate with the device on the Raspberry Pi, the Error 5 problem is resolved by explicitly selecting a suitable alternate setting with proper endpoints, e.g.
dev.set_interface_altsetting(interface = 0, alternate_setting = 1)
Then read/write requests no longer fail with Error 5, but with a timeout error (which is Error 110 for libusb1 and None for libusb0).
I now suspect that there is a problem with how the device is detected by the operating system, since not only the productID changes between Windows and Linux, but also wMaxPacketSize for the endpoints I'm interested in is given as 64 bytes on Windows and 512 bytes on Linux.
Once I'm further along in my troubleshooting, I will either open a new question and add the link as a comment to this answer (or post the whole solution here in case I can solve everything right away).
Probably doing something very silly here, but I'm having some trouble authenticating automatically through Tor.
I'm using 32 bit ubuntu 12.04 with obfuscated bridges.
This should be all the relevant code, but let me know if there's something else that would be useful in debugging this issue:
import socket
import socks
import httplib
def connectTor():
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050, True)
#9050 is the Tor proxy port
socket.socket = socks.socksocket
def newIdentity():
socks.setdefaultproxy() #Disconnect from Tor network
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 46594))
s.send("AUTHENTICATE\r\n")
response = s.recv(128)
#128 bytes of data for now, just to see how Tor responds
print response
if response.startswith("250"): #250 is the code for a positive response from Tor
s.send("SIGNAL NEWNYM\r\n") #Use a new identity
s.close()
connectTor() #Just to make sure we're still connected to Tor
Whenever I run this I get the following error:
515 Authentication failed: Password did not match HashedControlPassword value from configuration. Maybe you tried a plain text password
I tried using the --hash-password option and pasting that in the place of the AUTHENTICATE string, but that just caused the script to hang. Thoughts?
That error means that you set the HashedControlPassword option in your torrc. I would suggest option for CookieAuthentication 1 instead then using a controller library rather than doing this from scratch.
What you're trying to do here (issue a NEWNYM) is a very, very common request (1, 2) so I just added a FAQ entry for it. Here's an example using stem...
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)