Probably doing something very silly here, but I'm having some trouble authenticating automatically through Tor.
I'm using 32 bit ubuntu 12.04 with obfuscated bridges.
This should be all the relevant code, but let me know if there's something else that would be useful in debugging this issue:
import socket
import socks
import httplib
def connectTor():
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 9050, True)
#9050 is the Tor proxy port
socket.socket = socks.socksocket
def newIdentity():
socks.setdefaultproxy() #Disconnect from Tor network
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 46594))
s.send("AUTHENTICATE\r\n")
response = s.recv(128)
#128 bytes of data for now, just to see how Tor responds
print response
if response.startswith("250"): #250 is the code for a positive response from Tor
s.send("SIGNAL NEWNYM\r\n") #Use a new identity
s.close()
connectTor() #Just to make sure we're still connected to Tor
Whenever I run this I get the following error:
515 Authentication failed: Password did not match HashedControlPassword value from configuration. Maybe you tried a plain text password
I tried using the --hash-password option and pasting that in the place of the AUTHENTICATE string, but that just caused the script to hang. Thoughts?
That error means that you set the HashedControlPassword option in your torrc. I would suggest option for CookieAuthentication 1 instead then using a controller library rather than doing this from scratch.
What you're trying to do here (issue a NEWNYM) is a very, very common request (1, 2) so I just added a FAQ entry for it. Here's an example using stem...
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
Related
I have used Eclipse, Jython & Pydev for a long time. Upgrading as and when new releases arrived. All has worked very well until recently when the Pydev code completion started marking common items such as dir or print as 'undefined variable'. But the program ran correctly.
The error log showed:
'The python client still hasn't connected back to the eclipse java vm (will retry..)'
'Attempt: 2 of 5 failed, trying again...(socket connected still null)'
and more attempts to 5 out of 5
'Error connecting to python process(most likely cause for failure is firewall blocking...misconfigured network)'
Also, attempting to create a live jython console CTRL+ALT+ENTER gives the following error:
'Create interactive Console' has encountered a problem
Error initializing console.
Unexpected error connecting to console.
Failed to receive suitable Hello response from pydevconsole. Last msg received: Console already exited with value: 1 while waiting for an answer.
I have spent a lot of time looking for answers here and elsewhere that have included suggestions to check:
Mixed 32/64 bit installs; Firewall problems; IPV4 preference; localhost entries; path issues and others: all I've checked out with no success so far.
Software is Windows 10, Eclipse 4.21.0, Pydev 9.1.0.2021, Java JDK 11.0.13, Jython 2.7.2
I should be most grateful for any further help on this problem.
Many thanks
Well, you get that message because PyDev does spawn a shell and communicate with it for collecting code-completion results and the same happens for the interactive console.
Now, in both cases it seems like the socket communication is being prevented in your use case (as the messages states, the usual culprit is some firewall -- or possibly an anti-virus -- or some network misconfiguration).
Unfortunately, it's really hard for me to diagnose this as it's pretty much machine-dependent (all I can say is that I checked things here and Jython is working well, so, the issue lies in some misconfiguration on your machine).
I actually have plans to stop requiring that socket communication (for getting the completions from a shell) and using stdin/stdout (https://www.brainwy.com/tracker/PyDev/1183), but this still isn't done.
As Jython does run for you, you can try to create a simple server to verify a connection works.
i.e.: Create an echo_server.py with:
import socket
if __name__ == '__main__':
HOST = '127.0.0.1'
PORT = 65432
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print('Connected by', addr)
data = conn.recv(1024)
conn.sendall(data)
print('Echo server finished')
and a client.py with:
import socket
if __name__ == '__main__':
HOST = '127.0.0.1'
PORT = 65432
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(b'Hello, world')
data = s.recv(1024)
print('Received %r' % (data,))
Then run the server part and then the client part and check if that does work properly for you.
I'm trying to set up a secure socket server on esp32 with micropython. I used/tried the latest bulid (esp32-idf3-20200117-v1.12-68-g3032ae115.bin) with a self-signed certificate.
I saw a lot of memory leak related problem with ssl.wrap_socket() on esp32/esp8266 but what I got is different:
mbedtls_ssl_handshake error: -4310
Traceback (most recent call last):
File "boot.py", line 100, in <module>
OSError: [Errno 5] EIO
and the connection fails of course. I try to connect from my laptop. The exact same code of client side works if I start a secure socket server on the laptop itself (127.0.0.1) so I suppose that the client side is OK and the problem is on the server side.
I could not find any solution for this problem yet. I tried certificate and key both in 'der' and 'pem' format the result is the same.
The toy server code I tried:
import esp
esp.osdebug(None)
import gc
gc.collect()
import usocket as socket
import ssl
import machine
import network
KEY_PATH = 'server_key.der'
CERT_PATH = 'server_cert.der'
ssid= "AP_local"
pw="passwd"
ap = network.WLAN(network.AP_IF) # create access-point interface
ap.config(essid=ssid, password=pw, authmode=4) # set the ESSID of the access point
ap.ifconfig(('192.168.3.1', '255.25.255.0', '192.168.3.1', '8.8.8.8'))
ap.active(True)
with open(KEY_PATH, 'rb') as f:
key = f.read()
print(key)
print(" ")
with open(CERT_PATH, 'rb') as f:
cert = f.read()
#s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
ad = ap.ifconfig()
addr = socket.getaddrinfo(ad[0], 8443)[0][-1]
s.bind(addr)
s.listen(5)
import gc
gc.collect()
while True:
print("ssl connection started")
cl, addr = s.accept()
scl = ssl.wrap_socket(cl, server_side=True, cert=cert, key=key)
print(gc.mem_free())
l = 0
while True:
req = scl.read(1024)
print(req)
if not req or b'\r\n' in req:
break
response = '\r\n'.join(['HTTP/1.1 200 OK',
'Content-Type: text/plain',
'OK',
'Connection: close', '\r\n'])
scl.write(response.encode("utf-8"))
scl.close()
I hope someone could help me with this, thanks!
OK, after spending a few days on searching and reading and testing I got a solution and a possible reason for this issue. I hope it will help others a little.
I want to put forward that I didn't dig into the source code of mbedtls so the reasoning is somewhat phenomenological.
First, I tried to connect from a python program with default settings which failed as shown above. Then I tried with the openssl cli, which also failed.
I tried with the example given at https://github.com/micropython/micropython/blob/master/examples/network/http_server_ssl.py and failed again but with different error codes.
Finally I found a very useful page:
https://tls.mbed.org/api/ssl_8h.html#a55ed67b6e414f9b381ff536d9ea6b9c0
which helped to understand where the problem occures.
Now depending on requirements different errors occured from -7900,-7780, -7380, -7d00.
It turned out that although in the documentation the cipher suit being used is automatically agreed during handshake but there is some bug in it or the implementation of some of them in micropython is different.
I didn't tested all the available ciphers on my laptop but e.g.: AES_256_GCM_SHA384
cipher works.
For me it is enough now.
Context
I'm creating Flask app connected to mongodb using MongoEngine via flask-mongoengine extension. I create my app using application factory pattern as specified in configuration instructions.
Problem
While running test(s), I specified testing database named datazilla_test which is passed to mongo instance via mongo.init_app(app). Even though my app.config['MONGODB_DB'] and mongo.app.config['MONGODB_DB'] instance has correct value (datazilla_test), this value is not reflected in mongo instance. Thus, when I run assertion assert mongo.get_db().name == mongo.app.config['MONGODB_DB'] this error is triggered AssertionError: assert 'datazzilla' == 'datazzilla_test'
Question
What am I doing wrong? Why database connection persist with default database datazzilla rather, than datazilla_test? How to fix it?
Source Code
# __init__.py
from flask_mongoengine import MongoEngine
mongo = MongoEngine()
def create_app(config=None):
app = Flask(__name__)
app.config['MONGODB_HOST'] = 'localhost'
app.config['MONGODB_PORT'] = '27017'
app.config['MONGODB_DB'] = 'datazzilla'
# override default config
if config is not None:
app.config.from_mapping(config)
mongo.init_app(app)
return app
# conftest.py
import pytest
from app import mongo
from app import create_app
#pytest.fixture
def app():
app = create_app({
'MONGODB_DB': 'datazzilla_test',
})
assert mongo.get_db().name == mongo.app.config['MONGODB_DB']
# AssertionError: assert 'datazzilla' == 'datazzilla_test'
return app
Mongoengine is already connected when your fixture is called, when you call app = create_app from your fixture, it tries to re-establish the connection but fails silently (as it sees that there is an existing default connection established).
This got reworked in the development version of mongoengine (see https://github.com/MongoEngine/mongoengine/pull/2038) but wasn't released yet (As of 04-JUN-2019). When that version gets out, you'll be able to disconnect any existing mongoengine connection by calling disconnect_all
In the meantime, you can either:
- check where the existing connection is created and prevent it
- try to disconnect the existing connection by using the following:
from mongoengine.connection import disconnect, _connection_settings
#pytest.fixture
def app():
disconnect()
del _connection_settings['default']
app = create_app(...)
...
But it may have other side effects
Context
Coincidentally, I figure-out fix for this problem. #bagerard answer is correct! It works for MongoClient where client's connect is set to True -this is/should be default value.
MongoClient(host=['mongo:27017'], document_class=dict, tz_aware=False, connect=False, read_preference=Primary())
If that is the case, then you have to disconnect database and delete connection settings as #bagerard explains.
Solution
However, if you change MongoClient connection to False, then you don't have to disconnect database and delete connection settings. At the end solution that worked for me was this solution.
def create_app(config=None):
...
app.config['MONGODB_CONNECT'] = False
...
Notes
As I wrote earlier. I found this solution coincidentally, I was trying to solve this problem MongoClient opened before fork. Create MongoClient only after forking. It turned out that it fixes both problems :)
P.S If there are any side effects I'm not aware of them at this point! If you find some then please share them in comments section.
Compiler error when using example provided in Flink documentation. The Flink documentation provides sample Scala code to set the REST client factory parameters when talking to Elasticsearch, https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html.
When trying out this code i get a compiler error in IntelliJ which says "Cannot resolve symbol restClientBuilder".
I found the following SO which is EXACTLY my problem except that it is in Java and i am doing this in Scala.
Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
I tried copy pasting the solution code provided in the above SO into IntelliJ, the auto-converted code also has compiler errors.
// provide a RestClientFactory for custom configuration on the internally created REST client
// i only show the setMaxRetryTimeoutMillis for illustration purposes, the actual code will use HTTP cutom callback
esSinkBuilder.setRestClientFactory(
restClientBuilder -> {
restClientBuilder.setMaxRetryTimeoutMillis(10)
}
)
Then i tried (auto generated Java to Scala code by IntelliJ)
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
import org.apache.http.auth.AuthScope
import org.apache.http.auth.UsernamePasswordCredentials
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClientBuilder
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
esSinkBuilder.setRestClientFactory((restClientBuilder) => {
def foo(restClientBuilder) = restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = { // elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}
})
foo(restClientBuilder)
})
The original code snippet produces the error "cannot resolve RestClientFactory" and then Java to Scala shows several other errors.
So basically i need to find a Scala version of the solution described in Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
Update 1: I was able to make some progress with some help from IntelliJ. The following code compiles and runs but there is another problem.
esSinkBuilder.setRestClientFactory(
new RestClientFactory {
override def configureRestClientBuilder(restClientBuilder: RestClientBuilder): Unit = {
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
// elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
httpClientBuilder.setSSLContext(trustfulSslContext)
}
})
}
}
The problem is that i am not sure if i should be doing a new of the RestClientFactory object. What happens is that the application connects to the elasticsearch cluster but then discovers that the SSL CERT is not valid, so i had to put the trustfullSslContext (as described here https://gist.github.com/iRevive/4a3c7cb96374da5da80d4538f3da17cb), this got me past the SSL issue but now the ES REST Client does a ping test and the ping fails, it throws an exception and the app shutsdown. I am suspecting that the ping fails because of the SSL error and maybe it is not using the trustfulSslContext i setup as part of new RestClientFactory and this makes me suspect that i should not have done the new, there should be a simple way to update the existing RestclientFactory object and basically this is all happening because of my lack of Scala knowledge.
Happy to report that this is resolved. The code i posted in Update 1 is correct. The ping to ECE was not working for two reasons:
The certificate needs to include the complete chain including the root CA, the intermediate CA and the cert for the ECE. This helped get rid of the whole trustfulSslContext stuff.
The ECE was sitting behind an ha-proxy and the proxy did the mapping for the hostname in the HTTP request to the actual deployment cluster name in ECE. this mapping logic did not take into account that the Java REST High Level client uses the org.apache.httphost class which creates the hostname as hostname:port_number even when the port number is 443. Since it did not find the mapping because of the 443 therefore the ECE returned a 404 error instead of 200 ok (only way to find this was to look at unencrypted packets at the ha-proxy). Once the mapping logic in ha-proxy was fixed, the mapping was found and the pings are now successfull.
I am attempting to send a broadcast packet to a certain port, but it seems that the code gives some weird errors for which I can't find a fix.
I've tried using net.socket:connect() and then calling the "send()" method, which didn't work, then I said that I should use the net.socket:on('connection') and send there, since I suppose net.socket:connect() isn't synchronous. But that gave a weird error too...
For this code:
function sendBroadcastPacket()
bip = wifi.sta.getbroadcast()
srv = net.createConnection(net.UDP,0)
print('Trying to connect on: ', bip)
srv:connect('9001', bip)
srv:send("Broadcast packet from: "..NODE_ID, function(sent)
print("Broadcasted packet! "..sent)
end)
end
I receive the following error:
PANIC: unprotected error in call to Lua API (init.lua:24: attempt to
call method 'connect' (a nil value))
Line 24 is the srv:connect line.
After that I tried listening for the connection event first, to see if it worked that way:
function sendBroadcastPacket()
bip = wifi.sta.getbroadcast()
srv = net.createConnection(net.UDP,0)
print('Trying to connect on: ', bip)
srv:on('connection', function(sck, c)
sck:send("Broadcast packet from: "..NODE_ID, function(sent)
print("Broadcasted packet! "..sent)
end)
end)
srv:connect('9001', bip)
end
I receive the following error:
PANIC: unprotected error in call to Lua API (init.lua:24: invalid
callback name)
Line 24 is the srv:on('connection') line.
What seems to be happening here? There aren't many google results, since a lot of other people use the Arduino-IDE version(btw, is that still nodemcu or do you need a different firmware for the ESP8266?).
I am getting a broadcast IP, and it connects to the AP. I didn't post here the wifi connection part because it works, I've used it to test mqtt and http connections which worked.
The build I'm using, Lua 5.1.4 on SDK 2.1.0(116b762), has the net module included.
Your attempts all failed because you don't seem to consider that UDP is a connection-less protocol. Hence, there's no connect() or on('connection'... for UDP.
The first attempt failed because send() needs to be called on a socket and not on the connection. The second failed because the connection callback name is only available for TCP, not for UDP.
Try something like this:
function sendBroadcastPacket()
local port = 9001
local bip = wifi.sta.getbroadcast()
print(string.format("Broadcasting to %s:%d", bip, port))
net.createUDPSocket():send(port, bip, "foo bar")
end
Documentation: https://nodemcu.readthedocs.io/en/latest/en/modules/net/#netudpsocket-module