I met an error when running codes at the bottom. It's like a simple ftp.
I use python2.6.6 and CentOS release 6.8
In most linux server, it gets right results like this:(I'm very sorry that I have just sign up and couldn't )
Clinet:
[root#Test ftp]# python client.py
path:put|/home/aaa.txt
Server:
[root#Test ftp]# python server.py
connected...
pre_data:put|aaa.txt|4
cmd: put
file_name: aaa.txt
file_size: 4
upload successed.
But I get errors in some server(such as my own VM in my PC). I have done lots of tests(python2.6/python2.7, Centos6.5/Centos6.7) and found this error is not because them. Here is the error imformation:
[root#Lewis-VM ftp]# python server.py
connected...
pre_data:put|aaa.txt|7sdfsdf ###Here gets the wrong result, "sdfsdf" is the content of /home/aaa.txt and it shouldn't be sent here to 'file_size' and so it cause the "ValueError" below
cmd: put
file_name: aaa.txt
file_size: 7sdfsdf
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 10699)
Traceback (most recent call last):
File "/usr/lib64/python2.6/SocketServer.py", line 570, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib64/python2.6/SocketServer.py", line 332, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib64/python2.6/SocketServer.py", line 627, in __init__
self.handle()
File "server.py", line 30, in handle
if int(file_size)>recv_size:
ValueError: invalid literal for int() with base 10: '7sdfsdf\n'
What's more, I found that if I insert a time.sleep(1) between sk.send(cmd+"|"+file_name+'|'+str(file_size)) and sk.send(data) in client.py, the error will disappear. I have said that I did tests in different system and python versions and the error is not because them. So I guess that is it because of some system configs? I have check about socket.send() and socket.recv() in python.org but fount nothing helpful. So could somebody help me to explain why this happend?
The code are here:
#!/usr/bin/env python
#coding:utf-8
################
#This is server#
################
import SocketServer
import os
class MyServer(SocketServer.BaseRequestHandler):
def handle(self):
base_path = '/home/ftp/file'
conn = self.request
print 'connected...'
while True:
#####receive pre_data: we should get data like 'put|/home/aaa|7'
pre_data = conn.recv(1024)
print 'pre_data:' + pre_data
cmd,file_name,file_size = pre_data.split('|')
print 'cmd: ' + cmd
print 'file_name: '+ file_name
print 'file_size: '+ file_size
recv_size = 0
file_dir = os.path.join(base_path,file_name)
f = file(file_dir,'wb')
Flag = True
####receive 1024bytes each time
while Flag:
if int(file_size)>recv_size:
data = conn.recv(1024)
recv_size+=len(data)
else:
recv_size = 0
Flag = False
continue
f.write(data)
print 'upload successed.'
f.close()
instance = SocketServer.ThreadingTCPServer(('127.0.0.1',9999),MyServer)
instance.serve_forever()
#!/usr/bin/env python
#coding:utf-8
################
#This is client#
################
import socket
import sys
import os
ip_port = ('127.0.0.1',9999)
sk = socket.socket()
sk.connect(ip_port)
while True:
input = raw_input('path:')
#####we should input like 'put|/home/aaa.txt'
cmd,path = input.split('|')
file_name = os.path.basename(path)
file_size=os.stat(path).st_size
sk.send(cmd+"|"+file_name+'|'+str(file_size))
send_size = 0
f= file(path,'rb')
Flag = True
#####read 1024 bytes and send it to server each time
while Flag:
if send_size + 1024 >file_size:
data = f.read(file_size-send_size)
Flag = False
else:
data = f.read(1024)
send_size+=1024
sk.send(data)
f.close()
sk.close()
The TCP is a stream of data. That is the problem. TCP do not need to keep message boundaries. So when a client calls something like
connection.send("0123456789")
connection.send("ABCDEFGHIJ")
then a naive server like
while True;
data = conn.recv(1024)
print data + "_"
may print any of:
0123456789_ABCDEFGHIJ_
0123456789ABCDEFGHIJ_
0_1_2_3_4_5_6_7_8_9_A_B_C_D_E_F_G_H_I_J_
The server has no chance to recognize how many sends client called because the TCP stack at client side just inserted data to a stream and the server must be able to process the data received in different number of buffers than the client used.
Your server must contain a logic to separate the header and the data. All of application protocols based on TCP use a mechanism to identify application level boundaries. For example HTTP separates headers and body by an empty line and it informs about the body length in a separate header.
Your program works correctly when server receives a header with the command, name and size in a separate buffer it it fails when client is fast enough and push the data into stream quickly and the server reads header and data in one chunk.
I'm trying to configure AKKA SSL connection to use my keystore and trustore files, and I want it to be able to find them on a classpath.
I tried to set application.conf to:
...
remote.netty.ssl = {
enable = on
key-store = "keystore"
key-store-password = "passwd"
trust-store = "truststore"
trust-store-password = "passwd"
protocol = "TLSv1"
random-number-generator = "AES128CounterSecureRNG"
enabled-algorithms = ["TLS_RSA_WITH_AES_128_CBC_SHA"]
}
...
This works fine if keystore and trustore files are in the current directory. In my application these files get packaged into WAR and JAR archives, and because of that I'd like to read them from the classpath.
I tried to use getResource("keystore") in application.conf as described here without any luck. Config reads it literally as a string.
I also tried to parse String conf and force it to read the value:
val conf: Config = ConfigFactory parseString (s"""
...
"${getClass.getClassLoader.getResource("keystore").getPath}"
...""")
In this case it finds proper path on the classpath as file://some_dir/target/scala-2.10/server_2.10-1.1-one-jar.jar!/main/server_2.10-1.1.jar!/keystore which is exactly where it's located (in the jar). However, underlying Netty SSL transport can't find the file given this path, and I get:
Oct 03, 2013 1:02:48 PM org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink
WARNING: Failed to initialize an accepted socket.
45a13eb9-6cb1-46a7-a789-e48da9997f0fakka.remote.RemoteTransportException: Server SSL connection could not be established because key store could not be loaded
at akka.remote.netty.NettySSLSupport$.constructServerContext$1(NettySSLSupport.scala:113)
at akka.remote.netty.NettySSLSupport$.initializeServerSSL(NettySSLSupport.scala:130)
at akka.remote.netty.NettySSLSupport$.apply(NettySSLSupport.scala:27)
at akka.remote.netty.NettyRemoteTransport$PipelineFactory$.defaultStack(NettyRemoteSupport.scala:74)
at akka.remote.netty.NettyRemoteTransport$PipelineFactory$$anon$1.getPipeline(NettyRemoteSupport.scala:67)
at akka.remote.netty.NettyRemoteTransport$PipelineFactory$$anon$1.getPipeline(NettyRemoteSupport.scala:67)
at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.registerAcceptedChannel(NioServerSocketPipelineSink.java:277)
at org.jboss.netty.channel.socket.nio.NioServerSocketPipelineSink$Boss.run(NioServerSocketPipelineSink.java:242)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.io.FileNotFoundException: file:/some_dir/server/target/scala-2.10/server_2.10-1.1-one-jar.jar!/main/server_2.10-1.1.jar!/keystore (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at java.io.FileInputStream.<init>(FileInputStream.java:97)
at akka.remote.netty.NettySSLSupport$.constructServerContext$1(NettySSLSupport.scala:118)
... 10 more
I wonder if there is any way to configure this in AKKA without implementing custom SSL transport. Maybe I should configure Netty in the code?
Obviously I can hardcode the path or read it from an environment variable, but I would prefer a more flexible classpath solution.
I decided to look at the akka.remote.netty.NettySSLSupport at the code where exception is thrown from, and here is the code:
def initializeServerSSL(settings: NettySettings, log: LoggingAdapter): SslHandler = {
log.debug("Server SSL is enabled, initialising ...")
def constructServerContext(settings: NettySettings, log: LoggingAdapter, keyStorePath: String, keyStorePassword: String, protocol: String): Option[SSLContext] =
try {
val rng = initializeCustomSecureRandom(settings.SSLRandomNumberGenerator, settings.SSLRandomSource, log)
val factory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm)
factory.init({
val keyStore = KeyStore.getInstance(KeyStore.getDefaultType)
val fin = new FileInputStream(keyStorePath)
try keyStore.load(fin, keyStorePassword.toCharArray) finally fin.close()
keyStore
}, keyStorePassword.toCharArray)
Option(SSLContext.getInstance(protocol)) map { ctx ⇒ ctx.init(factory.getKeyManagers, null, rng); ctx }
} catch {
case e: FileNotFoundException ⇒ throw new RemoteTransportException("Server SSL connection could not be established because key store could not be loaded", e)
case e: IOException ⇒ throw new RemoteTransportException("Server SSL connection could not be established because: " + e.getMessage, e)
case e: GeneralSecurityException ⇒ throw new RemoteTransportException("Server SSL connection could not be established because SSL context could not be constructed", e)
}
It looks like it must be a plain filename (String) because that's what FileInputStream takes.
Any suggestions would be welcome!
I also got stuck in similar issue and was getting similar errors. In my case I was trying to hit an https server with self-signed certificates using akka-http, with following code I was able to get through:
val trustStoreConfig = TrustStoreConfig(None, Some("/etc/Project/keystore/my.cer")).withStoreType("PEM")
val trustManagerConfig = TrustManagerConfig().withTrustStoreConfigs(List(trustStoreConfig))
val badSslConfig = AkkaSSLConfig().mapSettings(s => s.withLoose(s.loose
.withAcceptAnyCertificate(true)
.withDisableHostnameVerification(true)
).withTrustManagerConfig(trustManagerConfig))
val badCtx = Http().createClientHttpsContext(badSslConfig)
Http().superPool[RequestTracker](badCtx)(httpMat)
At the time of writing this question there was no way to do it AFAIK. I'm closing this question but I welcome updates if new versions provide such functionality or if there are other ways to do that.
I try to execute simple PostgreSQL query with erlang and epgsql.
I do:
{ok, C} = pgsql:connect("localhost", "shk", "qwerty", [{database, "mydb"}]).
>> {ok,<0.34.0>}
Then:
{ok, Cols, Rows} = pgsql:squery(C, "select * from users").
But i got error:
=ERROR REPORT==== 27-Apr-2012::17:58:23 ===
** State machine <0.34.0> terminating
** Last message in was {'EXIT',<0.32.0>,
{{badmatch,
{error,
{error,'ð\236ð¨ð\230ð\221ð\232ð\220',<<"42P01">>,
<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>,
[{position,<<"15">>}]}}},
[{erl_eval,expr,3}]}}
** When State == ready
** Data == {state,undefined,<0.35.0>,5000,
[{<<"application_name">>,<<>>},
{<<"client_encoding">>,<<"UTF8">>},
{<<"DateStyle">>,<<"ISO, DMY">>},
{<<"integer_datetimes">>,<<"on">>},
{<<"IntervalStyle">>,<<"postgres">>},
{<<"is_superuser">>,<<"off">>},
{<<"server_encoding">>,<<"UTF8">>},
{<<"server_version">>,<<"9.0.7">>},
{<<"session_authorization">>,<<"shk">>},
{<<"standard_conforming_strings">>,<<"off">>},
{<<"TimeZone">>,<<"posixrules">>}],
undefined,undefined,undefined,
{30932,488494147},
{statement,undefined,undefined,undefined},
73}
** Reason for termination =
** {{badmatch,{error,{error,'ð\236ð¨ð\230ð\221ð\232ð\220',<<"42P01">>,
<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>,
[{position,<<"15">>}]}}},
[{erl_eval,expr,3}]}
What's wrong i do? How can i fix it?
Thank you.
The error string seems to be in russian if I recognize the characters.
To view the response you can use the following command:
io:format("~ts",[<<208,190,209,130,208,189,208,190,209,136,208,181,
208,189,208,184,208,181,32,34,109,121,100,98,34,
32,208,189,208,181,32,209,129,209,131,209,137,
208,181,209,129,209,130,208,178,209,131,208,181,
209,130>>]).
отношение "mydb" не существует
A quick google translate makes me think the database mydb does not exist or you do not have permissions to use it.
try simply doing
Response = pgsql:squery(C, "select * from mydb"),
io:format("~p~n",[Response]).
And see what he is giving back from the server, maybe you have typo or table don't exists etc.
also check this out http://www.erlangatwork.com/2009/01/erlang-and-postgresql.html
From epgsql docs:
Errors
Errors originating from the PostgreSQL backend are returned as {error, #error{}},
see pgsql.hrl for the record definition. epgsql functions may also return
{error, What} where What is one of the following:
{unsupported_auth_method, Method} - required auth method is unsupported
timeout - request timed out
closed - connection was closed
sync_required - error occured and pgsql:sync must be called
Try to include pgsql.hrl, capture the error and print the error message, that should point you to the right direction.
I'm trying to send push notifications to an iPhone using Python. I've exported my certificate and private key into a p12 file from keychain access and then converted it into pem file using the following command:
openssl pkcs12 -in cred.p12 -out cert.pem -nodes -clcerts
I'm using APNSWrapper in Python for the connection.
I run the following code:
deviceToken = 'Qun\xaa\xd ... c0\x9c\xf6\xca'
# create wrapper
wrapper = APNSNotificationWrapper('/path/to/cert/cert.pem', True)
# create message
message = APNSNotification()
message.token(deviceToken)
message.badge(5)
# add message to tuple and send it to APNS server
wrapper.append(message)
wrapper.notify()
And then I get the error message:
ssl.SSLError: (1, '_ssl.c:485:
error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown')
Can anyone help me out on this?
I recently did this using Django - http://leecutsco.de/2009/07/14/push-on-the-iphone/
May be useful? It's making use of no extra libraries other than those included with Python already. Wouldn't take much to extract the send_message() method out.
Have you considered the Twisted package? The below code is taken from here:
from struct import pack
from OpenSSL import SSL
from twisted.internet import reactor
from twisted.internet.protocol import ClientFactory, Protocol
from twisted.internet.ssl import ClientContextFactory
APNS_SERVER_HOSTNAME = "<insert the push hostname from your iPhone developer portal>"
APNS_SERVER_PORT = 2195
APNS_SSL_CERTIFICATE_FILE = "<your ssl certificate.pem>"
APNS_SSL_PRIVATE_KEY_FILE = "<your ssl private key.pem>"
class APNSClientContextFactory(ClientContextFactory):
def __init__(self):
self.ctx = SSL.Context(SSL.SSLv3_METHOD)
self.ctx.use_certificate_file(APNS_SSL_CERTIFICATE_FILE)
self.ctx.use_privatekey_file(APNS_SSL_PRIVATE_KEY_FILE)
def getContext(self):
return self.ctx
class APNSProtocol(Protocol):
def sendMessage(self, deviceToken, payload):
# notification messages are binary messages in network order
# using the following format:
# <1 byte command> <2 bytes length><token> <2 bytes length><payload>
fmt = "!cH32cH%dc" % len(payload)
command = 0
msg = struct.pack(fmt, command, deviceToken,
len(payload), payload)
self.transport.write(msg)
class APNSClientFactory(ClientFactory):
def buildProtocol(self, addr):
print "Connected to APNS Server %s:%u" % (addr.host, addr.port)
return APNSProtocol()
def clientConnectionLost(self, connector, reason):
print "Lost connection. Reason: %s" % reason
def clientConnectionFailed(self, connector, reason):
print "Connection failed. Reason: %s" % reason
if __name__ == '__main__':
reactor.connectSSL(APNS_SERVER_HOSTNAME,
APNS_SERVER_PORT,
APNSClientFactory(),
APNSClientContextFactory())
reactor.run()
there were a few bugs in the originally posted code, so here's a corrected version that works for me.
from struct import pack
from OpenSSL import SSL
from twisted.internet import reactor
from twisted.internet.protocol import ClientFactory, Protocol
from twisted.internet.ssl import ClientContextFactory
import binascii
import struct
APNS_SERVER_HOSTNAME = "gateway.sandbox.push.apple.com"
APNS_SERVER_PORT = 2195
APNS_SSL_CERTIFICATE_FILE = "<your ssl certificate.pem>"
APNS_SSL_PRIVATE_KEY_FILE = "<your ssl private key.pem>"
DEVICE_TOKEN = "<hexlified device token>"
MESSAGE = '{"aps":{"alert":"twisted test"}}'
class APNSClientContextFactory(ClientContextFactory):
def __init__(self):
self.ctx = SSL.Context(SSL.SSLv3_METHOD)
self.ctx.use_certificate_file(APNS_SSL_CERTIFICATE_FILE)
self.ctx.use_privatekey_file(APNS_SSL_PRIVATE_KEY_FILE)
def getContext(self):
return self.ctx
class APNSProtocol(Protocol):
def connectionMade(self):
print "connection made"
self.sendMessage(binascii.unhexlify(DEVICE_TOKEN), MESSAGE)
self.transport.loseConnection()
def sendMessage(self, deviceToken, payload):
# notification messages are binary messages in network order
# using the following format:
# <1 byte command> <2 bytes length><token> <2 bytes length><payload>
fmt = "!cH32sH%ds" % len(payload)
command = '\x00'
msg = struct.pack(fmt, command, 32, deviceToken,
len(payload), payload)
print "%s: %s" %(binascii.hexlify(deviceToken), binascii.hexlify(msg))
self.transport.write(msg)
class APNSClientFactory(ClientFactory):
def buildProtocol(self, addr):
print "Connected to APNS Server %s:%u" % (addr.host, addr.port)
return APNSProtocol()
def clientConnectionLost(self, connector, reason):
print "Lost connection. Reason: %s" % reason
def clientConnectionFailed(self, connector, reason):
print "Connection failed. Reason: %s" % reason
if __name__ == '__main__':
reactor.connectSSL(APNS_SERVER_HOSTNAME,
APNS_SERVER_PORT,
APNSClientFactory(),
APNSClientContextFactory())
reactor.run()
Try to update to latest APNSWrapper version (0.4). There is build-in support of openssl command line tool (openssl s_client) now.
I tried both APNSWrapper and Lee Peckham's code and couldn't get it to work under Snow Leopard with Python 2.6. After a lot of trial and error it finally worked with pyOpenSSL.
I already did a post with details and code snippets here so I'll just refer you there.