Incorrect throughput value in python - sockets

I am writing a python code to find throughput between server and client. It is based on speedtest.net functionality where I am sending a dummy file to calculate the speed. The problem I am facing is unreliable throughput output. I will appreciate your suggestions on the same. Here is the code.
server.py
import socket
import os
port = 60000
s = socket.socket()
host = socket.gethostname()
s.bind((host, port))
s.listen(5)
print 'Server listening....'
while True:
conn, addr = s.accept()
print 'Got connection from', addr
data = conn.recv(1024)
print('Server received', repr(data))
filename='akki.txt'
b = os.path.getsize(filename)
f = open(filename,'rb')
l = f.read(b)
while (l):
conn.send(l)
l = f.read(b)
f.close()
print('Done sending')
conn.send('Thank you for connecting')
conn.close()
Client.py
import socket
import time
import os
s = socket.socket()
host = socket.gethostname()
port = 60000
t1 = time.time()
s.connect((host, port))
s.send("Hello server!")
with open('received_file', 'wb') as f:
print 'file opened'
t2 = time.time()
while True:
data = s.recv(1024)
if not data:
break
f.write(data)
t3 = time.time()
print data
print 'Total:', t3 - t1
print 'Throughput:', round((1024.0 * 0.001) / (t3 - t1), 3),
print 'K/sec.'
f.close()
print('Successfully received the file')
s.close()
print('connection closed')
Output when sending akki.txt
Server Output
Server listening....
Got connection from ('10.143.47.165', 60902)
('Server received', "'Hello server!'")
Done sending
Client output
file opened
Raw timers: 1503350568.11 1503350568.11 1503350568.11
Total: 0.00499987602234
**Throughput: 204.805 K/sec.**
Successfully received the file
connection closed
Output for ak.zip ( which is bigger file)
Client output
file opened
Total: 0.0499999523163
**Throughput: 20.48 K/sec.**
Successfully received the file
connection closed

Short Answer: you need to take the file size into consideration.
More Details:
Throughput is data/time. Your calculation:
round((1024.0 * 0.001) / (t3 - t1), 3)
Doesn't take the file size into account. Since sending a large file takes more time, 't3-t1' is bigger so your throughput is smaller (same numerator with larger denominator). Try adding the file size to the formula and you should get much more constant results.
Hope this helps.

Related

Socket recvfrom function hanging, not recognizing ICMP packet

So I'm trying to do a mock Traceroute function where a UDP packet is sent to an IP address. I'm trying to design the program in such a way where a packet is sent each time the packet makes it to a router. I am trying to do this by making a very short TTL. However, the recvfrom function is stalling.
Here's the code:
host_addr = gethostbyname(host)
port = 33434
max_hops = 30
ttl = 1
while True:
recv_socket = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP)
send_socket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP)
send_socket.setsockopt(0, 4, ttl)
recv_socket.bind(("", port))
send_socket.sendto("waddup".encode(), (host_addr, port))
cur_addr = None
cur_name = None
path_end = False
cur_bytes = None
attempts_left = 3
timeReceived = 0
pingStartTime = 0
while not path_end and attempts_left > 0:
try:
pingStartTime = time.time()
cur_bytes, cur_addr = recv_socket.recvfrom(1024)
timeReceived = time.time()
path_end = True
cur_addr = cur_addr[0]
try:
cur_name = gethostbyaddr(cur_addr)[0]
except error:
cur_name = cur_addr
except error:
attempts_left -= 1
send_socket.close()
recv_socket.close()
if not path_end:
pass
if cur_addr is not None:
cur_host = "%s (%s) " % (cur_name, cur_addr)
else:
cur_host = ""
print("%d: %.0f ms " %
(
ttl,
(timeReceived - pingStartTime) * 1000,
) + cur_host
)
ttl += 1
if cur_addr == host_addr or ttl > max_hops:
break
I have set up the receiving socket for an ICMP packet as far as I can tell but it just hangs on recvfrom. I've allowed all ICMP connections on Windows Defender and when I run Wireshark, an appropriate ICMP packet is sent to my router.
The packet with the TTL expired message is the one I want to receive
I'm new to networking but all the code I've seen online has this exact setup. Would anyone be able to tell me why it's stalling and what I can do to fix it? I want the program to read the ICMP packet with the TTL expiration message.

How do I make Simpy simulation to depict a markovian M/M/1 process?

output printing the len of arrival and service timesI am trying to implement an M/M/1 markovian process with exponential inter arrival and exponential service times using simpy. The code runs fine but I dont quite get the expected results. Also the number of list items in arrival times is lesser than the number of list items in service time after the code is run.
# make a markovian queue
# make a server as a resource
# make customers at random times
# record the customer arrival time
# customer gets the resource
# record when the customer got the resource
# serve the customers for a random time using resource
# save this random time as service time
# customer yields the resource and next is served
import statistics
import simpy
import random
arrival_time = []
service_time = []
mean_service = 2.0
mean_arrival = 1.0
num_servers = 1
class Markovian(object):
def __init__(self, env, num_servers):
self.env = env
self.servers = simpy.Resource(env, num_servers)
#self.action = env.process(self.run())
def server(self,packet ):
#timeout after random service time
t = random.expovariate(1.0/mean_service)
#service_time.append(t)
yield self.env.timeout(t)
def getting_service(env, packet, markovian):
# new packet arrives in the system
arrival_time = env.now
with markovian.servers.request() as req:
yield req
yield env.process(markovian.server(packet))
service_time.append(env.now - arrival_time)
def run_markovian(env,num_servers):
markovian = Markovian(env,num_servers)
packet = 0
#generate new packets
while True:
t = random.expovariate(1.0/mean_arrival)
arrival_time.append(t)
yield env.timeout(t)
packet +=1
env.process(Markovian.getting_service(env,packet,markovian))
def get_average_service_time(service_time):
average_service_time = statistics.mean(service_time)
return average_service_time
def main():
random.seed(42)
env= simpy.Environment()
env.process(Markovian.run_markovian(env,num_servers))
env.run(until = 50)
print(Markovian.get_average_service_time(service_time))
print (arrival_time)
print (service_time)
if __name__ == "__main__":
main()
Hello there were basically one bug in your code and two queuing theory misconceptions:
Bug 1) the definition of the servers were inside the class, this makes the model behaves as a M/M/inf not M/M/1
Answer: I put the definition of your resources out the the class, and pass the servers not the num_servers from now on.
Misconception 1: with the times as you defined:
mean_service = 2.0
mean_arrival = 1.0
The system will generate much more packets and it is able to serve. That's why the size of the lists were so different.
Answer:
mean_service = 1.0
mean_arrival = 2.0
Misconception 2:
What you call service time in your code is actually system time.
I also put some prints in your code so we could see that is doing. Fell free to comment them. And there is no need for the library Statistics, so I commented it too.
I hope this answer is useful to you.
# make a markovian queue
# make a server as a resource
# make customers at random times
# record the customer arrival time
# customer gets the resource
# record when the customer got the resource
# serve the customers for a random time using resource
# save this random time as service time
# customer yields the resource and next is served
#import statistics
import simpy
import random
arrivals_time = []
service_time = []
waiting_time = []
mean_service = 1.0
mean_arrival = 2.0
num_servers = 1
class Markovian(object):
def __init__(self, env, servers):
self.env = env
#self.action = env.process(self.run())
#def server(self,packet ):
#timeout after random service time
# t = random.expovariate(1.0/mean_service)
#service_time.append(t)
# yield self.env.timeout(t)
def getting_service(env, packet, servers):
# new packet arrives in the system
begin_wait = env.now
req = servers.request()
yield req
begin_service = env.now
waiting_time.append(begin_service - begin_wait)
print('%.1f Begin Service of packet %d' % (begin_service, packet))
yield env.timeout(random.expovariate(1.0/mean_service))
service_time.append(env.now - begin_service)
yield servers.release(req)
print('%.1f End Service of packet %d' % (env.now, packet))
def run_markovian(env,servers):
markovian = Markovian(env,servers)
packet = 0
#generate new packets
while True:
t = random.expovariate(1.0/mean_arrival)
yield env.timeout(t)
arrivals_time.append(t)
packet +=1
print('%.1f Arrival of packet %d' % (env.now, packet))
env.process(Markovian.getting_service(env,packet,servers))
def get_average_service_time(service_time):
average_service_time = statistics.mean(service_time)
return average_service_time
def main():
random.seed(42)
env= simpy.Environment()
servers = simpy.Resource(env, num_servers)
env.process(Markovian.run_markovian(env,servers))
env.run(until = 50)
print(Markovian.get_average_service_time(service_time))
print ("Time between consecutive arrivals \n", arrivals_time)
print("Size: ", len(arrivals_time))
print ("Service Times \n", service_time)
print("Size: ", len(service_time))
print ("Waiting Times \n", service_time)
print (waiting_time)
print("Size: ",len(waiting_time))
if __name__ == "__main__":
main()

How can I take my captured information from a network/port scan and write that to a file?

I wrote an IP and port scanning program and I want to take the captured data and output it to a text file. I've been trying to figure it out for a while and haven't had any luck applying what I can find in searches. At the end, I commented out how I thought it should work to write the information to a file.
Any help or suggestions would be greatly appreciated, I'm still somewhat new to Python and trying to learn.
#!/usr/bin/env python
import ipaddress
import sys, time
import os
import subprocess
import socket
from datetime import datetime
FNULL = open(os.devnull, 'w')
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
print ('Welcome to the IP/Port Scanner and Logger')
address = input('Enter starting IP address: ')
split1 = first,second,third,fourth = str(address).split('.')
start = int(fourth)
host = first+'.'+second+'.'+third+'.'+str(start)
end_address = input('Enter the ending IP address: ')
split2 = first,second,third,fourth = str(end_address).split('.')
end = int(fourth)
network = first+'.'+second+'.'+third+'.'
min_port = input("Enter starting port range: ")
max_port = input("Enter ending port range: ")
remoteserver = host
remoteserverIP = socket.gethostbyname(remoteserver)
def port_scan():
print ('Port scanning function initialized:')
try:
for port in range(int(min_port),int(max_port)):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
result = sock.connect_ex((remoteserverIP, port))
if result == 0:
print ('Port ' + str(port) + ': Open')
sock.close()
except KeyboardInterrupt:
print ("You halted the process")
sys.exit()
except socket.gaierror:
print ('Hostname could not be resolved. Exiting')
sys.exit()
except socket.error:
print ("Couldn't connect to server")
sys.exit()
return port
def check_up():
for ip_address in range(int(start), int(end)):
try:
subprocess.check_call(['ping', '-c', '2',
network + str(ip_address)],
stdout=FNULL,stderr=FNULL)
except (OSError, subprocess.CalledProcessError):
print ("{}{}".format(network,ip_address), "is down")
except KeyboardInterrupt:
print ("You halted the process")
sys.exit()
else:
print ("{}{}".format(network,ip_address), "is up")
return ip_address
check_up()
time1 = datetime.now()
time2 = datetime.now()
scantime = time2-time1
print ('Scan completed in: ', scantime)
while True:
print ('Would you like to write information to file?')
answer = input()
if answer in ['yes', 'y', 'yeah']:
print ('Alright, writing to file')
print ('Program will exit upon scan completion.')
break
elif answer in ['no', 'n']:
print ('Okay, exiting now..')
sys.exit()
break
else:
print ('Please enter a yes or no value')
###Output File
##with open('ipscan.txt', 'w+') as ip:
## print (ip_address, port)
##
##sys.exit()

Need help in Python3

I have the below code that i am trying the run using python3.2 interpreter.
import socket #for sockets
import sys #for exit
#from UserString import MutableString
#create an INET, STREAMing socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print ('Failed to create socket')
sys.exit()
print ('Socket Created')
host = 'www.google.com';
port = 80;
try:
remote_ip = socket.gethostbyname( host )
except socket.gaierror:
#could not resolve
print ('Hostname could not be resolved. Exiting')
sys.exit()
#Connect to remote server
s.connect((remote_ip , port))
print ('Socket Connected to ' + host + ' on ip ' + remote_ip)
#Send some data to remote server
message = "GET / HTTP/1.0\r\n\r\n"
try :
#Set the whole string
s.sendall(message.encode())
except socket.error:
#Send failed
print ('Send failed')
sys.exit()
print ('Message send successfully')
#Now receive data
messageParts = []
remaining = 4096
chunk = s.recv(remaining)
messageParts.append(chunk)
while (len(chunk) > 0):
chunk = s.recv(remaining)
messageParts.append(chunk.decode())
finalMessage = b"".join(messageParts)
print('Printing the html contents ...')
print(finalMessage)
Upon running the above code, with python version 3.2, i get the below error:
Socket Created
Socket Connected to www.google.com on ip 74.125.201.147
Message send successfully
Traceback (most recent call last):
File "TestMainServerV2.py", line 73, in <module>
finalMessage = b"".join(messageParts)
TypeError: sequence item 1: expected bytes, str found
Could anybody let me know what is the issue?
Thanks!
messageParts.append(chunk.decode())
is appending strs to messageParts.
chunk = s.recv(remaining)
messageParts.append(chunk)
is appending bytes to messageParts. (Everything that comes through a socket is bytes so chunk is a bytes object.)
Especially in Python3, one should never mix strs and bytes.
b"".join(messageParts)
raises a TypeError since b"".join expects only bytes.
You can avoid the problem by not decoding chunk. Use
messageParts.append(chunk)
instead of
messageParts.append(chunk.decode())

Python 3: Server socket not closing properly, infinite loop of blank messages received

EDIT: I solved this question myself, look at the first answer if you have a similar issue
I am new to sockets and made a client-server test modifying one example, it sends and receives data ok but when closing the server get caught in a loop receiving blank messages and I can't find out why even though I use shutdown(socket.SHUT_RDWR) to close the connection immediately according to the documentation.
I had to use two cmd of windows for each one (server and client) because when running first the server in idle and then the client I got in client_example ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it here is the code. Anyways, here it is the sequence I tried:
(in the client console):
c:\python33\python client_example.py
SEND( TYPE q or Q to Quit):Hi
SEND( TYPE q or Q to Quit):q
(in the server console):
c:\python33\python server_example3.py
TCPServer Waiting for client on port 7000
I got a connection from ('127.0.0.1', 49263)
RECEIVED: b'Hi'
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
RECEIVED: b''
.... <---After I did CTRL+C
Traceback (most recent call last):
File "server_example.py", line 19, in <module>
print("RECEIVED:",data)
File "c:\python33\lib\encodings\cp850.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
KeyboardInterrupt
Here is the code of the server and client:
server_example.py:
#TCP server example
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("", 7000))
server_socket.listen(5)
print("TCPServer Waiting for client on port 7000")
while 1:
client_socket, address = server_socket.accept()
print("I got a connection from ", address)
while 1:
data = client_socket.recv(32)
if (data == 'q' or data == 'Q'):
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
else:
print("RECEIVED:",data)
client_example.py:
# TCP client example
import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("localhost", 7000))
while 1:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data != 'Q' and data != 'q'):
my_bytes = bytearray()
for c in data:
my_bytes.append(ord(c))
client_socket.send(my_bytes)
else:
client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
Well, I feel dumb that I solved this thing after 11 minutes of posting the question when I was unable to solve it for 2 hours...
The problem was that I was comparing data with the string 'q' which was correct in python 2 but in python 3 what is being sent are bytes, b'q' <> 'q' ...
Here is the corrected code:
server_example.py
#TCP server example
import socket
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.bind(("", 7000))
server_socket.listen(5)
print("TCPServer Waiting for client on port 7000")
while 1:
client_socket, address = server_socket.accept()
print("I got a connection from ", address)
while 1:
data = client_socket.recv(32)
print("RECEIVED:",data)
if (data == b'q' or data == b'Q'):
## client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
break;
client_example.py
# TCP client example
import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(("localhost", 7000))
while 1:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data != 'Q' and data != 'q'):
my_bytes = bytearray()
for c in data:
my_bytes.append(ord(c))
client_socket.send(my_bytes)
else:
my_bytes = bytearray()
my_bytes.append(ord(data))
client_socket.send(my_bytes)
## client_socket.shutdown(socket.SHUT_RDWR)
client_socket.close()
break;
You're assuming you're going to get the Q. You should also engage in the same processing when you get EOS from read, whatever form that takes in Python.
#EJP : By EOS you mean end of stream? If so you are right, this example was a very simple one to get started in which I overlooked details like these.
Anyway I found that using this check in the server it is solved:
data = input( "SEND( TYPE q or Q to Quit):" )
if (data == b''):
break;