Using private mibs with snmpv3 - pysnmp

I am using below mentioned example to send v3 trap with custom notification mib
http://pysnmp.sourceforge.net/examples/current/v3arch/agent/ntforg/trap-v3.html
But the sendNotification function is accepting only the dotted format as in the example ..
What should I do to pass the symbols instead ?
E.g.(' MY-MIB:testTrap') instead of (1,3,2,0....)

You are linking a historical pysnmp documentation. Please, refer to the actual one.
To pass MIB symbols you should use ObjectIdentifier class which turns MIB names into OIDs behind the scenes.
But SNMP notifications can be quite complicated because they may imply certain other MIB objects to be automatically included with the notification being sent.
But you can still add whatever MIB objects you want through NotificationType.addVarBinds.
Here's a simple example:
from pysnmp.hlapi import *
errorIndication, errorStatus, errorIndex, varBinds = next(
sendNotification(
SnmpEngine(),
CommunityData('public'),
UdpTransportTarget(('demo.snmplabs.com', 162)),
ContextData(),
'trap',
NotificationType(
ObjectIdentity('MY-MIB', 'testTrap')
)
)
)

"Hi again below is a snippet of the way I am using the code . "
from pysnmp.entity import engine, config
from pysnmp.carrier.asynsock.dgram import udp
from pysnmp.entity.rfc3413 import ntforg, context
from pysnmp.proto.api import v2c
from pysnmp.proto import rfc1902
from pysnmp.smi import builder
file = open("."+"/file.txt","r")
line = file.read()
fields = line.split(",")
file.close()
file = open("."+"/credentials.txt","r")
cred = file.read()
fields = cred.split(",")
file.close()
snmpEngine = engine.SnmpEngine(snmpEngineID=v2c.OctetString(hexValue=cred[0]))
config.addV3User(snmpEngine, cred[1], config.usmHMACSHAAuthProtocol, cred[2], config.usmAesCfb128Protocol, cred[3])
config.addTargetParams(snmpEngine, 'my-creds', 'traptest', 'authPriv')
config.addSocketTransport(snmpEngine, udp.domainName, udp.UdpSocketTransport().openClientMode())
config.addTargetAddr(snmpEngine, 'my-nms', udp.domainName, ('127.0.0.1', 162), 'my-creds', tagList='all-my-managers')
config.addNotificationTarget(snmpEngine, 'my-notification', 'my-filter', 'all-my-managers', 'trap')
config.addContext(snmpEngine, '')
config.addVacmUser(snmpEngine, 3, 'traptest', 'authPriv', (), (), (1,3,6))
snmpContext = context.SnmpContext(snmpEngine)
ntfOrg = ntforg.NotificationOriginator()
ntfOrg.snmpContext = snmpContext
ntfOrg.sendNotification(
snmpEngine,
'my-notification',
(1,3,6,1,4,1,46033,1,1,2,2),
(((1,3,6,1,4,1,46033,1,1,1,1), v2c.OctetString(fields[0])),)
)
print('Notification is scheduled to be sent')

Related

Using input function with remote files in snakemake

I want to use a function to read inputs file paths from a dataframe and send them to my snakemake rule. I also have a helper function to select the remote from which to pull the files.
from snakemake.remote.GS import RemoteProvider as GSRemoteProvider
from snakemake.remote.SFTP import RemoteProvider as SFTPRemoteProvider
from os.path import join
import pandas as pd
configfile: "config.yaml"
units = pd.read_csv(config["units"]).set_index(["library", "unit"], drop=False)
TMP= join('data', 'tmp')
def access_remote(local_path):
""" Connnects to remote as defined in config file"""
provider = config['provider']
if provider == 'GS':
GS = GSRemoteProvider()
remote_path = GS.remote(join("gs://" + config['bucket'], local_path))
elif provider == 'SFTP':
SFTP = SFTPRemoteProvider(
username=config['user'],
private_key=config['ssh_key']
)
remote_path = SFTP.remote(
config['host'] + ":22" + join(base_path, local_path)
)
else:
remote_path = local_path
return remote_path
def get_fastqs(wc):
"""
Get fastq files (units) of a particular library - sample
combination from the unit sheet.
"""
fqs = units.loc[
(units.library == wc.library) &
(units.libtype == wc.libtype),
"fq1"
]
return {
"r1": list(map(access_remote, fqs.fq1.values)),
}
# Combine all fastq files from the same sample / library type combination
rule combine_units:
input: unpack(get_fastqs)
output:
r1 = join(TMP, "reads", "{library}_{libtype}.end1.fq.gz")
threads: 12
run:
shell("cat {i1} > {o1}".format(i1=input['r1'], o1=output['r1']))
My config file contains the bucket name and provider, which are passed to the function. This works as expected when running simply snakemake.
However, I would like to use the kubernetes integration, which requires passing the provider and bucket name in the command line. But when I run:
snakemake -n --kubernetes --default-remote-provider GS --default-remote-prefix bucket-name
I get this error:
ERROR :: MissingInputException in line 19 of Snakefile:
Missing input files for rule combine_units:
bucket-name/['bucket-name/lib1-unit1.end1.fastq.gz', 'bucket-name/lib1-unit2.end1.fastq.gz', 'bucket-name/lib1-unit3.end1.fastq.gz']
The bucket is applied twice (once mapped correctly to each element, and once before the whole list (which gets converted to a string). Did I miss something ? Is there a good way to work around this ?

An error in my code to be a simple ftp

I met an error when running codes at the bottom. It's like a simple ftp.
I use python2.6.6 and CentOS release 6.8
In most linux server, it gets right results like this:(I'm very sorry that I have just sign up and couldn't )
Clinet:
[root#Test ftp]# python client.py
path:put|/home/aaa.txt
Server:
[root#Test ftp]# python server.py
connected...
pre_data:put|aaa.txt|4
cmd: put
file_name: aaa.txt
file_size: 4
upload successed.
But I get errors in some server(such as my own VM in my PC). I have done lots of tests(python2.6/python2.7, Centos6.5/Centos6.7) and found this error is not because them. Here is the error imformation:
[root#Lewis-VM ftp]# python server.py
connected...
pre_data:put|aaa.txt|7sdfsdf ###Here gets the wrong result, "sdfsdf" is the content of /home/aaa.txt and it shouldn't be sent here to 'file_size' and so it cause the "ValueError" below
cmd: put
file_name: aaa.txt
file_size: 7sdfsdf
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 10699)
Traceback (most recent call last):
File "/usr/lib64/python2.6/SocketServer.py", line 570, in process_request_thread
self.finish_request(request, client_address)
File "/usr/lib64/python2.6/SocketServer.py", line 332, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/lib64/python2.6/SocketServer.py", line 627, in __init__
self.handle()
File "server.py", line 30, in handle
if int(file_size)>recv_size:
ValueError: invalid literal for int() with base 10: '7sdfsdf\n'
What's more, I found that if I insert a time.sleep(1) between sk.send(cmd+"|"+file_name+'|'+str(file_size)) and sk.send(data) in client.py, the error will disappear. I have said that I did tests in different system and python versions and the error is not because them. So I guess that is it because of some system configs? I have check about socket.send() and socket.recv() in python.org but fount nothing helpful. So could somebody help me to explain why this happend?
The code are here:
#!/usr/bin/env python
#coding:utf-8
################
#This is server#
################
import SocketServer
import os
class MyServer(SocketServer.BaseRequestHandler):
def handle(self):
base_path = '/home/ftp/file'
conn = self.request
print 'connected...'
while True:
#####receive pre_data: we should get data like 'put|/home/aaa|7'
pre_data = conn.recv(1024)
print 'pre_data:' + pre_data
cmd,file_name,file_size = pre_data.split('|')
print 'cmd: ' + cmd
print 'file_name: '+ file_name
print 'file_size: '+ file_size
recv_size = 0
file_dir = os.path.join(base_path,file_name)
f = file(file_dir,'wb')
Flag = True
####receive 1024bytes each time
while Flag:
if int(file_size)>recv_size:
data = conn.recv(1024)
recv_size+=len(data)
else:
recv_size = 0
Flag = False
continue
f.write(data)
print 'upload successed.'
f.close()
instance = SocketServer.ThreadingTCPServer(('127.0.0.1',9999),MyServer)
instance.serve_forever()
#!/usr/bin/env python
#coding:utf-8
################
#This is client#
################
import socket
import sys
import os
ip_port = ('127.0.0.1',9999)
sk = socket.socket()
sk.connect(ip_port)
while True:
input = raw_input('path:')
#####we should input like 'put|/home/aaa.txt'
cmd,path = input.split('|')
file_name = os.path.basename(path)
file_size=os.stat(path).st_size
sk.send(cmd+"|"+file_name+'|'+str(file_size))
send_size = 0
f= file(path,'rb')
Flag = True
#####read 1024 bytes and send it to server each time
while Flag:
if send_size + 1024 >file_size:
data = f.read(file_size-send_size)
Flag = False
else:
data = f.read(1024)
send_size+=1024
sk.send(data)
f.close()
sk.close()
The TCP is a stream of data. That is the problem. TCP do not need to keep message boundaries. So when a client calls something like
connection.send("0123456789")
connection.send("ABCDEFGHIJ")
then a naive server like
while True;
data = conn.recv(1024)
print data + "_"
may print any of:
0123456789_ABCDEFGHIJ_
0123456789ABCDEFGHIJ_
0_1_2_3_4_5_6_7_8_9_A_B_C_D_E_F_G_H_I_J_
The server has no chance to recognize how many sends client called because the TCP stack at client side just inserted data to a stream and the server must be able to process the data received in different number of buffers than the client used.
Your server must contain a logic to separate the header and the data. All of application protocols based on TCP use a mechanism to identify application level boundaries. For example HTTP separates headers and body by an empty line and it informs about the body length in a separate header.
Your program works correctly when server receives a header with the command, name and size in a separate buffer it it fails when client is fast enough and push the data into stream quickly and the server reads header and data in one chunk.

AttributeError 'IdLookup' object has no attribute 'rel'

I try to use the django REST-framework's tutorial http://django-rest-framework.org/#django-rest-framework to administrate users. (I also use the Neo4j database and the neo4django mapper https://github.com/scholrly/neo4django to acces data via python.)
Whatever, wen I call localhost:8000/users an AttributeError appears.
models.py
from django.utils import timezone
from django.conf import settings
from django.contrib.auth import models as django_auth_models
from ..db import models
from ..db.models.manager import NodeModelManager
from ..decorators import borrows_methods
class UserManager(NodeModelManager, django_auth_models.UserManager):
pass
# all non-overriden methods of DjangoUser are called this way instead.
# inheritance would be preferred, but isn't an option because of conflicting
# metaclasses and weird class side-effects
USER_PASSTHROUGH_METHODS = (
"__unicode__", "natural_key", "get_absolute_url",
"is_anonymous", "is_authenticated", "get_full_name", "set_password",
"check_password", "set_unusable_password", "has_usable_password",
"get_group_permissions", "get_all_permissions", "has_perm", "has_perms",
"has_module_perms", "email_user", 'get_profile','get_username')
#borrows_methods(django_auth_models.User, USER_PASSTHROUGH_METHODS)
class User(models.NodeModel):
objects = UserManager()
username = models.StringProperty(indexed=True, unique=True)
first_name = models.StringProperty()
last_name = models.StringProperty()
email = models.EmailProperty(indexed=True)
password = models.StringProperty()
is_staff = models.BooleanProperty(default=False)
is_active = models.BooleanProperty(default=False)
is_superuser = models.BooleanProperty(default=False)
last_login = models.DateTimeProperty(default=timezone.now())
date_joined = models.DateTimeProperty(default=timezone.now())
USERNAME_FIELD = 'username'
REQUIRED_FIELDS=['email']
serializers.py
from neo4django.graph_auth.models import User
from rest_framework import serializers
class UserSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = User
fields = ('url', 'username', 'email')
views.py
from neo4django.graph_auth.models import User
from rest_framework import viewsets
from api.serializers import UserSerializer
class UserViewSet(viewsets.ModelViewSet):
"""
API endpoint that allows users to be viewed or edited.
"""
queryset = User.objects.all()
serializer_class = UserSerializer
I get this:
Environment:
Request Method: GET
Request URL: http://localhost:8000/users/
Django Version: 1.5.3
Python Version: 2.7.3
Installed Applications:
('core.models',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'neo4django.graph_auth',
'rest_framework')
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')
Traceback:
File "/opt/phaidra/env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
115. response = callback(request, *callback_args, **callback_kwargs)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/viewsets.py" in view
78. return self.dispatch(request, *args, **kwargs)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
77. return view_func(*args, **kwargs)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
399. response = self.handle_exception(exc)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
396. response = handler(request, *args, **kwargs)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/mixins.py" in list
92. serializer = self.get_pagination_serializer(page)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/generics.py" in get_pagination_serializer
113. return pagination_serializer_class(instance=page, context=context)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/pagination.py" in __init__
85. self.fields[results_field] = object_serializer(source='object_list', **context_kwarg)
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/serializers.py" in __init__
162. self.fields = self.get_fields()
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/serializers.py" in get_fields
198. default_fields = self.get_default_fields()
File "/opt/phaidra/env/local/lib/python2.7/site-packages/rest_framework/serializers.py" in get_default_fields
599. while pk_field.rel and pk_field.rel.parent_link:
Exception Type: AttributeError at /users/
Exception Value: 'IdLookup' object has no attribute 'rel'
I am rather new on the Python/Django/REST service area. I hope someone could help. Thanks in advance.
Firstly I would recommend to either use Tastypie - which is out of the box supported by neo4django - instead of Django Rest Framework - or use Django Rest Framework Serializer instead of the ModelSerializer or (worst choice in my opinion) HyperlinkedModelSerializer.
As for the error you get, the problem is that neo4django does not return the record id, therefore you get the error.
One solution is to override the restore_object function, like this, to include the ID.
# User Serializer
class UserSerializer(serializers.Serializer):
id = serializers.IntegerField()
username = serializers.CharField(max_length=30)
first_name = serializers.CharField(max_length=30)
last_name = serializers.CharField(max_length=30)
email = serializers.EmailField()
password = serializers.CharField(max_length=128)
is_staff = serializers.BooleanField()
is_active = serializers.BooleanField()
is_superuser = serializers.BooleanField()
last_login = serializers.DateTimeField()
date_joined = serializers.DateTimeField()
def restore_object(self, attrs, instance=None):
"""
Given a dictionary of deserialized field values, either update
an existing model instance, or create a new model instance.
"""
if instance is not None:
instance.id = attrs.get('ID', instance.pk)
instance.username = attrs.get('Username', instance.username)
instance.first_name = attrs.get('First Name', instance.first_name)
instance.last_name = attrs.get('First Name', instance.last_name)
instance.email = attrs.get('email', instance.email)
instance.password = attrs.get('Password', instance.password)
instance.is_staff = attrs.get('Staff', instance.is_staff)
instance.is_active = attrs.get('Active', instance.is_active)
instance.is_superuser = attrs.get('Superusers', instance.is_superuser)
instance.last_login = attrs.get('Last Seen', instance.last_login)
instance.date_joined = attrs.get('Joined', instance.date_joined)
return instance
return User(**attrs)
But I still think it's better to use Tastypie with ModelResource.
Here's a gist by Matt Luongo (or Lukas Martini ?) https://gist.github.com/mhluongo/5789513
TIP. Where Serializer in Django Rest Framework, is Resource in Tastypie and always make sure you are using the github version of neo4django (pip install -e git+https://github.com/scholrly/neo4django/#egg=neo4django)

Apple store receipt validation through Twisted server

I'm trying to validate a transaction receipt from an inApp purchase with the Apple store server from my Twisted server. I have sent the (SKPaymentTransaction *)transaction.transactionReceipt from my app to my server.
But now, sending the JSON object to the Apple server, I keep getting an unhandled error in Deferred from my Agent.request(). I suspect this is because I'm not listening on port 443 for response from Apple store, but I don't want my app to communicate with my Twisted server on port 443 also. Here is my code:
from twisted.application import internet, service
from twisted.internet import protocol, reactor
from zope.interface import implements
from twisted.web.iweb import IBodyProducer
from twisted.internet import defer
from twisted.web.client import Agent
from twisted.web.http_headers import Headers
import json
import base64
class StringProducer(object):
implements(IBodyProducer)
def __init__(self, body):
self.body = body
self.length = len(body)
def startProducing(self, consumer):
consumer.write(self.body)
return succeed(None)
def pauseProducing(self):
pass
def stopProducing(self):
pass
def printResponse(response):
print response # just testing to see what I have
def httpRequest(url, values, headers={}, method='POST'):
agent = Agent(reactor)
d = agent.request(method,
url,
Headers(headers),
StringProducer(values)
)
d.addCallback(printResponse)
class storeServer(protocol.Protocol):
def dataReceived(self, data):
receiptBase64 = base64.standard_b64encode(data)
jsonReceipt = json.dumps({'receipt-data':receiptBase64})
print jsonReceipt # verified that my data is correct
d = httpRequest(
"https://buy.itunes.apple.com/verifyReceipt",
jsonReceipt,
{'Content-Type': ['application/x-www-form-urlencoded']}
)
factory = protocol.Factory()
factory.protocol = storeServer
tcpServer = internet.TCPServer(30000, factory)
tcpServer.setServiceParent(application)
How can I fix this error? Do I have to create another service listening on port 443? If so, how might I have the service connecting to my app communicate with the service connecting through https?
The comment style in your code sample is incorrect. Python uses # for comments, not //.
After fixing that and running the snippet through pyflakes, I see these errors:
program.py:1: 'service' imported but unused
program.py:6: 'defer' imported but unused
program.py:21: undefined name 'succeed'
program.py:48: local variable 'd' is assigned to but never used
program.py:57: undefined name 'application'
It seems likely that the undefined name on line 21 is the cause of the NameError you've encountered. NameError is how Python signals this sort of bug:
x = y
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined

web automation - auto check link

I'm new to web app and I want to check when there's a new version of dota map, I'll check links in getdota.com.
How can I do this and which language, I want it checks every time you start warcraft, and auto download new map to specific folder.
My question is : Can you give a link to a specific article about web automation or something like that.
Thanks first :)
Below is an example in Python.
It parses getdota.com page, reads parameters for POST request for downloading a map, gets the file and saves it in configured directory (by default current directory).
#!/usr/bin/env python
import urllib
import urllib2
import sgmllib
from pprint import pprint
import os.path
import sys
url = 'http://www.getdota.com/'
download_url = 'http://www.getdota.com/app/getmap/'
chunk = 10000
directory = '' #directory where file should be saved, if empty uses current dir
class DotaParser(sgmllib.SGMLParser):
def parse(self, s):
self.feed(s)
self.close()
def __init__(self, verbose=0):
sgmllib.SGMLParser.__init__(self, verbose)
self.URL = ''
self.post_args = {}
def getArgs(self):
return self.post_args
def start_input(self, attributes):
d = dict(attributes)
if d.get('id', None) == None:
return
if d['id'] in ["input_mirror2", "input_file_name2", "input_map_id2", "input_language2", "input_language_id2"]:
self.post_args[d['name']] = d['value']
if __name__ == '__main__':
dotap = DotaParser()
data = urllib2.urlopen(urllib2.Request('http://www.getdota.com/')).read()
dotap.parse(data)
data = urllib.urlencode(dotap.getArgs())
request = urllib2.Request(download_url, data)
response = urllib2.urlopen(request)
page = response.read()
#download file
fname = directory + page.split('/')[-1]
if os.path.isfile(fname):
print "No newer file available"
sys.exit(0)
f = open(fname, 'w')
print "New file available. Saving in: %s" % fname
webFile = urllib.urlopen(page)
c = webFile.read(chunk)
while(c):
f.write(c)
c = webFile.read(chunk)
f.close()
webFile.close()