pcap2har for a pcap that was generated by tshark(-w option) - pcap

My pcap file is generated via a command like:
cmd = """tshark -r "%s" -R "frame.time_relative >= %f" -w "%s" """ % (pcap_name, first_dns_query_time, normalized_pcap_name)
subprocess.Popen(cmd)
And that normalized pcap is given input to pcap2har. I get this error:
Traceback (most recent call last):
File "main.py", line 65, in <module>
dispatcher = pcap.EasyParsePcap(filename=inputfile)
File "/path/to/pcap2har/pcap2har/pcap.py", line 80, in EasyParsePcap
ParsePcap(dispatcher, filename=filename, reader=reader)
File "/path/to/pcap2har/pcap2har/pcap.py", line 27, in ParsePcap
pcap = ModifiedReader(f)
File "/path/to/pcap2har/pcap2har/pcaputil.py", line 105, in __init__
raise ValueError, 'invalid tcpdump header'
ValueError: invalid tcpdump header
The portion of pcaputil.py that throws the error is:
elif self.__fh.magic != dpkt.pcap.TCPDUMP_MAGIC:
raise ValueError, 'invalid tcpdump header'
For my pcap(and for any pcap generated by the tshark command), self.__fh.magic is 168627466 and dpkt.pcap.TCPDUMP_MAGIC is 2712847316.
I commented the line that throws exception in pcaputil.py but after that I get this:
Traceback (most recent call last):
File "main.py", line 65, in <module>
dispatcher = pcap.EasyParsePcap(filename=inputfile)
File "/path/to/pcap2har/pcap2har/pcap.py", line 80, in EasyParsePcap
ParsePcap(dispatcher, filename=filename, reader=reader)
File "/path/to/pcap2har/pcap2har/pcap.py", line 27, in ParsePcap
pcap = ModifiedReader(f)
File "/path/to/pcap2har/pcap2har/pcaputil.py", line 108, in __init__
self.dloff = dpkt.pcap.dltoff[self.__fh.linktype]
KeyError: 4294967295L
I have already submitted the issue on github

As of Wireshark 1.8, the default output file format is pcap-ng, not pcap. If pcap2har had used one of the Python wrappers for libpcap, and you were running on a system with libpcap 1.0 or later (which also means "not running on Windows", as there's no version of WinPcap based on libpcap 1.0 or later), it would automatically be able to read many pcap-ng files, as libpcap can read them, but it's probably using its own code to read libpcap files.
Try running tshark with "-F pcap" to get it to generate a pcap file.

You can use editcap to change the format of a ".pcapng" file:
editcap teste.pcapng teste.pcap -F pcap

Related

Overwrite anaconda3 unsuccessful installation

I deleted the anaconda directory under the home and bashrc configurations.
Now, I need to install it again, but it occurs a problem evenif overwrites unsuccessful installation on Linux.
Should I delete some additional config files? How can I handle this?
sh Downloads/Anaconda3-2022.10-Linux-x86_64.sh -u -p /home/user/anaconda3/
PREFIX=/home/user/anaconda3
Unpacking payload ...
concurrent.futures.process._RemoteTraceback:
'''
Traceback (most recent call last):
File "concurrent/futures/process.py", line 384, in wait_result_broken_or_wakeup
File "multiprocessing/connection.py", line 256, in recv
TypeError: __init__() missing 1 required positional argument: 'msg'
'''
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "entry_point.py", line 69, in <module>
File "concurrent/futures/process.py", line 559, in _chain_from_iterable_of_lists
File "concurrent/futures/_base.py", line 608, in result_iterator
File "concurrent/futures/_base.py", line 445, in the result
File "concurrent/futures/_base.py", line 390, in __get_result
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
[8382] Failed to execute script entry_point
Make sure deleted .conda directory under home and have enough disk space.
No need to delete .cache or any bin libraries.

Running the Tensorflow2 upgrade Script triggers an encoding error

I am currently trying to upgrade a script to be compatible with tensorflow2 by using the jupyter notebook that is provided in https://www.tensorflow.org/guide/upgrade, so that I can run it in google colab, however when I run this cell (from an Anaconda environment in windows):
!tf_upgrade_v2 \
--infile D:/Multiclass.py\
--outfile /tmp/upgraded_Multiclass_CNN_V01_31012020Copy.py
I get an encoding error:
Traceback (most recent call last):
File "d:\programdata\miniconda3\envs\env_dlexp1\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "d:\programdata\miniconda3\envs\env_dlexp1\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\ProgramData\Miniconda3\envs\Env_DLexp1\Scripts\tf_upgrade_v2.exe\__main__.py", line 7, in <module>
File "C:\Users\user\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\tools\compatibility\tf_upgrade_v2_main.py", line 139, in main
args.input_file, output_file, upgrade)
File "C:\Users\user\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\tools\compatibility\tf_upgrade_v2_main.py", line 40, in process_file
upgrader.process_file(in_filename, out_filename)
File "C:\Users\user\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\tools\compatibility\ast_edits.py", line 900, in process_file
temp_file)
File "C:\Users\user\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\tools\compatibility\ast_edits.py", line 958, in process_opened_file
lines = in_file.readlines()
File "d:\programdata\miniconda3\envs\env_dlexp1\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 14: character maps to <undefined>
I would normally change the encoding to utf-8 by doing this: open(path, mode, encoding='UTF8')
,but in this case I am clueless. I am not a programmer, so my proficiency is very limited. Please help me understand this error.
Try editing with open in line 918 of function process_file in ast_edits.py and specify encoding='utf-8', so that in the function process_opened_file the file can be read with utf-8.

Librabbitmq 2.0.0 with Python 3 gives TypeError: can't pickle memoryview objects

I am using the latest master branch of the git repo https://github.com/celery/librabbitmq and installing librabbitmq==2.0.0 for Python 3.6 by following the instructions in the readme
Using the development version
You can clone the repository by doing the following:
$ git clone git://github.com/celery/librabbitmq.git
Then install it by doing the following:
$ cd librabbitmq
$ make install # or make develop
This works fine (after installing certain binaries for c compliation in the OS), but when I then make a small a+b add task and call it with add.delay(2,2) it fails with the following error. I looked up and saw that Celery 4 uses json as serializer, so clearly it is not because if pickle serialization
Changing from librabbitmq to pyamqp broker works normally
Same exact situation in both MacOS and Ubuntu 16
[2018-04-30 23:40:02,956: CRITICAL/MainProcess] Unrecoverable error:
SystemError(' returned a result with an error set',) Traceback (most
recent call last): File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/messaging.py",
line 624, in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message) File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 570, in on_task_received
callbacks, File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/strategy.py",
line 145, in task_message_handler
handle(req) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 221, in _process_task_sem
return self._quick_acquire(self._process_task, req) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/async/semaphore.py",
line 62, in acquire
callback(*partial_args, **partial_kwargs) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 226, in _process_task
req.execute_using_pool(self.pool) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/request.py",
line 531, in execute_using_pool
correlation_id=task_id, File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/concurrency/base.py",
line 155, in apply_async
**options) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/billiard/pool.py",
line 1486, in apply_async
self._quick_put((TASK, (result._job, None, func, args, kwds))) File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/concurrency/asynpool.py",
line 813, in send_job
body = dumps(tup, protocol=protocol) TypeError: can't pickle memoryview objects
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/worker.py",
line 203, in start
self.blueprint.start(self) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 119, in start
step.start(parent) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 370, in start
return self.obj.start() File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 320, in start
blueprint.start(self) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/bootsteps.py",
line 119, in start
step.start(parent) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 596, in start
c.loop(*c.loop_args()) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/celery/worker/loops.py",
line 88, in asynloop
next(loop) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/async/hub.py",
line 354, in create_loop
cb(*cbargs) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/transport/base.py",
line 236, in on_readable
reader(loop) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/kombu/transport/base.py",
line 218, in _read
drain_events(timeout=0) File "/Users/somghosh/.virtualenvs/ctdb/lib/python3.6/site-packages/librabbitmq-2.0.0-py3.6-macosx-10.6-intel.egg/librabbitmq/init.py",
line 227, in drain_events
self._basic_recv(timeout) SystemError: returned a result with an error set
This library is not recommended to use as rabbitmq broker with celery. Instead please try py-amqp. this is more maintained and less buggy.

No handlers could be found for logger "mongo_connector.util"

I always got the error below when I try to run mongo-connector with neo4j doc manager. I also tried with a config file as in https://github.com/mongodb-labs/mongo-connector/blob/master/config.json
Where is the problem?
mongo-connector -m localhost:27017 -t http://localhost:7474/db/data -d neo4j_doc_manager
No handlers could be found for logger "mongo_connector.util"
Traceback (most recent call last):
File "/usr/bin/mongo-connector", line 11, in <module>
sys.exit(main())
File "/usr/lib/python2.6/site-packages/mongo_connector/util.py", line 85, in wrapped
func(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 1041, in main
conf.parse_args()
File "/usr/lib/python2.6/site-packages/mongo_connector/config.py", line 118, in parse_args
option, dict((k, values.get(k)) for k in option.cli_names))
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 824, in apply_doc_managers
module = import_dm_by_name(dm['docManager'])
File "/usr/lib/python2.6/site-packages/mongo_connector/connector.py", line 803, in import_dm_by_name
module = __import__(full_name, fromlist=(name,))
File "/usr/lib/python2.6/site-packages/mongo_connector/doc_managers/neo4j_doc_manager.py", line 16, in <module>
from py2neo import Graph, authenticate
File "/usr/lib/python2.6/site-packages/py2neo/__init__.py", line 28, in <module>
from py2neo.database import *
File "/usr/lib/python2.6/site-packages/py2neo/database/__init__.py", line 65
parameters = {k: v for k, v in parameters.items() if k not in presub_parameters}
It looks like you are using Python 2.6. I'm not sure if that version is officially supported for this project. I would suggest upgrading to Python 2.7 or preferably 3.4 and trying to reproduce.

Doxyclean Error

I'm trying to run doxyclean but can't get it to work, any help would be appreciated...
I'm running from terminal :
./doxyclean.py --input=./xml/ --output=./clean/ --name="MyProject" --phone -v
I have my doxygen xml in the folder xml, in the same directory as doxyclean.py
The result is :
Checking arguments
Cleaning XML files:
Traceback (most recent call last):
File "./doxyclean.py", line 1220, in <module>
sys.exit(main())
File "./doxyclean.py", line 1171, in main
cleanXML(filePath, xmlOutputDirectory)
File "./doxyclean.py", line 93, in cleanXML
if not fileIsDocumented(filePath):
File "./doxyclean.py", line 62, in fileIsDocumented
originaldoc = minidom.parse(filePath)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xml/dom/minidom.py", line 1918, in parse
return expatbuilder.parse(file)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xml/dom/expatbuilder.py", line 924, in parse
result = builder.parseFile(fp)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/xml/dom/expatbuilder.py", line 207, in parseFile
parser.Parse(buffer, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 17, column 155
thanks
did you try other ways, my suggestions
use different version of doxyclean
change the arguments to different style, maybe give full paths to the folders.
./doxyclean.py -i Users/xxx/doxyclean/xml/ -o ./clean/ -p
regenerate xml from doxygen
doxyclean works fine. I got this error too and I tried my suggestion 2 and it worked.