I am trying to inspect wireshark logs at the moment captured from an in-field system. We have some protection internally for broadcast messages on embedded devices to ensure they dont get swamped at the application layer by too many broadcasts. We suspect this protection is at the root issue of this problem we are seeing.
However, I am looking through the logs trying to figure out how many times there are more than 10 messages in a 10mS interval.
Is there anyway to automatically do this in wireshark? I have been processing manually using the filters to determine what messages are going to which destinations and which sources, but I am unsure how to have (essentially) a sliding time window to highlight the number of messages within the defined interval.
Is this something I have to do outside of Wireshark? Process the log in csv or similar?
I have tried to google sliding window/timeframe filters in wireshark, but nothing relevant comes up, only filtering tutorials in Wireshark.
I ended up using the IO Graph in the Statistics menu to do this. I was able to set the time intervals to 10mS, and apply a filter for broadcasts/messages that were applicable
Related
I have a program using a bi-directional TCP socket to send messages from the host PC to a VLinx ethernet-to-serial converter and then on to a PLC via RS-232. During heavy traffic the socket will intermittently stop communicating although all soft tests of the connection show that it is connected, active and writeable. I suspect that something is interrupting the connection causing the socket to close with out FIN/ACK. How can I test to see where this disconnect might be occuring?
The program itself is written in VB6 and uses Catalyst SocketTools/SocketWrench as opposed to the standard Winsock library. The methodology, properties and code seem to be sound since the same setup works reliably at two other sites. It's just this one site in particular where this problem occurs. It only happens during production when there is traffic on the network and can lose connection anywhere between 20 - 100 times per 10-hour day.
There are redundant tests in place to catch this loss of communication and keep the system running. We have tests on ACK messages, message queue size, time between transmissions (tokens on 2s interval), etc. Typically, the socket will not be unresponsive for more than 30 seconds before it is caught, closed and re-established which works properly >99% of the time.
Previously I had enabled the SocketTools logging capabilities which did not capture any relevant information. Most recently I have tried to have the system ping the VLinx on the first sign of a missed message (2.5 seconds). Those pings have always been successful, meaning that if there is a momentary loss of connection at a switch or AP it does not stay disconnected for long.
I do not have access to the network hardware aside from the PC and VLinx that we own. The facility's IT is also not inclined to help track these kinds of things down because they work on a project-based model.
Does anyone have any suggestions what I can do to try and determine where the problem is occurring so that I can then try to come up with a permanent solution to this issue rather than the band-aid of reconnecting multiple times per day?
A tool like Wireshark may be helpful in seeing what's going on at the network level. The logging facility in SocketTools/SocketWrench can only report what's going on at the API level, and it sounds like whatever the underlying problem is occurs at a lower level in the TCP stack.
If this is occurring after periods of relative inactivity, followed by a burst of activity, one thing you could try doing is enabling keep-alive and see if that makes any difference.
We use mirth as our interface engine and the ActiveMQ and Spring Inbound listener to process messages.
Our customers reported that some of the messages are missing from the mirth console but found in the ActiveMQ queue and the Spring listener application.
Initially we thought that someone may have removed manually from Mirth. But when checked the event logs there's no sign of removing a message.
We found this happening on some of the messages but could not identify the cause of the issue or pattern of messages.
Have anyone face and issue like with Mirth Admin console ?
We have the client DB as well but unable to open except through Mirth to check whether data is available.
Highly appreciate if someone can help on this.
Thanks
I have found some channels don't display "filtered" messages properly. But I have never seen successful messages go "missing".
If you don't trust the Mirth Admin then I would recommend querying the Mirth DB.
This can be done outside the confines of Mirth provided that Mirth is writing to an external DB such as MS-SQL Sever.
The data you get from it is VERY rich, but if you are sending 1000's of messages an hour (or more) you'll probably want to limit the time-range you search. Free-text searching like
select * from message m where m.raw_data like ('%needle%')
is NOT recommended and will take a long time to execute.
Being able search Mirth via the DB has opened up a tone of analysis for us that we don't have through the admin interface.
Just to chime on this question: if you are are running quite a number of channels or if you have quite a high volume of messages mirth may have trouble keeping up with its database updates due to row/table locks and inefficient conversions or data types (this should be resolved now).
We do, however, at peak times every so often, see a message or two processed through the engine with log entries indicating it was unable to insert the message and it was rolled back. I would say we have around 10 per year like that. Hopefully this is non-issue in Mirth 3 with the new backend ...
I am using Tigase(XMPP) server. I want to block every incoming message from a particular JID. At the moment i am blocking a particular JID by dropping it's packet in Message.java inside
/tigase/xmpp/impl
package. Is it the right way to do this, if not please guide me.
Thanks
An advantage of blocking messages in Message plugin is that the performance penalty for this filtering is reduced to minimal. However there are quite a few disadvantages of doing it this way:
You modify Tigase's code which makes you version update painful and time consuming
It does not allow you to filter out any other packets (such as presence or iq)
Even if you block messages in Message plugin this message may still be processed by other plugins which intercept messages (such as offline message, message archiver, etc...)
Now, what is the best way to implement such a filtering it depends on what you really want to do and why do you want to do it. Have you heard of privacy lists? Please take a look at it. Tigase fully implements privacy lists, why you do not want to use them? Why you cannot use them?
Usually such a filtering you speak about is done in the Tigase filter called preprocessor. Please take a look at the privacy lists plugin or domain filter for a code example.
Uploads from my app are too slow, and I'd like to gather some real data as to where the time is being spent.
By way of example, here are a few stages a request goes through:
Initial radio connection (significant source of latency in EDGE)
DNS lookup (if not cached)
SSL/TLS handshake.
HTTP request upload, including data.
Server processing time.
HTTP response download.
I can address most of these (e.g. by powering up the radio earlier via a dummy request, establishing a dummy HTTP 1.1 connection, etc.), but I'd like to know which ones are actually contributing to network slowness, on actual devices, with my actual data, using actual cell towers.
If I were using WiFi, I could track a bunch of these with Wireshark and some synchronized clocks, but I need cellular data.
Is there any good way to get this detailed breakdown, short of having to (gak!) use very low level socket functions to reproduce my vanilla http request?
Ok, the method I would use is not easy, but it does work. Maybe you're already tried this, but bear with me.
I get a time-stamped log of the sending time of each message, the time each message is received, and the time it is acted upon. If this involves multiple processes or threads, I have each one generate a log, and then merge them into a common timeline.
Then I plot out the timeline. (A tool would be nice, but I did it by hand.)
What I look for is things like 1) messages re-transmitted due to timeouts, 2) delays between the time a message is received and the time it's acted upon.
Usually, this identifies problems that I can fix in the code I can control. This improves things, but then I do it all over again, because chances are pretty good that I missed something the last time.
The result was that a system of asynchronous message-passing can be made to run quite fast, once preventable sources of delay have been eliminated.
There is a tendency in posting questions about performance to look for magic fixes to improve the situation. But, the real magic fix is to refine your diagnostic technique so it tells you what to fix, because it will be different from anyone else's.
An easy solution to this would be once the application get's fired, make a Long Polling connection with the server (you can choose when this connection need's to establish prior hand, and when to disconnect), but that is a kind of a hack if you want to avoid all the sniffing of packets with less api exposure iOS provides.
I do a lot of work in the grid and HPC space and one of the biggest challenges we have with a system distributed across hundreds (or in some case thousands) of servers is analysing the log files.
Currently log files are written locally to the disk on each blade but we could also consider publishing logging information using for example a UDP Appender and collect it centally.
Given that the objective is to be able to identify problems in as close to real time as possible, what should we do?
First, synchronize all clocks in the system using NTP.
Second, if you are collecting the logs in a single location (like the UDP appender you mention) make sure the logs have enough information to actually help. I would include at least the server that generated the log, the time it happened, and the message. If there is any sort of transaction id, or job id type concept, include that also.
Since you mentioned a UDP Appender I am guessing you are using log4j (or one of it's siblings). Log4j has an MDC class that allows extra information to be passed along through a processing thread. it can help collect some of the extra information and pass it along.
Are you using Apache? If so you could have a look at mod_log_spread Though you may have too big an infrastructure to make it maintainable. The other option is to look at "broadcasting" or "multicasting" your log messages and having dedicated logging servers subscribing to those feeds and collating them