We are using IBM MQ Series 9 and we are facing a decoding problem.
The messages are being sent from a mainframe with an encoding of 424 (Hebrew) to a Windows-based system. The system pulls the messages out of the queue and parses the messages, and after that, cuts the messages in different parts for advanced parsing.
All messages might include Hebrew characters, hence I am obligated to use Hebrew encoding.
A message in the MQ can look like this:
9921388ABC.........3323DDFF.....43332FFF...2321......
After reading the message and parsing it using different code pages the message either doesn't reach the system (using 424, 916) or reaches the system but looks like this:
9921388ABC3323DDFF43332FFF2321
The messages are shorter and are unparseable.
I ahve tried to consult with our MQ people but they are clueless about this problem.
Would very appreciate any kind of help.
Thank you.
Related
I was reading the original RPC about the SMTP Protocol and came across this section:
SMTP indicates the end of the
mail data by sending a line containing only a period.
Why did Postel decide to use the period as the terminator? Would it not be easier to use the already existing null terminator?
I see, that he would not want the users content to interfere with the protocol, but I would naively assume, that a user is more likely to use a period in one line than a null terminator?
Added to that, would the implementation of the mail client not just cut of the text if the user came to use the null terminator his mail contents?
IMHO: SMTP has be designed long time ago to be human readable/writable.
It is pretty simple to test (send simple SMTP messages) typing them by hand via telnet program.
"Human readable" makes null terminator a suboptimal choice.
EMSMTP design is a fossil of pre-spam era. It is bad (by current standards) but it is so widely implemented and sufficiently good (after fixes) to make any quick revolution "not sufficiently urgent".
Extra info: Seen RFC 3030 for BDAT alternative to DATA command.
I am getting this error while using SOAP web service client with axis 1. I had created stub from the wsdl file and tried to consume it then I got this error. wsdl is given to me by someone else.
error in msg parsing: xml was empty, did't parse!
below is the error message and stack trace for the same. Anyone can help.?
In order to fix the javax.activation.DataHandler issue you must add the JavaBeans Activation Framework activation.jar in your classpath.
In order to fix the javax.mail.internet.mimeMultipart issue you must add the Java Mail API mail.jar in your classpath.
The warning messages printed in your console shows that the above jars are not in the classpath.
There are several common reasons to receive the message:
error in msg parsing: xml was empty, did't parse!
The most obvious is that no message was sent. If you have some way of inspecting your transport channel, that would be worth looking at.
Also, the xml message could have been sent in an unexpected character set, e.g. A header declares it to be "Utf-8" but it is really "Win-1252", sometimes you can get away with that if you only use 7-bit ASCII characters, but anything in the 8-bit plane will cause it to bomb.
Also, the xml message could have had a byte order mark unexpectedly inserted at the beginning of the message.
Also, the xml message might not have the document declaration ( starting in the first byte of the message, that violates the specification, and often causes parsers to puke and claim that no message was found.
All things considered with this error message, the parser was not able to find a valid xml message that it could parse, so it didn't. You need to grab the data on the transport channel and figure out what exactly is wrong to resolve the issue.
I'm using Wireshark packet analyzer & when I filter for all "Websocket" packets I see what I am sending /receiving to the host. When I check individual packets mine always show as [MASKED], but you can 'Umask Payload' which shows the data in clear text that looks like this:
<IC sid="52ccc752-6080-4668-8f55-662020d83979" msqid="120l93l9l114l30l104"/>
However, if I 'Follow TCP stream & look at that same packet, the data shows up as encoded in some way like this:
....K#....../...y#..|...}...f...s...~...}...{G..r...kN.."G..z...r...'...'...z...d.
The problem is all Websocket packets I receive from the host come as encoded, it is NOT SSL & I can't figure out how to decode them, I have no idea what they are even encoded as (but yet my browser can decode it).
I assume that whatever method they are coming back to me as encoded data is the same method that my data is encoded when I use 'Follow TCP stream'.
Can someone please help me figure out how to decode the data the host is sending me? See host data below
~.^jVpZc9y4Ef4ryFQ5+yJpeB+JJJdmNPI6G++mrN249kkFkuChIQmG5Fgj//p0AyAJypzxyi5T6P76
QKPRuHz9cUu6IrlZuVYcx75rXXpGYFw6nhdcBqnrXnqeZVhGEtihH65Id7NKWEoPZb8iVfc/FDQt
owztMixN0yltozQNZ3V7ncZkbxrAXZE8vFkZK2g66msJchLIjyuoiWQmvvyApGUY+JsJKPGLkrIF
IHcFALVJNXtTWsl9adMDPlAtQ1AZME0XvoFsShDz5McVn0J6y2z5ceTHlB9pnEltheQVEllIXiGR
z7Ifz6Cz4c2h6XkDLTDUFlkOQYuk/5EUimTnIykUyc5HoeTJjlHVMgWPwifv++Yf6/XLy8uVadlp
Sbs8zml/FfMKAKA4Z2WzLuqEHa/yviqBCEZXJJXeUzC25c2rcIhAEM1LyzBt8jtvtp8+kUee9i+0
ZWTL2+aKkLuyJJ8R2pHPrGPtV5ZcgRIXNVLoF6vh62tpkToy9LIzexnxvRydWEY8lhGPZRxjGccY
IDBEezkMsZSLlRZLtmQQYhm8WCBvr2lAMhFVyDqPpKDmPy1Pi5KtSGaM4Xrlh/aFRV3Rs3Uj+VdN
3rw/QJ9u3v3xuPv8DhSsUw/+ocHtdeKRDNz0wF4GfjpesJrM+CQDx5ACHtFkHdG6Zq159dxkQLPO
jxFa8Ucl7hsl1l9Sss5518vRPa/Ovupe0r+i7qXnTzT5ytq+6Io6e5LiSybMtzacUzbK4ivDZFzo
tmm8UeL+NUeBAKNYsa5jdcbay5TR/tCymZ/rBAYxCbWsuP2ZlSUn7/787Y/Pv9592r27IB9/qsi7
T3+KFklbXpHu0DS87UnPaHVBICKkoq/kI8EeEEif9zI7UFsxU/UCzpGEI4bUjCUT8AsrwWlGek6e
eVGTQ3dBNFHyN5VwSQhwc3I4kA4DN5Ct4oL0OWvZ3yYS+IfTFI0moDt7P2Pl9KvkRYzVGgvI9U89
6YAq2ClvCc1YNyn+gnYm+bxIEsD2kHCMJPS1e080KO1/6sih+Z6W06ZhNbr1HatmL5ND9+g6yThP
wASt950KJ434oUcH2o6V6YT/lcMAcU4imlwQWifDwEjuXUW/gb6pMx9ayI+piYOeSIvIuBoZW34o
EwxMxOBv37N2EvrXoYOAcfg74T+Squg6ESDgVIc413kll8GbaB+E29DPkfI7LfdIkip4PWEfmYx8
ScENzUXax/nEmPzbvDKFWqcmUCxRuBxjqFy+O1WudWoBwiY5TD0Hlkmojz585KKkVVExRaGKYzV9
rGQtBRExF1nF+4LXa5iv6w55auZ6b/h9fqgiDXE7TAuh3ZfK/8uroj+h/CvyziqfEIvKH5sifiUP
kFyn3EfAefdHxNxCqCyUjDWvJ7R/+/btrO6BP9fsKM05j/en3EbeebdHxFy5pbR/weyj5J5nJ0y8
AOCshREwN4B1XSzBOe/5Cd0N8s4qnxBvtKuw/0yr6nR4csk9a0HHLMYfKgdLTo1sJphnDWiQuX5X
6n+gRXtKfYq8s9onxFx5MMSnaV7Jpmj7HEr2CSuRYp81NAMt2trmjLWwBpywEiP70Jw1omPeDPhg
5GSs4h9EKl6M05Cmd3V2UjNF3lndE2JxiH+pYf/TndC+F8yz6jXIYvH5Nz1k+Qn1JfLOap8Qc+We
ch7Wt1OuA+u84wNgOeiPL7ACnyptyDtf2kbEYjp+grX0hO4Kl9lzqkfAYkXYwJZhT/44legRsp9+
kOkz0NyKvRrOLg3vaHmqdir2+fKpgxZXRhxd2OjkcEok20Pb0+JU0GLJ/eGYv8UtDg6sCjWDulSe
7B4CniIA/Gh90GHLtvietafMIO+8hRHxJofVpuj3HPt6Mo17ZJ81MCGWty6HumOnNkadYJ6fJRNk
sXY8yOuI5eUHeeeXnxGxHJ0t5/uCkQ9Fe2qgY4E4n1ETZDiYpzaYckkWSlMZXpEYF6Z5YVoXpn1h
Oheme2F6citrCjH39jp39B2uOd0TfKBdI05AePRri07f5Xb8UCfrDBBXVWNr/RSCRZaV36NzOO0u
oB/hGJnjeW5BAKgLEls8NM7qmMIfWlhwvsf/UsR7OEVULOLJK4GT03eSe0AsCkIdGAQXhGAyL/Qn
hipWfYfuBHkB/yd9qWWY3+6WpeAr8JfM1LydzzAJx22zhl7nzu114ZK9J4cYciI6RBEOT+GpLCgg
C55N8jy7XLjES4VLPBAfmLw8G09Jz3COKnyyN2XaFKAzO7Cux3tGeQdVyAutZ3mn9jwIFv7t9d4y
yd4CU5sVNmxowHnstzSd3UcV/aGGxR02aqWwvj50a2is9RuPdZG4pm/aoed478vuxvw7rZp/Vv1N
gLZANep3FvR3YKApYdcGB+tM6e963jI00a0TBqW67N4XyQ3sI6/Q1Cce4TltIxU74l8TIxzfXncx
yfE67hg+bOytq250jw/iR97FGsdduLt/gNKbOwIpfuR1rHF0LN+59+WtrCaHkTxuLfwjj6LG0RY/
6kR6NIwwxGNsbgkLAVjwVehHZCHkNg/37m4rRwrlPA/DUfgzpKd7VrjSl/BO8BzZCsSlc2HP5KwZ
T3gd24YbYKn2dGTq6/1Lg1lrFjMqWnd3Gx/jSc0ZT9i7e9ia2x3ICd7O2W0eDHmTaxxT8aNudI+g
I8ToUmfGs/URo6K327t7f4clWvYofNg+7OQtrtYHV9dC/ZmcM/Mz0PuQzUYlE3LW1ts6d3JOajxh
3XZtx7bk5DaOgRVu7h5kAmstT/clc/UeZTLPdtBy1MwdPUuEL574kQ8NqMXzxDvCrH+JylbbNobX
B8gewxBaVLYa8iHCmPHMWcsask4g7YGHecaEL5bnBRtPPUCMkWCz/GRqHDwPdTJX7xETOr3dxtxt
5OOEcfTFj3yjgDEy7u8fQvlUoekUXj84MNb36nHiaO683b16o5gi8SzkAtsW9p5lPL3At/BRQ2Wy
729hjj2Hw9wUcrMxehY9cu3A2NjqseMY2hv/fiPfPMBP7z7EmRrN+ideQEavn2e1QDyLTBbMGW+W
u5HMQce7uzPkA4rWmus09QzZ23qP9o4ut1dRklmwn/V27+tREk8dR8f0/PBePnVM2RMLnYbpbjAL
4ll9iUM9l/bCszTxUsNTzyCCJ/y09JjFvs6LVQ3B/JFPJJoF0XfL9bydhwW+w6IuZklcMto+wYZB
7sRwqiiSOinhdIGFnZelhnIm2gDzJ1KaalBvTh/g4BBvcK+n2uA8bMC+0h529WAdVv5qOOPg5NJ5
UTlurXGm5QUubOqcjktRy3kFW98eHzFGP3BVmjGUYVyDMlqxp4ZmQr0iG4ocg9UM1D8V8ShiK95I
sAbCYA3nfA0brhqp0V5jBG8YSgUWArG5mQUPC8JEHaDuQKw15BQIjehoyEEa+suORa+hLE10QBkD
ShFwYau6rKF46lLHc1zeovIgB6VkqdKH69xIbqeTBNaCuOTdFDesB+KNVDWxY6xOol7rGNYFJTWS
wBfxOqCaniY2aA7HlFEE8LXkNBEPWyp5cAnGlCih85OkoRNHGxA13CQVdXPoR3kTdWbqIhMXbdHb
ST90WHcKF3I5FWZhxFV71mVc1IcAKghEJuGTk7hgi/Yggqs0vmNiWikKdLihXffC20QdsnBc9lmL
CTE87kmGr4ZhM9y7gGzLMogAawcaVrOqqIvRh1jNtKfuUCmKmmJP/GXUjYVO0MrxPharXUWPk/NY
4gRh0OwPpkbEvDoooqVU67UCC92I3A/9sUTyPqk6FE2+4N5D44x3N8Y0lk8R73teTQdV2EpHfHMA
4nCX5H6vXisXuE/R2XQ0bs8YCYRNnXVTW+wrh9EaqI7Ym92P7+wAEWcgZ+K18lJCHLzJaqbFMMeT
9CTwyOgcZdpLqEaew3SgFSwAN5wqxyL4bQHwiVHop4RU4vclc3A2Ge11srEIg0nPaJwPQNVc8usV
X0/J2NuO0a5ImI4UVwuPv3z89enu8+ffvpCvFKYgFJq2nXyc2LiG3l7H+R6yH/9PjGGw0LfTyLGN
0Ehdzw4DM6apZaXM8pMA/+NMfLNyXc+IAwcgIaNBEDhmStPYMSPT9izHSMA5QCUA8tMoCFxGHWpa
phn5lhVYNAlTljoueiItwy8ft7f/Bw==
Client to server data is XORed with a mask (included in the dataframe). Some people suggest this is in order to throw off bad caching mechanisms responding to new websocket requests with server messages from older sessions. The masking makes sure that even messages containing identical data will appear differently to applications that do not understand websockets.
Also note that there are many different size options for the headers themselves.
Refer to RFC 6455 Section 5 which defines the masking/unmasking process for payloads sent from the client to the server.
https://www.rfc-editor.org/rfc/rfc6455
If you find any freeware VBA code to do the job of forming packets let me know! :-)
Hi I need help to understand, if 35=X message should contain Symbol/SecID within the repeating group.
The FIX Specification indicates that under the repeating group both 55 and 48/22 are optional.
I received a message from my client without a symbol tag, please help me undersatnd if that was a bad formed message
20150923-15:06:14.976 : 8=FIXT.1.19=33635=X34=19153349=SENDER52=20150923-15:06:14.63756=RECEIVER268=8279=0269=1270=99.609375271=289279=0269=1270=99.6171875271=241279=0269=1270=99.625271=154279=0269=1270=99.6328125271=139279=0269=0270=99.6015625271=268279=0269=0270=99.59375271=244279=0269=0270=99.5859375271=171279=0269=0270=99.578125271=21610=198
You are advised to treat the default FIX message and field definitions as a set of suggested definitions.
In practice, no commercial FIX counterparty uses these definitions as-is. Every counterparty I've connected to makes modifications, adding or removing fields from messages or groups, creating new fields, or sometimes adding entirely new messages. No counterparty supports every message and field.
When connecting to a counterparty, do not assume anything. Your counterparty should provide documentation on how they expect their interface to be used, and which messages and fields they will send and which they expect to receive from you.
You need to read their specs and modify your FIXnn.xml DataDictionary file to match what they will be sending you.
If their spec says they will send you Symbol and/or SecurityID in a 35=X message, you need to make sure your DD file matches that.
This page might be helpful to you. (It's technically for the C# QuickFIX/n, but the DD file is the same for all QF versions.)
http://quickfixn.org/tutorial/custom-fields-groups-and-messages.html
I have a rather puzzling problem with Amazon SQS and Zend(1.11.2). I am sending a message to the queue that I have setup with a snip-it that looks like this:
$sqs->send($queueURL, "opt1=foo opt2=bar");
The message comes in to the app at the far side and is seen as:
"opt1%3Dfoo+opt2%3Dbar"
The receiving app is written in Java using the com.xerox.amazonws.sqs2 library and is in production now.
There was similar sending code in an older php module that used Tarzan under Drupal that worked just fine. I have searched high and low and read the documentation for Zend, Amazon and the Java library and I am stuck.
The encoding of the string is understandable but I don't recognize the method being used. Further tests show that single quotes, angle brackets, etc. are also escaped as hex.
Any ideas?
Ken
Typica (com.xerox.amazonaws) has a 'encoded' option that internally uses Base64 to encode messages. This is on by default.
In general, we have found that encoding messages using Base64 is less troublesome then URL/percent encoding when using SQS.
I would Base64 the message text in PHP, write it to the queue, and Typica should decode just fine using the default config.