I am maintaining some existing code
I see this fragment:
msgsnd( $mQueue, pack("l! a*", length($msg), $msg), 0)
|| ... error handling ...
I would like to understand the call to pack() as the second argument to msgsnd.
I find the following documentation for msgsend
Calls the System V IPC function msgsnd to send the message MSG to the
message queue ID. MSG must begin with the native long integer message
type, be followed by the length of the actual message, and then
finally the message itself. This kind of packing can be achieved with
pack("l! a*", $type, $message) . Returns true if successful, false on
error. See also SysV IPC in perlipc and the documentation for
IPC::SysV and IPC::Msg .
This gives the second parameter to pack as $type, but does not explain what $type is. The code I'm trying to understand instead passes the message length.
What's going on? So far as I can tell the existing code is working reliably.
The application in question decided to use the type field to store the length of the message.
(This is rather odd, since the size of the message is already available to the reader.)
When the receiver request a message from the system, they may constrain the request to specific message types.
If msgtyp is 0, then the first message in the queue is read.
If msgtyp is greater than 0, then the first message in the queue of type msgtyp is read, unless MSG_EXCEPT was specified in msgflg, in which case the first message in the queue of type not equal to msgtyp will be read.
If msgtyp is less than 0, then the first message in the queue with the lowest type less than or equal to the absolute value of msgtyp will be read.
If the receiver specifies 0 for msgtyp, then the message type provided by the sender isn't used by the system, and can therefore be used to carry other information.
The man page for msgsnd says "The mtext field is an array (or other structure) whose size is specified by msgsz, a nonnegative integer value. Messages of zero length (i.e., no mtext field) are permitted. The mtype field must have a strictly positive integer value. This value can be used by the receiving process for message selection (see the description of msgrcv() below)."
Thus type is not used by sndmsg itself and the length that appears in the type field might or might not be used on the receiving side.
it creates a binary representation of the msg: len msg. Check it by:
perl -e '$a= "abcde"; print(pack("l! a*", length($a), $a))' | od -c
Gives:
0000000 005 \0 \0 \0 \0 \0 \0 \0 a b c d e
0000015
Related
My QuickFIX client is complaining that the body length is not expected.
After checking, it is found that it receives a message which actually contains 2 messages (2 different MsgTypes <35>). Also, 2 BeginStrings <8>
Is it a valid message?
The error is reported by QuickFIX, instead of my own code.
Hence, it looks like an invalid message to me although I cannot find any official doc, saying that it is not allowed.
I would expect that QuickFIX could parse the messages as long as the body length of the first message is correct.
You could check if the body length is correct by using the following:
counting the number of characters in the message following the BodyLength (9) field up to, and including, the delimiter immediately preceding the CheckSum (10) field. ALWAYS SECOND FIELD IN MESSAGE. (Always unencrypted) For example, for message 8=FIX 4.4^9=5^35=0^10=10^, the BodyLength is 5 for 35=0^
Source: https://btobits.com/fixopaedia/fixdic44/index.html?tag_9_BodyLength.html
Its completely depends on your fix engine whether to support multiple messages in one go or not.
Using BodyLength[9] and CheckSum[10] fields.
BodyLength is calculated starting from field starting after BodyLenght and
before CheckSum field.
CheckSum is calculated from ‘8= upto SOH before the checksum field.
Binary value of each character is calculated and compared to the LSB of the calculated value to the checksum value.
If the checksum has been calculated to be 274 then the modulo 256 value is 18 (256 + 18 = 274). This value would be transmitted a 10=018 where
"10="is the tag for the checksum field.
I have an unformatted binary file generated using the Compaq Visual Fortran compiler (big endian).
Here's what the little bit of documentation states about it:
The binary file is written in a general format consisting of data arrays, headed by a descriptor record:
An 8-character keyword which identifies the data in the block.
A 4-byte signed integer defining the number of elements in the block.
A 4-character keyword defining the type of data. (INTE, REAL, LOGI, DOUB, or CHAR)
The header items are read in as a single record. The data follows the descriptor on a new record. Numerical arrays are divided into block of up to 1000 items. The physical record size is the same as the block size.
Attempts to read such data
module modbin
type rectype
character(len=8)::key
integer::data_count
character(len=4)::data_type
logical::is_int
integer, allocatable:: idata(:)
real(kind=8), allocatable::rdata(:)
end type
contains
subroutine rec_read(in_file, out_rec)
integer, intent(in):: in_file
type (rectype), intent(inout):: out_rec
!
! You need to play around with this figure. It may not be
! entirely accurate - 1000 seems to work, 1024 does not
integer, parameter:: bsize = 1000
integer:: bb, ii, iimax
! read the header
out_rec%data_count = 0
out_rec%data_type = ' '
read(in_file, end = 20) out_rec%key, out_rec%data_count,
out_rec%data_type
! what type is it?
select case (out_rec%data_type)
case ('INTE')
out_rec%is_int = .true.
allocate(out_rec%idata(out_rec%data_count))
case ('DOUB')
out_rec%is_int = .false.
allocate(out_rec%rdata(out_rec%data_count))
end select
! read the data in blocks of bsize
bb = 1
do while (bb .lt. out_rec%data_count)
iimax = bb + bsize - 1
if (iimax .gt. out_rec%data_count) iimax = out_rec%data_count
if (out_rec%is_int) then
read(in_file) (out_rec%idata(ii), ii = bb, iimax)
else
read(in_file) (out_rec%rdata(ii), ii = bb, iimax)
end if
bb = iimax + 1
end do
20 continue
end subroutine rec_read
subroutine rec_print(in_recnum, in_rec)
integer, intent(in):: in_recnum
type (rectype), intent(in):: in_rec
print *, in_recnum, in_rec%key, in_rec%data_count, in_rec%data_type
! print out data
open(unit=12, file='reader.data' , status='old')
write(12,*)key
!write(*,'(i5')GEOMINDX
!write(*,'(i5')ID_BEG
!write(*,'(i5')ID_END
!write(*,'(i5')ID_CELL
!write(*,'(i5')TIME_BEG
!write(*,'(i5')SWAT
!format('i5')
!end do
close(12)
end subroutine rec_print
end module modbin
program main
use modbin
integer, parameter:: infile=11
! fixed size for now - should really be allocatable
integer, parameter:: rrmax = 500
type (rectype):: rec(rrmax)
integer:: rr, rlast
open(unit=infile, file='TEST1603.SLN0001', form='UNFORMATTED',
status='OLD', convert='BIG_ENDIAN')
rlast = 0
do rr = 1, rrmax
call rec_read(infile, rec(rr))
if (rec(rr)%data_type .eq. ' ') exit
rlast = rr
call rec_print(rr, rec(rr))
end do
close(infile)
end program main
This code compiles and runs smoothly showing
and produces no errors but this is written in the output file
shows me no useful numerical values
The file in question is available here
And the right WRITE statement should produce a file like this one here
Is my WRITE STATEMENT to output this file type wrong? , and if so, what is the best way?
thank you
The comments above are trying to direct you to one of (at least) two problems in your code. In the subroutine rec_print you have write(12,*)key where you meant to write write(12,*)in_rec%key (at least I think that's what you wanted.)
The other problem I spotted is that rec_print opens reader.data with status='old' and then closes it after writing key. (The use of old here suggests that the file already exists.) Each time rec_print is called, the file is opened, the first record is overwritten, and the file is closed. One solution to this would be to use status='unknown'. position='append', though it would be more efficient to open the file once in the main program and just let the subroutine write to it.
If I make these changes, I get in the data file:
INTEHEAD
GEOMETRY
GEOMINDX
ID_BEG
ID_END
ID_CELL
TIME_BEG
SWAT
A side-comment about CONVERT= and derived types: Your program isn't affected by this, but there are compiler differences with how reading a derived type record with CONVERT= is handled. I think gfortran converts each component according to its type, but I know that Intel Fortran doesn't convert reads (nor writes) of an entire derived type. You are reading individual components, which works in both compilers, so that's fine, but I thought it was worth mentioning.
If you're wondering why Intel Fortran does it this way, it's due to the VAX FORTRAN (where CONVERT= came from) heritage with STRUCTURE/RECORD and the possible use of UNION/MAP (not available in standard Fortran). With unions, there's no way to know how a particular component should be converted, so it just transfers the bytes. I had suggested to the Intel team that this could be relaxed if no UNIONs were present, but that I'm sure is very low priority.
I got some account issues in the SCN so I make a attempt here.
We switched to Unicode and got some issues with that. INFTY_TAB = PS+2. This coding gets an error that "the offset + length is exceeding".
I found some hints but couldn't really figure out how to fix this. And even when I manage to fix those errors I got a new error called 'Iclude-Report %HR_P9002 not found'. The IT is still there so is there something else I can check?
Definition of PS:
DATA: BEGIN OF PS OCCURS 0.
*This indicates if a record was read with disabled authority check.
data: authc_disabled(1) type c.
DATA: TCLAS LIKE PSPAR-TCLAS.
INCLUDE STRUCTURE PRELP.
DATA: ACRCD LIKE SY-SUBRC.
DATA: END OF PS.
TCLAS is a char(1) field.
This is the part where the error pops up:
INFTY_TAB = PS+2.
Error: I had to translate so sorry for some mistakes that could appear.
Offset and Length (=2432) exceed the length of the character based beginning (=2430) of the structure.
Depends on the length of INFTY_TAB. You have to explicitly set length:
INFTY_TAB = PS+2(length).
Official information is here. The important point to note is that the inclusion of SY-SUBRC (which is an INT4 field) places a limit to the range of fields you can access using this (discouraged) method of access.
ASSIGN field+off TO is generally forbidden from a syntactical
point of view since any offset <> 0 would cause the range to be
exceeded.
Although the sentence above is related to ASSIGN command, it is also valid for this situation.
If I try to replace or cancel an order I get a message
58=Conditionally Required Field Missing
and the next message contains
58=Invalid MsgType
Here are the logs:
Replacing an order (tgFZctx200U61 is my side. FG is an exchange.):
20170203-15:44:04.225 : 8=FIX.4.49=15135=G34=349=tgFZctx200U6152=20170203-15:44:04.22556=FG1=U6111=270071221=138=240=241=2700744=11640054=155=RTS-3.1760=20170203-18:44:04.20510=028
20170203-15:44:04.225 : 8=FIX.4.49=23235=849=FG56=tgFZctx200U6134=352=20170203-15:43:56.98137=572984433198=F:572984433526=$01$11=270071241=2700717=exec-201702031001027616150=E39=E55=RTS-3.17461=FXXXXX54=138=140=2151=114=06=060=19700101-00:00:00.00010=213
20170203-15:44:04.275 : 8=FIX.4.49=11535=j34=449=tgFZctx200U6152=20170203-15:44:04.27556=FG45=358=Conditionally Required Field Missing372=8380=510=065
20170203-15:44:04.275 : 8=FIX.4.49=33335=849=FG56=tgFZctx200U6134=452=20170203-15:43:56.98237=572984753198=F:572984753526=$01$11=270071241=27007453=1448=tgFZctx200U61447=C452=317=3355471052150=539=01=FZ00U6155=RTS-3.1754=138=240=244=116400.00000336=9291151=214=06=060=20170203-15:43:56.98920008=-922337203685372211120018=[51000-3355471052-0]10=100
20170203-15:44:04.285 : 8=FIX.4.49=10335=349=FG56=tgFZctx200U6134=552=20170203-15:43:57.03345=4371=372373=1158=Invalid MsgType372=810=164
cancelling an order:
20170203-15:26:19.178 : 8=FIX.4.49=15435=F34=349=tgFZctx200U6152=20170203-15:26:19.17856=FG11=270061237=57286383038=141=2700644=116470.0000054=155=RTS-3.1760=20170203-18:26:19.17810=013
20170203-15:26:19.188 : 8=FIX.4.49=20735=849=FG56=tgFZctx200U6134=352=20170203-15:26:11.92437=572863830198=F:572863830526=$01$11=270061241=2700617=exec-201702031001027615150=639=655=RTS-3.17461=FXXXXX54=138=140=2151=114=06=010=239
20170203-15:26:19.418 : 8=FIX.4.49=11535=j34=449=tgFZctx200U6152=20170203-15:26:19.41856=FG45=358=Conditionally Required Field Missing372=8380=510=070
20170203-15:26:19.418 : 8=FIX.4.49=33335=849=FG56=tgFZctx200U6134=452=20170203-15:26:11.92437=572863830198=F:572863830526=$01$11=270061241=27006453=1448=tgFZctx200U61447=C452=317=3354681208150=439=41=FZ00U6155=RTS-3.1754=138=140=244=116470.00000336=9291151=014=06=060=20170203-15:26:11.93120008=-922337203685267353520018=[51000-3354681208-0]10=080
20170203-15:26:19.418 : 8=FIX.4.49=10335=349=FG56=tgFZctx200U6134=552=20170203-15:26:12.16445=4371=372373=1158=Invalid MsgType372=810=161
Best regards, Mikhail
"Conditionally Required Field Missing" means you are trying to extract an optional field that isn't present. (It's not required by the DD, but the user's logic expects it to be there, hence "conditionally required".)
The first 35=j message says:
45=3 - sequence number of message where these happened
58=Conditionally Required Field Missing
372=8 - type of message where this happened
380=5 - same code as explained in 58
Unfortunately, the message doesn't say which field is the problem, but basically, you're doing this (forgive my pseudocode):
var x = msg.getSomeOptionalField()
but you need to do this:
var x = null;
if (msg.checkIfSomeOptionalFieldIsPresent())
x = msg.getSomeOptionalField();
In order to parse your own FIX messages use FIXimate.
58 is a text field. The text after 58 in this case is the error message. The tag value pair 372=83 means: The message referred to (i.e. the missing tag) is tag 83.
Tag 83 is the sequence number of message within report series. FIXimate says that 83 is "Used to carry reporting sequence number of the fill as represented on the Trade Report Side."
This is your FIX engine sending an error back to the exchange. You can tell by looking at the SenderCompID and TargetCompID for each message.
You send a message:
20170203 15:44:04.225:8=FIX.4.49=15135=G34=349=tgFZctx200U6152=20170203-15:44:04.22556=FG
You get an Execution Report (35=8, probably acknowledging order cancellation/replace):
20170203-15:44:04.225 : 8=FIX.4.49=23235=8 49=FG 56=tgFZctx200U61lo9
You send an Business Reject (35=j):
20170203-15:44:04.275 : 8=FIX.4.49=115 35=j 34=4 49=tgFZctx200U61 52=20170203-15:44:04.275 56=FG 45=358=Conditionally Required Field Missing372=8380=510=065
What this last message coming in from the exchange is, is hard to tell without further analysis, but it most likely the execution report for the replaced order. It seems to have been sent 1 ms after the original execution report.
Your FIX engine expects certain data to be present within the messages. The expectation is set in your data dictionary, an xml file which should be provided by your counterparty. Sometimes (like now) there are errors in this file and you have to open it up, find the message in question (in this case the original execution report), and tell your data dictionary not to expect tag 83.
That should clear things up. Let me know if it doesn't work.
I'm looking to create a Perl script that looks at an snmp MIB, pulls out parts of that MIB given reference points, and adds the results into a string.
For example, this a portion of the data I'm working with:
udpOutDatagrams OBJECT-TYPE
SYNTAX Counter
ACCESS read-only
STATUS mandatory
DESCRIPTION
"The total number of UDP datagrams sent from this
entity."
::= { udp 4 }
-- the UDP Listener table
-- The UDP listener table contains information about this
-- entity's UDP end-points on which a local application is
-- currently accepting datagrams.
udpTable OBJECT-TYPE
SYNTAX SEQUENCE OF UdpEntry
ACCESS not-accessible
STATUS mandatory
DESCRIPTION
"A table containing UDP listener information."
::= { udp 5 }
udpEntry OBJECT-TYPE
SYNTAX UdpEntry
ACCESS not-accessible
STATUS mandatory
DESCRIPTION
"Information about a particular current UDP
listener."
INDEX { udpLocalAddress, udpLocalPort }
::= { udpTable 1 }
I'm looking to pull out the object names, add a comma, and then the the descriptions :
udpOutDatagrams, The total number of UDP datagrams sent from this entity.
udpTable, A table containing UDP listener information.
udpEntry, Information about a particular current UDP listener.
There are some reference points that could be used:
All names are immediately followed by the string "OBJECT-TYPE".
All descriptions are surrounded by double-quotes and immediately follow a line that says "DESCRIPTION"
I've put together a basic frame to handle this. Just need some advice on the logic that would be used to create name/description strings.
UPDATE: Basic script based on the answer by Sinan Ünür. It can all be done with this.
#!/usr/local/bin/perl -w
use strict;
use SNMP;
$SNMP::save_descriptions = 1;
my $mib = $ARGV[0];
my $object;
&SNMP::addMibDirs("/usr/share/snmp/mibs/allMibs/");
&SNMP::loadModules($mib);
&SNMP::initMib();
foreach my $key ( keys %SNMP::MIB )
{
print "$SNMP::MIB{$key}{label}, $SNMP::MIB{$key}{description}, $SNMP::MIB{$key}{objectID}\n"
};
Have you considered using SNMP? It gives you a %SNMP::MIB hash:
a tied hash to access parsed MIB information. After the MIB has been loaded this hash allows access to to the parsed in MIB meta-data(the structure of the MIB (i.e., schema)). The hash returns blessed references to SNMP::MIB::NODE objects which represent a single MIB attribute. The nodes can be fetched with multiple 'key' formats - the leaf name (e.g.,sysDescr) or fully/partially qualified name (e.g., system.sysDescr) or fully qualified numeric OID.
Regular expressions seems the easiest way to go.
Just parse the file looking for the line "OBJECT-TYPE" then take the text before "OBJECT-TYPE" as the name.
After that look for "DESCRIPTION" then look for the text between the quotation marks that occurs right after.
Then repeat!