I have a FixMessage and I want to calculate the checksum manually.
8=FIX.4.2|9=49|35=5|34=1|49=ARCA|52=20150916-04:14:05.306|56=TW|10=157|
The body length here is calculated:
8=FIX.4.2|9=49|35=5|34=1|49=ARCA|52=20150916-04:14:05.306|56=TW|10=157|
0 + 0 + 5 + 5 + 8 + 26 + 5 + 0 = 49(correct)
The checksum is 157 (10=157). How to calculate it in this case?
You need to sum every byte in the message up to but not including the checksum field. Then take this number modulo 256, and print it as a number of 3 characters with leading zeroes (e.g. checksum=13 would become 013).
Link from the FIX wiki: FIX checksum
An example implementation in C, taken from onixs.biz:
char *GenerateCheckSum( char *buf, long bufLen )
{
static char tmpBuf[ 4 ];
long idx;
unsigned int cks;
for( idx = 0L, cks = 0; idx < bufLen; cks += (unsigned int)buf[ idx++ ] );
sprintf( tmpBuf, "%03d", (unsigned int)( cks % 256 ) );
return( tmpBuf );
}
Ready-to-run C example adapted from here
8=FIX.4.2|9=49|35=5|34=1|49=ARCA|52=20150916-04:14:05.306|56=TW|10=157|
#include <stdio.h>
void GenerateCheckSum( char *buf, long bufLen )
{
unsigned sum = 0;
long i;
for( i = 0L; i < bufLen; i++ )
{
unsigned val = (unsigned)buf[i];
sum += val;
printf("Char: %02c Val: %3u\n", buf[i], val); // print value of each byte
}
printf("CheckSum = %03d\n", (unsigned)( sum % 256 ) ); // print result
}
int main()
{
char msg[] = "8=FIX.4.2\0019=49\00135=5\00134=1\00149=ARCA\00152=20150916-04:14:05.306\00156=TW\001";
int len = sizeof(msg) / sizeof(msg[0]);
GenerateCheckSum(msg, len);
}
Points to Note
GenerateCheckSum takes the entire FIX message except CheckSum field
Delimiter SOH is written as \001 which has ASCII value 1
static void Main(string[] args)
{
//10=157
string s = "8=FIX.4.2|9=49|35=5|34=1|49=ARCA|52=20150916-04:14:05.306|56=TW|";
byte[] bs = GetBytes(s);
int sum=0;
foreach (byte b in bs)
sum = sum + b;
int checksum = sum % 256;
}
//string to byte[]
static byte[] GetBytes(string str)
{
byte[] bytes = new byte[str.Length * sizeof(char)];
System.Buffer.BlockCopy(str.ToCharArray(), 0, bytes, 0, bytes.Length);
return bytes;
}
Using BodyLength[9] and CheckSum[10] fields.
BodyLength is calculated starting from field starting after BodyLenght and
before CheckSum field.
CheckSum is calculated from ‘8= upto SOH before the checksum field.
Binary value of each character is calculated and compared to the LSB of the calculated value to the checksum value.
If the checksum has been calculated to be 274 then the modulo 256 value is 18 (256 + 18 = 274). This value would be transmitted a 10=018 where
"10="is the tag for the checksum field.
In Java there is a method from QuickFixJ.
String fixStringMessage = "8=FIX.4.29=12535=81=6090706=011=014=017=020=322=837=038=4.39=054=155=ALFAA99=20220829150=0151=06020=06021=06022=F9014=Y";
int checkSum = quickfix.MessageUtils.checksum(fixStringMessage);
System.out.prinln(checkSum);
Output: 127
Hope it can help you.
Related
I implemented a Galois Linear-Feedback Shift-Regiser in Verilog (and also in MATLAB, mainly to emulate the HDL design). It's been working great, and as of know I use MATLAB to calculate CRC-32 fields, and then include them in my HDL simulations to verify a data packet has arrived correctly (padding data with CRC-32), which produces good results.
The thing is I want to be able to calculate the CRC-32 I've implemented in software, because I'll be using a Raspberry Pi to input data through GPIO in my FPGA, and I haven't been able to do so. I've tried this online calculator, using the same parameters, but never get to yield the same result.
This is the MATLAB code I use to calculate my CRC-32:
N = 74*16;
data = [round(rand(1,N)) zeros(1,32)];
lfsr = ones(1,32);
next_lfsr = zeros(1,32);
for i = 1:length(data)
next_lfsr(1) = lfsr(2);
next_lfsr(2) = lfsr(3);
next_lfsr(3) = lfsr(4);
next_lfsr(4) = lfsr(5);
next_lfsr(5) = lfsr(6);
next_lfsr(6) = xor(lfsr(7),lfsr(1));
next_lfsr(7) = lfsr(8);
next_lfsr(8) = lfsr(9);
next_lfsr(9) = xor(lfsr(10),lfsr(1));
next_lfsr(10) = xor(lfsr(11),lfsr(1));
next_lfsr(11) = lfsr(12);
next_lfsr(12) = lfsr(13);
next_lfsr(13) = lfsr(14);
next_lfsr(14) = lfsr(15);
next_lfsr(15) = lfsr(16);
next_lfsr(16) = xor(lfsr(17), lfsr(1));
next_lfsr(17) = lfsr(18);
next_lfsr(18) = lfsr(19);
next_lfsr(19) = lfsr(20);
next_lfsr(20) = xor(lfsr(21),lfsr(1));
next_lfsr(21) = xor(lfsr(22),lfsr(1));
next_lfsr(22) = xor(lfsr(23),lfsr(1));
next_lfsr(23) = lfsr(24);
next_lfsr(24) = xor(lfsr(25), lfsr(1));
next_lfsr(25) = xor(lfsr(26), lfsr(1));
next_lfsr(26) = lfsr(27);
next_lfsr(27) = xor(lfsr(28), lfsr(1));
next_lfsr(28) = xor(lfsr(29), lfsr(1));
next_lfsr(29) = lfsr(30);
next_lfsr(30) = xor(lfsr(31), lfsr(1));
next_lfsr(31) = xor(lfsr(32), lfsr(1));
next_lfsr(32) = xor(data2(i), lfsr(1));
lfsr = next_lfsr;
end
crc32 = lfsr;
See I use a 32-zeroes padding to calculate the CRC-32 in the first place (whatever's left in the LFSR at the end is my CRC-32, and if I do the same replacing the zeroes with this CRC-32, my LFSR becomes empty at the end too, which means the verification passed).
The polynomial I'm using is the standard for CRC-32: 04C11DB7. See also that the order seems to be reversed, but that's just because it's mirrored to have the input in the MSB. The results of using this representation and a mirrored one are the same when the input is the same, only the result will be also mirrored.
Any ideas would be of great help.
Thanks in advance
Your CRC is not a CRC. The last 32 bits fed in don't actually participate in the calculation, other than being exclusive-or'ed into the result. That is, if you replace the last 32 bits of data with zeros, do your calculation, and then exclusive-or the last 32 bits of data with the resulting "crc32", then you will get the same result.
So you will never get it to match another CRC calculation, since it isn't a CRC.
This code in C replicates your function, where the data bits come from the series of n bytes at p, least significant bit first, and the result is a 32-bit value:
unsigned long notacrc(void const *p, unsigned n) {
unsigned char const *dat = p;
unsigned long reg = 0xffffffff;
while (n) {
for (unsigned k = 0; k < 8; k++)
reg = reg & 1 ? (reg >> 1) ^ 0xedb88320 : reg >> 1;
reg ^= (unsigned long)*dat++ << 24;
n--;
}
return reg;
}
You can immediately see that the last byte of data is simply exclusive-or'ed with the final register value. Less obvious is that the last four bytes are just exclusive-or'ed. This exactly equivalent version makes that evident:
unsigned long notacrc_xor(void const *p, unsigned n) {
unsigned char const *dat = p;
// initial register values
unsigned long const init[] = {
0xffffffff, 0x2dfd1072, 0xbe26ed00, 0x00be26ed, 0xdebb20e3};
unsigned xor = n > 3 ? 4 : n; // number of bytes merely xor'ed
unsigned long reg = init[xor];
while (n > xor) {
reg ^= *dat++;
for (unsigned k = 0; k < 8; k++)
reg = reg & 1 ? (reg >> 1) ^ 0xedb88320 : reg >> 1;
n--;
}
switch (n) {
case 4:
reg ^= *dat++;
case 3:
reg ^= (unsigned long)*dat++ << 8;
case 2:
reg ^= (unsigned long)*dat++ << 16;
case 1:
reg ^= (unsigned long)*dat++ << 24;
}
return reg;
}
There you can see that the last four bytes of the message, or all of the message if it is three or fewer bytes, is exclusive-or'ed with the final register value at the end.
An actual CRC must use all of the input data bits in determining when to exclusive-or the polynomial with the register. The inner part of that last function is what a CRC implementation looks like (though more efficient versions make use of pre-computed tables to process a byte or more at a time). Here is a function that computes an actual CRC:
unsigned long crc32_jam(void const *p, unsigned n) {
unsigned char const *dat = p;
unsigned long reg = 0xffffffff;
while (n) {
reg ^= *dat++;
for (unsigned k = 0; k < 8; k++)
reg = reg & 1 ? (reg >> 1) ^ 0xedb88320 : reg >> 1;
n--;
}
return reg;
}
That one is called crc32_jam because it implements a particular CRC called "JAMCRC". That CRC is the closest to what you attempted to implement.
If you want to use a real CRC, you will need to update your Verilog implementation.
I need to implement my own packets to send over UDP. I decided that I would do this by sending a char buffer which has the sequence number, checksum, size, and the data of the packet which is bytes from a file. The string i'm sending separates each field by a semicolon. Then, when I receive the string (which is my packet) I want to extract each felid, use them accordingly (the sequence number, size, and checksum) and write the bytes to a file. So far I have wrote a method to create 100 packets, and I'm trying to extract and write the bytes to a file (I'm not doing it in the receiver yet, first I'm testing the parsing in the sender). For some reason, the bytes written to my file are incorrect and I'm getting "JPEG DATATSTREAM CONTAINS NO IMAGE" error when I try to open it.
struct packetNode{
char packet[1052]; // this is the entire packet data including the header
struct packetNode *next;
};
This is how I'm creating my packets:
//populate initial window of size 100
for(i = 0; i < 100; i++){
memset(&data[0], 0, sizeof(data));
struct packetNode *p; // create packet node
p = (struct packetNode *)malloc(sizeof(struct packetNode));
bytes = fread(data, 1, sizeof(data), fp); // read 1024 bytes from file into data buffer
int b = fwrite(data, 1, bytes, fpNew);
printf("read: %d\n", bytes);
memset(&p->packet[0], 0, sizeof(p->packet));
sprintf(p->packet, "%d;%d;%d;%s", s, 0, numPackets, data); // create packet
//calculate checksum
int check = checksum8(p->packet, sizeof(p->packet));
sprintf(p->packet, "%d;%d;%d;%s", s, check, numPackets, data); //put checksum in packet
s++; //incremenet sequence number
if(i == 0){
head = p;
tail = p;
tail->next = NULL;
}
else{
tail->next = p;
tail = p;
tail->next = NULL;
}
}
fclose(fp);
and this is where I parse and write the bytes to a file:
void test(){
FILE *fpNew = fopen("test.jpg", "w");
struct packetNode *ptr = head;
char *tokens;
int s, c, size;
int i = 0;
char data[1024];
while(ptr != NULL){
memset(&data[0], 0, sizeof(data));
tokens = strtok(ptr->packet,";");
s = atoi(tokens);
tokens = strtok(NULL, ";");
c = atoi(tokens);
tokens = strtok(NULL, ";");
size = atoi(tokens);
tokens = strtok(NULL, ";");
if(tokens != NULL)
strcpy(data, tokens);
printf("sequence: %d, checksum: %d, size: %d\n", s,c,size);
int b = fwrite(data, 1, sizeof(data), fpNew);
ptr = ptr->next;
i++;
}
fclose(fpNew);
}
Since there is transfer of binary data, a JPEG stream, this data cannot be treated as a string. It's better to go all binary. For instance, instead of
sprintf(p->packet, "%d;%d;%d;%s", s, 0, numPackets, data); // create packet
you would do
sprintf(p->packet, "%d;%d;%d;", s, 0, numPackets);
memcpy(&p->packet[strlen(p->packet)], data, bytes);
but this leads to parsing problems: we would need to change this:
tokens = strtok(NULL, ";");
if(tokens != NULL)
strcpy(data, tokens);
to something like this:
tokens += 1 + ( size < 10 ? 1 : size < 100 ? 2 : size < 1000 ? 3 : size < 10000 ? 4 : 5 );
memcpy(data, tokens, sizeof(data));
#Binary Protocol
It's easier to use a binary packet:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#pragma push(pack,1)
typedef struct Packet {
int seq, maxseq, size;
unsigned short cksum;
unsigned char payload[];
} Packet;
#pragma pop(pack)
typedef struct PacketNode{
struct PacketNode * next;
Packet packet;
} PacketNode;
PacketNode * allocPacketNode(int maxPayloadSize) {
void * ptr = malloc(sizeof(PacketNode) + maxPayloadSize); // FIXME: error checking
memset(ptr, 0, sizeof(PacketNode) + maxPayloadSize); // mallocz wouldn't cooperate
return (PacketNode*) ptr;
}
PacketNode * prepare(FILE * fp, int fsize, int chunksize)
{
PacketNode * head = allocPacketNode(chunksize);
PacketNode * pn = head;
int rd, seq = 0;
int maxseq = fsize / chunksize + ( fsize % chunksize ? 1 : 0 );
while ( ( rd = fread(pn->packet.payload, 1, chunksize, fp ) ) > 0 )
{
printf("read %d bytes\n", rd);
pn->packet.seq = seq++;
pn->packet.maxseq = maxseq;
pn->packet.size = rd + sizeof(Packet);
pn->packet.cksum = 0;
pn->packet.cksum = ~checksum(&pn->packet, pn->packet.size);
if ( rd == chunksize )
pn = pn->next = allocPacketNode(chunksize);
}
return head;
}
int checksum(unsigned char * data, int len)
{
int sum = 0, i;
for ( i = 0; i < len; i ++ )
sum += data[i];
if ( sum > 0xffff )
sum = (sum & 0xffff) + (sum>>16);
return sum;
}
void test( PacketNode * ptr ) {
FILE *fpNew = fopen("test.jpg", "w");
while (ptr != NULL)
{
printf("sequence: %d/%d, checksum: %04x, size: %d\n",
ptr->packet.seq,
ptr->packet.maxseq,
ptr->packet.cksum,
ptr->packet.size - sizeof(Packet)
);
int b = fwrite(ptr->packet.payload, ptr->packet.size - sizeof(Packet), 1, fpNew);
ptr = ptr->next;
}
fclose(fpNew);
}
void fatal( const char * msg ) { printf("FATAL: %s\n", msg); exit(1); }
int main(int argc, char** argv)
{
if ( ! argv[1] ) fatal( "missing filename argument" );
FILE * fp = fopen( argv[1], "r" );
if ( ! fp ) fatal( "cannot open file" );
fseek( fp, 0, SEEK_END );
long fsize = ftell(fp);
fseek( fp, 0, SEEK_SET );
printf("Filesize: %d\n", fsize );
test( prepare(fp, fsize, 1024) );
}
The #pragma push(pack,1) changes how the compiler aligns fields of the struct. We want them to be compact, for network transport. Using 1 is byte-aligned. The #pragma pop(pack) restores the previous setting of the pack pragma.
A note on the checksum method
First we calculate the sum of all the bytes in the packet:
int sum = 0, i;
for ( i = 0; i < len; i ++ )
sum += data[i];
Since the packet uses an unsigned short (16 bits, max value 65535 or 0xffff) to store the checksum, we make sure that the result will fit:
if ( sum > 0xffff ) // takes up more than 16 bits.
Getting the low 16 bits of this int is done using sum & 0xffff, masking out everything but the low 16 bits. We could simply return this value, but we would loose the information from higher checksum bits. So, we will add the upper 16 bits to the lower 16 bits. Accessing the higher 16 bits is done by shifting the int to the right 16 bits, like so: sum >> 16. This is the same as sum / 65536, since 65536 = 216 = 1 << 16.
sum = (sum & 0xffff) + (sum>>16); // add low 16 bits and high 16 bits
I should note that network packet checksums are usually computed 2 bytes (or 'octets' as they like to call them there) at a time. For that, the data should be cast to an unsigned short *, and len should be divided by 2. However! len may be odd, so in that case we'll need to take special care of the last byte. For instance, assuming that the maximum packet size is even, and that the len argument is always <= max_packet_size:
unsigned short * in = (unsigned short *) data;
if ( len & 1 ) data[len] = 0; // make sure last byte is 0
len = (len + 1) / 2;
The rest of the checksum method can remain the same, except that it should operate on in instead of data.
The following is extremely slow for what I need.
CFStringCreateWithFormat(NULL, NULL, CFSTR("%d"), i);
Currently this takes 20,000ns in my tests to execute on my 3gs. Perhaps that sounds fast, but I can create and release two NSMutableDictionaries in the time this executes. My C is weak, but there must be something equivalent to itoa that I can use on IOS.
This is the faster I can get:
CFStringRef TECFStringCreateWithInteger(NSInteger integer)
{
size_t size = 21; // long enough for 64 bits integer
char buffer[size];
char *characters = buffer + size;
*(--characters) = 0; // NULL-terminated string
int sign = integer < 0 ? -1 : 1;
do {
*(--characters) = '0' + (integer % 10) * sign;
integer /= 10;
}
while ( integer );
if ( sign == -1 )
*(--characters) = '-';
return CFStringCreateWithCString(NULL, characters, kCFStringEncodingASCII);
}
I have used the following code for converting the bigint in decimal to bytearray (raw data), but I'm getting wrong result.
What is the mistake here?
I'm trying this in Apple Mac ( for Iphone app)
COMP_BYTE_SIZE is 4
Is there any bigendian/ little endian issue, please Help.
void bi_export(BI_CTX *ctx, bigint *x, uint8_t *data, int size)
{
int i, j, k = size-1;
check(x);
memset(data, 0, size); /* ensure all leading 0's are cleared */
for (i = 0; i < x->size; i++)
{
for (j = 0; j < COMP_BYTE_SIZE; j++)
{
comp mask = 0xff << (j*8);
int num = (x->comps[i] & mask) >> (j*8);
data[k--] = num;
if (k < 0)
{
break;
}
}
}
Thanks.
The argument size is at least x->size*4, ie. the target array is big enough? Also use
comp mask = (comp)0xff << (j*8);
num should be cast to uint8_t before copy
data[k--] = (uint8_t)num;
Say I have a large number (integer or float) like 12345 and I want it to look like 12,345.
How would I accomplish that?
I'm trying to do this for an iPhone app, so something in Objective-C or C would be nice.
Here is the answer.
NSNumber* number = [NSNumber numberWithDouble:10000000];
NSNumberFormatter *numberFormatter = [[NSNumberFormatter alloc] init];
[numberFormatter setNumberStyle:kCFNumberFormatterDecimalStyle];
[numberFormatter setGroupingSeparator:#","];
NSString* commaString = [numberFormatter stringForObjectValue:number];
[numberFormatter release];
NSLog(#"%# -> %#", number, commaString);
Try using an NSNumberFormatter.
This should allow you to handle this correctly on an iPhone. Make sure you use the 10.4+ style, though. From that page:
"iPhone OS: The v10.0 compatibility mode is not available on iPhone OS—only the 10.4 mode is available."
At least on Mac OS X, you can just use the "'" string formatter with printf(3).
$ man 3 printf
`'' Decimal conversions (d, u, or i) or the integral portion
of a floating point conversion (f or F) should be
grouped and separated by thousands using the non-mone-
tary separator returned by localeconv(3).
as in printf("%'6d",1000000);
Cleaner C code
// write integer value in ASCII into buf of size bufSize, inserting commas at tousands
// character string in buf is terminated by 0.
// return length of character string or bufSize+1 if buf is too small.
size_t int2str( char *buf, size_t bufSize, int val )
{
char *p;
size_t len, neg;
// handle easy case of value 0 first
if( val == 0 )
{
a[0] = '0';
a[1] = '\0';
return 1;
}
// extract sign of value and set val to absolute value
if( val < 0 )
{
val = -val;
neg = 1;
}
else
neg = 0;
// initialize encoding
p = buf + bufSize;
*--p = '\0';
len = 1;
// while the buffer is not yet full
while( len < bufSize )
{
// put front next digit
*--p = '0' + val % 10;
val /= 10;
++len;
// if the value has become 0 we are done
if( val == 0 )
break;
// increment length and if it's a multiple of 3 put front a comma
if( (len % 3) == 0 )
*--p = ',';
}
// if buffer is too small return bufSize +1
if( len == bufSize && (val > 0 || neg == 1) )
return bufSize + 1;
// add negative sign if required
if( neg == 1 )
{
*--p = '-';
++len;
}
// move string to front of buffer if required
if( p != buf )
while( *buf++ = *p++ );
// return encoded string length not including \0
return len-1;
}
I did this for an iPhone game recently. I was using the built-in LCD font, which is a monospaced font. I formatted the numbers, ignoring the commas, then stuck the commas in afterward. (The way calculators do it, where the comma is not considered a character.)
Check out the screenshots at RetroJuJu. Sorry--they aren't full-sized screenshots so you'll have to squint!
Hope that helps you (it's in C) :
char* intToFormat(int a)
{
int nb = 0;
int i = 1;
char* res;
res = (char*)malloc(12*sizeof(char));
// Should be enough to get you in the billions. Get it higher if you need
// to use bigger numbers.
while(a > 0)
{
if( nb > 3 && nb%3 == 0)
res[nb++] = ',';
// Get the code for the '0' char and add it the position of the
// number to add (ex: '0' + 5 = '5')
res[nb] = '0' + a%10;
nb++;
a /= 10;
}
reverse(&res);
return res;
}
There might be a few errors I didn't see (I'm blind when it comes to this...)
It's like an enhanced iToA so maybe it's not the best solution.
Use recursion, Luke:
#include <stdio.h>
#include <stdlib.h>
static int sprint64u( char* buffer, unsigned __int64 x) {
unsigned __int64 quot = x / 1000;
int chars_written;
if ( quot != 0) {
chars_written = sprint64u( buffer, quot);
chars_written += sprintf( buffer + chars_written, ".%03u", ( unsigned int)( x % 1000));
}
else {
chars_written = sprintf( buffer, "%u", ( unsigned int)( x % 1000));
}
return chars_written;
}
int main( void) {
char buffer[ 32];
sprint64u( buffer, 0x100000000ULL);
puts( buffer);
return EXIT_SUCCESS;
}