I am building a tool to replay logs. Manually parsing the logs is annoying, so I'm wondering if there is a way to simply load a message from the log.
Also, I am not against just using a third-party replay tool if one exists.
First read the log file by any mean you want, getting the individual lines (there is one message per line).
Then build a Data Dictionary:
// Use the version of the XML dictionary that is right for you
FIX::DataDictionary dd("FIX44.XML");
Then, for each line (as std::string str), build a message:
FIX::Message msg(str, dd, false);
Finally, handle the message just like your FIX::Application does, or better, call
yourFixApplication.fromApp(msg, mySessionID);
ValidFIX Log analyzer is an online log parser that makes a good job:
http://www.validfix.com/fix-log-analyzer.html
Related
The message that is posted every time an execution fails is too verbose and creates too much noise. Is there a way to disable or hide it somehow? We have our own error messages within the scripts and don't need a red chunk of extra text displayed in the logs. Adding an error handler that will exit with code 0 is not an option because we still need the job to fail if a step fails.
That message is Rundeck standard output in case of failure (you can manipulate the loglevel but that doesn't affect the NonZeroResultCode last line, just the text from your commands/scripts), you can suggest that here. The rest are just workarounds as you mentioned.
This occurs even using the Quiet output.
I'm trying to use Casablanca to consume a REST api.
I've been following the microsoft tutorial, how ever i'm getting a crash and I cannot figure it out.
I'm using visual studio 2017 with C++11
I've codded a function GetRequest() that do work when used in a new empty project, but when I try to use it on my Project (Very big project with millions of code lines).
I'm crashing in the constructor of http_client, in the file xmemory0 line 118.
const uintptr_t _Ptr_container = _Ptr_user[-1];
This is a link to the callstack : https://i.imgur.com/lBm0Hv7.png
void RestManager::GetRequest()
{
auto fileStream = std::make_shared<ostream>();
// Open stream to output file.
pplx::task<void> requestTask = fstream::open_ostream(U("results.html")).then([=](ostream outFile)
{
*fileStream = outFile;
// Create http_client to send the request.
http_client client(U("XXX/XXX.svc/"));
// Build request URI and start the request.
uri_builder builder(U("/IsLive"));
builder.append_query(U("q"), U("cpprestsdk github"));
return client.request(methods::GET, builder.to_string());
})
// Handle response headers arriving.
.then([=](http_response response)
{
printf("Received response status code:%u\n", response.status_code());
// Write response body into the file.
return response.body().read_to_end(fileStream->streambuf());
})
// Close the file stream.
.then([=](size_t)
{
return fileStream->close();
});
// Wait for all the outstanding I/O to complete and handle any exceptions
try
{
requestTask.wait();
}
catch (const std::exception &e)
{
printf("Error exception:%s\n", e.what());
}
}
EDIT : I just want to add that the http_client constructor is the issue. It always crash inside it no matter what I send as parameter.
The wierd thing is that it's not crashing when i just make a main() that call this function.
I guess it must be due to some memory issues, however I have no idea how could I debug that.
Does anyone would have an idea about it?
Thanks and have a great day!
I've experienced a similar issue on ubuntu. It works in an empty project, but crashes randomly when put into an existing large project, complaining memory corruptions.
Turns out that the existing project loaded a proprietary library, which is using cpprestsdk (casablanca) internally. Even cpprestsdk is static linked, its symbols are still exported as Weak Symbols. So either my code crashes, or the proprietary library crashes.
Ideally, my project can be divided into several libraries, and load them with RTLD_LOCAL to avoid symbol clashes. But the proprietary library in my project only accept RTLD_GLOBAL, otherwise it crashes... So the import order and flags become important:
dlopen("my-lib-uses-cpprest", RTLD_LOCAL); //To avoid polluting the global
dlopen("proprietary-lib-with-built-in-cpprest", RTLD_GLOBAL); //In my case, this lib must be global
dlopen("another-lib-uses-cpprest", RTLD_DEEPBIND); //To avoid being affected by global
"it will probably never concern anyone."
I agree with that.
I guess this issues was very specific, and it will probably never concern anyone, but still I'm going to update on everything I found out about it.
On this project, we are using custom allocator, if i'm not wrong, it's not possible to give our custom allocator to this lib, which result to many random crash.
A good option to fix it would be to use the static version to this lib, however, since we are using a lot of dynamic lib, this option wasn't possible for us.
If you are on my case, I would advice to use the libcurl and rapidjson, it's a bit harder to use, but you can achieve the same goal.
I'm looking to fetch recorded data using LogBook in a custom Movesense firmware. How do I get the correct byte stream offset for the next GET call when receiving HTTP_CONTINUE?
I'm trying to implement these steps as described in DataStorage.md:
### /Logbook usage ###
To get recording from the Movesense sensors EEPROM storage, you need to:
1. Do **GET** on */Logbook/Entries*. This returns a list of LogEntry objects. If the status was HTTP_OK, the list is complete. If the result code is HTTP_CONTINUE, you must GET again with the parameter StartAfterId set to the Id of the last entry you received and you'll get the next entries.
2. Choose the Log that you are interested in and notice the Id of it.
3. Fetch the descriptors with **GET** to */Logbook/byId/<Id>/Descriptors*. This returns a bytestream with the similar HTTP_CONTINUE handling as above. However you **must** keep re-requesting the **GET** until you get error or HTTP_OK, or the Logbook service will stay "in the middle of the stream" (we hope to remove this limitation in the future).
4. Fetch the data with **GET** to */Logbook/byId/<Id>/Data*. This returns also a bytestream (just like the */Logbook/Descriptors* above).
5. Convert the data using the converter tools or classes. (To Be Continued...)
The problem is basically the same for step 3 and 4. I receive a whiteboard::ByteStream object in the onGetResult callback function but I don't know how to get the correct offset information from it.
I've found a number of different methods seemingly concerning different aspects of number of bytes in ByteStream.h (length, fullSize, transmitted, payloadSize and serializationLength) but I just can't get it working properly.
Basically I would like to do something like this in onGetResult:
if (resultCode == whiteboard::HTTP_CODE_CONTINUE) {
const whiteboard::ByteStream &byteStream = rResultData.convertTo<const whiteboard::ByteStream &>();
currentEntryOffset += byteStream.length();
asyncGet(WB_RES::LOCAL::MEM_LOGBOOK_BYID_LOGID_DESCRIPTORS(), AsyncRequestOptions::Empty, currentEntryIdToFetch, currentEntryOffset);
return;
}
The basic idea is to do the same call again.
So if you do:
asyncGet(WB_RES::LOCAL::MEM_LOGBOOK_BYID_LOGID_DESCRIPTORS(),AsyncRequestOptions::Empty, currentEntryIdToFetch);
and get the response HTTP_CONTINUE, do:
asyncGet(WB_RES::LOCAL::MEM_LOGBOOK_BYID_LOGID_DESCRIPTORS(),AsyncRequestOptions::Empty, currentEntryIdToFetch);
Until you get HTTP_CONTINUE or an error.
If the result code is HTTP_CONTINUE, you must GET again with the parameter StartAfterId set to the Id of the last entry you received and you'll get the next entries.
Might be a bit cryptic but do another asyncGet to the exact same resource until you get HTTP_OK or an http error code.
Also, note that you need to decode the data, a python script can be found here in this answer
I am trying to make a patch that plays audio when a bang is pressed. I have put a symbol so that I don't need to keep reimporting the file. However it works sometimes but not all the time.
A warning in the Pd console reads: Start requested with no prior open
However I have imported an audio file
Is there something that I have done wrong?
Use [trigger] to get the order-of-execution correct.
One problem is, that whenever you send a [1( to [readsf~] you must have sent an [open ...( message directly beforehand.
Even if you have just successfully opened a file, but then stopped it (with [0() or played it through (so it has been closed automatically), you have to send the filename again.
The real problem is, that your messages are out of order: you should never have a fan-out (that is: connecting a message outlet to multiple inlets), as this will create undefined behavior.
Use [trigger] to get the order-of-execution correct.
(Mastering [trigger] is probably the single most important step in learning to program Pd)
How would I validate that a jpg file is a valid image file. We are having files written to a directory using FTP, but we seem to be picking up the file before it has finished writing it, creating invalid images. I need to be able to identify when it is no longer being written to. Any ideas?
Easiest way might just be to write the file to a temporary directory and then move it to the real directory after the write is finished.
Or you could check here.
JPEG::Error
[arguments: none] If the file reference remains undefined after a call to new, the file is to be considered not parseable by this module, and one should issue some error message and go to another file. An error message explaining the reason of the failure can be retrieved with the Error method:
EDIT:
Image::TestJPG might be even better.
You're solving the wrong problem, I think.
What you should be doing is figuring out how to tell when whatever FTPd you're using is done writing the file - that way when you come to have the same problem for (say) GIFs, DOCs or MPEGs, you don't have to fix it again.
Precisely how you do that depends rather crucially on what FTPd on what OS you're running. Some do, I believe, have hooks you can set to trigger when an upload's done.
If you can run your own FTPd, Net::FTPServer or POE::Component::Server::FTP are customizable to do the right thing.
In the absence of that:
1) try tailing the logs with a Perl script that looks for 'upload complete' messages
2) use something like lsof or fuser to check whether anything is locking a file before you try and copy it.
Again looking at the FTP issue rather than the JPG issue.
I check the timestamp on the file to make sure it hasn't been modified in the last X (5) mins - that way I can be reasonably sure they've finished uploading
# time in seconds that the file was last modified
my $last_modified = (stat("$path/$file"))[9];
# get the time in secs since epoch (ie 1970)
my $epoch_time = time();
# ensure file's not been modified during the last 5 mins, ie still uploading
unless ( $last_modified >= ($epoch_time - 300)) {
# move / edit or what ever
}
I had something similar come up once, more or less what I did was:
var oldImageSize = 0;
var currentImageSize;
while((currentImageSize = checkImageSize(imageFile)) != oldImageSize){
oldImageSize = currentImageSize;
sleep 10;
}
processImage(imageFile);
Have the FTP process set the readonly flag, then only work with files that have the readonly flag set.