I have two web requests which I need to poll to find out when they return. Ideally I don't want to keep testing them in a tight loop. I would like to free up the CPU so other processes can execute.
I'm currently using Perl's Time::HiRes::sleep(0.100) function to release the CPU before testing whether or not the web requests have returned.
During testing under load I can see that the sleep duration 'stretches'. Ideally I want to make sure that the sleep duration is adhered to but that CPU is freed up. Should I be calling a different function to achieve this?
I'm coding Perl on Linux 2.6.
Rather than polling, see if you can't get file-descriptors and do a select call.
Then you'll get control back as soon as anything happens, without occupying the CPU at all.
Somewhere in the web-request will be some sockets, and attached to the sockets will be file-descriptors that you can use in select.
In any case your program can be interrupted at any point for any amount of time; if this is a real problem you need a real-time operating system, but since you're dealing with web-requests I doubt you need that level of responsiveness.
In fact what you want is a high level interface that does the select call for you. As suggested in the comments: http://search.cpan.org/dist/HTTP-Async/ looks like it'll do precisely what you need.
It sounds like you really want an event loop. There is
POE,
EV, and abstraction layers over
both.
Either way, don't implement this yourself. This wheel has already
been invented.
I don't think sleep duration can be guaranteed on regular Linux. That's pretty much the point of a "Real Time" operating system, and regular Linux is not "Real Time."
I agree with #Douglas Leeder: use a select call to have the kernel notify you when something changes. You can also emulate sub-second sleeps with a select call, but Time::HiRes is a cleaner interface (and you're still not going to avoid stretching the wait).
Related
From what I understand the main q thread monitors it socket descriptors for requests and respond to them.
I want to use a while loop in my main thread that will go on for an indefinite period of time. This would mean, that I will not be able to use hopen on the process port and perform queries.
Is there any way to manually check requests within the while loop.
Thanks.
Are you sure you need to use a while loop? Is there any chance you could, for instance, instead use the timer functionality of KDB+?
This could allow you to run a piece of code periodically instead of looping over it continually. Depending on your use case, this may be more appropriate as it would allow you to repeatedly run a piece of code (e.g. that could be polling something periodically), without using the main thread constantly.
KDB+ is by default single-threaded, which makes it tricky to do what you want to do. There might be something you can do with slave threads.
If you're interested in using timer functionality, but the built-in timer is too limited for your needs, there is a more advanced set of timer functionality available free from AquaQ Analytics (disclaimer: I work for AquaQ). It is distributed as part of the TorQ KDB framework, the specific script you'd be interested in is timer.q, which is documented here. You may be able to use this code without the full TorQ if you like, you may need some of the other "common" code from TorQ to provide functions used within timer.q
I have yet to find a definition of an RTOS that is specific enough to have meaning. The best one I can find is on wiki:
https://en.wikipedia.org/wiki/Real-time_operating_system
However I have some critical comments/questions:
"Real Time" seems to be undefined in all the definitions for RTOS I've found. Nothing can be fast as actual real time (infinitesimally small!). Therefore, I believe "real time" only makes sense in the context of the observer. Real time for a human using an iPhone user might be <20ms because human eye sight cannot detect changes faster than that. For an air bag deployment it might be <1ms. All definitions on the internet seem to gloss over the definition of "real time"!
If RTOS is defined by the requirement to execute something within a specific time frame ("deadline"), why does jitter come into the definition? If the iPhone response jitters between 12-14ms, is it no longer responding in real time? It meets the 20ms requirement, right? If one time the response went to 100ms, the user might notice, at which point the system is not an RTOS
How can there possibly be a "soft" RTOS?! The definition of RTOS is meeting a particular deadline time requirement. If it doesn't meet it, than its not an RTOS! The very definition of RTOS prohibits a "soft" RTOS
To me it seems there is no formal and precise definition of RTOS. It's a general term to explain the characteristic of an OS who's main priority is the appearance of "real time" (per requirement number) to a particular type of observer. It also seems like the name has taken on implementation meaning such as how things are processed, multi-tasking, message passing, semaphores, etc... all which may NOT be part of an RTOS at all if the system fails to respond within the "deadline" requirement, right?
Sorry about such a ubiquitous question, but I can't get a clear picture in my brain. All definitions I've found are simply not precise enough or cloud the definition with implementation details.
You're right that no definition defines the exact time bounds. That's not the goal of a definition. Real time isn't dependent on the observer, though, but the application. As applications differ, time bounds differ, and therefore a definition cannot give that bound as a number.
Jitter is irrelevant as long as the application's time bound is met. You're absolutely right about the example. If the deadline is 20 ms, taking 100 ms is a failure. If the OS is to blame for the delay, it's not an RTOS.
"Soft realtime" has a very specific meaning, and this is probably the only thing you really got wrong. The concept at work here is, what do you do when a task exceeds its deadline? (Note: this could be either the fault of the task itself or the RTOS.) In a hard realtime system, the task simply has no value anymore. A late outcome is as good as no outcome, and you cancel the task. No point in risking other tasks.
Soft RTOS is actually more complex. Finishing the task still has value, although diminished. So the RTOS cannot hard kill the task, but the OS still has to ensure other tasks meet their deadlines. That requires extra care, which wouldn't have been necessary if you'd just kill the task.
There is an Embedded Systems Dictionary. Here are some excerpts:
real-time adj. Having timeliness requirements, typically in the form of deadlines that can’t be missed.
real-time operating system n. An operating system designed specifically for use in real-time systems. Abbreviated RTOS.
real-time system n. Any computer system, embedded or otherwise, that has timeliness requirements. The following question can be used
to distinguish real-time systems from the rest: “Is a late answer as
bad, or even worse, than a wrong answer?” In other words, what happens
if the computation doesn’t finish in time? If nothing bad happens,
it’s not a real-time system. If someone dies or the mission fails,
it’s generally considered “hard” real-time, which is meant to imply
that the system has hard deadlines. Everything in between is “soft”
real-time.
in short my questions are:
Is there anything built-in into drools that allows/facilitates detection of endless loops?
Is there a way to programmatically halt sessions (e.g. for the case of a detected endless loop)?
More details:
I'm planning to have drools (6.2 or higher) run within a server/platform where users will create and execute their own rules. One of the issues I'm facing is that carelessly/faulty rule design can easily result in endless loops (whether its just a forgotten "no-loop true" statement or the more complex rule1 triggers rule2 triggers rule3 (re)triggers rule1 circles that lead to endless loops.
If this happens, drools basically slows down my server/platform to a halt.
I'm currently looking into how to detect and/or terminate sessions that run in an endless loop.
Now as a (seemingly) endless loop is nothing that is per-se invalid or in certain cases maybe even desired I can imagine that there is not a lot of built-in detection mechanism for this case (if any). But as I am not an expert I'd be happy to know if there is anything built-in to detect endless loops?
In my use case I would be ok to determine a session as "endlessly looped" based on a threshold of how often any rule might have been activated.
As I understand I could use maybe AgendaEventListeners that keep track of how often any rule has been fired and if a threshold is met either insert a control fact or somehow trigger a rule that contains the drools.halt() for this session.
I wonder (and couldn't find a lot of details) if it is possible to programmatically halt/terminate sessions.
I've only come across a fireUntilHalt() method but that didn't seem like the way to go (or I didnt understand it really).
Also, at this point I was only planning to use stateless session (but if it's well encapsulated I could also work with stateful sessions if that makes my goal easier to achieve).
Any answers/ideas/feedback to my initial approach is highly welcome :)
Thanks!
A fundamental breaking point of any RBS implementation is created where the design lets "users create and design their own rules". I don't know why some marketing hype opens the door for non-programmers to write what is program code, without any safeguarding.
Detecting whether a session halts is theoretically impossible. Google "Halting problem".
For certain contexts you might come up with a limit of the number of rules that might be executed at most or something similar. And you can use listeners to count and raise an exception, etc etc.
Basically you have very bad cards once you succumb to the execution of untested code created by amateurs.
Is it actually important for a programmer to know if the standard library function he/she is using is actually executing a system call? If so, why?
Intuitively I'm guessing the only importance is in knowing if the general standard function is a library function or a system call itself. In other cases, I'm guessing there isn't much of a need to know if a library functions uses internally a system call?
It is not always possible to know (for sure) if a library function wraps a system call. But in one way or another, this knowledge can help improve the portability and (or) efficiency of your program. At least in the following two cases, knowing the syscall-level behaviours of your program is helpful.
When your program is time critical. Some system calls are expensive, and the library functions that wrap them are even more expensive. Thus time-critical tasks may need to switch to equivalent functions that do not enter kernel space at all.
It is also worth noticing the vsyscall (or vdso) mechanism of linux, which accelerates some system calls (i.e. gettimeofday) through mapping their implementations into user-space memory. See this for more details.
When your program needs to be deployed to some restricted environments with system call auditing. In order for your programs to survive such environments, it could be necessary to profile your program for any potential policy violations, or perhaps less tough if you are aware of the restrictions when you wrote the program.
Sometimes it might be important, and sometimes it isn't. I don't think there's any universal answer to this question. Reasons I can think of that might be important in some contexts are: if the system call requires user permissions that the user might not have; in performance critical code a system call might be too heavyweight; if you're writing a signal-handler where most system calls are forbidden; if it might use some system resource (e.g. reading from /dev/random for every random number could use up the whole entropy pool - you'd want to know if that's going to happen every time you call rand()).
The system time substraction may be one method, but it includes the running time for all the tasks/threads, the function is only in one thread of one task.
Instrument in Xcode may be another method, but how to measure the time for a specific function?
You need to understand a few things here: firstly, the concept of a 'CPU cycle' isn't very useful. It's in fact fairly meaningless. You're never going to get an accurate result. You can use valgrind to get detailed output regarding the number of instructions being executed, and in theory (that's a big 'in theory') you could use this information to derive cycle counts. Realistically it's impossible, and not worth the effort.
One would have to ask why you'd want to find this out in the first place.