Anybody have experience or thoughts on a scheduler to couple with a persistent Gearman queue?
Obviously there's crontab available, but i'd like some more functionality. We're looking at Java Quartz.
Please weigh in if you've done this before in any capacity!
While quartz is quite capable of handling this, it is a pretty large package so you might want to get started a bit lighter at least at first. I have been exploring using either cron or at for scheduling and I'm leaning toward "at". I've never found a Unix-like os that didn't have at.
I'm using the MySQL Event Scheduler. Full timezone support, etc. Gearman has MySQL UDF's you can use.
Related
Although I have implemented many projects in C, I am completely new to operating systems. I tried real time linux on Discovery board (STM32) and got the correct results for blinking LED but I didn't really understand the whole process since I just followed the steps and could not find whole description for each step on the internet.
I want to implement scheduling on real time linux. What is the best way to start? Any sites, books, tutorials available?
Complete RTLinux process description will be appreciated.
Thanks in adv.
The transition from "bare metal" to OS based programming is something that I experienced in reverse. I started out a complete software guy, totally into the OS side of things and over time I have moved to the opposite of that (even designing circuits in VHDL!). My advice would be to start simple. Linux is pretty complex, and everywhere you look there are many layers of things all working together to deliver the final product. If you are dead set on a real time linux extension, I'd be happy to suggest https://xenomai.org/ which is a real time extension for linux.
However, to more specifically address your question about implementing scheduling in Linux, you can, but it will be a large amount of work and can be very complicated. The OS uses a completely fair scheduling process ( http://en.wikipedia.org/wiki/Completely_Fair_Scheduler ) and whenever you spin up a thread, it simply gets added to the list to run. This can differ slightly if you implement your code in kernel space as a driver, rely on hardware interrupts, etc., but in general, this is how Linux works. Real time generally means that it has the ability to assign threads one of several different priorities and utilize thread preemption fully at any given time which are concepts that aren't really a part of vanilla Linux. It has some notion of this, but it has limitations that can cause problems when you are looking for real time behavior from Linux.
What may be helpful to you is an RTOS. If you are looking for a full on Real Time Operating System, check out FreeRTOS http://www.freertos.org/ . It has a large community and supports a lot of different devices out of the box with a large amount of example code. They even support your specific board with an example package, so you can give it a shot with nothing to lose! http://www.freertos.org/FreeRTOS-for-Cortex-M3-STM32-STM32F100-Discovery.html . It gives you access to many OS ish constructs like network APIs, memory management, and threading without the overhead and latency of a huge OS. With an RTOS, you create tasks and assign them priorities so you become the scheduler and are no longer at the mercy of the OS. You run the OS, not the OS runs you (if that makes sense). Plus, the constructs offered within an RTOS will feel much like bare metal code and thus will be much easier to follow, understand, and fully learn. It is a more simple world to learn the base building blocks of a full blown OS such as Linux or Windows. If this option sounds good, I would suggest looking through the supported devices on FreeRTOS website and picking one you would like to experiment with and then go for it. I would highly recommend this as a way to learn about scheduling and OS constructs in general as it is as simple as you can get and open source. Once you have the basics of an RTOS down, buying a book about Linux specifically wouldn't be a bad idea. Although there are many free resources on the web related to learning about Linux, they are commonly contradictory, and can be misleading. Pile on learning Linux specific knowledge along with OS in general, and it can feel overwhelming. Starting simpler will help keep you from getting burnt out and minimize the amount of time you spend feeling lost. Linux is definitely a learning process, but like with any learning process, start simple, keep your ultimate goal in mind, make a plan, and take small, manageable steps along that plan until you look up and find yourself exactly where you want to be. Then go tackle the next mountain!
The real-time Linux landscape is quite confusing. 99.99% of the information out there is just plain obsolete.
First, there are lots of "microkernels" that run Linux as one task. (Such as the defunct RTLinux). The problem is that you must write your real-time task to a different API, and can't depend on anything in Linux, because Linux will be frozen in the background while your task runs. So unless your task is dead-simple ("stop the motors when I press this button"), this approach will cause more pain than gain.
Next, there is the realtime Linux patch set. This hasn't been doing so well. because of the next item:
Lastly, the current Linux kernel has gotten rid of the problems that caused people to need realtime in the past. You can even turn off Linux on one of your processors to have full control of the CPU. See also this paper.
To answer your question: I see two different paths you could take:
1) Start with a normal 3.xx Linux kernel and explore the various APIs and realtime techniques (i.e. realtime priorities, memory pinning, etc.) This can get you "close enough" for 99% of what people want "realtime" for. If it's good enough for high frequency trading, it's probably good enough for you.
2) If you have a hard realtime requirement and you are worried that Linux won't cut it, then (as Nick mentioned above), just go buy a processor and write your realtime code with no OS. By splitting up your "realtime" and "non-realtime" code onto different CPUs, you will make the whole system simpler and much more robust.
If you want to learn real-time operating systems then I suggest that you get an FPGA, for example the Altera DE2, and experiment with your own operating system and ucos. You can read a good text about embedded RTOS here.
You could also get a Linux Raspberry and write your own operating system for that.
I would like to run Erlang application on a centOS server for stress testing. I would like to know what I must accomplish to deploy a Erlang application before I send it to the server admin to install and run.
What will I need to tell my admin to properly run an Erlang app on a server? (Basically how can I send him a file and give him an really stupid easy instructions to install it?)
Is their a way to send my app in a executable format that the admin can easily install with? Kinda like PyPy?
How can I perform code swapping? Will the app need to be in it's code state still in order to do this?
I would appreciate any help to make this process easy. Thank you!
For anyone just passing by: Even though this is my first Erlang app, I highly recommend anyone wanting to build a easy to scale server user Erlang. The benefits are amazing, if you know how to use Erlang properly.
Cheers!
The usual way of deploying an Erlang program (I'm avoiding the word application here, as it's somewhat ambiguous in an Erlang context) involves building a release (see this for some background). A release is essentially a packaging of your Erlang code (which often consists of multiple applications - hence the ambiguity) and the parts of the runtime system required to run it. The Erlang release system was designed with code upgrades in mind.
If you are using Rebar to build your program, you may find this article helpful. In addition to describing how to build a release (the process is a bit tricky) it also explains how you can deploy an upgrade.
A similar guide is also available for users of erlang.mk. If you're not using one of them already, or a comparable build tool, I strongly encourage you to do so as it makes release handling much less painful.
You may also find the answers to this question helpful. They list some additional options.
I am thinking of writing my own code (in order to practice and also to serve my company need) to monitor my 300 routers. I want to do the following
1) icmp monitoring. Email when packet loss is above 60%
2) Latency monitoring, email when latency is above 30 ms.
Now quite a time ago, when i asked similar question on cisco forum, they recommended me perl and freebsd. From that time it was in my mind that perl can do it. But i am just confused what approach shall i take. Do i need to do some multi threaded programming here ? as you know i am a little noob here, so if someone can pls guide me a little how to do it. I know there are enterprise class NMS for this but still i thought writing my own would be a good learning experience for me. Also perl is not must but i just thought perl would do it.
EDIT: I am sorry but a dear friend pointed me my mistake. Can someone tell me can i use multithreaded programming in perl ?
Thanks
Yes, multi-threaded programming is possible with e.g. threads, threads::lite and Coro.
Threads are not the only means of concurrent programming.
Perl is much stronger with the traditional Unix mechanisms building on the fork system call. Nice wrapper modules include Proc::Fork, Forks::Super and Parallel::ForkManager.
Another way to solve your problem is employing an event loop, such as AnyEvent, Reflex or POE.
I have looked online at making a multi-threaded server in perl, but i cannot find any good ones that clearly explain how to create one. I have created a server and client in perl but i do not seem to be able to make it multi-threaded. Can someone help me?
Assuming you are using the Net::Server CPAN module (rather than rolling your own), you could make use of Net::Server::Coro to build a multi-threded server. Note that Net::Server can handle (pre)forking (i.e. multi-processes) servers and these may be easer to use.
I think POE might be what you're looking for.
See the POE Cookbook for examples.
Hi guys we are on the way to start developing a big web platform. For db server we choosen postgresql. Would you suggest an OS for the postgresql server (we are looking for the maximum performance)?
Thanks
P.S. sorry for the bad english
I would suggest a platform that you feel comfortable with. As Jeff suggested, it is usually easier to throw faster hardware at a problem than human time.
This reasoning is based on theses main ideas :
Usually the database is only marginally faster given different OS.
The high order optimisations are usually in tuning the database or the requests. Not really in switching OS.
If you have more knowledge on a OS, you can take usually more juice out from it. Whereas if you take an OS that you are not really familiar with, but that is supposed to be faster, it might kick your back in unexpected ways.
That said, as answered before an *NIX-based OS would be better right now, since PostgreSQL has still deep roots in a *NIX world. But this is becoming less and less an issue with the 8.x line.
I would suggest *nix based, Linux would be great if it is possible because you can get the package easier with the built-in package manager (e.g apt for debian, yum for fedora, etc). Because Postgres is originally made for *nix based OS. The port to windows is only recently and as you can see on several threads here on Stackoverflow, Postgres does not perform as good on Windows as it is on *nix based OS.