Can a hypervisor with paravirtualized guests benefit from virtualization extensions? - virtualization

A hypervisor developer claimed that, even though they only support paravirtualized guests, the virtualization would still be faster on a platform with hardware virtualization extensions.
Does this make sense? And through which mechanism could there even be a performance gain?

Yes. In a very abstract sense, there are two things a VMM needs to do: allow the guest to do things it needs to, and prevent it from doing things it shouldn't. Paravirtualization solves the first part. Hardware virtualization extensions are very effective at solving the second part.

Related

Does an operating system design for specific processors or all types of processors?

I need to know Does a operating system design for specific processors category?
and also can any operating system run on any microprocessor?
Generally speaking, an operating system is not designed for a specific processor; though some do make assumptions about the hardware and computer system over all that might not be available in all systems. That said, for an operating system to run on a partical architecture, there is usually code that performs some specific, critical functions that is implemented for a specific architecture, frequently being written in assembly (I know of no OS that doesn't do this). To enable a new architecture, this code needs to be rewritten for the new machine, so that means new assembly most of the time. As mentioned in the comments, there are operating systems that only run on a single architecture like Windows, while others have these specific components for a number of architectures and thus can run on a number of processors like Linux. Note however the same exactly binary will not run across architectures, the operating system needs to be rebuilt for each architecture and possibly even for the same architecture if the system itself is different enough (as can be the case with some small MCUs).
So to answer your two questions directly: no, an OS is not uaually designed for a specific processor, and no, any OS cannot run on any processor.
Historically, operating systems have been designed for specific hardware. In some cases, such as eunuchs, the system was reworked so that it could be ported to multiple systems.
M$ ported Windoze to the Alpha processor in order to placate Digital and avoid lawsuits.
[C]an any operating system run on any microprocessor?
No.

UML equivalent for one of the BSDs?

On Linux there exists a thing called a User Mode Linux kernel, which is a kernel built in such a way that it will run as an ordinary, unprivileged user process (not even needing root). It's a cheap and easy virtualisation method that'll even run on non-Linux platforms (e.g. CoPilot is based around UML built for Windows).
Does such a thing exist for the BSD world? I don't mind which OS.
I've had a look myself but 'user mode bsd' is a completely ungoogleable term. I've discovered NetBSD rump kernels, but they seem to be solving a different problem --- it looks like it's not possible to run arbitrary process trees on them.
I think DragonFly BSD's vkernel is what you are looking for.
NetBSD has usermode too. I don't know how to configure/build it.
Look :
http://mail-index.netbsd.org/current-users/2015/02/06/msg026632.html
The best true virtualisation tool in the NetBSD world is Xen.
It's not all that difficult to set up any more either. Don't get put off by the length and volume of information in the "how-to" -- the basics boil down to a few simple steps.
NetBSD/xen HowTo
In the FreeBSD world I've heard good things about The BSD Hypervisor
DragonFly BSD's vkernel is indeed similar to User-Mode-Linux, but from what I've heard it wouldn't be very suitable as a "cheap and easy" virtualisation method. It is actually intended for the same types of uses as NetBSD's rump kernels, though it does seem to go so far as supporting a full virtual OS environment. The performance will likely be pitiful though.
It probably wouldn't hurt to pester the VirtualBox folks about supporting VBox as an application on the BSDs too, but don't hold your breath waiting for it to happen.
On FreeBSD you have jails. They basically are a (almost) no-overhead virtual machine running on the same kernel as the host.

what operating system concepts should every programmer be aware of? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am compiling various lists of competencies that self taught programmers must have.
Among all subjects, Operating Systems is the trickiest one, because creating even a toy operating system is a rather non-trivial task. However, at the same time an application developer (who may not have formally learned CS) must at least be aware of and hopefully should have implemented some key concepts to appreciate how an OS works, and to be a better developer.
I have a few specific questions:
What key concepts of operating systems are important for a self taught programmer to understand so they can be better software developers (albeit working on regular application development)?
Is it even remotely possible to learn such a subject in byte sized practical pieces ? (Even a subject like compiler construction can be learned in a hands on way, at a rather low level of complexity)
I would suggest reading Andrew S. Tanenbaum ( http://en.wikipedia.org/wiki/Andrew_S._Tanenbaum ) book on Modern Operating Systems (ISBN 978-0-13-600663-3) as everything is there.
However from the book index we can identify the minimum key topics:
Processes
Memory management
File systems
Input/output
And the easiest way to start playing with this topics will be to download MINIX:
http://www.minix3.org/
and study the code. Older versions of this operating system might be easier to understand.
Another useful resource is Mike Saunders How to write a simple operating system that shows you how to write and build your first operating system in x86 assembly language:
http://mikeos.sourceforge.net/write-your-own-os.html
Every OS designer must understand the concepts behind Multics. One of the most brilliant ideas is the notion of of a vast virtual memory partioned into directly readable and writable segments with full protections, and multiprocessor support to boot; with 64 bit pointers, we have enough bits to address everything on the planet directly. These ideas are from the 1960s yet timeless IMHO.
The apparent loss of such knowledge got us "Eunuchs" now instantiated as Unix then Linux and an equally poor design from Microsoft, both of which organize the world as a flat process space and files. Those who don't know history are doomed to doing something dumber.
Do anything you can to get a copy of Organick's book on Multics, and read it, cover to cover. (Elliott I. Organick, The Multics System: An Examination of Its Structure).
The wikipedia site has some good information; Corbato's papers are great.
I believe it depends on the type of application you are developing and the OS platform you are developing for. For example if you are developing a website you don't need to know too much about the OS. In this example you need to know more about your webserver. There are different things you need to know when you are working on Windows, Linux or Android or some embedded system or sometimes you need to know nothing beyond what your API provides. In general it is always good for a developer or CS person to know following.
What lies in the responsibility of application, toolchain and then OS.
Inter process communication and different IPC mechanism the OS system calls provides.
OS is quite an interesting subject but mostly consist of theory but this theory comes to action when you working on embedded systems. On average for desktop applications you don't see where all that theory fits in.
Ok, operating system concepts that a good programmer should be aware of.
practically speaking. Unless you are concerned about performance. If you are writing in a cross os language. None.
If you care about performance.
The cost of user/system transitions
How the os handles locking/threads/deadlocks and how to best use them.
Virtual Memory/Paging/thrashing and the cost thereof.
Memory allocation, how the os does it, and how you should take advantage of that to when A, use the OS allocator ( see 1) and when to allocate from the os and sub allocate.
As earlier put, process creation/ and inter process communication.
How the os writes/reads to disk by default to read/write optimally ( see why databases use B-trees)
Bonus, sub-os, what cache size and cache lines can mean to you in terms of performance.
but generally it would boil down to what does the OS provide you that isn't generic, and what and why does it cost, and what will cost too much ( too much cpu, too much disk usage, too much io, too much network ect).
Well that depends on the need of the developer like:-
Point.
Applications such as web browsers and email tools are
performing an increasingly important role inmodern desktop computer
systems. To fulfill this role, they should be incorporated as part of the
operating system. By doing so, they can provide better performance
and better integration with the rest of the system. In addition, these
important applications can have the same look-and-feel as the operating
system software.
Counterpoint.
The fundamental role of the operating system is to manage
system resources such as the CPU, memory, I/O devices, etc. In addition,
it’s role is to run software applications such as web browsers and
email applications. By incorporating such applications into the operating
system, we burden the operating system with additional functionality.
Such a burdenmay result in the operating system performing a less-thansatisfactory
job at managing system resources. In addition, we increase
the size of the operating system thereby increasing the likelihood of
system crashes and security violations.
Also there are many other important points which one must understand to get a better grip of Operating System like Multithreading, Multitasking, Virtual Memory, Demand Paging, Memory Management, Processor Management, and more.
I would start with What Every Programmer Should Know About Memory. (Not completely OS, but all of it is useful information. And chapter 4 covers virtual memory, which is the first thing that came to mind reading your question.)
To learn the rest piecemeal, pick any system call and learn exactly what it does. This will often mean learning about the kernel objects it manipulates.
Of course, the details will differ from OS to OS... But so does the answer to your question.
Simply put:
Threads and Processes.
Kernel space/threads vs user space/threads (probably some kernel level programming)
Followed by the very fundamental concepts of process deadlocks.
And thereafter monitors vs semaphores vs mutex
How Memory works and talks to process and devices.
Every self-taught programmer and computer scientist alike should know the OSI model and know it well. It helps to identify where a problem could lie and who to contact if there are problems. The scope is defined here and many issues could be filtered out here.
This is because there is just too much in an operating system to simply learn it all. As a web developer I usually work in the application level when an issue ever goes out of this scope I know when i need help. Also many people simply do not care about certain components they want to create thing as quickly as possible. The OSI model is a place where someone can find their computer hot spot.
http://en.wikipedia.org/wiki/OSI_model

Understanding Virtualization

I want to know about Virtualization in detail. But start from basics, like what is virtualization, with real world examples and scenarios. When ever I search, i got Virtualization technologies, but I want to learn and in fact know what exactly is virtualization, its types and all...
Please help me in getting practical details of term "Virtualization"
Virtualization is a big topic, but very roughly speaking there are three main levels of virtualization:
OS virtualization with a type 1 hypervisor
Virtual machine that run on an existing hardware/OS, e.g. VMWare
High-level virtual machines, e.g. JVM
I would recommend this book: Virtual Machines, by J. E. Smith and R. Nair.
It covers pretty much all virtual execution environments: process virtualization, high level virtual machines, system virtual machines, multiprocessor virtualization, etc.
Note that depending on the level which you are interested in, it gets pretty much related to system administration / hardware, so you might also ask the question on serverfault.com.
The question is pretty generic so it can lead to different replies, but I think that you first need to define and understand what is an actual hypervisor and why it is used:
The hypervisor is used to do the abstraction between the physical and the virtual resources. In other words, it is responsible to create a virtualized layer and share the hardware components (CPU, RAM, NIC, storage) to the virtual machines that will be used. Therefore, it is a lot simpler to centralize and manage multiple sources of heterogeneous elements. It is possible to deploy two types of hypervisors:
Type 1 Hypervisor: There is no actual operating system installed on the bare metal server except the software used to deploy the hypervisor. It is especially used in an enterprise context considering that you maximize the resources to share (you do not have the limitation of running an underlying operating system), but it also possible to deploy one in a home lab if you have the required hardware. A classic example is a VMWare ESXi host.
Type 2 Hypervisor: It consists of installing the hypervisor on top of the actual operating system (Windows, Linux, MacOS). It is used especially for testing, deploying simple services and to extend your software capabilities (you can run multiple different operating systems simultaneously). A classic example is Oracle’s Virtual Box.
In your case, I think that the best choice to actually learn about virtualization would be to download a type 2 Hypervisor (Virtual Box or VMWare Workstation Player for instance) in order to deploy a few multiple virtual machines and gain your first hands-on experience.
Specifically, try to explore the different virtual network and storage configurations that are available to differentiate every component.

Comparison of embedded operating systems?

I've been involved in embedded operating systems of one flavor or another, and have generally had to work with whatever the legacy system had. Now I have the chance to start from scratch on a new embedded project.
The primary constraints on the system are:
It needs a web-based interface.
Inputs are required to be processed in real-time (so a true RTOS is needed).
The memory available is 32MB of RAM and FLASH.
The operating systems that the team has used previously are VxWorks, ThreadX, uCos, pSOS, and Windows CE.
Does anyone have a comparison or trade study regarding operating system choice?
Are there any other operating systems that we should consider? (We've had eCos and RT-Linux suggested).
Edit - Thanks for all the responses to date. A pity I can't flag all as "accepted".
I think it would be wise to evaluate carefully what you mean by "RTOS". I have worked for years at a large company that builds high-performance embedded systems, and they refer to them as "real-time", although that's not what they really are. They are low-latency and have deterministic schedulers, and 9 times out of 10, that's what people are really after when they say RTOS.
True real-time requires hardware support and is likely not what you really mean. If all you want is low latency and deterministic scheduling (again, I think this is what people mean 90% of the time when they say "real-time"), then any Linux distribution would work just fine for you. You could probably even get by with Windows (I'm not sure how you control the Windows scheduler though...).
Again, just be careful what you mean by "Real-time".
It all depends on how much time was allocated for your team has to learn a "new" RTOS.
Are there any reasons you don't want to use something that people already have experience with?
I have plenty of experience with vxWorks and I like it, but disregard my opinion as I work for WindRiver.
uC/OS II has the advantage of being fully documented (as in the source code is actually explained) in Labrosse's Book. Don't know about Web Support though.
I know pSos is no longer available.
You can also take a look at this list of RTOSes
I worked with QNX many years ago, and have nothing but great things to say about it. Even back then, QNX 4 (which is positively chunky compared to the Neutrino microkernel) was perfectly suited for low memory situations (though 32MB is oodles compared to the 1-2MB that we had to play with), and while I didn't explicitly play with any web-based stuff, I know Apache was available.
I purchased some development hardware from netburner
It has been very easy to work with and very well documented. It is an RTOS running uCLinux. The company is great to work with.
It might be a wise decision to select an OS that your team is experienced with. However I would like to promote two good open source options:
eCos (has you mentioned)
RTEMS
Both have a lot of features and drivers for a wide variety of architectures. You haven't mentioned what architecture you will be using. They provide POSIX layers which is nice if you want to stay as portable as possible.
Also the license for both eCos and RTEMS is GPL but with an exception so that the executable that is produced by linking against the kernel is not covered by GPL.
The communities are very active and there are companies which provide commercial support and development.
We've been very happy with the Keil RTX system....light and fast and meets all of our tight real time constraints. It also has some nice debugging features built in to monitor stack overflow, etc.
I have been pretty happy with Windows CE, although it is 'heavier'.
Posting to agree with Ben Collins -- your really need to determine if you have a soft real-time requirement (primarily for human interaction) or hard real-time requirement (for interfacing with timing-sensitive devices).
Soft can also mean that you can tolerate some hiccups every once in a while.
What is the reliability requirements? My experience with more general-purpose operating systems like Linux in embedded is that they tend to experience random hiccups due to their smart average-case optimizations that try to avoid starvation and similar for individual tasks.
VxWorks is good:
good documentation;
friendly developing tool;
low latency;
deterministic scheduling.
However, I doubt that WindRiver would convert their major attention to Linux and WindRiver Linux would break into the market of WindRiver VxWorks.
Less market, less requirement of engineers.
Here is the latest study. The last one was done more than 8 years ago so this is most relevant. The tables can be used to add additional RTOS choices. You'll note that this comparison is focused on lighter machines but is equally applicable to heavier machines provided virtual memory is not required.
http://www.embedded.com/design/operating-systems/4425751/Comparing-microcontroller-real-time-operating-systems