I wonder how virtualization software such as VirtualBox or VMWare Workstation works? How can they create a virtual environment that is taken as a separate computer by operating systems? I'm almost sure the answer to this question is very deep, but I'd be well satisfied with basic theory.
How does VMWare work:
http://www.extremetech.com/article2/0,2845,1624080,00.asp
How does virtualizaton work:
http://blog.tmcnet.com/voip-enterprise/tmcnet/how-does-virtualization-work-and-why-is-now-a-good-time-to-check-it-o.asp
Server Virtualization FAQ
http://www.itmanagement.com/faq/server-virtualization/
In the simplest sense, a virtualised environment is to a native environment, what an interpreted language, like PHP, Javascript or Basic, is to a compiled language like C, C++ or assembler.
When a compiled binary executes, the binary machine code, is passed straight to the CPU. However when an interpreted language runs, the language application reads in the code, then it decides what that meant and execute binary procedures to reflect that.
So virtualisation software like Qemu, while compiled to run on, say an x86 processor, will read the binary file, intended for say a Mac, and it will interpret the binary it receives, switch it from big, to little endian, and then know that op code X on mac corresponds to op code Y on x86, and that op code A on mac, doesn't have an equivalent on x86, so will need to call function B on x86 and so on.
The really clever bit, is the hardware interpretation, where someone has to write a driver, that will run on Qemu, on x86, but will present an interface to the Mac face of Qemu, to make Mac applications think they're talking to Mac hardware.
In the most basic sense, virtualization software puts a computer within another computer... kind of. (Here's a link that's very, very basic: http://blog.capterra.com/virtualization-software)
In a more complex sense, virtualization software (also called a hypervisor) abstracts the characteristics of a server. This allows several OSs to run on a single physical server.
Related
Do interpreted languages such as Java and Python need an operating system to work?
For example, on a bare-metal ARM microcontroller, can an interpreter be installed such that we can have both compiled code such as C, and interpreted code such as Python working together, Or is an OS needed to support this?
Of course you can write an interpreter that runs on bare-metal, it is just that if the platform does not have an OS any run-time support the language needs must be part of the interpreter. To the extent in some cases that such an interpreter might essentially be an OS. That is if it provides the services to operate a system, it could be called an operating system.
It is not perhaps as simple as interpreted vs compiled. Java for example runs on a virtual machine and is "compiled" to bytecode. The bytecode is interpreted (or just-in-time compiled in some cases), rather then the Java source directly. In an embedded system, it is possible that you would deploy cross-compiled bytecode on the target rather then the source. Certainly however JVMs exist for bare-metal. Some support multi-threading through a third party RTOS, others either have that support built-in or do not support threading at all.
There are interpreters for cut-down subsets of JavaScript and Python that run on bare-metal microcontrollers. I am not sure about full implementations, but it is technically possible given sufficient run-time support even if not explicitly implemented. To fully support some of these languages along with all the standard and third-party libraries and frameworks a developer might expect, may require so much run-time support and resource that it is simpler to deploy and OS, so implementations for resource constrained systems are often subsets or have restricted libraries.
Java needs a VM - virtual machine. It isn't interpreted, but executes byte code. Interpreted would mean grabbing the source in run-time as it goes, like BASIC.
When Java was new and exciting around year 2000, everyone thought it would be the next big general-purpose language, replacing C++. The syntax was so clean, it was "pure OO" and not some "filthy hybrid".
It was the major buzz word of the time. Schools stopped teaching C and C++. MCU manufacturers started to make chips with Java VM in hardware. Microsoft made their own Java "standard". Everyone was high on the Java hype.
Then as the Internet hype as whole collapsed in 2002, it took the Java hype with it. In the sober hang-over afterwards, people started to realize that things like byte code, VMs and garbage collection probably don't belong on bare metal systems.
They went back to using compiled C for hardware-related programming. Or in fact they never stopped, since Java never quite made it there, save for some oddball exotic architectures.
Java remained used only in the areas were it was suitable, namely web, desktop and mobile development. And so it got a second golden age when the smart phone hype struck around 2010.
No. See for example picoJava, which is one of several solutions for running Java natively. You can't get closer to bare metal than running bytecode on the CPU.
No. Some 8-bit computers had interpreted languages in ROM despite not having anything reasonably resembling a modern operating system. The Apple 2 is one example. You could boot the system without any disks or tapes, and it would go straight to a BASIC prompt, where you could write basic (no pun intended) programs.
Note that an operating system is somewhat of a vague term when speaking about these days - these 8-bit computers did have some level of firmware, and this firmware did provide some OS-type functionality like access to basic peripherals. In these days, what we now know as an OS was more commonly called a "DOS" - a Disk Operating System. MS-DOS is one of them, as well as Apple's ProDOS. These DOS's evolved into our modern-day operating systems (e.g. Windows 95 was based on top of MS-DOS, while modern Windows versions derive from a separate branch that was largely re-implemented with more modern techniques), so one could claim that their ancestors are the closest they had to what we now call an OS.
But what is an interpreter but a piece of software?
In a more theoretical sense, an interpreter is simply software - a program that takes input and produces output. Suppose you were to implement a custom solid-state Turing Machine. In this case, your "input" would be the program to be interpreted, and the "output" would be the program's behavior. If "software" can run without an operating system, then an interpreter can.
Is this model a little simplified? Of course. The difference is a matter of degree, not nature. Add very basic user input and output capabilities (e.g. a TTY) and you have the foundation to implement all, or nearly all, of the basic functionality of a language such as Java byte code, Python, or BASIC. The main things you would be missing are libraries and whatnot that depend on things like screen manipulation, multiprocessing, and networking, but you could handle them with time too.
I want to clarify before the question that I am not an established professional programmer in any position at any firm. This is solely to satisfy curiosity, and will not pertain to any task or project at this time.
As I understand it, firmware is software placed on hardware to grant it autonomous functionality from instructions, which is given through some form of input; As long as the input stream is readable, which is made possible through drivers. Drivers are software packages with pre-written reference libraries that recognize a specific set of instructions for each possible function in the attached device.
NOTE: not quoted, so I'm aware that this could be inaccurate.
What I want to know is how firmware or drivers are placed on devices without installation through an OS or storage medium; such as a DVD or USB? Specifically firmware installed by manufacturers, like bios and keyboard drivers that are present on all computers. I'm assuming these are less or not reliant on compilation in order to function properly, which is the sole reason I'm asking this question.
Can firmware be developed without compilation?
References
Demystifying Firmware
C++ Kernel Development
Starting Firmware Development
These just explain that an OS is a type of firmware, and that firmware is primarily developed in C with Assembly and C++ as plausible alternatives; pertaining to kernel development as well.
Yes, especially in the larger components. An example involving lua is http://nodelua.org/doc/index/
However, firmware development is typically an extremely memory (and frequently CPU) constrained environment.
C (or traditionally, assembler) is often preferred because it can produce extremely small executables, and is very efficient in stack usage. This matters when you're counting memory in bytes, or kilobytes.
Using a non-compiled language means you need to include a tiny interpreter, and you might not be able to set aside enough memory for this.
You've made an edit, wherein you suggest that an "OS is a type of firmware".
This can be true, in a manner of speaking.
Often firmware itself can consist of an operating system, with components. As an example, the firmware in some home internet routers will contain an OS (which might very well be linux!), however it is still regarded as firmware. There is a bit of a grey area between a computer that is an "embedded device with firmware", vs that of a 'regular computer with regular software', but generally firmware is a computer system running in a very constrained environment, often with very specific uses.
NetBSD includes Lua in it's kernel. Many systems have been developed that do not use Assembly (except for a small part of it), C, or C++, but instead use some other language - though it is typically still compiled for size and performance reasons.
As for the actual transfer of firmware (whatever the form it may be in), this depends on the device in question.
Some devices require that the firmware be burned into the components. (In ROM, though there are various types of ROM and some can be rewritten).
Other devices require that the firmware be transferred when the device is turned on.
And yet others have SDCards or battery-backed RAM or whatever that allow storing the firmware across reboots.
I have been programming in java for 3 years but I have no experience with other languages. I want to know what I need to study in order to be able to make an operating system. Am most likely going to make my operating system based on Linux kernel. What programming languages should I be familiar with and what aspects of the pc hardware should I study. if you know any online tutorials or good books please mention them.
The answer depends on how far you want to go and how much you want to write yourself vs using existing code.
The most straightforward way is to have a look at Arch Linux or Gentoo and build your own, custom Linux setup. Approximate time needed to create a minimal working system: ~2 hours
You could, otherwise, compile the Linux kernel, build some software packages and put it all together yourself and create your own Linux distro - i.e your own operating system in a sense. Linux From Scratch will be an invaluable resource if this is what you decide to do. Approximate time needed to create a minimal working system: ~2-5 days
Say that's too easy for you but you're not ready to delve into the nitty-gritty of kernel development. You can write your own applications that run on top of the Linux kernel. Typically, you'd need to know C/C++ but any language that supports running on Linux/compiling to a Linux executable will work. Heck, you could chuck a (or write your own) Java runtime on and write your whole 'operating system'/user-space in Java. Approximate time needed to create a minimal working system: ~6-12 months
What about if you want to get down to programming your own kernel from scratch? Heck, Linux is overrated and you want to write your own kernel and it's going to be the next best thing! You would want to learn a bit about the platform you're developing on. At the very least, you'll need to know the assembly instructions of some special operations for your platform that can't be done natively in C like switching CPU modes. You'll definitely need to read up on the OS dev wiki for this and you should have a fair decent computer science background.
At this point, you shouldn't need too much other than a good tutorial, C, a little assembly, reference manuals for the hardware you're hoping to support and 3+ years of computer science background to get you started. Your boot loader that's booting your kernel should handle most of the hardware initialisation. Bootloaders like GRUB (I'm assuming you're developing on a x86 system) does so much for you that it can probably just jump to your kernel main function straight away without you having to do too much. Again, if you wanted, you could port or write your own Java runtime in C and write the rest of your kernel in Java! Approximate time needed to create a minimal working system: ~3-5 years
But, let's just say you're screaming for more pain and you want to write an operating system really from scratch and you don't want no bootloader doing a lot of the legwork. What do you need? Firstly, you'll need a lot more reference manuals. And you'll need to read up on a lot more assembly. Especially for Intel processors, there's a lot of work involved bringing the system up from 16bit mode to 32/64bit protected mode with paging (which is what I assume you want). You'll also want to know about every tiny quirk and weirdness of your platform that will affect your OS (these are often not documented; hooray!). Plus, all of the above. In short, you will need to study everything. Approximate time needed to create a minimal working system: 5+ years
Of course, this post is just scraping the surface of what is needed to even bring up a basic operating system capable of say, opening a web browser. The approximates roughly assumes a minimal working system is something capable of running a graphical web browser and will vary depending on how much you want to write.
I don't mean to be condescending but this is the kind of reality you'll be facing if you decide to write your own operating system. Nevertheless, it is a valuable learning experience if you can break the initial barrier or even just trying to set up your own Linux system.
First, I would say that you install any Linux OS on your system and get accustomed with it.
Second, for OS development you have to know C language. As for the Assembly language, it depends from where you start the OS development. If you will be using available bootloaders, then I don't think that you will be requiring to learn assembly language.
This is a website on OS development: http://wiki.osdev.org/Main_Page
There you will find all the stuff you need to know for the OS development. And also how to develop OS step-by-step.
Now-a-days, a "Eudyptula Challenge" is going on. It is a series of programming exercise for the linux kernel. You can find more info here: http://eudyptula-challenge.org/
As a personal learning project, I want to port an existing x86 emulator library to JavaScript and then run a very simple operating system on top of it.
My only requirement for the library is that it should be written in C\C++, as I want to use emscripten and asm.js to compile the source files to JavaScript. The library should at least be able to interpret x86 opcodes; the specific environment details for JavaScript (memory, GUI, interrupts etc.) I can implement myself. The operating system doesn't have any requirement, as long as it is simple enough and can run on the emulator.
So far in my research, Softx86 seems like the most viable solution, as it seems simple enough to port to JavaScript (it only does CPU emulation). The simplest 16 bit real-mode operating system I can find is MikeOS, the only problem being that it requires a 386 processor, which Softx86 cannot emulate and I am not sure if they are completely compatible.
My question is if anyone knows an alternate combination of emulator-OS that is simple enough to be ported to JavaScript.
For anyone who will come across this thread in the future, I've decided to port Fake86 to JavaScript, and use MS-DOS as the operating system.
You can see a live demo here
It runs very, very slowly, and currently only runs on Firefox, but it works nonetheless.
You can see the full source code here
Can anybody suggest any experimental, easy to understand operating system written in a high level language on which I can do some interesting stuff as a learning experience.
There are some OS like MikeOS, TomOS etc but these are written in assembly languages.
Basically, I want an OS which has driver support for power management but should not be very large.
House, written in Haskell ontop of the H interface and GHC runtime system. Alternatively, consider a paravirtual machine that runs on Xen like mini-os (written in C, lives in the Xen repository) or HaLVM (a Haskell cross compiler to the Xen ABI).
Moving away from the Xen / Haskell realm, you could look at Haiku - a reimplementation of BeOS in C++.
While I haven't looked at the code myself, MINIX has a reputation as an easy-to-understand educational operating system. You'll have to see for yourself whether it has power management support, though the web size does say MINIX 3 is targeted at embedded systems, so I'd say your chances are good.