As per this SO post and several resources online, I am aware that ROS is a meta-operating system that runs on top of a real OS.
However, what is the best way to identify all the actual Operating System (OS) abstractions/services that ROS 2 applications actually depend on?
Specifically, I'm interested in understanding the OS abstractions necessary to run ROS applications and not the OS abstractions required to develop ROS 2 applications.
Examples of OS abstractions I am referring to include:
File System (which to my knowledge ROS 2 requires).
Network Drivers (which to my knowledge ROS 2 also requires given that ROS 2 nodes need to communicate with each other over some network).
My goal is to understand ROS 2's Operating System abstractions to a level deep enough where I potentially may be able to attempt to port ROS 2 applications to an unsupported OS as an educational exercise.
Some strategies that I have considered (but have yet to attempt) include:
Going through each of the libraries outlined in the ROS architecture diagram below such as the rcl and rclpp libraries and examining the header files included in those libraries to determine what Operating System services they require.
Running strace on a ROS 2 executable and examining the System Calls made by the ROS 2 program. However, this seems to me a less structured version of the previous strategy.
The following diagram also raise further questions like, what is the best way to identify Operating System abstractions required by Data Distribution Service (DDS) libraries?
However, I'm unsure if this is the best way to go about it and if there are any resources out there (research papers, documentation I haven't read, blog posts etc.) that may speed this discovery/understanding process up.
Would appreciate some starting advice from anyone who has:
Ported ROS 2 applications to a different OSs in the past.
Knowledge about ROS 2's internal architecture.
Experience porting other things to new OSs and doing this kind of discovery work!
General advice on how to get started.
Related
From my research I cannot find what kernel type is being used in eCos, such as monolithic or micro-kernel. All I could find from my research is that the kernel is a real-time one or websites just describe it as the eCos kernel, does this mean it is a custom made kernel?
What I know about eCos is that it is a hard RTOS although is somewhat vulnerable in terms of security, uses priority, queue based scheduling.
A micro-kernel is:
... the near-minimum amount of software that can provide the mechanisms
needed to implement an operating system (OS). These mechanisms include
low-level address space management, thread management, and
inter-process communication (IPC).
(Wikipedia 11 Dec 2018)
The eCos kernel is described in its Reference Manual thus:
It provides the core functionality needed for developing
multi-threaded applications:
The ability to create new threads in the system, either during startup
or when the system is already running.
Control over the various threads in the system, for example
manipulating their priorities.
A choice of schedulers, determining which thread should currently be
running.
A range of synchronization primitives, allowing threads to interact
and share data safely.
Integration with the system's support for interrupts and exceptions.
It is quite clearly by comparison of these descriptions a micro-kernel. Other services provided by eCos such as file-systems, networking and device drivers are external and separable from the kernel. That is to say, you can deploy the kernel alone without such services and it remains viable.
In a monolithic kernel, these services are difficult or impossible to separate as they are an intrinsic part of the whole. Unlike eCos mand most other RTOS they do not scale well to small hardware platforms common in embedded systems. Monolithic kernels are suited to desktop and general purpose computing platforms, because the platforms themselves are monolithinc - a PC without a filesystem, display, keyboard etc, is not really viable, whereas in an embedded system that is not the case.
While Linux, and even Windows are used in embedded systems, a micro-kernel is deployable on platforms with a few tens of kilo-bytes of memory, whereas a minimal embedded Linux for example requires several mega-bytes and will include a great deal of code that your application may never use.
Ultimately the distinction is perhaps irrelevant, as is the terminology. It is what it is. You do not choose your kernel or OS on this criteria, but rather whether it provides the services you require, runs on your target, and fits in the available resource.
I think it is a monolithic kernel. If you review this page: http://ecos.sourceware.org/getstart.html
It is used instead of linux kernel and linux kernel support monolithic kernels. In addition, if it was a micro kernel , they would highlight the kernel type like QNX Kernel type which is micro kernel
I have been programming in java for 3 years but I have no experience with other languages. I want to know what I need to study in order to be able to make an operating system. Am most likely going to make my operating system based on Linux kernel. What programming languages should I be familiar with and what aspects of the pc hardware should I study. if you know any online tutorials or good books please mention them.
The answer depends on how far you want to go and how much you want to write yourself vs using existing code.
The most straightforward way is to have a look at Arch Linux or Gentoo and build your own, custom Linux setup. Approximate time needed to create a minimal working system: ~2 hours
You could, otherwise, compile the Linux kernel, build some software packages and put it all together yourself and create your own Linux distro - i.e your own operating system in a sense. Linux From Scratch will be an invaluable resource if this is what you decide to do. Approximate time needed to create a minimal working system: ~2-5 days
Say that's too easy for you but you're not ready to delve into the nitty-gritty of kernel development. You can write your own applications that run on top of the Linux kernel. Typically, you'd need to know C/C++ but any language that supports running on Linux/compiling to a Linux executable will work. Heck, you could chuck a (or write your own) Java runtime on and write your whole 'operating system'/user-space in Java. Approximate time needed to create a minimal working system: ~6-12 months
What about if you want to get down to programming your own kernel from scratch? Heck, Linux is overrated and you want to write your own kernel and it's going to be the next best thing! You would want to learn a bit about the platform you're developing on. At the very least, you'll need to know the assembly instructions of some special operations for your platform that can't be done natively in C like switching CPU modes. You'll definitely need to read up on the OS dev wiki for this and you should have a fair decent computer science background.
At this point, you shouldn't need too much other than a good tutorial, C, a little assembly, reference manuals for the hardware you're hoping to support and 3+ years of computer science background to get you started. Your boot loader that's booting your kernel should handle most of the hardware initialisation. Bootloaders like GRUB (I'm assuming you're developing on a x86 system) does so much for you that it can probably just jump to your kernel main function straight away without you having to do too much. Again, if you wanted, you could port or write your own Java runtime in C and write the rest of your kernel in Java! Approximate time needed to create a minimal working system: ~3-5 years
But, let's just say you're screaming for more pain and you want to write an operating system really from scratch and you don't want no bootloader doing a lot of the legwork. What do you need? Firstly, you'll need a lot more reference manuals. And you'll need to read up on a lot more assembly. Especially for Intel processors, there's a lot of work involved bringing the system up from 16bit mode to 32/64bit protected mode with paging (which is what I assume you want). You'll also want to know about every tiny quirk and weirdness of your platform that will affect your OS (these are often not documented; hooray!). Plus, all of the above. In short, you will need to study everything. Approximate time needed to create a minimal working system: 5+ years
Of course, this post is just scraping the surface of what is needed to even bring up a basic operating system capable of say, opening a web browser. The approximates roughly assumes a minimal working system is something capable of running a graphical web browser and will vary depending on how much you want to write.
I don't mean to be condescending but this is the kind of reality you'll be facing if you decide to write your own operating system. Nevertheless, it is a valuable learning experience if you can break the initial barrier or even just trying to set up your own Linux system.
First, I would say that you install any Linux OS on your system and get accustomed with it.
Second, for OS development you have to know C language. As for the Assembly language, it depends from where you start the OS development. If you will be using available bootloaders, then I don't think that you will be requiring to learn assembly language.
This is a website on OS development: http://wiki.osdev.org/Main_Page
There you will find all the stuff you need to know for the OS development. And also how to develop OS step-by-step.
Now-a-days, a "Eudyptula Challenge" is going on. It is a series of programming exercise for the linux kernel. You can find more info here: http://eudyptula-challenge.org/
Recently I heard the term meta-operating system while I was learning ros. Could you please help me to differentiate between operating system and meta-operating system?
What ROS is and is not is best explained in this paper.
To cite the intodruction at ros.org:
It provides the services you would expect from an operating system, including hardware abstraction, low-level device control, implementation of commonly-used functionality, message-passing between processes, and package management. It also provides tools and libraries for obtaining, building, writing, and running code across multiple computers.
You'll also find a good explanation in this wiki. The basic difference is that a meta operating system is built on top of the operating system and allows different processes (nodes) to communicate with each other at runtime.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I want to learn linux kernel device driver programming. So can anyone please post good tutorials pages or links here. I am new to linux kernel environment. I have searched for it but I don't know how to start and which one to read for easy understanding basics. Thanks in advance.
Depends on your current skills. If you're really new to Linux, perhaps you should start with user space system programming with Advanced Linux Programming. You'll get good knowledge of Unix system calls and other concepts such as signals, processes/threads and so on with this free resource. This is a must (understanding the user space API) if you're developing on the kernel side since the role of a kernel is providing services to users in a secure way.
Otherwise one often cited book is Linux Device Drivers, Third Edition (LDD3). Keep in mind that this edition was written at the time of Linux 2.6.10 and some things changed since then. This article shows the differences as 2.6 evolved (until 2.6.31, that is, so not very useful). I should mention martinezjavier/ldd3, which contains example drivers of LDD3 updated for more recent kernels (thanks to 42n4 for pointing that out).
Another interesting book that's not as often cited is Essential Linux Device Drivers. You won't find a free version of this one, but it still features an interesting approach. What I like about this one is it covers lots of different device types and is up-to-date as of 2.6.24, which is a bit better than LDD.
Finally, one great book about the kernel itself (not specifically for drivers) is Understanding the Linux Kernel, 3rd Edition. This covers in-depth kernel facilities and internal mechanisms. It's up-to-date as of 2.6.11.
As for online tutorials, I found this post on Pete's Blog is a really great example. Not only does it show how to create a character device (the most easy kernel driver type, i.e. the one you should start with), it uses modern Linux kernel features in an easy to understand fashion, including:
use of udev
use of a kernel data structure (FIFO)
use of kernel synchronization (mutex)
use of Sysfs with custom attributes
module options for insmod
Plus: it's aimed at Linux 3.0, which means it's more up-to-date compared to other resources.
You might also like this post about how to create Sysfs entries manually, although the Linux device model will take care of registering your device as a Sysfs entry if you don't need additional nodes or attributes.
Edit: I should add that the best way to learn real Linux device driver programming is to look at actual drivers. There are thousands of drivers in drivers. Start reading and understanding the concept of simple ones like drivers/leds and you will see how rewarding this is.
The site with very important links:
http://elinux.org/Device_drivers
Great tutorial with real examples (ends with an usbpen formatted with our own file system - search for author further articles):
http://www.linuxforu.com/tag/linux-device-drivers-series/
http://sysplay.in/index.php?pagefile=lfy_articles
Source code of famous LDD3 book ported to newest kernels:
https://github.com/martinezjavier/ldd3/
Videos:
https://www.youtube.com/playlist?list=PL16941B715F5507C5
Maybe other cources:
http://www.skilledup.com/courses?keyword=linux+kernel&price_filter=0..0
Some examples:
http://www.staerk.de/thorsten/index.php/My_Tutorials/Writing_Linux_kernel_modules
http://www.freesoftwaremagazine.com/articles/drivers_linux
http://techblog.aasisvinayak.com/kernel-module-programming-tutorial/
http://www.linuxforu.com/2009/05/a-voyage-to-the-kernel-day-11/
Books (without mentioned in this thread LDD3):
http://www.coopj.com/ with an updated source code http://www.coopj.com/LPD/
KernelHacking:
http://kernelnewbies.org/KernelHacking
Visual map of the linux kernel: http://www.makelinux.net/kernel_map/
Linux and kernel books for beginners:
http://www.kroah.com/lkn/
http://swift.siphos.be/linux_sea/
You might be interested in the newly released Linux Driver Templates. As the name suggests, it provides templates and demonstrates frequently used Linux facilities to get started quickly.
I understand it is a delayed response !!
You can pick any book, those are really great books suggested above.
But you need to really work practically. Try to be involved into Kernel as much as possible.
Mostly you need to look into kernel source code itself.
And the most interesting document you can find in Documentation folder under Kernel tree.
best book for learning device driver programming is
"LDD"
Professional Linux Kernel Architecture is a good read also.
The best source is the linux man pages but they are somewhat critical to understand for a beginner, Directly programming device drivers is not a easy task. I recommend you to go through pointers and structures through following books
Basic C Books 1. Programming C - Byron gottfried 2. The C Programming Language - Dennis Ritchie
Intermediate Books 1. Pointers on C Kenneth Reek 2. Expert C Programming Deep Secrets - Linden
Coming to device Drivers I have uploaded the Kernel Source documentation in pdf format https://drive.google.com/folderview?id=0B7iRyndFhHldR3hjOHpOZTdKTjA&usp=sharing Youcan download from this link.
Device Drivers Basics (User mode Programming)
Linux Programming interface - Michael Kerrisk
Beginning Linux Programming Wrox Publishers
Device Drivers (Kernel Deleopment) 1. Linux Kernel Development - Robert Love 2. Linux Kernel Internals - m beck
Device Drivers (Driver Programming) 1. Linux Device Drivers - Third Edition (Free Download is available for 2.6 Kernel) 2. Essential Linux Device Drivers - Venkateswaran
For Basic Driver Knowledge Follow this site http://www.tldp.org/LDP/khg/HyperNews/get/devices/devices.html
My Experience is First of all we must gain a sound knowledge on C Programming, mainly Structures and Pointers before going through Driver Programming otherwise it will be bit cumbersome to understand driver programming.
Comming to Startup :-
User space programming
Basics of gcc generating proprocessed, assembler and object files using gcc commands analysis of object files using different object tools, Basic programming in linux, threads(locks, synchronization techniques), signals, processes(fork, exec), timers(Posix). This will help you lot in understanding kernel concepts.
Kernel Programming
Understanding the kernel source tree, different branches and their importance, using ctags and cscope to browse kernel source code, this will help you in kernel source browsing.
Driver Programming
First of all Learn different driver classes, char, block etc.,
Next learn the device protocols, transmission and reception, the product manual is the best source for going through this.
Then start writing the simple char driver for example RTC (a port mapped device), IOAPIC (A memory mapped device), then go to USB, PCI, Network, I2C, SPI etc., if you want the help you can find the skeleton code for all this in kernel source tree.
I recently posted a question about Azure... is it really an OS? I understand the technical details, and I got a lot of fuzzy answers... I really want to know... what do you think is the difference between an OS and a Framework?
Just for reference, Azure will be built on top of Hyper-V servers and the virtual machines will be running vanilla Windows Server 2008. It will run services that creates a cloud on top of the many virtual machines which is called Azure. Windows is calling Azure an OS.
I am trying to understand how to define the difference between an OS and a framework.
Operating System: The infrastructure software component of a computer system
Framework: A re-usable design for a software system (or subsystem).
By these definitions it seems to me, that an operating system can be built using a framework, and a framework can be built to interact with an operating system.
Singularity is an example of an experimental OS that is built using managed code.
Framework is a very broad term, it can be used to describe many types of subsystems. It could even describe an operating system.
Operating System is more specific, it implies facilitation of interaction with a computers or group of computers hardware layer, through the use of human user interfaces. I think Azure fits this description.
It's up to marketing - I don't think the terms have a definite meaning any more.
Is a JVM a framework?
What if it's running on a raw uC or even an FPGA - is it an OS?
An OS is the thing that directly interfaces with the machine, be it virtual or real. It has to expose syscalls that handle input devices, output devices, sound, networking, and all the other things that we take for granted these days. It also often provides some kind of UI which uses these services to make it easy to use/useful for an end-user. It needs to have device drivers to work with video cards, sound cards, etc. (Once again, these can be virtualized).
A framework is... something built on top of the OS. It, too, exposes an API, but they are often not so low-level as the one the OS exposes.
frameworks provide api contracts that oses usually don't - meaning they sit atop the os, hide and manage the differences, and consequently give you that platform independence goodness that can broaden our target audience
A framework thought to be as a development environment,a helping platform for further developments and you can work additively to create some other application using components of framework, while OS is system software is an environment to operate a system.