I want to have clarity on user space program and OS interaction. In the context of a file I/O, I want to know how a user app like a java file i/o API read() may work.
Since file operations are accessed by POSIX system calls like open() close() read() write(), how does the java code calls these system calls ?
When we compile the java code, what kind of instructions the java read() API would be compiled to ?
Its said, user programs raise traps/software interrupts to make system calls. So does Java read() API is also raise traps ? Are there APIs in java to Raise traps ? If yes, so those APIs might be compiled to trap instructions like 'INT' ? But then does INT calls higher layer POSIX system calls or some fixed ISRs (Interrupt Service Routines) ?
I am confused and trying to know step by step... from compilation to execution- how system calls are done in this scenario.
Please help me with this simple concept.
Perhaps I got a very good clarification on http://pages.cs.wisc.edu/~remzi/OSFEP/intro-syscall.pdf
Hence I enfer the following flow: Java Code -> JNI -> read() syscall in C -> kernel subroutines.
But last doubt: How JNI which is in Java, calls C codes ?
The answer to this question is java run time environment.It provide a system call interface which intercepts java function call in the api and invokes the necessary system call
Related
Using Module and ModuleManager with ThreadX in a MPU-enabled platform
is "default_module_start" considered part of ModuleManager and can call Tx APIs even though, it is in the app_module.c?
E.G tx_thread_create works in default_module_start but doesn't work in the modules threads and throughs an exception;
Another question why is the ModuleManager is not just using the Tx APIs to handle threats for example, instead it uses custom functions that doesn't call Tx APIs at all
Function demo_module_start is part of a module (this function is in sample_threadx_module.c). It runs in the module context. This function gets called by txm_module_thread_shell_entry.c when a module is started.
Modules run in unprivileged mode, but they call ThreadX APIs (a.k.a. kernel functions). In order to execute ThreadX APIs, the module uses the SVC instruction (for ARM processors) to get into supervisor (privileged) mode. Thus, in the module library, all of the kernel calls are just simple calls that pass the function parameters to the kernel, and the actual ThreadX function is executed in kernel (privileged) mode.
Let me know if this answers your questions or if you have more questions.
Edit:
You can call TX APIs from module threads. By default, they trap into the kernel via the SVC instruction. If you want to call TX APIs directly from a module (i.e. without trapping), the module needs to be in privileged mode execution, which you can configure by modifying the module properties in the preamble of the module (e.g. see https://github.com/azure-rtos/threadx/blob/master/ports_module/cortex_m7/gnu/example_build/txm_module_preamble.S - change the properties from 0x00000007 to 0x00000000).
Creating a module thread is a bit different than creating a normal thread. The manager puts the TXM_MODULE_THREAD_ENTRY_INFO into the module thread stack, allocates a kernel stack for the thread, builds the module thread stack (which has a different return mode than a normal thread).
The manager can have whatever priority you want to assign it. Most if not all of our module manager examples assign a priority of 1 (https://github.com/azure-rtos/threadx/blob/master/ports_module/cortex_m7/gnu/example_build/sample_threadx_module_manager.c).
I am going through the book by Galvin on OS . There is a section at the end of chapter 2 where the author writes about "adding a system call " to the kernel.
He describes how using asmlinkage we can create a file containing a function and make it qualify as a system call . But in the next part about how to call the system call he writes the following :
" Unfortunately, these are low-level operations that cannot be performed using C language statements and instead require assembly instructions. Fortunately, Linux provides macros for instantiating wrapper functions that contain the appropriate assembly instructions. For instance, the following C program uses the _syscallO() macro to invoke the newly defined system call:
Basically , I want to understand how syscall() function generally works . Now , what I understand by Macros is a system for text substitution .
(Please correct me If I am wrong)
How does a macro call an assembly language instruction ?
Is it so that syscallO() when compiled is translated into the address(op code) of the instruction to execute a trap ?(But this somehow doesn't fit with concept or definition of macros that I have )
What exactly are the wrapper functions that are contained inside and are they also written in assembly language ?
Suppose , I want to create a function of my own which performs the system call then what are the things that I need to do . Do , I need to compile it to generate the machine code for performing Trap instructions ?
Man, you have to pay $156 dollars to by the thing, then you actually have to read it. You could probably get an VMS Internals and Data Structures book for under $30.
That said, let me try to translate that gibberish into English.
System calls do not use the same kind of linkage (i.e. method of passing parameters and calling functions) that other functions use.
Rather than executing a call instruction of some kind, to execute a system service, you trigger an exception (which in Intel is bizarrely called an interrupt).
The CPU expects the operating system to create a DISPATCH TABLE and store its location and size in a special hardware register(s). The dispatch table is an array of pointers to handlers for exceptions and interrupts.
Exceptions and interrupts have numbers so, when exception or interrupt number #1 occurs, the CPU invokes the 2d exception handler (not #0, but #1) in the dispatch table in kernel mode.
What exactly are the wrapper functions that are contained inside and are they also written in assembly language ?
The operating system devotes usually one (but sometimes more) exceptions to system services. You need to do some thing like this in assembly language to invoke a system service:
INT $80 ; Explicitly trigger exception 80h
Because you have to execute a specific instruction, this has to be one in assembly language. Maybe your C compiler can do assembly language in line to call system service like that. But even if it could, it would be a royal PITA to have to do it each time you wanted to call a system service.
Plus I have not filled in all the details here (only the actual call to the system service). Normally, when you call functions in C (or whatever), the arguments are pushed on the program stack. Because the stack usually changes when you enter kernel mode, arguments to system calls need to be stored in registers.
PLUS you need to identify what system service you want to execute. Usually, system services have numbers. The number of the system service is loaded into the first register (e.g., R0 or AX).
The full process when you need to invoke a system service is:
Save the registers you are going to overwrite on the stack.
Load the arguments you want to pass to the system service into hardware registers.
Load the number of the system service into the lowest register.
Trigger the exception to enter kernel mode.
Unload the arguments returned by the system service from registers
Possibly do some error checking
Restore the registers you saved before.
Instead of doing this each time you call a system service, operating systems provide wrapper functions for high level languages to use. You call the wrapper as you would normally call a function. The wrapper (in assembly language) does the steps above for you.
Because these wrappers are pretty much the same (usually the only difference is the result of different numbers of arguments), wrappers can be created using macros. Some assemblers have powerful macro facilities that allow a single macro to define all wrappers, even with different numbers of arguments.
Linux provides multiple _syscall C macros that create wrappers. There is one for each number of arguments. Note that these macros are just for operating system developers. Once the wrapper is there, everyone can use it.
How does a macro call an assembly language instruction ?
These _syscall macros have to generate in line assembly code.
Finally, note that these wrappers do not define the actual system service. That has to be set up in the dispatch table and the system service exception handler.
MSDN says closesocket() is the function to use. However, I couldn't help wondering if _close() will work also?
MSDN appears to say no in their description of SOCKET type: (http://msdn.microsoft.com/en-us/windows/ms740516(v=vs.80)):
In Winsock applications, a socket descriptor is not a file descriptor and must be used with the Winsock functions.
and more specifically from its note on renamed socket functions:
Sockets are represented by standard file descriptors in Berkeley Sockets, so the close function can be used to close sockets as well as regular files. While nothing in Windows Sockets prevents an implementation from using regular file handles to identify sockets, nothing requires it either. On Windows, sockets must be closed by using the closesocket routine. On Windows, using the close function to close a socket is incorrect and the effects of doing so are undefined by this specification.
However, even despite the above, some of the Windows file functions might work with sockets in practice:
Given that ReadFile and WriteFile work on sockets, I suspect _read and _write, for instance, might also work with sockets as well as file handles.
MSDN's overview of socket handles states:
A socket handle can optionally be a file handle in Windows Sockets 2. A socket handle from a Winsock provider can be used with other non-Winsock functions such as ReadFile, WriteFile, ReadFileEx, and WriteFileEx.
The short answer is no. Sockets handles on Windows are not file handles as they are on Unix. There's special support such that the low level Win32 APIs, ReadFile and WriteFile, can work with a socket handle. But that's likely where it ends.
With regards to _open_osfhandle, yes, that will possibly work in a very limited sense, but there's good reasons why you shouldn't do this. Most of the following I inferred just by browsing the sources of open, read,write, close, and open_osfhandle in the CRT sources (that comes with Visual Studio).
There's a lot of buffering that goes on with the CRT read/write calls. Any attempt to mix read/write with recv/send will be going into undefined behavior.
Performance. Just look at the source of read() and write() as seen in the CRT sources. A lot of wrapper code to eventually call ReadFile and WriteFile, which in turn have to forward to the actual socket API.
Socket error codes may not bubble to the file API as you think. Remember socket API errors get returned through WSAGetLastError. Win32 file IO calls get bubbled up through GetLastError. So if your call to write() hits a socket error, it might try to map the return value via GetLastError, which still returns success.
close() won't properly close the socket handle since it maps to CloseHandle, not closesocket.
More insightful than MSDN: [http://tangentsoft.net/wskfaq/articles/bsd-compatibility.html]
No they are definitely different in Windows 3.x, 9x:
'In Windows 3.1 and 95 Windows sockets and file handles are
completely distinct. In Windows NT, however, it appears they may usually be one and the same.'
And no, you can't use them together on Windows NT - at least not by default:
'The Visual C++ RTL emulates POSIX functions, except that they’re named
with a leading underscore: for example, _read() instead of read(). The
_read() function uses ReadFile() internally, so you’d think it would work with sockets. The problem is, the first argument is an
RTL-specific handle, not an operating system file handle. If you pass
a socket handle to _read() or _write(), the RTL will realize that it
isn’t an RTL handle and the call will fail.
'Fortunately, there is a bridge function in Visual C++’s RTL:
_open_osfhandle(). (If you’re not using Visual C++, you’ll have to check its RTL source for a similar function.) I’ve not tried it, but
it appears to take an operating system file handle (including socket
handles) and return a handle you can use with the POSIX emulation
functions in the RTL. I’m told that this will work with sanely-coded
non-Microsoft Winsock stacks, but since I haven’t tried it, you should
if you want to support these alternate stacks.'
Trying to understand why there are ioctl calls in socket.c ? I can see a modified kernel that I am using, it has some ioctl calls which load in the required modules when the calls are made.
I was wondering why these calls ended up in socket.c ? Isn't socket kind of not-a-device and ioctls are primarily used for device.
Talking about 2.6.32.0 heavily modified kernel here.
ioctl suffers from its historic name. While originally developed to perform i/o controls on devices, it has a generic enough construct that it may be used for arbitrary service requests to the kernel in context of a file descriptor. A file descriptor is an opaque value (just an int) provided by the kernel that can be associated with anything.
Now if you treat a file descriptor and think of things as files, which most *nix constructs do, open/read/write/close isn't enough. What if you want to label a file (rename)? what if you want to wait for a file to become available (ioctl)? what if you want to terminate everything if a file closes (termios)? all the "meta" operations that don't make sense in the core read/write context are lumped under ioctls; fctls; etc. unless they are so frequently used that they deserve their own system call (e.g. flock(2) functionality in BSD4.2)
In Linux, when you can choose between a system call or a function call to do a task, which option is the better one due to a better performance?
We should note that in most of the cases we do not directly use system call. We use the interface provided by glibc.
http://www.kernel.org/doc/man-pages/online/pages/man2/syscalls.2.html
http://www.gnu.org/software/libc/manual/html_node/System-Calls.html
Now in cases like File Mangement/IPC/ process management etc which are the core resource management activities of the Operating System the only option is system call and not library functions.
In these cases, typically we use Library function which works as a wrapper over a system call. That is say for reading a file, we have many library functions like
fgetc/fgets/fscanf/fread - all should invoke read system call.
So shall we use read system call? or the other library functions?
This should depend on the particular application.If we are using read, then we again need to change the code to run this, on some other operating system where read is not available.
We are losing some flexibilty. It may be useful when we are sure of the platform and we can do some optimisations by using read only or may be the application must use only file descriptors and not file pointer etc.
Now in cases where we need to consider only say user level operations and invoke
no service from operating system , like say copying a string.(strcpy).
In this case definitely we shall not use any system call unnecessarily, if at
all something is there, since it should be an extra overhead due to operating
system intervention, which is not needed in this case.
So I feel choosing between a system call and a library function only occurs for cases where we have a library function built on top of a system call.
(like adding to examples above we can have say malloc which calls system call brk).
Here the choice will depend on the particular type of software, the platform on which it should run, the precise non functional requirements like speed (Though you cannot say with certainty that your code will run faster if you are using brk instead of malloc), portability etc.