Most posts i read just give info about the maximum file name length's. But, i want to understand why there's this limit. Why can't file name's be big. I see that few file systems have put a limit of 255 bytes. Why not 1 MB or anything more than 255 bytes. I probably would never have a file name of length more than 100 characters. But, this question is about why the limit?
long file name costs much more space and time than you can imagine
the 255 bytes limit of file name length is a long time trade off between human
onvenience and space/time efficiency
and backward compatibility , of course
back to the dark old days , the capacity of hard drive capacity was count by MB or a few GB
file name are often stored in some fixed length C structs ,
and the size of the struct was mostly round by the factor of 512 byte,
which is the size of a physical sector ,so that it can be read out by a single touch of the head
if the file system put a limit of 1MB on filename, it would run out of harddisk space with only a few hundred files. and memory limits also applys.....
Related
Consider the following parameters of a FAT based lesystem:
Blocks are 8KB (213 bytes) large
FAT entries are 32 bits wide, of which 24 bits are used to store a block address
A. How large does the FAT structure need to be accommodate a 1GB (2^30 bytes) disk?
B. What is the largest theoretical le size supported by the FAT structure from part (A)?
A. How large does the FAT structure need to be accommodate a 1GB (2^30 bytes) disk?
The FAT file system splits the space into clusters, then has a table (the "cluster allocation table" or FAT) with an entry for each cluster (to say if it's free, faulty or which cluster is the next cluster in a chain of clusters). To work out size of the "cluster allocation table" divide the total size of the volume by the size of a cluster (to determine how many clusters and how many entries in the "cluster allocation table"), then multiply by the size of one entry, then maybe round up to a multiple of the cluster size or not (depending on which answer you want - actual size or space consumed).
B. What is the largest theoretical le size supported by the FAT structure from part (A)?
The largest file size supported is determined by either (whichever is smaller):
the size of "file size" field in the file's directory entry (which is 32-bit for FAT32 and would therefore be 4 GiB); or
the total size of the space minus the space consumed by the hidden/reserved/system area, cluster allocation table, directories and faulty clusters.
For a 1 GiB volume formatted with FAT32, the max. size of a file would be determined by the latter ("total space - sum of areas not usable by the file").
Note that if you have a 1 GiB disk, this might (e.g.) be split into 4 partitions and a FAT file system might be given a partition with a fraction of 1 GiB of space. Even if there is only one partition for the "whole" disk, typically (assuming "MBR partitions" and not the newer "GPT partitions" which takes more space for partition tables, etc) the partition begins on the second track (the first track is "reserved" for MBR, partition table and maybe "boot manager") or a later track (e.g. to align the start of the partition to a "4 KiB physical sector size" and avoid performance problems caused by "512 logical sector size").
In other words, the size of the disk has very little to do with the size of the volume used for FAT; and when questions only tell you the size of the disk and don't tell you the size of the partition/volume you can't provide accurate answers.
What you could do is state your assumptions clearly in your answer, for example:
"I assume that a "1 GB" disk is 1000000 KiB (1024000000 bytes, and not 1 GiB or 1073741824 bytes, and not 1 GB or 1000000000 bytes); and I assume that 1 MiB (1024 KiB) of disk space is consumed by the partition table and MBR and all remaining space is used for a single FAT partition; and therefore the FAT volume itself is 998976 KiB."
I found this example.
Consider a system with a 32-bit logical address space. If the page
size in such a system is 4 KB (2^12), then a page table may consist of
up to 1 million entries (2^32/2^12). Assuming that
each entry consists of 4 bytes, each process may need up to 4 MB of physical address space for the page table alone.
What is the meaning of each entry consists of 4 bytes and why each process may need up to 4 MB of physical address space for the page table?
A page table is a table of conversions from virtual to physical addresses that the OS uses to artificially increase the total amount of main memory available in a system.
Physical memory is the actual bits located at addresses in memory (DRAM), while virtual memory is where the OS "lies" to processes by telling them where it's at, in order to do things like allow for 2^64 bits of address space, despite the fact that 2^32 bits is the most RAM normally used. (2^32 bits is 4 gigabytes, so 2^64 is 16 gb.)
Most default page table sizes are 4096 kb for each process, but the number of page table entries can increase if the process needs more process space. Page table sizes can also initially be allocated smaller or larger amounts or memory, it's just that 4 kb is usually the best size for most processes.
Note that a page table is a table of page entries. Both can have different sizes, but page table sizes are most commonly 4096 kb or 4 mb and page table size is increased by adding more entries.
As for why a PTE(page table entry) is 4 bytes:
Several answers say it's because the address space is 32 bits and the PTE needs 32 bits to hold the address.
But a PTE doesn't contain the complete address of a byte, only the physical page number. The rest of the bits contain flags or are left unused. It need not be 4 bytes exactly.
1) Because 4 bytes (32 bits) is exactly the right amount of space to hold any address in a 32-bit address space.
2) Because 1 million entries of 4 bytes each makes 4MB.
Your first doubt is in the line, "Each entry in the Page Table Entry, also called PTE, consists of 4 bytes". To understand this, first let's discuss what does page table contain?", Answer will be PTEs. So,this 4 bytes is the size of each PTE which consist of virtual address, offset,( And maybe 1-2 other fields if are required/desired)
So, now you know what page table contains, you can easily calculate the memory space it will take, that is: Total no. of PTEs times the size of a PTE.
Which will be: 1m * 4 bytes= 4MB
Hope this clears your doubt. :)
The page table entry is the number number of bits required to get any frame number . for example if you have a physical memory with 2^32 frames , then you would need 32 bits to represent it. These 32 bits are stored in the page table in 4 bytes(32/8) .
Now, since the number of pages are 1 million i.e. so the total size of the page table =
page table entry*number of pages
=4b*1million
=4mb.
hence, 4mb would be required to store store the table in the main memory(physical memory).
So, the entry refers to page table entry (PTE). The data stored in each entry is the physical memory address (PFN). The underlying assumption here is the physical memory also uses a 32-bit address space. Therefore, PTE will be at least 4 bytes (4 * 8 = 32 bits).
In a 32-bit system with memory page size of 4KB (2^2 * 2^10 B), the maximum number of pages a process could have will be 2^(32-12) = 1M. Each process thinks it has access to all physical memory. In order to translate all 1M virtual memory addresses to physical memory addresses, a process may need to store 1 M PTEs, that is 4MB.
Honestly a bit new to this myself, but to keep things short it looks like 4MB comes from the fact that there are 1 million entries (each PTE stores a physical page number, assuming it exists); therefore, 1 million PTE's, which is 2^20 = 1MB. 1MB * 4 Bytes = 4MB, so each process will require that for their page tables.
size of a page table entry depends upon the number of frames in the physical memory, since this text is from "OPERATING SYSTEM CONCEPTS by GALVIN" it is assumed here that number of pages and frames are same, so assuming the same, we find the number of pages/frames which comes out to be 2^20, since page table only stores the frame number of the respective page, so each page table entry has to be of atleast 20 bits to map 2^20 frame numbers with pages, here 4 byte is taken i.e 32 bits, because they are using the upper limit, since page table not only stores the frame numbers, but it also stores additional bits for protection and security, for eg. valid and invalid bit is also stored in the page table, so to map pages with frames we need only 20 bits, the rest are extra bits to store protection and security information.
I have searched some posts and cannot find what the maximum filesize is under iPhone.
max size of an iOS application
maximum size of sqlite or database on iOS
As the above posts said, the maximum filesize depends on the free disk space. So, can I store everything into sqlite file and it's filesize can exceed 4GB or 10GB ?
According to the following links I found,
Mac OS, HFS File System volumn and file limits
iOS filesystem, HFSX
HFS, Wiki
As the first link says, "The theoretical maximum file size for a Mac OS Extended file system is millions of terabytes. In practice, the maximum file size is equivalent to the maximum volume size, except for a small amount of disk space reserved for file system information."
Because the maximum filesize is equal to the maximum volumn size, and consider the factor about the free disk space.
So, in my conclusion, the maximum size of single file depends on the free disk space.
I have a question here that I do not know how to calculate the maximal size of a file that one can store on a disk that uses inodes and disk blocks.
Assuming a page size of 4096 bytes, a page table entry that points to a frame takes 8 bytes (4
bytes for the pointer plus some flags), and a page table entry that points to another page table
takes 4 bytes, how many levels of page tables would be required to map a 32-bit address space if
each level page table must fit into a single page?
What the maximal file size one can store on a disk that uses inodes and disk blocks that store 4096 bytes. Each inode can store 10 entries, and the first inode reserves the last two entries for cascading inode???
For the first part of the question, I got the total number of levels is 3, but I do not know how to do the second part.
What you're describing sounds like the EXT filesystem.
EXT3 uses a total of 15 pointers.
The first 12 entries are direct: they point directly to data blocks. The third to final entry is a level 1 indirect: it points to a block filled entirely with level 1 entries. The second to final entry is a level 2 indirect: it points to a block completely full of level 1 indirects. The last entry is a level 3 indirect.
The maximum file size on this system is usually a restriction of the operating system, and is usually between 16GB and 2TB.
The theoretical maximum is 12I + I^2/P + I^3/P^2 + I^4/P^3, where I is the inode size in bytes (typically 4096, though different values are possible), and P is the pointer size, in bytes (4). This yields a maximum theoretical size of 4,402,345,721,856 bytes.
EXT3 Inode pointer structure
What limits the size of a memory-mapped file? I know it can't be bigger than the largest continuous chunk of unallocated address space, and that there should be enough free disk space. But are there other limits?
You're being too conservative: A memory-mapped file can be larger than the address space. The view of the memory-mapped file is limited by OS memory constraints, but that's only the part of the file you're looking at at one time. (And I guess technically you could map multiple views of discontinuous parts of the file at once, so aside from overhead and page length constraints, it's only the total # of bytes you're looking at that poses a limit. You could look at bytes [0 to 1024] and bytes [240 to 240 + 1024] with two separate views.)
In MS Windows, look at the MapViewOfFile function. It effectively takes a 64-bit file offset and a 32-bit length.
This has been my experience when using memory-mapped files under Win32:
If your map the entire file into one segment, it normally taps out at around 750 MB, because it can't find a bigger contiguous block of memory. If you split it up into smaller segments, say 100MB each, you can get around 1500MB-1800MB depending on what else is running.
If you use the /3g switch you can get more than 2GB up to about 2700MB but OS performance is penalized.
I'm not sure about 64-bit, I've never tried it but I presume the max file size is then limited only by the amount of physical memory you have.
Under Windows: "The size of a file view is limited to the largest available contiguous block of unreserved virtual memory. This is at most 2 GB minus the virtual memory already reserved by the process. "
From MDSN.
I'm not sure about LINUX/OSX/Whatever Else, but it's probably also related to address space.
Yes, there are limits to memory-mapped files. Most shockingly is:
Memory-mapped files cannot be larger than 2GB on 32-bit systems.
When a memmap causes a file to be created or extended beyond its current size in the filesystem, the contents of the new part are unspecified. On systems with POSIX filesystem semantics, the extended part will be filled with zero bytes.
Even on my 64-bit, 32GB RAM system, I get the following error if I try to read in one big numpy memory-mapped file instead of taking portions of it using byte-offsets:
Overflow Error: memory mapped size must be positive
Big datasets are really a pain to work with.
The limit of virtual address space is >16 Terabyte on 64Bit Windows systems. The issue discussed here is most probably related to mixing DWORD with SIZE_T.
There should be no other limits. Aren't those enough? ;-)