Mmap fails when length is larger the 4G?

If you do not actually have significantly more than 8G of swap configured, then that large mapping is likely to fail You can add the MAP_NORESERVE flag to mmap() to tell it not to reserve any swap space for the mapping up front.

If you do not actually have significantly more than 8G of swap configured, then that large mapping is likely to fail. You can add the MAP_NORESERVE flag to mmap() to tell it not to reserve any swap space for the mapping up front.

Great and thanks! After adding this flag, mmap will success. Actually I only have 2GB mem + 2GB swap.

– Zhiqiang Ma Jan 26 at 12:24 You should be aware that MAP_NORESERVE is completely ignored on systems configured for strict commit charge (no overcommit). – R.. Sep 22 at 16:20.

Just ran your code on Fedora 13 and it produces result 2. Check errno when mmap() returns MAP_FAILED (-1). You can also stick the following line before and after mmap call to see if you've got space in the virtual address space of the process for a 4GB region: system("pmap -x $$"); Update: The above actually prints the map of the child process.

Correct code: char buf0x100; snprintf(buf, sizeof buf, "pmap -x %u", (unsigned)getpid()); system(buf).

Very clever trick :) Thanks! – sarnold Jan 26 at 10:12 @Maxim Thanks! I have added system("pmap -x $$") before and after mmap call.

I have posted the updated result (update 2). I am not sure I use it in the right way. Please help me to take a look at it.

– Zhiqiang Ma Jan 26 at 10:18 Use the updated code please. – Maxim Yegorushkin Jan 26 at 10:34 1 Ooops, silly me; the $$ gets the shell pid. The pmap is dumping the wrong map.

Replace the system("pmap -x $$") with something like char pid10; sprintf(pid, "%i", getpid()); execlp("pmap",pid, (char *)NULL); – sarnold Jan 26 at 10:35 @sarnold: I have changed the code and print the new result (Update 3). I generate the right command and execute it by calling system. The code you give seems missing the '-x' parameter.

I think the range (0x100000000 to 0x200000000) isn't be used according to the output of pmap. – Zhiqiang Ma Jan 26 at 10:50.

Linux has two distinct modes for address space allocation: Memory allocation on write (i.e. Overcommit mode) or memory allocation on address space allocation. You can check by reading two files in procfs: cat /proc/sys/vm/overcommit_memory cat /proc/sys/vm/overcommit_ratio If overcommit_memory is not 0, then every address space allocation must be backed by physical memory (RAM + swap space), if overcommit_memory is 0, then memory is overcommited, i.e.

The kernel will happily hand out address space, but the memory will be only allocated if data is writen to the allocated address space. And then memory is not allocated for the full reserved address space, but only for those pages that are touched. This is kinda like booking a flight ticket: Airlines usually sell more tickets than there are seats on a flight, expecting not all booked passengers will actually show up.

Now you may wonder, what happens if all programs make use of the full space… Well then some nasty thing kicks in: The Linux Out Of Memory Killer will wreak havoc on your system and very likely kill those processes you need the most, due to it's arcane heuristics. Overcommit_ratio tells the kernel in overcommit mode to which ratio physical memory may be overcommited, i.e. How much more address space may be handed out, than there is physical memory.

In non-overcommit-mode how much spare memory to keep So maybe the overcommit mode just differs between the systems.

1 Linux VM accounting actually has three modes: Heuristic overcommit (0), Always overcommit (1) and Never overcommit (2). – caf Jan 26 at 12:14 Thanks a lot! On both systems, the overcommit_ratio is 50 while the overcommit_memory is 0.

I only have 2GB memory + 2GB swap. After adding MAP_NORESERVE flag as suggested by @caf, it can mmap 8GB memory. – Zhiqiang Ma Jan 26 at 12:23.

Since you try to map to a specific address, it will depend on the current memory layout for your process when you call mmap. The strategy at which address the request is fulfill is system dependent, the linux man page says something of a "hint". So maybe in the first case there simply not enough room in the virtual address space of your process to fulfill the request, since there is already another mapping in the way in that range.

A good idea to check if this is related to that would be to check if you succeed when you don't give the addr hint.

Thanks for the suggestion! I change the addr to NULL in mmap call, but it still fails and gives the "Cannot allocate memory" error message. – Zhiqiang Ma Jan 26 at 10:35.

Try adding system("ulimit -m -v"); to print out the amount of memory and address space that may be allocated. EDIT: Well, I'm out of ideas.Sorry. After cleaning up the errors and warnings in the code, I have this source: #include #include #include #include int main(void) { uint64_t* rr_addr = 0; uint64_t I = 17179869184; printf("\nsizeof(size_t): %lu\n", sizeof(size_t)); printf("(uint64_t)0x100000000: %lx\n", (uint64_t)0x100000000); printf("1L So 00007f37b3fa0000 0 16 16 r---- libc-2.12.1.

So 00007f37b3fa4000 0 4 4 rw--- libc-2.12.1. So 00007f37b3fa5000 0 12 12 rw--- anon 00007f37b3faa000 0 108 0 r-x-- ld-2.12.1.So 00007f37b41aa000 0 12 12 rw--- anon 00007f37b41c7000 0 12 12 rw--- anon 00007f37b41ca000 0 4 4 r---- ld-2.12.1. So 00007f37b41cb000 0 4 4 rw--- ld-2.12.1.

So 00007f37b41cc000 0 4 4 rw--- anon 00007fff70cf8000 0 12 12 rw--- stack 00007fff70dff000 0 4 0 r-x-- anon ffffffffff600000 0 0 0 r-x-- anon ---------------- ------ ------ ------ total kB 3912 464 88 pmap -x 23819 23819: . /zhiqiang Address Kbytes RSS Dirty Mode Mapping 0000000000400000 0 4 0 r-x-- zhiqiang 0000000000600000 0 4 4 r---- zhiqiang 0000000000601000 0 4 4 rw--- zhiqiang 0000000400000000 0 0 0 rw--- anon 00007f37b3c27000 0 260 0 r-x-- libc-2.12.1. So 00007f37b3da1000 0 0 0 ----- libc-2.12.1.

So 00007f37b3fa0000 0 16 16 r---- libc-2.12.1.So 00007f37b3fa4000 0 4 4 rw--- libc-2.12.1. So 00007f37b3fa5000 0 12 12 rw--- anon 00007f37b3faa000 0 108 0 r-x-- ld-2.12.1. So 00007f37b41aa000 0 12 12 rw--- anon 00007f37b41c7000 0 12 12 rw--- anon 00007f37b41ca000 0 4 4 r---- ld-2.12.1.So 00007f37b41cb000 0 4 4 rw--- ld-2.12.1.

So 00007f37b41cc000 0 4 4 rw--- anon 00007fff70cf8000 0 12 12 rw--- stack 00007fff70dff000 0 4 0 r-x-- anon ffffffffff600000 0 0 0 r-x-- anon ---------------- ------ ------ ------ total kB 8392520 464 88 rr_addr: 0x400000000, 17179869184 And details of my system: Linux haig 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux gcc version 2.12 (Ubuntu/Linaro 4.4. 4-14ubuntu5) GNU C Library (Ubuntu EGLIBC 2.12.1-0ubuntu10.1) stable release version 2.12.1, by Roland McGrath et al.

Looks like it: ENOMEM No memory is available, or the process's maximum number of mappings would have been exceeded. – Maxim Yegorushkin Jan 26 at 11:02 I have added this command just before calling mmap. The output is shown in 'Update 4'.

It shows 'unlimited' for both "max memory size" and "virtual memory". – Zhiqiang Ma Jan 26 at 11:04 Thanks all the same for your help! I have also tried different systems with different versions of kernel/gcc/libc.

Only on my laptop with Fedora 14 (system 1), this program fails. – Zhiqiang Ma Jan 26 at 11:24 @Zhiqiang Ma, if @datenwolf doesn't have the right answer, I think it's time to file a bug report with Fedora :) – sarnold Jan 26 at 11:30 caf's gives the right solution. Mmap tries to reserver space for the 8GB memory while my computer don't have.

Adding MAP_NORESERVE flag will avoid reserving the space. That's not a bug from Fedora, it is my code's bug. But it's nice to find it out.

– Zhiqiang Ma Jan 26 at 12:27.

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.