lk/linux-kernel-test
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

[PATCH] __vm_enough_memory() signedness fix

Browse Source 
We have found what seems to be a small bug in __vm_enough_memory() when
sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.

When this bug occurs the systems fails to boot, with /sbin/init whining
about fork() returning ENOMEM.

We hunted down the problem to this:

The deferred update mecanism used in vm_acct_memory(), on a SMP system,
allows the vm_committed_space counter to have a negative value.

This should not be a problem since this counter is known to be inaccurate.

But in __vm_enough_memory() this counter is compared to the `allowed'
variable, which is an unsigned long.  This comparison is broken since it
will consider the negative values of vm_committed_space to be huge positive
values, resulting in a memory allocation failure.

Signed-off-by: <Jean-Marc.Saffroy@ext.bull.net>
Signed-off-by: <Simon.Derr@bull.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
This commit is contained in:
Simon Derr
2005-08-04 19:52:03 -07:00
committed by Linus Torvalds
parent b68e9f8572
commit 2f60f8d357
2 changed files with 10 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
mm/mmap.c
View File

@@ -143,7 +143,11 @@ int __vm_enough_memory(long pages, int cap_sys_admin)
leave 3% of the size of this process for other processes */ leave 3% of the size of this process for other processes */
allowed -= current->mm->total_vm / 32; allowed -= current->mm->total_vm / 32;
if (atomic_read(&vm_committed_space) < allowed) /*
* cast `allowed' as a signed long because vm_committed_space
* sometimes has a negative value
*/
if (atomic_read(&vm_committed_space) < (long)allowed)
return 0; return 0;
vm_unacct_memory(pages); vm_unacct_memory(pages);

6
mm/nommu.c
View File

@@ -1167,7 +1167,11 @@ int __vm_enough_memory(long pages, int cap_sys_admin)
leave 3% of the size of this process for other processes */ leave 3% of the size of this process for other processes */
allowed -= current->mm->total_vm / 32; allowed -= current->mm->total_vm / 32;
if (atomic_read(&vm_committed_space) < allowed) /*
* cast `allowed' as a signed long because vm_committed_space
* sometimes has a negative value
*/
if (atomic_read(&vm_committed_space) < (long)allowed)
return 0; return 0;
vm_unacct_memory(pages); vm_unacct_memory(pages);