lk/linux-kernel-test
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
9b2e4f1880
linux-kernel-test/kernel
History
Paul E. McKenney 9b2e4f1880 rcu: Track idleness independent of idle tasks 
Earlier versions of RCU used the scheduling-clock tick to detect idleness
by checking for the idle task, but handled idleness differently for
CONFIG_NO_HZ=y.  But there are now a number of uses of RCU read-side
critical sections in the idle task, for example, for tracing.  A more
fine-grained detection of idleness is therefore required.

This commit presses the old dyntick-idle code into full-time service,
so that rcu_idle_enter(), previously known as rcu_enter_nohz(), is
always invoked at the beginning of an idle loop iteration.  Similarly,
rcu_idle_exit(), previously known as rcu_exit_nohz(), is always invoked
at the end of an idle-loop iteration.  This allows the idle task to
use RCU everywhere except between consecutive rcu_idle_enter() and
rcu_idle_exit() calls, in turn allowing architecture maintainers to
specify exactly where in the idle loop that RCU may be used.

Because some of the userspace upcall uses can result in what looks
to RCU like half of an interrupt, it is not possible to expect that
the irq_enter() and irq_exit() hooks will give exact counts.  This
patch therefore expands the ->dynticks_nesting counter to 64 bits
and uses two separate bitfields to count process/idle transitions
and interrupt entry/exit transitions.  It is presumed that userspace
upcalls do not happen in the idle loop or from usermode execution
(though usermode might do a system call that results in an upcall).
The counter is hard-reset on each process/idle transition, which
avoids the interrupt entry/exit error from accumulating.  Overflow
is avoided by the 64-bitness of the ->dyntick_nesting counter.

This commit also adds warnings if a non-idle task asks RCU to enter
idle state (and these checks will need some adjustment before applying
Frederic's OS-jitter patches (http://lkml.org/lkml/2011/10/7/246).
In addition, validation of ->dynticks and ->dynticks_nesting is added.

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
2011-12-11 10:31:24 -08:00
..
debug Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
events perf: Do no try to schedule task events if there are none 2011-12-07 16:31:22 +01:00
gcov gcov: disable CONSTRUCTORS for UML 2011-07-26 16:49:45 -07:00
irq genirq: Fix race condition when stopping the irq thread 2011-12-02 11:54:24 +01:00
power PM / Hibernate: Do not leak memory in error/test code paths 2011-11-23 21:03:38 +01:00
time rcu: Track idleness independent of idle tasks 2011-12-11 10:31:24 -08:00
trace ftrace: Fix hash record accounting bug 2011-12-05 13:28:47 -05:00
.gitignore
acct.c
async.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
audit_tree.c audit_tree,rcu: Convert call_rcu(__put_tree) to kfree_rcu() 2011-07-20 14:10:11 -07:00
audit_watch.c kill path_lookup() 2011-03-14 09:15:23 -04:00
audit.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
audit.h
auditfilter.c netlink: kill loginuid/sessionid/sid members from struct netlink_skb_parms 2011-03-03 10:55:40 -08:00
auditsc.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
backtracetest.c
bounds.c memcg: remove direct page_cgroup-to-page pointer 2011-03-23 19:46:28 -07:00
capability.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
cgroup_freezer.c cgroup_freezer: fix freezing groups with stopped tasks 2011-11-24 11:58:22 -08:00
cgroup.c memcg: replace ss->id_lock with a rwlock 2011-11-02 16:07:03 -07:00
compat.c kernel: Fix files explicitly needing EXPORT_SYMBOL infrastructure 2011-10-31 19:30:05 -04:00
configs.c kernel/configs.c: include MODULE_*() when CONFIG_IKCONFIG_PROC=n 2011-07-25 20:57:15 -07:00
cpu_pm.c cpu_pm: call notifiers during suspend 2011-09-23 12:05:29 +05:30
cpu.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
cpuset.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
crash_dump.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
cred.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
delayacct.c KVM: Steal time implementation 2011-07-14 12:59:14 +03:00
dma.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
elfcore.c
exec_domain.c
exit.c oom: remove oom_disable_count 2011-10-31 17:30:45 -07:00
extable.c extable, core_kernel_data(): Make sure all archs define _sdata 2011-05-20 08:56:56 +02:00
fork.c writeback: remove vm_dirties and task->dirties 2011-11-17 20:49:06 +08:00
freezer.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
futex_compat.c userns: user namespaces: convert several capable() calls 2011-03-23 19:47:08 -07:00
futex.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
groups.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
hrtimer.c Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2011-11-28 08:43:52 -08:00
hung_task.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
irq_work.c kernel: fix two implicit header assumptions in irq_work.c 2011-10-31 09:20:12 -04:00
itimer.c
jump_label.c jump_label: jump_label_inc may return before the code is patched 2011-12-05 13:28:46 -05:00
kallsyms.c Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2011-03-25 17:52:22 -07:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks arch:Kconfig.locks Remove unused config option. 2011-04-10 17:01:05 +02:00
Kconfig.preempt sched: Isolate preempt counting in its own config option 2011-06-10 15:15:40 +02:00
kexec.c [S390] kdump: Add infrastructure for unmapping crashkernel memory 2011-10-30 15:16:42 +01:00
kfifo.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
kmod.c kmod: prevent kmod_loop_msg overflow in __request_module() 2011-10-26 13:10:39 +10:30
kprobes.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
ksysfs.c kernel: ksysfs.c is implicitly using stat.h 2011-10-31 09:20:13 -04:00
kthread.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
latencytop.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
lockdep_internals.h
lockdep_proc.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
lockdep_states.h
lockdep.c lockdep, kmemcheck: Annotate ->lock in lockdep_init_map() 2011-12-06 18:18:13 +01:00
Makefile Merge branch 'devel-stable' of http://ftp.arm.linux.org.uk/pub/linux/arm/kernel/git-cur/linux-2.6-arm 2011-10-28 12:02:27 -07:00
module.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
mutex-debug.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mutex-debug.h mutex: Use p->on_cpu for the adaptive spin 2011-04-14 08:52:33 +02:00
mutex.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
mutex.h mutex: Use p->on_cpu for the adaptive spin 2011-04-14 08:52:33 +02:00
notifier.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
nsproxy.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
padata.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
panic.c module,bug: Add TAINT_OOT_MODULE flag for modules not built in-tree 2011-11-07 07:54:42 +10:30
params.c kernel: params.c needs module.h not moduleparam.h 2011-10-31 09:20:13 -04:00
pid_namespace.c pidns: call pid_ns_prepare_proc() from create_pid_namespace() 2011-03-23 19:46:58 -07:00
pid.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
posix-cpu-timers.c Merge branch 'core-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2011-10-26 16:17:32 +02:00
posix-timers.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
printk.c printk: avoid double lock acquire 2011-12-09 07:50:28 -08:00
profile.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
ptrace.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
range.c range: fix bogus misuse of module.h to get printk() 2011-10-31 09:20:11 -04:00
rcu.h rcu: Add grace-period, quiescent-state, and call_rcu trace events 2011-09-28 21:38:21 -07:00
rcupdate.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rcutiny_plugin.h kernel: fix up module header handling in rcutiny files 2011-10-31 09:20:13 -04:00
rcutiny.c rcu: Track idleness independent of idle tasks 2011-12-11 10:31:24 -08:00
rcutorture.c rcu: Make rcu_torture_boost() exit loops at end of test 2011-09-28 21:38:46 -07:00
rcutree_plugin.h rcu: Make synchronize_sched_expedited() better at work sharing 2011-12-11 10:31:22 -08:00
rcutree_trace.c rcu: Track idleness independent of idle tasks 2011-12-11 10:31:24 -08:00
rcutree.c rcu: Track idleness independent of idle tasks 2011-12-11 10:31:24 -08:00
rcutree.h rcu: Track idleness independent of idle tasks 2011-12-11 10:31:24 -08:00
relay.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
res_counter.c memcg: res_counter_read_u64(): fix potential races on 32-bit machines 2011-03-23 19:46:22 -07:00
resource.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rtmutex_common.h
rtmutex-debug.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rtmutex-debug.h
rtmutex-tester.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rtmutex.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
rtmutex.h
rwsem.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
sched_autogroup.c Fix common misspellings 2011-03-31 11:26:23 -03:00
sched_autogroup.h sched: Skip autogroup when looking for all rt sched groups 2011-07-01 10:39:08 +02:00
sched_clock.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
sched_cpupri.c sched/cpupri: Remove cpupri->pri_active 2011-08-14 12:01:11 +02:00
sched_cpupri.h sched/cpupri: Remove cpupri->pri_active 2011-08-14 12:01:11 +02:00
sched_debug.c sched: Get rid of lock_depth 2011-04-24 13:18:38 +02:00
sched_fair.c sched: Fix buglet in return_cfs_rq_runtime() 2011-11-16 08:43:45 +01:00
sched_features.h sched, rt: Provide means of disabling cross-cpu bandwidth sharing 2011-11-14 12:50:40 +01:00
sched_idletask.c sched: Drop the rq argument to sched_class::select_task_rq() 2011-04-14 08:52:36 +02:00
sched_rt.c sched, rt: Provide means of disabling cross-cpu bandwidth sharing 2011-11-14 12:50:40 +01:00
sched_stats.h locking, sched: Annotate thread_group_cputimer as raw 2011-09-13 11:11:55 +02:00
sched_stoptask.c sched: Implement hierarchical task accounting for SCHED_OTHER 2011-08-14 12:01:13 +02:00
sched.c sched: Set the command name of the idle tasks in SMP kernels 2011-11-14 12:50:43 +01:00
seccomp.c
semaphore.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
signal.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
smp.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
softirq.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
spinlock.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
srcu.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
stacktrace.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
stop_machine.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
sys_ni.c Cross Memory Attach 2011-10-31 17:30:44 -07:00
sys.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
sysctl_binary.c ipv4: NET_IPV4_ROUTE_GC_INTERVAL removal 2011-10-03 14:13:01 -04:00
sysctl_check.c xfs: remove subdirectories 2011-08-12 16:21:35 -05:00
sysctl.c Merge branch 'akpm' (Andrew's incoming) 2011-10-31 17:46:07 -07:00
taskstats.c Make TASKSTATS require root access 2011-09-19 17:04:37 -07:00
test_kprobes.c
time.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
timeconst.pl
timer.c sys_getppid: add missing rcu_dereference 2011-12-09 07:50:29 -08:00
tracepoint.c Tracepoint: Dissociate from module mutex 2011-08-10 20:38:14 -04:00
tsacct.c Make taskstats round statistics down to nearest 1k bytes/events 2011-09-19 17:10:57 -07:00
uid16.c userns: user namespaces: convert several capable() calls 2011-03-23 19:47:08 -07:00
up.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
user_namespace.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
user-return-notifier.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
user.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
utsname_sysctl.c Merge branch 'modsplit-Oct31_2011' of git://git.kernel.org/pub/scm/linux/kernel/git/paulg/linux 2011-11-06 19:44:47 -08:00
utsname.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
wait.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00
watchdog.c watchdog: move watchdog_*_all_cpus under CONFIG_SYSCTL 2011-10-31 17:30:53 -07:00
workqueue_sched.h
workqueue.c kernel: Map most files to use export.h instead of module.h 2011-10-31 09:20:12 -04:00