Given some common flag combinations, particularly -Os, gcc will inline
rcu_read_unlock_special() despite its being in an unlikely() clause.
Use noinline to prohibit this misoptimization.
In addition, move the second barrier() in __rcu_read_unlock() so that
it is not on the common-case code path. This will allow the compiler to
generate better code for the common-case path through __rcu_read_unlock().
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
The RCU_BOOST commits for TREE_PREEMPT_RCU introduced an other-task
write to a new RCU_READ_UNLOCK_BOOSTED bit in the task_struct structure's
->rcu_read_unlock_special field, but, as noted by Steven Rostedt, without
correctly synchronizing all accesses to ->rcu_read_unlock_special.
This could result in bits in ->rcu_read_unlock_special being spuriously
set and cleared due to conflicting accesses, which in turn could result
in deadlocks between the rcu_node structure's ->lock and the scheduler's
rq and pi locks. These deadlocks would result from RCU incorrectly
believing that the just-ended RCU read-side critical section had been
preempted and/or boosted. If that RCU read-side critical section was
executed with either rq or pi locks held, RCU's ensuing (incorrect)
calls to the scheduler would cause the scheduler to attempt to once
again acquire the rq and pi locks, resulting in deadlock. More complex
deadlock cycles are also possible, involving multiple rq and pi locks
as well as locks from multiple rcu_node structures.
This commit fixes synchronization by creating ->rcu_boosted field in
task_struct that is accessed and modified only when holding the ->lock
in the rcu_node structure on which the task is queued (on that rcu_node
structure's ->blkd_tasks list). This results in tasks accessing only
their own current->rcu_read_unlock_special fields, making unsynchronized
access once again legal, and keeping the rcu_read_unlock() fastpath free
of atomic instructions and memory barriers.
The reason that the rcu_read_unlock() fastpath does not need to access
the new current->rcu_boosted field is that this new field cannot
be non-zero unless the RCU_READ_UNLOCK_BLOCKED bit is set in the
current->rcu_read_unlock_special field. Therefore, rcu_read_unlock()
need only test current->rcu_read_unlock_special: if that is zero, then
current->rcu_boosted must also be zero.
This bug does not affect TINY_PREEMPT_RCU because this implementation
of RCU accesses current->rcu_read_unlock_special with irqs disabled,
thus preventing races on the !SMP systems that TINY_PREEMPT_RCU runs on.
Maybe-reported-by: Dave Jones <davej@redhat.com>
Maybe-reported-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Reported-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
PREEMPT_RCU read-side critical sections blocking an expedited grace
period invoke rcu_report_exp_rnp(). When the last such critical section
has completed, rcu_report_exp_rnp() invokes the scheduler to wake up the
task that invoked synchronize_rcu_expedited() -- needlessly holding the
root rcu_node structure's lock while doing so, thus needlessly providing
a way for RCU and the scheduler to deadlock.
This commit therefore releases the root rcu_node structure's lock before
calling wake_up().
Reported-by: Ed Tomlinson <edt@aei.ca>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
.. compile options. This way the user can decide during runtime whether they
want the default 'vpci' (virtual pci passthrough) or where the PCI devices
are passed in without any BDF renumbering. The option 'passthrough' allows
the user to toggle the it from 0 (vpci) to 1 (passthrough).
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
The latter is easily fixed - by the developer compiling the
module with -DDEBUG. And during runtime - the loglvl provides
quite a lot of useful data.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
- Remove the slot and controller controller backend as they
are not used.
- Document the find pciback_[read|write]_config_[byte|word|dword]
to make it easier to find.
- Collapse the code from conf_space_capability_msi into pciback_ops.c
- Collapse conf_space_capability_[pm|vpd].c in conf_space_capability.c
[and remove the conf_space_capability.h file]
- Rename all visible functions from pciback to xen_pcibk.
- Rename all the printk/pr_info, etc that use the "pciback" to say
"xen-pciback".
- Convert functions that are not referenced outside the code to be
static to save on name space.
- Do the same thing for structures that are internal to the driver.
- Run checkpatch.pl after the renames and fixup its warnings and
fix any compile errors caused by the variable rename
- Cleanup any structs that checkpath.pl commented about or just
look odd.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
If the verbose_request is set (and loglevel high enough), print out
the MSI/MSI-X values that are sent to the guest. This should aid in
debugging issues.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
If we try to setup an fake IRQ handler for legacy interrupts
for devices that only have MSI-X (most if not all SR-IOV cards),
we will fail with this:
pciback[0000:01:10.0]: failed to install fake IRQ handler for IRQ 0! (rc:-38)
Since those cards don't have anything in dev->irq.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
We were using coarse spinlocks that could end up with a deadlock.
This patch fixes that and makes the spinlocks much more fine-grained.
We also drop be->watchding state spinlocks as they are already
guarded by the xenwatch_thread against multiple customers. Without
that we would trigger the BUG: scheduling while atomic.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
If the device that is to be shared with a guest is a level device and
the IRQ is shared with the initial domain we need to take actions.
Mainly we install a dummy IRQ handler that will ACK on the interrupt
line so as to not have the initial domain disable the interrupt line.
This dummy IRQ handler is not enabled when the device MSI/MSI-X lines
are set, nor for edge interrupts. And also not for level interrupts
that are not shared amongst devices. Lastly, if the user passes
to the guest all of the PCI devices on the shared line the we won't
install the dummy handler either.
There is also SysFS instrumentation to check its state and turn
IRQ ACKing on/off if necessary.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
In cases where the guest is abruptly killed and has not disabled
MSI/MSI-X interrupts we want to do it for it.
Otherwise when the guest is started up and enables MSI, we would
get a WARN() that the device already had been enabled.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
These changes are for PV guest to use Virtual Function. Because the VF's
vendor, device registers in cfg space are 0xffff, which are invalid and
ignored by PCI device scan. Values in 'struct pci_dev' are fixed up by
SR-IOV code, and using these values will present correct VID and DID to
PV guest kernel.
And command registers in the cfg space are read only 0, which means we
have to emulate MMIO enable bit (VF only uses MMIO resource) so PV
kernel can work properly.
Acked-by: Jan Beulich <jbeulich@novell.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
When the front-end and back-end start negotiating we register
the domain that will use the PCI device. Furthermore during shutdown
of guest or unbinding of the PCI device (and unloading of module)
from pciback we unregister the domain owner.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Checkpatch found some extra warnings and errors. This mega
patch fixes them all in one big swoop. We also spruce
up the pcistub_ids to use DEFINE_PCI_DEVICE_TABLE macro
(suggested by Jan Beulich).
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
This is the host side counterpart to the frontend driver in
drivers/pci/xen-pcifront.c. The PV protocol is also implemented by
frontend drivers in other OSes too, such as the BSDs.
The PV protocol is rather simple. There is page shared with the guest,
which has the 'struct xen_pci_sharedinfo' embossed in it. The backend
has a thread that is kicked every-time the structure is changed and
based on the operation field it performs specific tasks:
XEN_PCI_OP_conf_[read|write]:
Read/Write 0xCF8/0xCFC filtered data. (conf_space*.c)
Based on which field is probed, we either enable/disable the PCI
device, change power state, read VPD, etc. The major goal of this
call is to provide a Physical IRQ (PIRQ) to the guest.
The PIRQ is Xen hypervisor global IRQ value irrespective of the IRQ
is tied in to the IO-APIC, or is a vector. For GSI type
interrupts, the PIRQ==GSI holds. For MSI/MSI-X the
PIRQ value != Linux IRQ number (thought PIRQ==vector).
Please note, that with Xen, all interrupts (except those level shared ones)
are injected directly to the guest - there is no host interaction.
XEN_PCI_OP_[enable|disable]_msi[|x] (pciback_ops.c)
Enables/disables the MSI/MSI-X capability of the device. These operations
setup the MSI/MSI-X vectors for the guest and pass them to the frontend.
When the device is activated, the interrupts are directly injected in the
guest without involving the host.
XEN_PCI_OP_aer_[detected|resume|mmio|slotreset]: In case of failure,
perform the appropriate AER commands on the guest. Right now that is
a cop-out - we just kill the guest.
Besides implementing those commands, it can also
- hide a PCI device from the host. When booting up, the user can specify
xen-pciback.hide=(1:0:0)(BDF..) so that host does not try to use the
device.
The driver was lifted from linux-2.6.18.hg tree and fixed up
so that it could compile under v3.0. Per suggestion from Jesse Barnes
moved the driver to drivers/xen/xen-pciback.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Allow multiple workqueue items (locks with callbacks) to be
processed concurrently. There should be no reason not to
take advantage of this workqueue feature.
Signed-off-by: David Teigland <teigland@redhat.com>
The following commit renames irq_gc_ack() to irq_gc_ack_set_bit(),
and makes gpio-mxc and gpio-mxs fail to build.
659fb32d1b
genirq: replace irq_gc_ack() with {set,clr}_bit variants (fwd)
The patch fixed a couple of typo of comma to semicolon.
Signed-off-by: Shawn Guo <shawn.guo@linaro.org>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
open(2) must always include one of O_RDONLY, O_WRONLY, or O_RDWR. No need
for any O_APPEND special case.
Passing O_WRONLY|O_RDWR is undefined according to the man page, but the
Linux VFS interprets this as O_RDWR, so we'll do the same.
This fixes open(2) with flags O_RDWR|O_APPEND, which was incorrectly being
translated to readonly.
Reported-by: Fyodor Ustinov <ufm@ufm.su>
Signed-off-by: Sage Weil <sage@newdream.net>
create DCB related states in function state-machine
allow handling of DCB errors from FW
allow disablement of DCB in FW, when peer disappears or error
clean up unused functions/variables as pointed by
David Binderman <dcb314@hotmail.com>
Reported-by: David Binderman <dcb314@hotmail.com>
Signed-off-by: Dmitry Kravkov <dmitry@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
It may take some time to cnic to respond, this prevents tx_timeout
when it happens.
Signed-off-by: Dmitry Kravkov <dmitry@broadcom.com>
Signed-off-by: Eilon Greenstein <eilong@broadcom.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
e542a2269f (r8169: adjust the RxConfig settings)
broke the return from promiscuous mode to physical address match mode.
Signed-off-by: Francois Romieu <romieu@fr.zoreil.com>
Cc: Signed-off-by: Hayes Wang <hayeswang@realtek.com>
As we now only update used ring after enabling
the backend, we can write flags with __put_user:
as that's done on data path, it matters.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Fix get/put refcount imbalance with zero copy,
which caused qemu to hang forever on guest driver unload.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
We need to log writes when updating used flags and avail event
fields. Otherwise the guest may see a stale value after migration and
miss notifying the host.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Move the used ring initialization after backend was set. This
makes it possible to disable the backend and tweak the used ring,
then restart. This will also make it possible to log the used ring
write correctly.
Signed-off-by: Jason Wang <jasowang@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Introduces a new nfnetlink type that applies a given
verdict to all queued packets with an id <= the id in the verdict
message.
If a mark is provided it is applied to all matched packets.
This reduces the number of verdicts that have to be sent.
Applications that make use of this feature need to maintain
a timeout to send a batchverdict periodically to avoid starvation.
Signed-off-by: Florian Westphal <fw@strlen.de>
Signed-off-by: Patrick McHardy <kaber@trash.net>
Packet identifier is currently setup in nfqnl_build_packet_message(),
using one atomic_inc_return().
Problem is that since several cpus might concurrently call
nfqnl_enqueue_packet() for the same queue, we can deliver packets to
consumer in non monotonic way (packet N+1 being delivered after packet
N)
This patch moves the packet id setup from nfqnl_build_packet_message()
to nfqnl_enqueue_packet() to guarantee correct delivery order.
This also removes one atomic operation.
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
CC: Florian Westphal <fw@strlen.de>
CC: Pablo Neira Ayuso <pablo@netfilter.org>
CC: Eric Leblond <eric@regit.org>
Signed-off-by: Patrick McHardy <kaber@trash.net>
Since printk_ratelimit() shouldn't be used anymore (see comment in
include/linux/printk.h), replace it with printk_ratelimited().
Signed-off-by: Manuel Zerpies <manuel.f.zerpies@ww.stud.uni-erlangen.de>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Use the per port counter attached to all QPs created on that port to
implement port level packets/bytes performance counters a la IB.
Derived from a patch by Eli Cohen <eli@mellanox.co.il>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Allocate flow counter per Ethernet/IBoE port, and attach this counter
to all the QPs created on that port. Based on patch by Eli Cohen
<eli@mellanox.co.il>.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Move the various definitions and mad structures needed for software
implementation of IBA PM agent from the ipath and qib drivers into a
single include file, which in turn could be used by more consumers.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
ConnectX devices support a set of flow counters that can be attached
to a set containing one or more QPs. Each such counter tracks receive
and transmit packets and bytes of these QPs. This patch queries the
device to check support for counters, handles initialization of the
HCA to enable counters, and initializes a bitmap allocator to control
counter allocations. Derived from patch by Eli Cohen <eli@mellanox.co.il>.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Fix the address handle portion of the QP context structure to have the
correct bit location for the counter index field.
Signed-off-by: Eli Cohen <eli@mellanox.co.il>
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
Query another dword containing up to 32 extended device capabilities
and merge it into struct mlx4_caps.flags. Update the code that
handles the current extended device capabilities (e.g UDP RSS, WoL,
vep steering, etc) to use the extended device cap flags field instead
of a field per extended capability. Initial patch done by Eli Cohen
<eli@mellanox.co.il>.
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@purestorage.com>
