Use CPU_POKE hypervisor call to resume idle cpu if supported.
Signed-off-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
Reviewed-by: Anthony Yznaga <anthony.yznaga@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This adds a new hypercall CPU_POKE for quickly waking up an idle CPU.
CPU_POKE should only be sent to valid non-local CPUs.
Signed-off-by: Rob Gardner <rob.gardner@oracle.com>
Signed-off-by: Vijay Kumar <vijay.ac.kumar@oracle.com>
Reviewed-by: Anthony Yznaga <anthony.yznaga@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Nitin Gupta says:
====================
sparc64: Add 16GB hugepage support
SPARC architecture supports 16G hugepages but the kernel did not
support these. This patch series adds support for it and also cleanes
up some page walk/alloc functions.
Patch 1/3: Core changes needed to add 16G hugepage support: To map a
single 16G hugepage, two PUD entries are used. Each PUD entry maps
8G portion of a 16G page. This page table encoding scheme is same as
that used for hugepages at PMD level (8M, 256M and 2G pages) where
each PMD entry points successively to 8M regions within a page. No
page table entries below the PUD level are allocated for 16G
hugepage since those are not required.
TSB entries for a 16G page are created at every 4M boundary since
the HUGE_TSB is used for these pages which is configured with page
size of 4M. When walking page tables (on a TSB miss), bits [32:22]
are transferred from vaddr to PUD to resolve addresses at 4M
boundary. The resolved address mapping is then stored in HUGE_TSB.
Patch 2/3: get_user_pages() etc. are used for direct IO. These
functions were not aware of hugepages at the PUD level and would try
to continue walking page tables beyond the PUD level. Since 16G
hugepages have page tables allocated till PUD level only, these
accesses would result in invalid access. This patch adds the case
for PUD huge pages to these functions.
Patch 3/3: Patch 1 added the case of PUD entry being huge in page
table walk and alloc functions. This new case further increased
nesting in these functions and made them harder to follow. This
patch flattens these functions for better readability.
Cc: sparclinux@vger.kernel.org
Changelog v5 vs v4:
- Checking at PUD level for hugepage entry during page table walk is
patched out if 16GB hugepages are not being used.
Changelog v4 vs v3:
- Added cover letter (patch 0/4) for patch series.
Changelog v3 vs v2:
- Fixed email headers so the subject shows up correctly.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
Flatten out nested code structure in huge_pte_offset()
and huge_pte_alloc().
Signed-off-by: Nitin Gupta <nitin.m.gupta@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Adds support for 16GB hugepage size. To use this page size
use kernel parameters as:
default_hugepagesz=16G hugepagesz=16G hugepages=10
Testing:
Tested with the stream benchmark which allocates 48G of
arrays backed by 16G hugepages and does RW operation on
them in parallel.
Orabug: 25362942
Signed-off-by: Nitin Gupta <nitin.m.gupta@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
get_user_pages() is used to do direct IO. It already
handles the case where the address range is backed
by PMD huge pages. This patch now adds the case where
the range could be backed by PUD huge pages.
Signed-off-by: Nitin Gupta <nitin.m.gupta@oracle.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
The current context logic only updates the descriptor of context when
it's being pinned to graphics memory space. But this cannot satisfy the
requirement of shadow context. The addressing mode of the pinned shadow
context descriptor may be changed according to the guest addressing mode.
And this won't be updated, as the already pinned shadow context has no
chance to update its descriptor. And this will lead to GPU hang issue,
as shadow context is used with wrong descriptor. This patch fixes this
issue by letting the pinned shadow context descriptor update its
addressing mode on demand.
This patch fixes GPU HANG issue which happends after changing the
grub parameter i915.enable_ppgtt form 0x01 to 0x03 or vice versa and
then rebooting the guest.
Signed-off-by: Tina Zhang <tina.zhang@intel.com>
Signed-off-by: Kechen Lu <kechen.lu@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
This exposes vGPU context hw id in mdev sysfs which is used to
do vGPU based profiling. Retrieved vGPU context hw id can be set
through i915 perf ioctl to set profiling for target vGPU.
Cc: Jiao Pengyuan <pengyuan.jiao@intel.com>
Cc: Niu Bing <bing.niu@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
When doing the VGPU reset, we don't need to do the gtt/ppgtt reset.
This will make the GVT to do the ppgtt shadow every time for
a workload and caused really bad performance after a VGPU reset.
This patch will make sure ppgtt clean only happen at device module
level reset to fix this.
Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
When debugging the gtt code, found the intel_vgpu_gma_to_gpa() can
translate any given GMA though the GMA is not valid. This because
the GTT ops suppress the possible errors, which may result in an
invalid PT entry is retrieved by upper caller.
This patch changed the prototype of pte ops to propagate status to
callers. Then we make sure the GTT walker stop as early as when
a error is detected to prevent undefined behavior.
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Remove duplicated MMIO entries in the tracked MMIO list. -EEXIST
is returned if duplicated MMIO entries are found when new MMIO
entry is added.
v2:
- Use WARN(1, ...) for more verbose message. (Zhenyu)
Signed-off-by: Jian Jun Chen <jian.jun.chen@intel.com>
Cc: Zhi Wang <zhi.a.wang@intel.com>
Cc: Changbin Du <changbin.du@intel.com>
Cc: Zhenyu Wang <zhenyuw@linux.intel.com>
Reviewed-by: Yulei Zhang <yulei.zhang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Need to take runtime pm when do early scan/shadow of workload
for request operations.
Fixes: 7fa56bd159bc ("drm/i915/gvt: Audit and shadow workload during ELSP writing")
Cc: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Use the exist function intel_gvt_ggtt_validate_range to replace
these duplicated code that do the same thing.
Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
The function workload scan and shadow have to hold the drm.struct_mutex
before called. To avoid misusing of this function, add a lockdep assert
in it.
Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Let the workload audit and shadow ahead of vGPU scheduling, that
will eliminate GPU idle time and improve performance for multi-VM.
The performance of Heaven running simultaneously in 3VMs has
improved 20% after this patch.
v2:Remove condition current->vgpu==vgpu when shadow during ELSP
writing.
Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
To perform the workload scan and shadow in ELSP writing stage for
performance consideration, the workload scan and shadow stuffs
should be factored out from dispatch_workload().
v2:Put context pin before i915_add_request;
Refine the comments;
Rename some APIs;
v3:workload->status should set only when error happens.
v4:i915_add_request is must to have after i915_gem_request_alloc.
Signed-off-by: Ping Gao <ping.a.gao@intel.com>
Reviewed-by: Zhi Wang <zhi.a.wang@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
The I915_READ/WRITE is not only a mmio read/write, it also contains
debug checking and Forcewake domain lookup. This is too heavy for
GVT ring switch case which access batch of mmio registers on ring
switch. We can handle Forcewake manually and use the raw
i915_read/write instead. The benefit from this is 2x faster mmio
switch performance.
Before After
cycles ~550000 ~250000
v2: Use existing I915_READ_FW/I915_WRITE_FW macro. (zhenyu)
Signed-off-by: Changbin Du <changbin.du@intel.com>
Reviewed-by: Zhenyu Wang <zhenyuw@linux.intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
There are lots of POSTING_READ alongside each mmio write Op. While
actually this is not necessary. It just bring too much latency since
PCIe read Op is very slow which is of non-posted transaction.
For PCIe device, the mem transaction for strong ordering rules are:
o PCIe mmio write sequence is FIFO. Posted request cannot
pass previous posted request.
o PCIe mmio read will not go ahead of previous write.
Intel graphics doesn't support RO, so we can apply above rules. In
our case, we only need one POSTING_READ at last. This can remove
half of mmio read Op and then the average ring switch performance
is nearly doubled.
Before After
cycles ~970000 ~550000
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
It is better to use gvt_err when the gvt resource is not enough so
the user can be notified from the kernel dmesg. And this kind of
error message is gvt related.
Suggested-by: Bing Niu <bing.niu@intel.com>
Signed-off-by: Chuanxiao Dong <chuanxiao.dong@intel.com>
Cc: Bing Niu <bing.niu@intel.com>
Signed-off-by: Zhenyu Wang <zhenyuw@linux.intel.com>
The blk_mq_delay_kick_requeue_list() function is used by the device
mapper and only by the device mapper to rerun the queue and requeue
list after a delay. This function is called once per request that
gets requeued. Modify this function such that the queue is run once
per path change event instead of once per request that is requeued.
Fixes: commit 2849450ad3 ("blk-mq: introduce blk_mq_delay_kick_requeue_list()")
Signed-off-by: Bart Van Assche <bart.vanassche@wdc.com>
Cc: Mike Snitzer <snitzer@redhat.com>
Cc: Laurence Oberman <loberman@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
This gets us back to the behavior in 4.12 and earlier.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Fixes: 7c20f116 ("bio-integrity: stop abusing bi_end_io")
Signed-off-by: Jens Axboe <axboe@kernel.dk>
In dm-integrity target we register integrity profile that have
both generate_fn and verify_fn callbacks set to NULL.
This is used if dm-integrity is stacked under a dm-crypt device
for authenticated encryption (integrity payload contains authentication
tag and IV seed).
In this case the verification is done through own crypto API
processing inside dm-crypt; integrity profile is only holder
of these data. (And memory is owned by dm-crypt as well.)
After the commit (and previous changes)
Commit 7c20f11680
Author: Christoph Hellwig <hch@lst.de>
Date: Mon Jul 3 16:58:43 2017 -0600
bio-integrity: stop abusing bi_end_io
we get this crash:
: BUG: unable to handle kernel NULL pointer dereference at (null)
: IP: (null)
: *pde = 00000000
...
:
: Workqueue: kintegrityd bio_integrity_verify_fn
: task: f48ae180 task.stack: f4b5c000
: EIP: (null)
: EFLAGS: 00210286 CPU: 0
: EAX: f4b5debc EBX: 00001000 ECX: 00000001 EDX: 00000000
: ESI: 00001000 EDI: ed25f000 EBP: f4b5dee8 ESP: f4b5dea4
: DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
: CR0: 80050033 CR2: 00000000 CR3: 32823000 CR4: 001406d0
: Call Trace:
: ? bio_integrity_process+0xe3/0x1e0
: bio_integrity_verify_fn+0xea/0x150
: process_one_work+0x1c7/0x5c0
: worker_thread+0x39/0x380
: kthread+0xd6/0x110
: ? process_one_work+0x5c0/0x5c0
: ? kthread_worker_fn+0x100/0x100
: ? kthread_worker_fn+0x100/0x100
: ret_from_fork+0x19/0x24
: Code: Bad EIP value.
: EIP: (null) SS:ESP: 0068:f4b5dea4
: CR2: 0000000000000000
Patch just skip the whole verify workqueue if verify_fn is set to NULL.
Fixes: 7c20f116 ("bio-integrity: stop abusing bi_end_io")
Signed-off-by: Milan Broz <gmazyland@gmail.com>
[hch: trivial whitespace fix]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Fixes hitting WARN_ON() during initialisation of pre-NV50 GPUs, caused
by the recent changes to support pad macro routing on GM20x.
We currently don't use them here for older GPUs anyway.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Default config value for all other drivers is N.
Signed-off-by: Michał Mirosław <mirq-linux@rere.qmqm.pl>
Signed-off-by: Dave Airlie <airlied@redhat.com>
UAPI Changes:
- vc4: Add ioctl to allow attaching a label to a bo (Eric)
- Add new format/modifier blob plane property (Ben)
- armada: Use __u32/__u64 instead of uint32_t/uint64_t (Mikko)
- [kinda uapi] fb_helper: Expose display_info size via fb_info (David)
Core Changes:
- Default gem_dumb_[map_offset|destroy] as mmap/destroy implementations (Noralf)
- Simplify atomic properties by removing the helpers and handling in core (Daniel)
Driver Changes:
- stm: Add STM32 DSI controller driver (Phillipe)
- vc4: Add HDMI CEC support (Hans)
- rockchip: Refactor register init & soc version handling (Mark)
- misc: Remove .load_lut, .gamma_set, .gamma_get dead code (Peter)
- dw-hdmi: Add HDMI CEC support (Russell)
Cc: Philippe CORNU <philippe.cornu@st.com>
Cc: Hans Verkuil <hans.verkuil@cisco.com>
Cc: Eric Anholt <eric@anholt.net>
Cc: Noralf Trønnes <noralf@tronnes.org>
Cc: Ben Widawsky <ben@bwidawsk.net>
Cc: Mark yao <mark.yao@rock-chips.com>
Cc: Peter Rosin <peda@axentia.se>
Cc: Russell King <rmk+kernel@armlinux.org.uk>
Cc: Mikko Rapeli <mikko.rapeli@iki.fi>
Cc: David Lechner <david@lechnology.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
* tag 'drm-misc-next-2017-08-08' of git://anongit.freedesktop.org/git/drm-misc: (107 commits)
drm: Nuke drm_atomic_legacy_backoff
drm: Nuke drm_atomic_helper_connector_dpms
drm: Nuke drm_atomic_helper_connector_set_property
drm: Nuke drm_atomic_helper_plane_set_property
drm: Nuke drm_atomic_helper_crtc_set_property
drm: Handle properties in the core for atomic drivers
drm: Don't update property values for atomic drivers
drm/omap: Rework the rotation-on-crtc hack
drm/radeon: Use the drm_driver.dumb_destroy default
drm/i915: Use the drm_driver.dumb_destroy default
drm/sti: Use .dumb_map_offset and .dumb_destroy defaults
drm: bridge: synopsys/dw-hdmi: Provide default configuration function for HDMI 2.0 PHY
drm/fb-helper: pass physical dimensions to fbdev
uapi drm/armada_drm.h: use __u32 and __u64 instead of uint32_t and uint64_t
drm/bridge: dw-hdmi: remove CEC engine register definitions
drm/bridge: dw-hdmi: add cec driver
drm/bridge: dw-hdmi: add missing cec_notifier_put
drm: remove unused and redundant callbacks
staging: vboxvideo: remove dead gamma lut code
drm: dw-hdmi-i2s: add missing company name on Copyright
...
When cur_valid_map passes the f2fs_test_and_set(,clear)_bit test,
cur_valid_map_mir update is skipped unlikely, so fix it. The fix
now changes the mirror check together with cur_valid_map all the
time.
Signed-off-by: Yunlong Song <yunlong.song@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: Fix unused variable and add unlikely for corner condition.]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
drm/i915 fixes for v4.13-rc5
* tag 'drm-intel-fixes-2017-08-09-1' of git://anongit.freedesktop.org/git/drm-intel:
drm/i915: fix backlight invert for non-zero minimum brightness
drm/i915/shrinker: Wrap need_resched() inside preempt-disable
drm/i915/perf: fix flex eu registers programming
drm/i915: Fix out-of-bounds array access in bdw_load_gamma_lut
drm/i915/gvt: Change the max length of mmio_reg_rw from 4 to 8
drm/i915/gvt: Initialize MMIO Block with HW state
drm/i915/gvt: clean workload queue if error happened
drm/i915/gvt: change resetting to resetting_eng
Core Changes:
- dma-buf: Allow multiple sync_files to wrap a single dma-fence (Chris)
Driver Changes:
- rockchip: misc fixes to vop driver from the downstream rockchip tree (Mark)
- Error path cleanups to tc358767 & host1x (Lucas & Paul, respectively)
* tag 'drm-misc-fixes-2017-08-08' of git://anongit.freedesktop.org/git/drm-misc:
drm/rockchip: vop: report error when check resource error
drm/rockchip: vop: round_up pitches to word align
drm/rockchip: vop: fix NV12 video display error
drm/rockchip: vop: fix iommu page fault when resume
dma-buf/sync_file: Allow multiple sync_files to wrap a single dma-fence
drm/bridge: tc358767: fix probe without attached output node
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJZisBiAAoJEFc4NIkMQxK4yioP/RYKLRvm0bmHFVW5rhZLx/Gt
GBqn/2rJ9fWklW2iIzWjRGIkz93BDl1e6kaPz4liL2I/jKXaoKwnxsY0zOm7UxXd
np/CEPH0oevpKNUtEsUqEsVNzk3OULG5BCY7NPThCBtlKXtSzmF/HtZgyEBUlB4V
wsA/jWWgnken/JK9GPMZNfBOCC7AoTLtMAa2grVsW8XTZN9Lw80zDD8hhWbyd+Gf
PFsPLJwHRFMwn0TajNnK3dwvd07IKnctMn5P31B5XBmifBOhqbZI4IxXHcCgSv3H
vbcjLvzmGaSZvFQxXn7HNHC3iC4luzj/y3Wytb1B2/VEXQxIY0SxJVmFuYjg1ZUw
wwZ9qXPAHTTPIyKKCI54bU5g++EALPBkVqmeQztWx9PBR941M7z85l6xXcjldn0s
7q4EPV5QCgpDUzqVxEdrc+gg7X7h8qsJL1zVd7iEmzXbjjQrGFkZrRKg13Wkm+8B
sZ6wDaTiWxBOM+EDbTueYodJ7rz85ujmxTsapTFhrhmhR8GdHMwke1ymD8jJ8Meb
qqBJzbz8qD/6BvdhhldTVI5/uRmPCOSHYW9Yq5G5rBH8b8PNOaRYEBxLldsdZz8F
yBNCzZojkFLMeqJVM6DvI5x3Ke/eNYiZCCf2OPy9smjWYwcVWuOUDeoPS56frw7R
XUaYa3J7gs/dBg8bBsUT
=CUpE
-----END PGP SIGNATURE-----
Merge tag 'exynos-drm-fixes-for-v4.13-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/daeinki/drm-exynos into drm-fixes
Fix a issue to display system memory region outside a gem buffer.
* tag 'exynos-drm-fixes-for-v4.13-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/daeinki/drm-exynos:
drm/exynos: forbid creating framebuffers from too small GEM buffers
Bunch of msm fixes for 4.13
* 'msm-fixes-4.13-rc3' of git://people.freedesktop.org/~robclark/linux:
drm/msm: gpu: don't abuse dma_alloc for non-DMA allocations
drm/msm: gpu: call qcom_mdt interfaces only for ARCH_QCOM
drm/msm/adreno: Prevent unclocked access when retrieving timestamps
drm/msm: Remove __user from __u64 data types
drm/msm: args->fence should be args->flags
drm/msm: Turn off hardware clock gating before reading A5XX registers
drm/msm: Allow hardware clock gating to be toggled
drm/msm: Remove some potentially blocked register ranges
drm/msm/mdp5: Drop clock names with "_clk" suffix
drm/msm/mdp5: Fix typo in encoder_enable path
drm/msm: NULL pointer dereference in drivers/gpu/drm/msm/msm_gem_vma.c
drm/msm: fix WARN_ON in add_vma() with no iommu
drm/msm/dsi: Calculate link clock rates with updated dsi->lanes
drm/msm/mdp5: fix unclocked register access in _cursor_set()
drm/msm: unlock on error in msm_gem_get_iova()
drm/msm: fix an integer overflow test
drm/msm/mdp5: Fix compilation warnings
Florian Westphal says:
====================
rtnetlink: allow selected handlers to run without rtnl
Changes since v1:
In patch 6, don't make ipv6 route handlers lockless, they all have
assumptions on rtnl being held. Other patches are unchanged.
The RTNL mutex is used to serialize both rtnetlink calls and
dump requests.
Its also used to protect other things such as the list of current
net namespaces.
Unfortunately RTNL mutex is a performance issue, e.g. a cpu adding an
ip address prevents other cpus from seemingly unrelated tasks such as
dumping tc classifiers or doing rtnetlink route lookups.
This patch set adds basic infrastructure to start pushing the rtnl lock
down to those places that need it, or even elide it entirely in some cases.
Subsystems can now indicate that their doit() callback can run without
RTNL mutex, such callbacks can then run in parallel.
This will obviously need a lot of followup work; all current
users need to be audited/changed to benefit from this.
Initial no-rtnl spot is netns new/getid.
We have various 'get' handlers that are also a tempting target,
however, several of these depend on rtnl mutex to prevent information
from changing while objects are being read by rtnl handlers; however,
it doesn't appear impossible to change this.
Dumps are another problem entirely, see
commit 2907c35ff6 ("net: hold rtnl again in dump callbacks"),
this patchset doesn't touch dump requests.
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
Both functions take nsid_lock and don't rely on rtnl lock.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Allow callers to tell rtnetlink core that its doit callback
should be invoked without holding rtnl mutex.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Note that netlink dumps still acquire rtnl mutex via the netlink
dump infrastructure.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
instead of rtnl lock/unload at the top level, push it down
to the called function.
This is just an intermediate step, next commit switches protection
of the rtnl_link ops table to rcu, in which case (for dumps) the
rtnl lock is acquired only by the netlink dumper infrastructure
(current lock/unlock/dump/lock/unlock rtnl sequence becomes
rcu lock/rcu unlock/dump).
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
I don't see what prevents rmmod (unregister_all is called) while a dump
is active.
Even if we'd add rtnl lock/unlock pair to unregister_all (as done here),
thats not enough either as rtnl_lock is released right before the dump
process starts.
So this adds a refcount:
* acquire rtnl mutex
* bump refcount
* release mutex
* start the dump
... and make unregister_all remove the callbacks (no new dumps possible)
and then wait until refcount is 0.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
This change allows us to later indicate to rtnetlink core that certain
doit functions should be called without acquiring rtnl_mutex.
This change should have no effect, we simply replace the last (now
unused) calcit argument with the new flag.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
There is only a single place in the kernel that regisers the "calcit"
callback (to determine min allocation for dumps).
This is in rtnetlink.c for PF_UNSPEC RTM_GETLINK.
The function that checks for calcit presence at run time will first check
the requested family (which will always fail for !PF_UNSPEC as no callsite
registers this), then falls back to checking PF_UNSPEC.
Therefore we can just check if type is RTM_GETLINK and then do a direct
call. Because of fallback to PF_UNSPEC all RTM_GETLINK types used this
regardless of family.
This has the advantage that we don't need to allocate space for
the function pointer for all the other families.
A followup patch will drop the calcit function pointer from the
rtnl_link callback structure.
Signed-off-by: Florian Westphal <fw@strlen.de>
Reviewed-by: Hannes Frederic Sowa <hannes@stressinduktion.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Daniel Borkmann says:
====================
bpf: Add BPF_J{LT,LE,SLT,SLE} instructions
This set adds BPF_J{LT,LE,SLT,SLE} instructions to the BPF
insn set, interpreter, JIT hardening code and all JITs are
also updated to support the new instructions. Basic idea is
to reduce register pressure by avoiding BPF_J{GT,GE,SGT,SGE}
rewrites. Removing the workaround for the rewrites in LLVM,
this can result in shorter BPF programs, less stack usage
and less verification complexity. First patch provides some
more details on rationale and integration.
Thanks a lot!
v1 -> v2:
- Reworded commit msg in patch 1
====================
Signed-off-by: David S. Miller <davem@davemloft.net>
Add test cases to the verifier selftest suite in order to verify that
i) direct packet access, and ii) dynamic map value access is working
with the changes related to the new instructions.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Enable the newly added jump opcodes, main parts are in two
different areas, namely direct packet access and dynamic map
value access. For the direct packet access, we now allow for
the following two new patterns to match in order to trigger
markings with find_good_pkt_pointers():
Variant 1 (access ok when taking the branch):
0: (61) r2 = *(u32 *)(r1 +76)
1: (61) r3 = *(u32 *)(r1 +80)
2: (bf) r0 = r2
3: (07) r0 += 8
4: (ad) if r0 < r3 goto pc+2
R0=pkt(id=0,off=8,r=0) R1=ctx R2=pkt(id=0,off=0,r=0)
R3=pkt_end R10=fp
5: (b7) r0 = 0
6: (95) exit
from 4 to 7: R0=pkt(id=0,off=8,r=8) R1=ctx
R2=pkt(id=0,off=0,r=8) R3=pkt_end R10=fp
7: (71) r0 = *(u8 *)(r2 +0)
8: (05) goto pc-4
5: (b7) r0 = 0
6: (95) exit
processed 11 insns, stack depth 0
Variant 2 (access ok on fall-through):
0: (61) r2 = *(u32 *)(r1 +76)
1: (61) r3 = *(u32 *)(r1 +80)
2: (bf) r0 = r2
3: (07) r0 += 8
4: (bd) if r3 <= r0 goto pc+1
R0=pkt(id=0,off=8,r=8) R1=ctx R2=pkt(id=0,off=0,r=8)
R3=pkt_end R10=fp
5: (71) r0 = *(u8 *)(r2 +0)
6: (b7) r0 = 1
7: (95) exit
from 4 to 6: R0=pkt(id=0,off=8,r=0) R1=ctx
R2=pkt(id=0,off=0,r=0) R3=pkt_end R10=fp
6: (b7) r0 = 1
7: (95) exit
processed 10 insns, stack depth 0
The above two basically just swap the branches where we need
to handle an exception and allow packet access compared to the
two already existing variants for find_good_pkt_pointers().
For the dynamic map value access, we add the new instructions
to reg_set_min_max() and reg_set_min_max_inv() in order to
learn bounds. Verifier test cases for both are added in a
follow-up patch.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: John Fastabend <john.fastabend@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This work implements jiting of BPF_J{LT,LE} instructions with
BPF_X/BPF_K variants for the nfp eBPF JIT. The two BPF_J{SLT,SLE}
instructions have not been added yet given BPF_J{SGT,SGE} are
not supported yet either.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This work implements jiting of BPF_J{LT,LE,SLT,SLE} instructions
with BPF_X/BPF_K variants for the ppc64 eBPF JIT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Tested-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This work implements jiting of BPF_J{LT,LE,SLT,SLE} instructions
with BPF_X/BPF_K variants for the s390x eBPF JIT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Michael Holzheu <holzheu@linux.vnet.ibm.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This work implements jiting of BPF_J{LT,LE,SLT,SLE} instructions
with BPF_X/BPF_K variants for the sparc64 eBPF JIT.
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
