Add Niklas as co-maintainer of the ata subsystem (aka libata). The new
shared tree for libata will now be
"pub/scm/linux/kernel/git/libata/linux" on kernel.org GIT.
Niklas and I will alternate maintainership every release.
Signed-off-by: Damien Le Moal <dlemoal@kernel.org>
Acked-by: Niklas Cassel <cassel@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQGzBAABCgAdFiEE6fsu8pdIjtWE/DpLiiy9cAdyT1EFAmWM/+gACgkQiiy9cAdy
T1H6NAv/a/hJO2dQB8/7Ts7Y3XTDP0hqvJEu2RRx0LuK5ERSTr7zEUgDP0W8HYpO
IHvlOnOCa3n68EJGVa9OolYRWgAIX2qYXrGx4+iEWJaLIDsdPgyZdfU0YgwjdmgD
xHjadidNfmrahfc/r6s8C9HBE9J/LVpFyHnCmasAw7V4p+7mBL/Ycwkm3BmbX0Sp
/gSX5ITOPZdGJl97OS7elkA7t2ABLaJXeHPKCHgKYiwx9TbpH75WxM2Kpfsk2LaO
yaEWW/JTgr+Vg+EYUTumKGONhKpgx+rgqIGiZYwWu42aEXp+53JM8f67FWihDvL5
2t4uei8GaeuXRrYTSq/ylfy10V9BOcraf3SV/pJ6C9NvAWUoGiVKRFj62BjZSxJp
E8Pdj4wy97hFa9gaEyMg81w02CvGe19R1ikLzSXA8GJLnlg+rBLzEt/1g0gQ6YI/
vj8Ett6m5ezkW1Q7V3yCvjDN9IrYV0q138ZBtcjJm9P9EWHXzpDKXoUVcaXGNw31
gM5eRPNE
=/4UC
-----END PGP SIGNATURE-----
Merge tag '6.7rc7-smb3-srv-fix' of git://git.samba.org/ksmbd
Pull ksmbd server fix from Steve French:
- address possible slab out of bounds in parsing of open requests
* tag '6.7rc7-smb3-srv-fix' of git://git.samba.org/ksmbd:
ksmbd: fix slab-out-of-bounds in smb_strndup_from_utf16()
Constify few pointers to iomem, where the destination memory is not
modified, for code safety and readability.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org>
Reviewed-by: Linus Walleij <linus.walleij@linaro.org>
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Link: https://lore.kernel.org/r/20231223191902.22857-1-krzysztof.kozlowski@linaro.org
Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
Just a few fixes: besides a few one liners, we have a fix for snapshots
+ compression where the extent update path didn't account for the fact
that with snapshots, we might split an existing extent into three, not
just two; and a small fixup for promotes which were broken by the recent
changes in the data update path to correctly take into account device
durability.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEEKnAFLkS8Qha+jvQrE6szbY3KbnYFAmWMU08ACgkQE6szbY3K
bnZ22Q//dTTPeGkHDIT0lk/8Jbn6zcIh2nvrstR2BUBLlrFzeKJEL1nhMBnhbcxZ
99NcjKPirumINeOyH9c0ZzRn+Q/Mr799yOQ1FgmGyYcmP2DkxK/3eiTzb/lVD6PW
MKnHantKCDkaBptRl9pTzHlSir7IbyzRejTwrLbO+zSZxlEuticfaGhfnDBGeVnK
MZIwnYOFWs9mIzGaYMTBXjQD0/Q38Z+mPR+1kgcz/cAyoYSblRoWiZ3qfy1LMpRB
FVfKK2xXXa0z1Z3NAbtoiNmIvZOlwWNqApY5jIDX5EVKmQahXxxnR+kcFp545bd+
Lp78ULJhSDNxs1XZZM44lUV52ZPoVfqz6SXs2hs3kxFcdFdWtpMpfWLjqm4sZtvK
9gnXckurLtHVhdQdSbQ1n1iGVdx1qgs9znAjlvSzQEKr+ba3RzdUABsRvGjnqtgy
V5xBv02Xa/gF5Uk7YonDqm3/aY6QhhU4bDlGlRzveZOm8ktwElqrkeE54FveuEy1
9IasPoUNVMu6urDO4xr7LpY3s3u06TnNHhj0TKllqY1mWZoj4COr+IwizUyff0Nz
W1s0wKaSOX4eLcntJLmfzZ4429JTyR2AJbA1j3VnxGF3jaLhvyeSD1pstBqVicSJ
wXg5PHN/XM1AQlEcpNv2pSrt2u//9+UOPnk9H18EY0tXvo/B91g=
=JTkX
-----END PGP SIGNATURE-----
Merge tag 'bcachefs-2023-12-27' of https://evilpiepirate.org/git/bcachefs
Pull bcachefs fixes from Kent Overstreet:
"Just a few fixes: besides a few one liners, we have a fix for
snapshots + compression where the extent update path didn't account
for the fact that with snapshots, we might split an existing extent
into three, not just two; and a small fixup for promotes which were
broken by the recent changes in the data update path to correctly take
into account device durability"
* tag 'bcachefs-2023-12-27' of https://evilpiepirate.org/git/bcachefs:
bcachefs: Fix promotes
bcachefs: Fix leakage of internal error code
bcachefs: Fix insufficient disk reservation with compression + snapshots
bcachefs: fix BCH_FSCK_ERR enum
Add a helper function to check if there are listeners for
thermal_gnl_family multicast groups.
For now use it to avoid unnecessary allocations and sending
thermal genl messages when there are no recipients.
In the future, in conjunction with (not yet implemented) notification
of change in the netlink socket group membership, this helper can be
used to open/close hardware interfaces based on the presence of
user space subscribers.
Signed-off-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Use enum instead of hard-coded numbers for indexing multicast groups.
Signed-off-by: Stanislaw Gruszka <stanislaw.gruszka@linux.intel.com>
Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
When freeing an object that was allocated from KFENCE, we do that in the
slowpath __slab_free(), relying on the fact that KFENCE "slab" cannot be
the cpu slab, so the fastpath has to fallback to the slowpath.
This optimization doesn't help much though, because is_kfence_address()
is checked earlier anyway during the free hook processing or detached
freelist building. Thus we can simplify the code by making the
slab_free_hook() free the KFENCE object immediately, similarly to KASAN
quarantine.
In slab_free_hook() we can place kfence_free() above init processing, as
callers have been making sure to set init to false for KFENCE objects.
This simplifies slab_free(). This places it also above kasan_slab_free()
which is ok as that skips KFENCE objects anyway.
While at it also determine the init value in slab_free_freelist_hook()
outside of the loop.
This change will also make introducing per cpu array caches easier.
Tested-by: Marco Elver <elver@google.com>
Reviewed-by: Chengming Zhou <zhouchengming@bytedance.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
tilcdc currently just ioremaps its iomem, without doing the (a bit more
robust) request on the memory first. The devm_ functions provide a handy
way to both request and ioremap the memory with automatic cleanup.
Replace the manual ioremap with the devm_ version.
Suggested-by: Thomas Zimmermann <tzimmermann@suse.de>
Signed-off-by: Philipp Stanner <pstanner@redhat.com>
Reviewed-by: Jyri Sarha <jyri.sarha@iki.fi>
Tested-by: Jyri Sarha <jyri.sarha@iki.fi>
Signed-off-by: Jyri Sarha <jyri.sarha@iki.fi>
Link: https://patchwork.freedesktop.org/patch/msgid/20231222115216.19218-2-pstanner@redhat.com
The ___kcrctab section holds an array of 32-bit CRC values.
Add a .balign 4 to tell the linker the correct memory alignment.
Fixes: f3304ecd7f ("linux/export: use inline assembler to populate symbol CRCs")
Signed-off-by: Helge Deller <deller@gmx.de>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
The resume of thermal zones in thermal_pm_notify() is carried out
sequentially, which may be a problem if __thermal_zone_device_update()
takes a significant time to run for some thermal zones, because some
other thermal zones may need to wait for them to resume then and if
any other PM notifiers are going to be invoked after the thermal one,
they will need to wait for it either.
To address this, make thermal_pm_notify() switch the poll_queue delayed
work over to a one-shot thermal_zone_device_resume() work function that
will restore the original one during the thermal zone resume and queue
up poll_queue without a delay for each thermal zone.
Link: https://lore.kernel.org/linux-pm/20231120234015.3273143-1-radusolea@google.com/
Reported-by: Radu Solea <radusolea@google.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
In preparation for a subsequent change, move the initialization of the
poll_queue delayed work from thermal_zone_device_register_with_trips()
to thermal_zone_device_init() which is called by the former.
However, because thermal_zone_device_init() is also called by
thermal_pm_notify(), make the latter call cancel_delayed_work() on
poll_queue before invoking the former, so as to allow the work
item to be re-initialized safely.
Also move thermal_zone_device_check() which needs to be defined
before thermal_zone_device_init(), so the latter can pass it to the
INIT_DELAYED_WORK() macro.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
There are 3 synchronization issues with thermal zone suspend-resume
during system-wide transitions:
1. The resume code runs in a PM notifier which is invoked after user
space has been thawed, so it can run concurrently with user space
which can trigger a thermal zone device removal. If that happens,
the thermal zone resume code may use a stale pointer to the next
list element and crash, because it does not hold thermal_list_lock
while walking thermal_tz_list.
2. The thermal zone resume code calls thermal_zone_device_init()
outside the zone lock, so user space or an update triggered by
the platform firmware may see an inconsistent state of a
thermal zone leading to unexpected behavior.
3. Clearing the in_suspend global variable in thermal_pm_notify()
allows __thermal_zone_device_update() to continue for all thermal
zones and it may as well run before the thermal_tz_list walk (or
at any point during the list walk for that matter) and attempt to
operate on a thermal zone that has not been resumed yet. It may
also race destructively with thermal_zone_device_init().
To address these issues, add thermal_list_lock locking to
thermal_pm_notify(), especially arount the thermal_tz_list,
make it call thermal_zone_device_init() back-to-back with
__thermal_zone_device_update() under the zone lock and replace
in_suspend with per-zone bool "suspend" indicators set and unset
under the given zone's lock.
Link: https://lore.kernel.org/linux-pm/20231218162348.69101-1-bo.ye@mediatek.com/
Reported-by: Bo Ye <bo.ye@mediatek.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Since all of the ACPI EC driver code runs in thread context after recent
changes, it does not need to disable interrupts on the local CPU when
acquiring a spin lock.
Make it use the spin lock without disabling interrupts.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
After commit 7a36b901a6 ("ACPI: OSL: Use a threaded interrupt handler
for SCI") all of the EC code runs in thread context on all systems where
EC events are signaled through a GPE.
It may as well run in thread context on systems using a dedicated IRQ
for EC events signaling, so make it use a threaded handler for that IRQ.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
After commit 7a36b901a6 ("ACPI: OSL: Use a threaded interrupt handler
for SCI") any ACPICA code never runs in a hardirq handler, so it need
not dissable interrupts on the local CPU when acquiring a spin lock.
Make it use spin locks without disabling interrupts.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
All usages of this struct have been removed from the kernel tree.
The struct is still referenced by scripts/check-sysctl-docs but that
script is broken anyways as it only supports the register_sysctl_paths()
API and not the currently used register_sysctl() one.
Fixes: 0199849acd ("sysctl: remove register_sysctl_paths()")
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Reviewed-by: Joel Granados <j.granados@samsung.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
It seems it was never used.
Fixes: 2f2665c13a ("sysctl: replace child with an enumeration")
Signed-off-by: Thomas Weißschuh <linux@weissschuh.net>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
This commit comes at the tail end of a greater effort to remove the
empty elements at the end of the ctl_table arrays (sentinels) which
will reduce the overall build time size of the kernel and run time
memory bloat by ~64 bytes per sentinel (further information Link :
https://lore.kernel.org/all/ZO5Yx5JFogGi%2FcBo@bombadil.infradead.org/)
Remove empty sentinel from coda_table
Signed-off-by: Joel Granados <j.granados@samsung.com>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
This commit comes at the tail end of a greater effort to remove the
empty elements at the end of the ctl_table arrays (sentinels) which
will reduce the overall build time size of the kernel and run time
memory bloat by ~64 bytes per sentinel (further information Link :
https://lore.kernel.org/all/ZO5Yx5JFogGi%2FcBo@bombadil.infradead.org/)
Remove empty sentinel element from test_table and test_table_unregister.
Signed-off-by: Joel Granados <j.granados@samsung.com>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
This commit comes at the tail end of a greater effort to remove the
empty elements at the end of the ctl_table arrays (sentinels) which
will reduce the overall build time size of the kernel and run time
memory bloat by ~64 bytes per sentinel (further information Link :
https://lore.kernel.org/all/ZO5Yx5JFogGi%2FcBo@bombadil.infradead.org/)
Remove sentinel elements ctl_table struct. Special attention was placed in
making sure that an empty directory for fs/verity was created when
CONFIG_FS_VERITY_BUILTIN_SIGNATURES is not defined. In this case we use the
register sysctl call that expects a size.
Signed-off-by: Joel Granados <j.granados@samsung.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: "Darrick J. Wong" <djwong@kernel.org>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
This commit comes at the tail end of a greater effort to remove the empty
elements at the end of the ctl_table arrays (sentinels) which will reduce the
overall build time size of the kernel and run time memory bloat by ~64 bytes
per sentinel (further information Link :
https://lore.kernel.org/all/ZO5Yx5JFogGi%2FcBo@bombadil.infradead.org/)
Remove sentinel from cachefiles_sysctls
Signed-off-by: Joel Granados <j.granados@samsung.com>
Acked-by: Christian Brauner <brauner@kernel.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
In some cases the result of test were hidden inside the stdout and it
was difficult to identify when a test was skipped and why.
List of changes
1. Capitalize all the words that express a test result : "OK", "SKIPPED"
and "FAIL".
2. Place all test result text at the end of the message. This will
prevent the result from being hidden when stdout is verbose.
3. Any other explanation that comes after the result text will be placed
in a new line.
4. All failures are marked as "FAIL"
5. Pipped the failure to stderr in tests 8, 9, 10.
6. Replaced bogus "FAIL" with "SKIPPED" in test 0007
7. All "..." are prefixed and followed by a space.
Signed-off-by: Joel Granados <j.granados@samsung.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Basic test to ensure that empty directories can be registered and that
they in turn can serve as a base dir for other registrations.
Add one test to the sysctl selftest module. It first registers an empty
directory under "empty_add" and then uses that as a base to register
another empty dir.
The sysctl bash script then checks that "empty_add" is present and that
there an empty directory within it.
Signed-off-by: Joel Granados <j.granados@samsung.com>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
When registering tables to the sysctl subsystem there is a check to see
if header is a permanently empty directory (used for mounts). This check
evaluates the first element of the ctl_table. This results in an out of
bounds evaluation when registering empty directories.
The function register_sysctl_mount_point now passes a ctl_table of size
1 instead of size 0. It now relies solely on the type to identify
a permanently empty register.
Make sure that the ctl_table has at least one element before testing for
permanent emptiness.
Signed-off-by: Joel Granados <j.granados@samsung.com>
Reported-by: kernel test robot <oliver.sang@intel.com>
Closes: https://lore.kernel.org/oe-lkp/202311201431.57aae8f3-oliver.sang@intel.com
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Joel Granados has been doing quite a bit of the work to help us move
forward with the proc sysctl cleanups, and is keen on helping and
so has agreed to help with maintenance of proc sysctl. Add him as
a maintainer.
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Iurii Zaikin has moved on to other projects and has had no time to
help with proc sysctl maintenance.
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
The cpu_key was not initialized in reiserfs_delete_solid_item(), which triggered
this issue.
Reported-and-tested-by: <syzbot+b3b14fb9f8a14c5d0267@syzkaller.appspotmail.com>
Signed-off-by: Edward Adam Davis <eadavis@qq.com>
Link: https://lore.kernel.org/r/tencent_9EA7E746DE92DBC66049A62EDF6ED64CA706@qq.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
When running 'make htmldocs', I see the following warning:
Documentation/filesystems/api-summary:14: ./include/linux/fs.h:1659: WARNING: Definition list ends without a blank line; unexpected unindent.
The official guidance [1] seems to be to use lists, which will prevent
both the "unexpected unindent" warning as well as ensure that each line
is formatted on a separate line in the HTML output instead of being
all considered a single paragraph.
[1]: https://docs.kernel.org/doc-guide/kernel-doc.html#return-values
Fixes: 8802e580ee ("fs: create __sb_write_started() helper")
Cc: Amir Goldstein <amir73il@gmail.com>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
Link: https://lore.kernel.org/r/20231228100608.3123987-1-vegard.nossum@oracle.com
Signed-off-by: Christian Brauner <brauner@kernel.org>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEqG5UsNXhtOCrfGQP+7dXa6fLC2sFAmWNRBEACgkQ+7dXa6fL
C2vQPBAAh36FEN5497raWKakNfFnsvctNO4b7dYkYVZsKDxf/tZY/FoUe1VKYp6m
49ZwBHp5kwmr1baWBqa5JS/t4SkgRYwXB0qp2J7maCA7vqglDKMyPVJgh+bohVHU
5pUvinrtk5BzVfEqU2FW7fhyr2uFNN4N+9UeSZBz7GIAMFaULovVIRGtYGLG2Rbk
fCx05eLSEh3DKarLvU8A32jNI3TfPXIwPULS8VxcR/G7Tl6iqDXRysofizCtU+zL
Xy/8KLp1juw1BYbHzv24A/GxJqTSHV1GayfRaAOi1fSDNTz4Wzmjmg6lpwio52gU
Z8VbBPaaBnu+83paWsKYUFMbr/rncIw7W83NCnLvn8SqWPp1cXYFRycjA5FN+mjj
E+DZoGT/4Y/tEg/vxlF39RO+lWjI6zzQU9qc6dEj0FFaEDM31x6ssjHXTB6l9bW0
VG3vFfUPnNigOYbB/eJfcx96aAOlzJPZrX1qK9fwJAuyT8ZBwEnwZg8keYqpAgoO
jfyVLaPQ8fSEp/raFnTeyS1CUpeYq6VPCg2EKUgu+L/IestYkkP53zShXtzfIB1Y
LrZmZvX+8lIOJ/WHHYbGLUUqr/tOzzvg1mNyXWX7XaGfCN+HWpoobcAY2jTKKGnG
6aYz3uP/Tc6khy1gVSw2P82pk3OEiJy5mnHQh8cRrUY8R93n3pE=
=1ta4
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQRAhzRXHqcMeLMyaSiRxhvAZXjcogUCZY1PVgAKCRCRxhvAZXjc
oieIAP9PZUj2r7frbtczRiIC9VukSjiPuXvcvkX9E8p54GlyRwD/cRSlawwptR6F
Q0LvrjdgaxW5ppwkgJvVdm9+52BdZwY=
=9fJW
-----END PGP SIGNATURE-----
Merge tag 'netfs-lib-20231228' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs
Pull netfs updates from David Howells:
The main aims of these patches are to get high-level I/O and knowledge of
the pagecache out of the filesystem drivers as much as possible and to get
rid, as much of possible, of the knowledge that pages/folios exist.
Further, I would like to see ->write_begin, ->write_end and
->launder_folio go away.
Features that are added by these patches to that which is already there in
netfslib:
(1) NFS-style (and Ceph-style) locking around DIO vs buffered I/O calls to
prevent these from happening at the same time. mmap'd I/O can, of
necessity, happen at any time ignoring these locks.
(2) Support for unbuffered I/O. The data is kept in the bounce buffer and
the pagecache is not used. This can be turned on with an inode flag.
(3) Support for direct I/O. This is basically unbuffered I/O with some
extra restrictions and no RMW.
(4) Support for using a bounce buffer in an operation. The bounce buffer
may be bigger than the target data/buffer, allowing for crypto
rounding.
(5) ->write_begin() and ->write_end() are ignored in favour of merging all
of that into one function, netfs_perform_write(), thereby avoiding the
function pointer traversals.
(6) Support for write-through caching in the pagecache.
netfs_perform_write() adds the pages is modifies to an I/O operation
as it goes and directly marks them writeback rather than dirty. When
writing back from write-through, it limits the range written back.
This should allow CIFS to deal with byte-range mandatory locks
correctly.
(7) O_*SYNC and RWF_*SYNC writes use write-through rather than writing to
the pagecache and then flushing afterwards. An AIO O_*SYNC write will
notify of completion when the sub-writes all complete.
(8) Support for write-streaming where modifed data is held in !uptodate
folios, with a private struct attached indicating the range that is
valid.
(9) Support for write grouping, multiplexing a pointer to a group in the
folio private data with the write-streaming data. The writepages
algorithm only writes stuff back that's in the nominated group. This
is intended for use by Ceph to write is snaps in order.
(10) Skipping reads for which we know the server could only supply zeros or
EOF (for instance if we've done a local write that leaves a hole in
the file and extends the local inode size).
General notes:
(1) The fscache module is merged into the netfslib module to avoid cyclic
exported symbol usage that prevents either module from being loaded.
(2) Some helpers from fscache are reassigned to netfslib by name.
(3) netfslib now makes use of folio->private, which means the filesystem
can't use it.
(4) The filesystem provides wrappers to call the write helpers, allowing
it to do pre-validation, oplock/capability fetching and the passing in
of write group info.
(5) I want to try flushing the data when tearing down an inode before
invalidating it to try and render launder_folio unnecessary.
(6) Write-through caching will generate and dispatch write subrequests as
it gathers enough data to hit wsize and has whole pages that at least
span that size. This needs to be a bit more flexible, allowing for a
filesystem such as CIFS to have a variable wsize.
(7) The filesystem driver is just given read and write calls with an
iov_iter describing the data/buffer to use. Ideally, they don't see
pages or folios at all. A function, extract_iter_to_sg(), is already
available to decant part of an iterator into a scatterlist for crypto
purposes.
AFS notes:
(1) I pushed a pair of patches that clean up the trace header down to the
base so that they can be shared with another branch.
9P notes:
(1) Most of xfstests now pass - more, in fact, since upstream 9p lacks a
writepages method and can't handle mmap writes. An occasional oops
(and sometimes panic) happens somewhere in the pathwalk/FID handling
code that is unrelated to these changes.
(2) Writes should now occur in larger-than-page-sized chunks.
(3) It should be possible to turn on multipage folio support in 9P now.
All in all these patches remove a little over 800 lines from AFS, 300
from 9P, albeit with around 3000 lines added to netfs. Hopefully, I will
be able to remove a bunch of lines from Ceph too.
I've split the CIFS patches out to a separate branch, cifs-netfs, where
a further 2000+ lines are removed. I can run a certain amount of
xfstests on CIFS, though I'm running into ksmbd issues and not all the
tests work correctly because of issues between fallocate and what the
SMB protocol actually supports.
I've also dropped the content-crypto patches out for the moment as
they're only usable by the ceph changes which I'm still working on.
The patch to use PG_writeback instead of PG_fscache for writing to the
cache has also been deferred, pending 9p, afs, ceph and cifs all being
converted.
* tag 'netfs-lib-20231228' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs: (40 commits)
9p: Use netfslib read/write_iter
afs: Use the netfs write helpers
netfs: Export the netfs_sreq tracepoint
netfs: Optimise away reads above the point at which there can be no data
netfs: Implement a write-through caching option
netfs: Provide a launder_folio implementation
netfs: Provide a writepages implementation
netfs, cachefiles: Pass upper bound length to allow expansion
netfs: Provide netfs_file_read_iter()
netfs: Allow buffered shared-writeable mmap through netfs_page_mkwrite()
netfs: Implement buffered write API
netfs: Implement unbuffered/DIO write support
netfs: Implement unbuffered/DIO read support
netfs: Allocate multipage folios in the writepath
netfs: Make netfs_read_folio() handle streaming-write pages
netfs: Provide func to copy data to pagecache for buffered write
netfs: Dispatch write requests to process a writeback slice
netfs: Prep to use folio->private for write grouping and streaming write
netfs: Make the refcounting of netfs_begin_read() easier to use
netfs: Make netfs_put_request() handle a NULL pointer
...
Signed-off-by: Christian Brauner <brauner@kernel.org>
Use netfslib's read and write iteration helpers, allowing netfslib to take
over the management of the page cache for 9p files and to manage local disk
caching. In particular, this eliminates write_begin, write_end, writepage
and all mentions of struct page and struct folio from 9p.
Note that netfslib now offers the possibility of write-through caching if
that is desirable for 9p: just set the NETFS_ICTX_WRITETHROUGH flag in
v9inode->netfs.flags in v9fs_set_netfs_context().
Note also this is untested as I can't get ganesha.nfsd to correctly parse
the config to turn on 9p support.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: Eric Van Hensbergen <ericvh@kernel.org>
cc: Latchesar Ionkov <lucho@ionkov.net>
cc: Dominique Martinet <asmadeus@codewreck.org>
cc: Christian Schoenebeck <linux_oss@crudebyte.com>
cc: v9fs@lists.linux.dev
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
Export the netfs_sreq tracepoint so that it can be called directly from
client filesystems/cache backend modules.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Track the file position above which the server is not expected to have any
data (the "zero point") and preemptively assume that we can satisfy
requests by filling them with zeroes locally rather than attempting to
download them if they're over that line - even if we've written data back
to the server. Assume that any data that was written back above that
position is held in the local cache. Note that we have to split requests
that straddle the line.
Make use of this to optimise away some reads from the server. We need to
set the zero point in the following circumstances:
(1) When we see an extant remote inode and have no cache for it, we set
the zero_point to i_size.
(2) On local inode creation, we set zero_point to 0.
(3) On local truncation down, we reduce zero_point to the new i_size if
the new i_size is lower.
(4) On local truncation up, we don't change zero_point.
(5) On local modification, we don't change zero_point.
(6) On remote invalidation, we set zero_point to the new i_size.
(7) If stored data is discarded from the pagecache or culled from fscache,
we must set zero_point above that if the data also got written to the
server.
(8) If dirty data is written back to the server, but not fscache, we must
set zero_point above that.
(9) If a direct I/O write is made, set zero_point above that.
Assuming the above, any read from the server at or above the zero_point
position will return all zeroes.
The zero_point value can be stored in the cache, provided the above rules
are applied to it by any code that culls part of the local cache.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Provide a flag whereby a filesystem may request that cifs_perform_write()
perform write-through caching. This involves putting pages directly into
writeback rather than dirty and attaching them to a write operation as we
go.
Further, the writes being made are limited to the byte range being written
rather than whole folios being written. This can be used by cifs, for
example, to deal with strict byte-range locking.
This can't be used with content encryption as that may require expansion of
the write RPC beyond the write being made.
This doesn't affect writes via mmap - those are written back in the normal
way; similarly failed writethrough writes are marked dirty and left to
writeback to retry. Another option would be to simply invalidate them, but
the contents can be simultaneously accessed by read() and through mmap.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Make netfslib pass the maximum length to the ->prepare_write() op to tell
the cache how much it can expand the length of a write to. This allows a
write to the server at the end of a file to be limited to a few bytes
whilst writing an entire block to the cache (something required by direct
I/O).
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Provide an entry point to delegate a filesystem's ->page_mkwrite() to.
This checks for conflicting writes, then attached any netfs-specific group
marking (e.g. ceph snap) to the page to be considered dirty.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Institute a netfs write helper, netfs_file_write_iter(), to be pointed at
by the network filesystem ->write_iter() call. Make it handled buffered
writes by calling the previously defined netfs_perform_write() to copy the
source data into the pagecache.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Implement support for unbuffered writes and direct I/O writes. If the
write is misaligned with respect to the fscrypt block size, then RMW cycles
are performed if necessary. DIO writes are a special case of unbuffered
writes with extra restriction imposed, such as block size alignment
requirements.
Also provide a field that can tell the code to add some extra space onto
the bounce buffer for use by the filesystem in the case of a
content-encrypted file.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Implement support for unbuffered and DIO reads in the netfs library,
utilising the existing read helper code to do block splitting and
individual queuing. The code also handles extraction of the destination
buffer from the supplied iterator, allowing async unbuffered reads to take
place.
The read will be split up according to the rsize setting and, if supplied,
the ->clamp_length() method. Note that the next subrequest will be issued
as soon as issue_op returns, without waiting for previous ones to finish.
The network filesystem needs to pause or handle queuing them if it doesn't
want to fire them all at the server simultaneously.
Once all the subrequests have finished, the state will be assessed and the
amount of data to be indicated as having being obtained will be
determined. As the subrequests may finish in any order, if an intermediate
subrequest is short, any further subrequests may be copied into the buffer
and then abandoned.
In the future, this will also take care of doing an unbuffered read from
encrypted content, with the decryption being done by the library.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Allocate a multipage folio when copying data into the pagecache if possible
if there's sufficient data to warrant it.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
netfs_read_folio() needs to handle partially-valid pages that are marked
dirty, but not uptodate in the event that someone tries to read a page was
used to cache data by a streaming write.
In such a case, make netfs_read_folio() set up a bvec iterator that points
to the parts of the folio that need filling and to a sink page for the data
that should be discarded and use that instead of i_pages as the iterator to
be written to.
This requires netfs_rreq_unlock_folios() to convert the page into a normal
dirty uptodate page, getting rid of the partial write record and bumping
the group pointer over to folio->private.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Provide a netfs write helper, netfs_perform_write() to buffer data to be
written in the pagecache and mark the modified folios dirty.
It will perform "streaming writes" for folios that aren't currently
resident, if possible, storing data in partially modified folios that are
marked dirty, but not uptodate. It will also tag pages as belonging to
fs-specific write groups if so directed by the filesystem.
This is derived from generic_perform_write(), but doesn't use
->write_begin() and ->write_end(), having that logic rolled in instead.
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Dispatch one or more write reqeusts to process a writeback slice, where a
slice is tailored more to logical block divisions within the file (such as
crypto blocks, an object layout or cache granules) than the protocol RPC
maximum capacity.
The dispatch doesn't happen until throttling allows, at which point the
entire writeback slice is processed and queued. A slice may be written to
multiple destinations (one or more servers and the local cache) and the
writes to each destination might be split up along different lines.
The writeback slice holds the required folios pinned. An iov_iter is
provided in netfs_write_request that describes the buffer to be used. This
may be part of the pagecache, may have auxiliary padding pages attached or
may be a bounce buffer resulting from crypto or compression. Consequently,
the filesystem must not twiddle the folio markings directly.
The following API is available to the filesystem:
(1) The ->create_write_requests() method is called to ask the filesystem
to create the requests it needs. This is passed the writeback slice
to be processed.
(2) The filesystem should then call netfs_create_write_request() to create
the requests it needs.
(3) Once a request is initialised, netfs_queue_write_request() can be
called to dispatch it asynchronously, if not completed immediately.
(4) netfs_write_request_completed() should be called to note the
completion of a request.
(5) netfs_get_write_request() and netfs_put_write_request() are provided
to refcount a request. These take constants from the netfs_wreq_trace
enum for logging into ftrace.
(6) The ->free_write_request is method is called to ask the filesystem to
clean up a request.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
Prepare to use folio->private to hold information write grouping and
streaming write. These are implemented in the same commit as they both
make use of folio->private and will be both checked at the same time in
several places.
"Write grouping" involves ordering the writeback of groups of writes, such
as is needed for ceph snaps. A group is represented by a
filesystem-supplied object which must contain a netfs_group struct. This
contains just a refcount and a pointer to a destructor.
"Streaming write" is the storage of data in folios that are marked dirty,
but not uptodate, to avoid unnecessary reads of data. This is represented
by a netfs_folio struct. This contains the offset and length of the
modified region plus the otherwise displaced write grouping pointer.
The way folio->private is multiplexed is:
(1) If private is NULL then neither is in operation on a dirty folio.
(2) If private is set, with bit 0 clear, then this points to a group.
(3) If private is set, with bit 0 set, then this points to a netfs_folio
struct (with bit 0 AND'ed out).
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
cc: linux-cachefs@redhat.com
cc: linux-fsdevel@vger.kernel.org
cc: linux-mm@kvack.org
