2019-07-31 23:57:31 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2020-07-13 15:09:44 +02:00
|
|
|
|
2021-08-20 18:00:19 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2019-07-31 23:57:36 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2023-10-26 10:16:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2023-10-26 10:16:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2019-08-23 05:36:59 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2021-08-13 13:29:31 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2024-11-14 09:32:47 +08:00
|
|
|
|
2019-09-04 10:09:09 +08:00
|
|
|
|
2024-11-14 09:32:47 +08:00
|
|
|
|
2019-09-04 10:09:09 +08:00
|
|
|
|
2024-11-14 09:32:47 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-11-23 01:15:59 +08:00
|
|
|
|
2019-07-31 23:57:36 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:36 +08:00
|
|
|
|
2025-03-10 17:54:54 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:38 +08:00
|
|
|
|
erofs: add file-backed mount support
It actually has been around for years: For containers and other sandbox
use cases, there will be thousands (and even more) of authenticated
(sub)images running on the same host, unlike OS images.
Of course, all scenarios can use the same EROFS on-disk format, but
bdev-backed mounts just work well for OS images since golden data is
dumped into real block devices. However, it's somewhat hard for
container runtimes to manage and isolate so many unnecessary virtual
block devices safely and efficiently [1]: they just look like a burden
to orchestrators and file-backed mounts are preferred indeed. There
were already enough attempts such as Incremental FS, the original
ComposeFS and PuzzleFS acting in the same way for immutable fses. As
for current EROFS users, ComposeFS, containerd and Android APEXs will
be directly benefited from it.
On the other hand, previous experimental feature "erofs over fscache"
was once also intended to provide a similar solution (inspired by
Incremental FS discussion [2]), but the following facts show file-backed
mounts will be a better approach:
- Fscache infrastructure has recently been moved into new Netfslib
which is an unexpected dependency to EROFS really, although it
originally claims "it could be used for caching other things such as
ISO9660 filesystems too." [3]
- It takes an unexpectedly long time to upstream Fscache/Cachefiles
enhancements. For example, the failover feature took more than
one year, and the deamonless feature is still far behind now;
- Ongoing HSM "fanotify pre-content hooks" [4] together with this will
perfectly supersede "erofs over fscache" in a simpler way since
developers (mainly containerd folks) could leverage their existing
caching mechanism entirely in userspace instead of strictly following
the predefined in-kernel caching tree hierarchy.
After "fanotify pre-content hooks" lands upstream to provide the same
functionality, "erofs over fscache" will be removed then (as an EROFS
internal improvement and EROFS will not have to bother with on-demand
fetching and/or caching improvements anymore.)
[1] https://github.com/containers/storage/pull/2039
[2] https://lore.kernel.org/r/CAOQ4uxjbVxnubaPjVaGYiSwoGDTdpWbB=w_AeM6YM=zVixsUfQ@mail.gmail.com
[3] https://docs.kernel.org/filesystems/caching/fscache.html
[4] https://lore.kernel.org/r/cover.1723670362.git.josef@toxicpanda.com
Closes: https://github.com/containers/composefs/issues/144
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240830032840.3783206-1-hsiangkao@linux.alibaba.com
2024-08-30 11:28:37 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2025-05-17 17:05:43 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2025-03-10 17:54:53 +08:00
|
|
|
|
|
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
2021-12-06 22:35:52 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-07 15:02:23 +08:00
|
|
|
|
2020-05-29 18:48:36 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-03-02 15:17:51 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
2021-03-29 09:23:06 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-04-07 12:39:23 +08:00
|
|
|
|
|
|
|
|
|
2021-03-29 09:23:06 +08:00
|
|
|
|
|
|
|
|
|
2022-09-18 12:34:53 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
|
|
|
|
|
2023-02-09 14:39:13 +08:00
|
|
|
|
2023-02-09 14:39:11 +08:00
|
|
|
|
|
|
|
|
|
2022-09-18 19:01:50 +08:00
|
|
|
|
2023-02-09 14:39:11 +08:00
|
|
|
|
|
|
|
|
|
2022-09-18 19:01:50 +08:00
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
|
|
|
|
|
2023-04-07 22:17:08 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2024-12-16 20:53:08 +08:00
|
|
|
|
2021-10-07 15:02:23 +08:00
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
2018-07-26 20:22:03 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:22:04 +08:00
|
|
|
|
2018-07-26 20:22:03 +08:00
|
|
|
|
2020-02-20 10:46:42 +08:00
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
2026-01-12 11:43:30 +08:00
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:49 +08:00
|
|
|
|
|
|
|
|
|
2021-03-29 09:23:06 +08:00
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
2023-04-07 22:17:06 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2018-07-26 20:21:52 +08:00
|
|
|
|
|
|
|
|
|
2023-04-07 22:17:08 +08:00
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:25 +00:00
|
|
|
|
2023-04-07 22:17:08 +08:00
|
|
|
|
2023-07-22 17:45:38 +08:00
|
|
|
|
2018-07-26 20:21:52 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2023-03-13 21:53:08 +08:00
|
|
|
|
2023-03-13 21:53:09 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2021-03-29 18:00:12 +08:00
|
|
|
|
2025-03-10 17:54:54 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-04-07 22:17:05 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-09-23 15:01:12 +08:00
|
|
|
|
2019-11-04 10:49:37 +08:00
|
|
|
|
2019-09-04 10:08:53 +08:00
|
|
|
|
erofs: separate plain and compressed filesystems formally
The EROFS on-disk format uses a tiny, plain metadata design that
prioritizes performance and minimizes complex inconsistencies against
common writable disk filesystems (almost all serious metadata
inconsistency cannot happen in well-designed immutable filesystems like
EROFS). EROFS deliberately avoids artificial design flaws to eliminate
serious security risks from untrusted remote sources by design,
although human-made implementation bugs can still happen sometimes.
Currently, there is no strict check to prevent compressed inodes,
especially LZ4-compressed inodes, from being read in plain filesystems.
Starting with erofs-utils 1.0 and Linux 5.3, LZ4_0PADDING sb feature
is automatically enabled for LZ4-compressed EROFS images to support
in-place decompression. Furthermore, since Linux 5.4 LTS is no longer
supported, we no longer need to handle ancient LZ4-compressed EROFS
images generated by erofs-utils prior to 1.0.
To formally distinguish different filesystem types for improved
security:
- Use the presence of LZ4_0PADDING or a non-zero
`dsb->u1.lz4_max_distance` as a marker for compressed filesystems
containing LZ4-compressed inodes only;
- For other algorithms, use `dsb->u1.available_compr_algs` bitmap.
Note: LZ4_0PADDING has been supported since Linux 5.4 (the first formal
kernel version), so exposing it via sysfs is no longer necessary and is
now deprecated (but remain it for five more years until 2031):
`dsb->u1` has been strictly non-zero for all EROFS images containing
compressed inodes starting with erofs-utils v1.3 and it is actually
a much better marker for compressed filesystems.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-01-29 10:41:25 +08:00
|
|
|
|
2021-12-01 22:54:36 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-07-21 10:13:52 +08:00
|
|
|
|
2022-04-25 20:21:33 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-09-18 12:34:53 +08:00
|
|
|
|
2022-10-21 10:31:53 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:52 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-08-05 08:36:00 +08:00
|
|
|
|
|
|
|
|
|
2024-12-12 21:43:36 +08:00
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
2018-07-26 20:21:52 +08:00
|
|
|
|
2021-10-07 15:02:23 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
erofs: add file-backed mount support
It actually has been around for years: For containers and other sandbox
use cases, there will be thousands (and even more) of authenticated
(sub)images running on the same host, unlike OS images.
Of course, all scenarios can use the same EROFS on-disk format, but
bdev-backed mounts just work well for OS images since golden data is
dumped into real block devices. However, it's somewhat hard for
container runtimes to manage and isolate so many unnecessary virtual
block devices safely and efficiently [1]: they just look like a burden
to orchestrators and file-backed mounts are preferred indeed. There
were already enough attempts such as Incremental FS, the original
ComposeFS and PuzzleFS acting in the same way for immutable fses. As
for current EROFS users, ComposeFS, containerd and Android APEXs will
be directly benefited from it.
On the other hand, previous experimental feature "erofs over fscache"
was once also intended to provide a similar solution (inspired by
Incremental FS discussion [2]), but the following facts show file-backed
mounts will be a better approach:
- Fscache infrastructure has recently been moved into new Netfslib
which is an unexpected dependency to EROFS really, although it
originally claims "it could be used for caching other things such as
ISO9660 filesystems too." [3]
- It takes an unexpectedly long time to upstream Fscache/Cachefiles
enhancements. For example, the failover feature took more than
one year, and the deamonless feature is still far behind now;
- Ongoing HSM "fanotify pre-content hooks" [4] together with this will
perfectly supersede "erofs over fscache" in a simpler way since
developers (mainly containerd folks) could leverage their existing
caching mechanism entirely in userspace instead of strictly following
the predefined in-kernel caching tree hierarchy.
After "fanotify pre-content hooks" lands upstream to provide the same
functionality, "erofs over fscache" will be removed then (as an EROFS
internal improvement and EROFS will not have to bother with on-demand
fetching and/or caching improvements anymore.)
[1] https://github.com/containers/storage/pull/2039
[2] https://lore.kernel.org/r/CAOQ4uxjbVxnubaPjVaGYiSwoGDTdpWbB=w_AeM6YM=zVixsUfQ@mail.gmail.com
[3] https://docs.kernel.org/filesystems/caching/fscache.html
[4] https://lore.kernel.org/r/cover.1723670362.git.josef@toxicpanda.com
Closes: https://github.com/containers/composefs/issues/144
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240830032840.3783206-1-hsiangkao@linux.alibaba.com
2024-08-30 11:28:37 +08:00
|
|
|
|
|
|
|
|
|
2024-12-16 20:53:08 +08:00
|
|
|
|
erofs: add file-backed mount support
It actually has been around for years: For containers and other sandbox
use cases, there will be thousands (and even more) of authenticated
(sub)images running on the same host, unlike OS images.
Of course, all scenarios can use the same EROFS on-disk format, but
bdev-backed mounts just work well for OS images since golden data is
dumped into real block devices. However, it's somewhat hard for
container runtimes to manage and isolate so many unnecessary virtual
block devices safely and efficiently [1]: they just look like a burden
to orchestrators and file-backed mounts are preferred indeed. There
were already enough attempts such as Incremental FS, the original
ComposeFS and PuzzleFS acting in the same way for immutable fses. As
for current EROFS users, ComposeFS, containerd and Android APEXs will
be directly benefited from it.
On the other hand, previous experimental feature "erofs over fscache"
was once also intended to provide a similar solution (inspired by
Incremental FS discussion [2]), but the following facts show file-backed
mounts will be a better approach:
- Fscache infrastructure has recently been moved into new Netfslib
which is an unexpected dependency to EROFS really, although it
originally claims "it could be used for caching other things such as
ISO9660 filesystems too." [3]
- It takes an unexpectedly long time to upstream Fscache/Cachefiles
enhancements. For example, the failover feature took more than
one year, and the deamonless feature is still far behind now;
- Ongoing HSM "fanotify pre-content hooks" [4] together with this will
perfectly supersede "erofs over fscache" in a simpler way since
developers (mainly containerd folks) could leverage their existing
caching mechanism entirely in userspace instead of strictly following
the predefined in-kernel caching tree hierarchy.
After "fanotify pre-content hooks" lands upstream to provide the same
functionality, "erofs over fscache" will be removed then (as an EROFS
internal improvement and EROFS will not have to bother with on-demand
fetching and/or caching improvements anymore.)
[1] https://github.com/containers/storage/pull/2039
[2] https://lore.kernel.org/r/CAOQ4uxjbVxnubaPjVaGYiSwoGDTdpWbB=w_AeM6YM=zVixsUfQ@mail.gmail.com
[3] https://docs.kernel.org/filesystems/caching/fscache.html
[4] https://lore.kernel.org/r/cover.1723670362.git.josef@toxicpanda.com
Closes: https://github.com/containers/composefs/issues/144
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240830032840.3783206-1-hsiangkao@linux.alibaba.com
2024-08-30 11:28:37 +08:00
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:24 +00:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:32 +08:00
|
|
|
|
|
|
|
|
|
erofs: add file-backed mount support
It actually has been around for years: For containers and other sandbox
use cases, there will be thousands (and even more) of authenticated
(sub)images running on the same host, unlike OS images.
Of course, all scenarios can use the same EROFS on-disk format, but
bdev-backed mounts just work well for OS images since golden data is
dumped into real block devices. However, it's somewhat hard for
container runtimes to manage and isolate so many unnecessary virtual
block devices safely and efficiently [1]: they just look like a burden
to orchestrators and file-backed mounts are preferred indeed. There
were already enough attempts such as Incremental FS, the original
ComposeFS and PuzzleFS acting in the same way for immutable fses. As
for current EROFS users, ComposeFS, containerd and Android APEXs will
be directly benefited from it.
On the other hand, previous experimental feature "erofs over fscache"
was once also intended to provide a similar solution (inspired by
Incremental FS discussion [2]), but the following facts show file-backed
mounts will be a better approach:
- Fscache infrastructure has recently been moved into new Netfslib
which is an unexpected dependency to EROFS really, although it
originally claims "it could be used for caching other things such as
ISO9660 filesystems too." [3]
- It takes an unexpectedly long time to upstream Fscache/Cachefiles
enhancements. For example, the failover feature took more than
one year, and the deamonless feature is still far behind now;
- Ongoing HSM "fanotify pre-content hooks" [4] together with this will
perfectly supersede "erofs over fscache" in a simpler way since
developers (mainly containerd folks) could leverage their existing
caching mechanism entirely in userspace instead of strictly following
the predefined in-kernel caching tree hierarchy.
After "fanotify pre-content hooks" lands upstream to provide the same
functionality, "erofs over fscache" will be removed then (as an EROFS
internal improvement and EROFS will not have to bother with on-demand
fetching and/or caching improvements anymore.)
[1] https://github.com/containers/storage/pull/2039
[2] https://lore.kernel.org/r/CAOQ4uxjbVxnubaPjVaGYiSwoGDTdpWbB=w_AeM6YM=zVixsUfQ@mail.gmail.com
[3] https://docs.kernel.org/filesystems/caching/fscache.html
[4] https://lore.kernel.org/r/cover.1723670362.git.josef@toxicpanda.com
Closes: https://github.com/containers/composefs/issues/144
Reviewed-by: Sandeep Dhavale <dhavale@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Link: https://lore.kernel.org/r/20240830032840.3783206-1-hsiangkao@linux.alibaba.com
2024-08-30 11:28:37 +08:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:32 +08:00
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-01-02 12:00:13 +08:00
|
|
|
|
2024-04-25 00:15:46 -04:00
|
|
|
|
2024-11-15 07:49:05 +08:00
|
|
|
|
2025-05-17 17:05:43 +08:00
|
|
|
|
2022-01-02 12:00:13 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-03-10 17:54:54 +08:00
|
|
|
|
|
|
|
|
|
2023-03-13 21:53:08 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2021-03-29 09:23:05 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: separate plain and compressed filesystems formally
The EROFS on-disk format uses a tiny, plain metadata design that
prioritizes performance and minimizes complex inconsistencies against
common writable disk filesystems (almost all serious metadata
inconsistency cannot happen in well-designed immutable filesystems like
EROFS). EROFS deliberately avoids artificial design flaws to eliminate
serious security risks from untrusted remote sources by design,
although human-made implementation bugs can still happen sometimes.
Currently, there is no strict check to prevent compressed inodes,
especially LZ4-compressed inodes, from being read in plain filesystems.
Starting with erofs-utils 1.0 and Linux 5.3, LZ4_0PADDING sb feature
is automatically enabled for LZ4-compressed EROFS images to support
in-place decompression. Furthermore, since Linux 5.4 LTS is no longer
supported, we no longer need to handle ancient LZ4-compressed EROFS
images generated by erofs-utils prior to 1.0.
To formally distinguish different filesystem types for improved
security:
- Use the presence of LZ4_0PADDING or a non-zero
`dsb->u1.lz4_max_distance` as a marker for compressed filesystems
containing LZ4-compressed inodes only;
- For other algorithms, use `dsb->u1.available_compr_algs` bitmap.
Note: LZ4_0PADDING has been supported since Linux 5.4 (the first formal
kernel version), so exposing it via sysfs is no longer necessary and is
now deprecated (but remain it for five more years until 2031):
`dsb->u1` has been strictly non-zero for all EROFS images containing
compressed inodes starting with erofs-utils v1.3 and it is actually
a much better marker for compressed filesystems.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-01-29 10:41:25 +08:00
|
|
|
|
2021-03-29 18:00:12 +08:00
|
|
|
|
2021-04-07 12:39:22 +08:00
|
|
|
|
2021-12-01 22:54:36 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2021-12-01 22:54:36 +08:00
|
|
|
|
2021-12-28 13:46:04 +08:00
|
|
|
|
2022-09-23 10:11:22 +08:00
|
|
|
|
2022-09-23 09:49:15 +08:00
|
|
|
|
2023-04-08 06:28:08 +08:00
|
|
|
|
2025-03-10 17:54:54 +08:00
|
|
|
|
2025-07-17 15:08:03 +08:00
|
|
|
|
2021-03-29 09:23:05 +08:00
|
|
|
|
2023-07-22 17:45:38 +08:00
|
|
|
|
2025-07-17 15:08:03 +08:00
|
|
|
|
2025-09-12 03:27:11 +08:00
|
|
|
|
2026-01-23 01:31:25 +00:00
|
|
|
|
2021-03-29 09:23:05 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-02-18 15:19:04 +08:00
|
|
|
|
2019-09-04 10:08:56 +08:00
|
|
|
|
|
|
|
|
|
2019-02-18 15:19:04 +08:00
|
|
|
|
|
|
|
|
|
2019-09-04 10:08:56 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2025-07-21 10:13:52 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-09-04 10:08:56 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2019-02-18 15:19:04 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2019-09-04 10:08:54 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2025-03-10 17:54:55 +08:00
|
|
|
|
2023-04-14 14:18:10 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2023-07-22 17:45:38 +08:00
|
|
|
|
2019-07-15 17:51:27 +05:30
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2019-06-24 15:22:52 +08:00
|
|
|
|
2025-03-10 17:54:53 +08:00
|
|
|
|
2021-08-20 18:00:19 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-06-24 15:22:52 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: add encoded extent on-disk definition
Previously, EROFS provided both (non-)compact compressed indexes to
keep necessary hints for each logical block, enabling O(1) random
indexing. This approach was originally designed for small compression
units (e.g., 4KiB), where compressed data is strictly block-aligned via
fixed-sized output compression.
However, EROFS now supports big pclusters up to 1MiB and many users use
large configurations to minimize image sizes. For such configurations,
the total number of extents decreases significantly (e.g., only 1,024
extents for a 1GiB file using 1MiB pclusters), then runtime metadata
overhead becomes negligible compared to data I/O and decoding costs.
Additionally, some popular compression algorithm (mainly Zstd) still
lacks native fixed-sized output compression support (although it's
planned by their authors). Instead of just waiting for compressor
improvements, let's adopt byte-oriented extents, allowing these
compressors to retain their current methods.
For example, it speeds up Zstd compression a lot:
Processor: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz * 96
Dataset: enwik9
Build time Size Type Command Line
3m52.339s 266653696 FO -C524288 -zzstd,22
3m48.549s 266174464 FO -E48bit -C524288 -zzstd,22
0m12.821s 272134144 FI -E48bit -C1048576 --max-extent-bytes=1048576 -zzstd,22
0m14.528s 248987648 FO -C1048576 -zlzma,9
0m14.605s 248504320 FO -E48bit -C1048576 -zlzma,9
Encoded extents are structured as an array of `struct z_erofs_extent`,
sorted by logical address in ascending order:
__le32 plen // encoded length, algorithm id and flags
__le32 pstart_lo // physical offset LSB
__le32 pstart_hi // physical offset MSB
__le32 lstart_lo // logical offset
__le32 lstart_hi // logical offset MSB
..
Note that prefixed reduced records can be used to minimize metadata for
specific cases (e.g. lstart less than 32 bits, then 32 to 16 bytes).
If the logical lengths of all encoded extents are the same, 4-byte
(plen) and 8-byte (plen, pstart_lo) records can be used. Or, 16-byte
(plen .. lstart_lo) and 32-byte full records have to be used instead.
If 16-byte and 32-byte records are used, the total number of extents
is kept in `struct z_erofs_map_header`, and binary search can be
applied on them. Note that `eytzinger order` is not considerd because
data sequential access is important.
If 4-byte records are used, 8-byte start physical offset is between
`struct z_erofs_map_header` and the `plen` array.
In addition, 64-bit physical offsets can be applied with new encoded
extent format to match full 48-bit block addressing.
Remove redundant comments around `struct z_erofs_lcluster_index` too.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Acked-by: Chao Yu <chao@kernel.org>
Link: https://lore.kernel.org/r/20250310095459.2620647-8-hsiangkao@linux.alibaba.com
2025-03-10 17:54:57 +08:00
|
|
|
|
2025-03-10 17:54:58 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-25 19:40:38 +08:00
|
|
|
|
|
|
|
|
|
2019-06-24 15:22:52 +08:00
|
|
|
|
2019-07-31 23:57:36 +08:00
|
|
|
|
2019-06-24 15:22:52 +08:00
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-14 23:08:23 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-14 23:08:23 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2023-01-14 23:08:23 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
2023-01-14 23:08:23 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
|
|
|
|
|
2023-03-13 21:53:08 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2023-04-14 16:30:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2023-04-14 16:30:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2023-04-14 16:30:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2023-04-14 16:30:27 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
2024-07-03 20:00:48 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-09 04:08:39 +08:00
|
|
|
|
2024-07-03 20:00:48 +08:00
|
|
|
|
2021-10-09 04:08:39 +08:00
|
|
|
|
2024-07-03 20:00:48 +08:00
|
|
|
|
2021-10-09 04:08:39 +08:00
|
|
|
|
|
|
|
|
|
2026-04-10 16:48:37 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2026-04-10 16:48:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-09-23 10:11:22 +08:00
|
|
|
|
2026-04-10 16:48:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-07-12 03:58:26 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2022-01-02 12:00:17 +08:00
|
|
|
|
|
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2021-10-09 04:08:37 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-08-18 23:22:31 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-09 10:48:25 +08:00
|
|
|
|
2021-10-11 05:31:45 +08:00
|
|
|
|
2023-02-09 10:48:25 +08:00
|
|
|
|
2022-09-23 10:11:22 +08:00
|
|
|
|
2023-02-09 10:48:25 +08:00
|
|
|
|
2018-07-26 20:21:45 +08:00
|
|
|
|
2021-10-09 04:08:37 +08:00
|
|
|
|
|
|
|
|
|
2022-09-23 10:11:21 +08:00
|
|
|
|
2021-10-09 04:08:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2024-12-13 07:54:01 +08:00
|
|
|
|
|
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
|
|
|
|
|
2024-09-05 17:30:31 +08:00
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-08-05 08:35:59 +08:00
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-09 14:39:13 +08:00
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
2023-04-07 22:17:08 +08:00
|
|
|
|
|
|
|
|
|
2022-01-02 12:00:17 +08:00
|
|
|
|
2022-01-02 12:00:14 +08:00
|
|
|
|
2025-02-17 04:31:41 -05:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
|
|
|
|
|
2022-01-02 12:00:14 +08:00
|
|
|
|
2025-07-22 08:32:29 +08:00
|
|
|
|
2021-10-14 16:10:10 +08:00
|
|
|
|
2021-08-13 13:29:31 +08:00
|
|
|
|
|
|
|
|
|
2023-02-09 10:48:25 +08:00
|
|
|
|
2024-09-05 17:30:31 +08:00
|
|
|
|
|
|
|
|
|
2025-07-09 11:46:14 +08:00
|
|
|
|
2022-09-27 14:36:07 +08:00
|
|
|
|
2023-01-13 12:49:12 +01:00
|
|
|
|
2021-01-21 14:19:43 +01:00
|
|
|
|
|
|
|
|
|
2022-04-25 12:07:12 +08:00
|
|
|
|
2019-01-14 19:40:24 +08:00
|
|
|
|
|
|
|
|
|
erofs: support decompress big pcluster for lz4 backend
Prior to big pcluster, there was only one compressed page so it'd
easy to map this. However, when big pcluster is enabled, more work
needs to be done to handle multiple compressed pages. In detail,
- (maptype 0) if there is only one compressed page + no need
to copy inplace I/O, just map it directly what we did before;
- (maptype 1) if there are more compressed pages + no need to
copy inplace I/O, vmap such compressed pages instead;
- (maptype 2) if inplace I/O needs to be copied, use per-CPU
buffers for decompression then.
Another thing is how to detect inplace decompression is feasable or
not (it's still quite easy for non big pclusters), apart from the
inplace margin calculation, inplace I/O page reusing order is also
needed to be considered for each compressed page. Currently, if the
compressed page is the xth page, it shouldn't be reused as [0 ...
nrpages_out - nrpages_in + x], otherwise a full copy will be triggered.
Although there are some extra optimization ideas for this, I'd like
to make big pcluster work correctly first and obviously it can be
further optimized later since it has nothing with the on-disk format
at all.
Link: https://lore.kernel.org/r/20210407043927.10623-10-xiang@kernel.org
Acked-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
2021-04-07 12:39:26 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: allow sharing page cache with the same aops only
Inode with identical data but different @aops cannot be mixed
because the page cache is managed by different subsystems (e.g.,
@aops for compressed on-disk inodes cannot handle plain on-disk
inodes).
In this patch, we never allow inodes to share the page cache
among plain, compressed, and fileio cases. When a shared inode
is created, we initialize @aops that is the same as the initial
real inode, and subsequent inodes cannot share the page cache
if the inferred @aops differ from the corresponding shared inode.
This is reasonable as a first step because, in typical use cases,
if an inode is compressible, it will fall into compressed
inodes across different filesystem images unless users use plain
filesystems. However, in that cases, users will use plain
filesystems all the time.
Fixes: 5ef3208e3be5 ("erofs: introduce the page cache share feature")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-02-14 03:02:48 +00:00
|
|
|
|
|
|
|
|
|
2026-01-23 07:52:39 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: allow sharing page cache with the same aops only
Inode with identical data but different @aops cannot be mixed
because the page cache is managed by different subsystems (e.g.,
@aops for compressed on-disk inodes cannot handle plain on-disk
inodes).
In this patch, we never allow inodes to share the page cache
among plain, compressed, and fileio cases. When a shared inode
is created, we initialize @aops that is the same as the initial
real inode, and subsequent inodes cannot share the page cache
if the inferred @aops differ from the corresponding shared inode.
This is reasonable as a first step because, in typical use cases,
if an inode is compressible, it will fall into compressed
inodes across different filesystem images unless users use plain
filesystems. However, in that cases, users will use plain
filesystems all the time.
Fixes: 5ef3208e3be5 ("erofs: introduce the page cache share feature")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-02-14 03:02:48 +00:00
|
|
|
|
2026-01-23 07:52:39 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: allow sharing page cache with the same aops only
Inode with identical data but different @aops cannot be mixed
because the page cache is managed by different subsystems (e.g.,
@aops for compressed on-disk inodes cannot handle plain on-disk
inodes).
In this patch, we never allow inodes to share the page cache
among plain, compressed, and fileio cases. When a shared inode
is created, we initialize @aops that is the same as the initial
real inode, and subsequent inodes cannot share the page cache
if the inferred @aops differ from the corresponding shared inode.
This is reasonable as a first step because, in typical use cases,
if an inode is compressible, it will fall into compressed
inodes across different filesystem images unless users use plain
filesystems. However, in that cases, users will use plain
filesystems all the time.
Fixes: 5ef3208e3be5 ("erofs: introduce the page cache share feature")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-02-14 03:02:48 +00:00
|
|
|
|
2026-01-23 07:52:39 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
erofs: allow sharing page cache with the same aops only
Inode with identical data but different @aops cannot be mixed
because the page cache is managed by different subsystems (e.g.,
@aops for compressed on-disk inodes cannot handle plain on-disk
inodes).
In this patch, we never allow inodes to share the page cache
among plain, compressed, and fileio cases. When a shared inode
is created, we initialize @aops that is the same as the initial
real inode, and subsequent inodes cannot share the page cache
if the inferred @aops differ from the corresponding shared inode.
This is reasonable as a first step because, in typical use cases,
if an inode is compressible, it will fall into compressed
inodes across different filesystem images unless users use plain
filesystems. However, in that cases, users will use plain
filesystems all the time.
Fixes: 5ef3208e3be5 ("erofs: introduce the page cache share feature")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-02-14 03:02:48 +00:00
|
|
|
|
2026-01-23 07:52:39 +00:00
|
|
|
|
|
|
|
|
|
erofs: allow sharing page cache with the same aops only
Inode with identical data but different @aops cannot be mixed
because the page cache is managed by different subsystems (e.g.,
@aops for compressed on-disk inodes cannot handle plain on-disk
inodes).
In this patch, we never allow inodes to share the page cache
among plain, compressed, and fileio cases. When a shared inode
is created, we initialize @aops that is the same as the initial
real inode, and subsequent inodes cannot share the page cache
if the inferred @aops differ from the corresponding shared inode.
This is reasonable as a first step because, in typical use cases,
if an inode is compressible, it will fall into compressed
inodes across different filesystem images unless users use plain
filesystems. However, in that cases, users will use plain
filesystems all the time.
Fixes: 5ef3208e3be5 ("erofs: introduce the page cache share feature")
Signed-off-by: Hongbo Li <lihongbo22@huawei.com>
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-02-14 03:02:48 +00:00
|
|
|
|
|
|
|
|
|
2026-01-23 07:52:39 +00:00
|
|
|
|
|
|
|
|
|
2021-12-01 22:54:36 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-04-02 07:15:23 -06:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
2021-10-22 17:01:20 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-06-24 15:22:53 +08:00
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
2024-11-12 21:11:48 -07:00
|
|
|
|
|
|
|
|
|
2024-10-21 11:53:22 +08:00
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-09 17:41:05 +08:00
|
|
|
|
|
|
|
|
|
2025-03-17 13:48:40 +08:00
|
|
|
|
2024-10-21 11:53:22 +08:00
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
|
|
|
|
|
2024-04-02 04:00:36 -06:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-09 17:41:05 +08:00
|
|
|
|
|
|
|
|
|
2025-03-17 13:48:40 +08:00
|
|
|
|
2019-07-31 23:57:39 +08:00
|
|
|
|
erofs: separate plain and compressed filesystems formally
The EROFS on-disk format uses a tiny, plain metadata design that
prioritizes performance and minimizes complex inconsistencies against
common writable disk filesystems (almost all serious metadata
inconsistency cannot happen in well-designed immutable filesystems like
EROFS). EROFS deliberately avoids artificial design flaws to eliminate
serious security risks from untrusted remote sources by design,
although human-made implementation bugs can still happen sometimes.
Currently, there is no strict check to prevent compressed inodes,
especially LZ4-compressed inodes, from being read in plain filesystems.
Starting with erofs-utils 1.0 and Linux 5.3, LZ4_0PADDING sb feature
is automatically enabled for LZ4-compressed EROFS images to support
in-place decompression. Furthermore, since Linux 5.4 LTS is no longer
supported, we no longer need to handle ancient LZ4-compressed EROFS
images generated by erofs-utils prior to 1.0.
To formally distinguish different filesystem types for improved
security:
- Use the presence of LZ4_0PADDING or a non-zero
`dsb->u1.lz4_max_distance` as a marker for compressed filesystems
containing LZ4-compressed inodes only;
- For other algorithms, use `dsb->u1.available_compr_algs` bitmap.
Note: LZ4_0PADDING has been supported since Linux 5.4 (the first formal
kernel version), so exposing it via sysfs is no longer necessary and is
now deprecated (but remain it for five more years until 2031):
`dsb->u1` has been strictly non-zero for all EROFS images containing
compressed inodes starting with erofs-utils v1.3 and it is actually
a much better marker for compressed filesystems.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
2026-01-29 10:41:25 +08:00
|
|
|
|
2019-01-16 16:59:56 +08:00
|
|
|
|
2024-08-30 11:28:39 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:33 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
2024-03-08 17:41:59 +08:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:33 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
2022-04-25 20:21:33 +08:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
|
|
|
|
|
2023-02-04 17:30:40 +08:00
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
|
|
|
|
|
2022-09-18 12:34:52 +08:00
|
|
|
|
2022-04-25 20:21:34 +08:00
|
|
|
|
|
|
|
|
|
2024-03-08 17:41:59 +08:00
|
|
|
|
|
|
|
|
|
2022-04-25 20:21:33 +08:00
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:30 +00:00
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:30 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-23 01:31:28 +00:00
|
|
|
|
|
|
|
|
|
2025-09-23 15:01:12 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-07-31 23:57:36 +08:00
|
|
|
|