2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:07 -07:00
|
|
|
|
2010-07-06 15:39:16 -07:00
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2016-07-28 15:48:06 -07:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2010-07-28 15:43:02 +10:00
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-28 15:43:02 +10:00
|
|
|
|
2010-08-04 13:34:42 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
mem-hotplug: handle node hole when initializing numa_meminfo.
When parsing SRAT, all memory ranges are added into numa_meminfo. In
numa_init(), before entering numa_cleanup_meminfo(), all possible memory
ranges are in numa_meminfo. And numa_cleanup_meminfo() removes all
ranges over max_pfn or empty.
But, this only works if the nodes are continuous. Let's have a look at
the following example:
We have an SRAT like this:
SRAT: Node 0 PXM 0 [mem 0x00000000-0x5fffffff]
SRAT: Node 0 PXM 0 [mem 0x100000000-0x1ffffffffff]
SRAT: Node 1 PXM 1 [mem 0x20000000000-0x3ffffffffff]
SRAT: Node 4 PXM 2 [mem 0x40000000000-0x5ffffffffff] hotplug
SRAT: Node 5 PXM 3 [mem 0x60000000000-0x7ffffffffff] hotplug
SRAT: Node 2 PXM 4 [mem 0x80000000000-0x9ffffffffff] hotplug
SRAT: Node 3 PXM 5 [mem 0xa0000000000-0xbffffffffff] hotplug
SRAT: Node 6 PXM 6 [mem 0xc0000000000-0xdffffffffff] hotplug
SRAT: Node 7 PXM 7 [mem 0xe0000000000-0xfffffffffff] hotplug
On boot, only node 0,1,2,3 exist.
And the numa_meminfo will look like this:
numa_meminfo.nr_blks = 9
1. on node 0: [0, 60000000]
2. on node 0: [100000000, 20000000000]
3. on node 1: [20000000000, 40000000000]
4. on node 4: [40000000000, 60000000000]
5. on node 5: [60000000000, 80000000000]
6. on node 2: [80000000000, a0000000000]
7. on node 3: [a0000000000, a0800000000]
8. on node 6: [c0000000000, a0800000000]
9. on node 7: [e0000000000, a0800000000]
And numa_cleanup_meminfo() will merge 1 and 2, and remove 8,9 because the
end address is over max_pfn, which is a0800000000. But 4 and 5 are not
removed because their end addresses are less then max_pfn. But in fact,
node 4 and 5 don't exist.
In a word, numa_cleanup_meminfo() is not able to handle holes between nodes.
Since memory ranges in node 4 and 5 are in numa_meminfo, in
numa_register_memblks(), node 4 and 5 will be mistakenly set to online.
If you run lscpu, it will show:
NUMA node0 CPU(s): 0-14,128-142
NUMA node1 CPU(s): 15-29,143-157
NUMA node2 CPU(s):
NUMA node3 CPU(s):
NUMA node4 CPU(s): 62-76,190-204
NUMA node5 CPU(s): 78-92,206-220
In this patch, we use memblock_overlaps_region() to check if ranges in
numa_meminfo overlap with ranges in memory_block. Since memory_block
contains all available memory at boot time, if they overlap, it means the
ranges exist. If not, then remove them from numa_meminfo.
After this patch, lscpu will show:
NUMA node0 CPU(s): 0-14,128-142
NUMA node1 CPU(s): 15-29,143-157
NUMA node4 CPU(s): 62-76,190-204
NUMA node5 CPU(s): 78-92,206-220
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Xishi Qiu <qiuxishi@huawei.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Vladimir Murzin <vladimir.murzin@arm.com>
Cc: Fabian Frederick <fabf@skynet.be>
Cc: Alexander Kuleshov <kuleshovmail@gmail.com>
Cc: Baoquan He <bhe@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-09-08 15:02:03 -07:00
|
|
|
|
2011-10-31 17:09:15 -07:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:39 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2015-09-08 15:02:00 -07:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
mm/memblock.c: factor out of top-down allocation
[Problem]
The current Linux cannot migrate pages used by the kernel because of the
kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and keep the
va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not
migratable. For example, the physical address may be cached somewhere and
will be used. It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages
are used by the kernel, they are not migratable. As a result, memory used
by the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the
following way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of
the zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these
memory. To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The
memory affinities in SRAT record every memory range in the system, and
also, flags specifying if the memory range is hotpluggable. (Please refer
to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must
be unhotpluggable. We cannot prevent this anyway. So in a memory hotplug
system, we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable.
But memblock has already started to work. In the current kernel,
memblock allocates the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best
to allocate memory near the kernel image. Since the whole node the kernel
resides in won't be hotpluggable, and for a modern server, a node may have
at least 16GB memory, allocating several mega bytes memory around the
kernel image won't cross to hotpluggable memory.
[About this patchset]
So this patchset is the preparation for the problem 2 that we want to
solve. It does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in
bottom up direction.
3. Introduce "movable_node" boot option to enable and disable this
functionality.
This patch (of 6):
Create a new function __memblock_find_range_top_down to factor out of
top-down allocation from memblock_find_in_range_node. This is a
preparation because we will introduce a new bottom-up allocation mode in
the following patch.
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-12 15:07:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
mm/memblock.c: factor out of top-down allocation
[Problem]
The current Linux cannot migrate pages used by the kernel because of the
kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and keep the
va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not
migratable. For example, the physical address may be cached somewhere and
will be used. It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages
are used by the kernel, they are not migratable. As a result, memory used
by the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the
following way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of
the zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these
memory. To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The
memory affinities in SRAT record every memory range in the system, and
also, flags specifying if the memory range is hotpluggable. (Please refer
to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must
be unhotpluggable. We cannot prevent this anyway. So in a memory hotplug
system, we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable.
But memblock has already started to work. In the current kernel,
memblock allocates the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best
to allocate memory near the kernel image. Since the whole node the kernel
resides in won't be hotpluggable, and for a modern server, a node may have
at least 16GB memory, allocating several mega bytes memory around the
kernel image won't cross to hotpluggable memory.
[About this patchset]
So this patchset is the preparation for the problem 2 that we want to
solve. It does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in
bottom up direction.
3. Introduce "movable_node" boot option to enable and disable this
functionality.
This patch (of 6):
Create a new function __memblock_find_range_top_down to factor out of
top-down allocation from memblock_find_in_range_node. This is a
preparation because we will introduce a new bottom-up allocation mode in
the following patch.
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-12 15:07:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
mm/memblock.c: factor out of top-down allocation
[Problem]
The current Linux cannot migrate pages used by the kernel because of the
kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and keep the
va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not
migratable. For example, the physical address may be cached somewhere and
will be used. It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages
are used by the kernel, they are not migratable. As a result, memory used
by the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the
following way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of
the zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these
memory. To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The
memory affinities in SRAT record every memory range in the system, and
also, flags specifying if the memory range is hotpluggable. (Please refer
to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must
be unhotpluggable. We cannot prevent this anyway. So in a memory hotplug
system, we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable.
But memblock has already started to work. In the current kernel,
memblock allocates the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best
to allocate memory near the kernel image. Since the whole node the kernel
resides in won't be hotpluggable, and for a modern server, a node may have
at least 16GB memory, allocating several mega bytes memory around the
kernel image won't cross to hotpluggable memory.
[About this patchset]
So this patchset is the preparation for the problem 2 that we want to
solve. It does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in
bottom up direction.
3. Introduce "movable_node" boot option to enable and disable this
functionality.
This patch (of 6):
Create a new function __memblock_find_range_top_down to factor out of
top-down allocation from memblock_find_in_range_node. This is a
preparation because we will introduce a new bottom-up allocation mode in
the following patch.
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-12 15:07:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
mm/memblock.c: factor out of top-down allocation
[Problem]
The current Linux cannot migrate pages used by the kernel because of the
kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and keep the
va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not
migratable. For example, the physical address may be cached somewhere and
will be used. It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages
are used by the kernel, they are not migratable. As a result, memory used
by the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the
following way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of
the zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these
memory. To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The
memory affinities in SRAT record every memory range in the system, and
also, flags specifying if the memory range is hotpluggable. (Please refer
to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must
be unhotpluggable. We cannot prevent this anyway. So in a memory hotplug
system, we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable.
But memblock has already started to work. In the current kernel,
memblock allocates the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best
to allocate memory near the kernel image. Since the whole node the kernel
resides in won't be hotpluggable, and for a modern server, a node may have
at least 16GB memory, allocating several mega bytes memory around the
kernel image won't cross to hotpluggable memory.
[About this patchset]
So this patchset is the preparation for the problem 2 that we want to
solve. It does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in
bottom up direction.
3. Introduce "movable_node" boot option to enable and disable this
functionality.
This patch (of 6):
Create a new function __memblock_find_range_top_down to factor out of
top-down allocation from memblock_find_in_range_node. This is a
preparation because we will introduce a new bottom-up allocation mode in
the following patch.
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-12 15:07:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
mm/memblock.c: factor out of top-down allocation
[Problem]
The current Linux cannot migrate pages used by the kernel because of the
kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
When the pa is changed, we cannot simply update the pagetable and keep the
va unmodified. So the kernel pages are not migratable.
There are also some other issues will cause the kernel pages not
migratable. For example, the physical address may be cached somewhere and
will be used. It is not to update all the caches.
When doing memory hotplug in Linux, we first migrate all the pages in one
memory device somewhere else, and then remove the device. But if pages
are used by the kernel, they are not migratable. As a result, memory used
by the kernel cannot be hot-removed.
Modifying the kernel direct mapping mechanism is too difficult to do. And
it may cause the kernel performance down and unstable. So we use the
following way to do memory hotplug.
[What we are doing]
In Linux, memory in one numa node is divided into several zones. One of
the zones is ZONE_MOVABLE, which the kernel won't use.
In order to implement memory hotplug in Linux, we are going to arrange all
hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these
memory. To do this, we need ACPI's help.
In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The
memory affinities in SRAT record every memory range in the system, and
also, flags specifying if the memory range is hotpluggable. (Please refer
to ACPI spec 5.0 5.2.16)
With the help of SRAT, we have to do the following two things to achieve our
goal:
1. When doing memory hot-add, allow the users arranging hotpluggable as
ZONE_MOVABLE.
(This has been done by the MOVABLE_NODE functionality in Linux.)
2. when the system is booting, prevent bootmem allocator from allocating
hotpluggable memory for the kernel before the memory initialization
finishes.
The problem 2 is the key problem we are going to solve. But before solving it,
we need some preparation. Please see below.
[Preparation]
Bootloader has to load the kernel image into memory. And this memory must
be unhotpluggable. We cannot prevent this anyway. So in a memory hotplug
system, we can assume any node the kernel resides in is not hotpluggable.
Before SRAT is parsed, we don't know which memory ranges are hotpluggable.
But memblock has already started to work. In the current kernel,
memblock allocates the following memory before SRAT is parsed:
setup_arch()
|->memblock_x86_fill() /* memblock is ready */
|......
|->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
|->reserve_real_mode() /* allocate memory under 1MB */
|->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
|->dma_contiguous_reserve() /* specified by user, should be low */
|->setup_log_buf() /* specified by user, several mega bytes */
|->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
|->acpi_initrd_override() /* several mega bytes */
|->reserve_crashkernel() /* could be large, should reorder */
|......
|->initmem_init() /* Parse SRAT */
According to Tejun's advice, before SRAT is parsed, we should try our best
to allocate memory near the kernel image. Since the whole node the kernel
resides in won't be hotpluggable, and for a modern server, a node may have
at least 16GB memory, allocating several mega bytes memory around the
kernel image won't cross to hotpluggable memory.
[About this patchset]
So this patchset is the preparation for the problem 2 that we want to
solve. It does the following:
1. Make memblock be able to allocate memory bottom up.
1) Keep all the memblock APIs' prototype unmodified.
2) When the direction is bottom up, keep the start address greater than the
end of kernel image.
2. Improve init_mem_mapping() to support allocate page tables in
bottom up direction.
3. Introduce "movable_node" boot option to enable and disable this
functionality.
This patch (of 6):
Create a new function __memblock_find_range_top_down to factor out of
top-down allocation from memblock_find_in_range_node. This is a
preparation because we will introduce a new bottom-up allocation mode in
the following patch.
Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: Zhang Yanfei <zhangyanfei@cn.fujitsu.com>
Acked-by: Tejun Heo <tj@kernel.org>
Acked-by: Toshi Kani <toshi.kani@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Cc: Thomas Renninger <trenn@suse.de>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: Wen Congyang <wency@cn.fujitsu.com>
Cc: Lai Jiangshan <laijs@cn.fujitsu.com>
Cc: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
Cc: Taku Izumi <izumi.taku@jp.fujitsu.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Michal Nazarewicz <mina86@mina86.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Kamezawa Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2013-11-12 15:07:57 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:14 -08:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2014-01-21 15:50:14 -08:00
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2013-02-22 16:33:51 -08:00
|
|
|
|
2014-08-29 15:18:31 -07:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2013-02-22 16:33:51 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-03-17 14:19:47 -07:00
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2013-02-22 16:33:51 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2013-02-22 16:33:51 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-11-12 15:07:59 -08:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2010-07-28 15:43:02 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
2010-08-04 14:06:41 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
mm/memblock: properly handle overlaps and fix error path
Currently memblock_reserve() or memblock_free() don't handle overlaps of
any kind. There is some special casing for coalescing exactly adjacent
regions but that's about it.
This is annoying because typically memblock_reserve() is used to mark
regions passed by the firmware as reserved and we all know how much we can
trust our firmwares...
Also, with the current code, if we do something it doesn't handle right
such as trying to memblock_reserve() a large range spanning multiple
existing smaller reserved regions for example, or doing overlapping
reservations, it can silently corrupt the internal region array, causing
odd errors much later on, such as allocations returning reserved regions
etc...
This patch rewrites the underlying functions that add or remove a region
to the arrays. The new code is a lot more robust as it fully handles
overlapping regions. It's also, imho, simpler than the previous
implementation.
In addition, while doing so, I found a bug where if we fail to double the
array while adding a region, we would remove the last region of the array
rather than the region we just allocated. This fixes it too.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-03-22 16:33:43 -07:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
mm/memblock: properly handle overlaps and fix error path
Currently memblock_reserve() or memblock_free() don't handle overlaps of
any kind. There is some special casing for coalescing exactly adjacent
regions but that's about it.
This is annoying because typically memblock_reserve() is used to mark
regions passed by the firmware as reserved and we all know how much we can
trust our firmwares...
Also, with the current code, if we do something it doesn't handle right
such as trying to memblock_reserve() a large range spanning multiple
existing smaller reserved regions for example, or doing overlapping
reservations, it can silently corrupt the internal region array, causing
odd errors much later on, such as allocations returning reserved regions
etc...
This patch rewrites the underlying functions that add or remove a region
to the arrays. The new code is a lot more robust as it fully handles
overlapping regions. It's also, imho, simpler than the previous
implementation.
In addition, while doing so, I found a bug where if we fail to double the
array while adding a region, we would remove the last region of the array
rather than the region we just allocated. This fixes it too.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-03-22 16:33:43 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
mm/memblock: properly handle overlaps and fix error path
Currently memblock_reserve() or memblock_free() don't handle overlaps of
any kind. There is some special casing for coalescing exactly adjacent
regions but that's about it.
This is annoying because typically memblock_reserve() is used to mark
regions passed by the firmware as reserved and we all know how much we can
trust our firmwares...
Also, with the current code, if we do something it doesn't handle right
such as trying to memblock_reserve() a large range spanning multiple
existing smaller reserved regions for example, or doing overlapping
reservations, it can silently corrupt the internal region array, causing
odd errors much later on, such as allocations returning reserved regions
etc...
This patch rewrites the underlying functions that add or remove a region
to the arrays. The new code is a lot more robust as it fully handles
overlapping regions. It's also, imho, simpler than the previous
implementation.
In addition, while doing so, I found a bug where if we fail to double the
array while adding a region, we would remove the last region of the array
rather than the region we just allocated. This fixes it too.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-03-22 16:33:43 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2014-01-23 15:53:24 -08:00
|
|
|
|
2017-08-18 15:16:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2017-08-18 15:16:05 -07:00
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2017-08-18 15:16:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2017-08-25 15:55:46 -07:00
|
|
|
|
2017-08-18 15:16:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2014-01-23 15:53:11 -08:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 09:58:09 +02:00
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
|
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
2012-09-04 13:55:05 +05:30
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
2011-07-12 09:58:09 +02:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
2010-07-28 15:13:22 +10:00
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2012-07-31 16:42:40 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
|
|
|
|
|
2012-07-11 14:02:56 -07:00
|
|
|
|
2012-05-29 15:06:50 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-01-11 14:31:44 -08:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-29 15:08:41 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2016-08-04 15:31:46 -07:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2012-04-20 08:31:34 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2010-08-04 14:06:41 +10:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-08-04 15:31:46 -07:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-04 15:47:38 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 14:59:53 -07:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2016-07-26 15:24:56 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
2011-07-12 11:15:55 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
|
|
|
|
|
2016-05-20 16:57:35 -07:00
|
|
|
|
2015-04-15 16:14:44 -07:00
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-15 16:14:44 -07:00
|
|
|
|
2016-05-20 16:57:35 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-04-20 08:31:34 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:05 -07:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-11-05 18:47:00 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
mm/memblock: properly handle overlaps and fix error path
Currently memblock_reserve() or memblock_free() don't handle overlaps of
any kind. There is some special casing for coalescing exactly adjacent
regions but that's about it.
This is annoying because typically memblock_reserve() is used to mark
regions passed by the firmware as reserved and we all know how much we can
trust our firmwares...
Also, with the current code, if we do something it doesn't handle right
such as trying to memblock_reserve() a large range spanning multiple
existing smaller reserved regions for example, or doing overlapping
reservations, it can silently corrupt the internal region array, causing
odd errors much later on, such as allocations returning reserved regions
etc...
This patch rewrites the underlying functions that add or remove a region
to the arrays. The new code is a lot more robust as it fully handles
overlapping regions. It's also, imho, simpler than the previous
implementation.
In addition, while doing so, I found a bug where if we fail to double the
array while adding a region, we would remove the last region of the array
rather than the region we just allocated. This fixes it too.
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: Yinghai Lu <yinghai@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2011-03-22 16:33:43 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:06 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:06 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:16:06 +02:00
|
|
|
|
2016-10-11 13:55:11 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2016-05-20 16:57:35 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2016-05-20 16:57:35 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
2015-09-08 15:04:02 -07:00
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 15:04:02 -07:00
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 15:04:02 -07:00
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-12-12 16:54:59 -08:00
|
|
|
|
2014-01-21 15:49:23 -08:00
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 15:04:02 -07:00
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-11-30 13:28:15 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
2017-04-03 11:23:54 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
memblock: introduce a for_each_reserved_mem_region iterator
Struct page initialisation had been identified as one of the reasons why
large machines take a long time to boot. Patches were posted a long time ago
to defer initialisation until they were first used. This was rejected on
the grounds it should not be necessary to hurt the fast paths. This series
reuses much of the work from that time but defers the initialisation of
memory to kswapd so that one thread per node initialises memory local to
that node.
After applying the series and setting the appropriate Kconfig variable I
see this in the boot log on a 64G machine
[ 7.383764] kswapd 0 initialised deferred memory in 188ms
[ 7.404253] kswapd 1 initialised deferred memory in 208ms
[ 7.411044] kswapd 3 initialised deferred memory in 216ms
[ 7.411551] kswapd 2 initialised deferred memory in 216ms
On a 1TB machine, I see
[ 8.406511] kswapd 3 initialised deferred memory in 1116ms
[ 8.428518] kswapd 1 initialised deferred memory in 1140ms
[ 8.435977] kswapd 0 initialised deferred memory in 1148ms
[ 8.437416] kswapd 2 initialised deferred memory in 1148ms
Once booted the machine appears to work as normal. Boot times were measured
from the time shutdown was called until ssh was available again. In the
64G case, the boot time savings are negligible. On the 1TB machine, the
savings were 16 seconds.
Nate Zimmer said:
: On an older 8 TB box with lots and lots of cpus the boot time, as
: measure from grub to login prompt, the boot time improved from 1484
: seconds to exactly 1000 seconds.
Waiman Long said:
: I ran a bootup timing test on a 12-TB 16-socket IvyBridge-EX system. From
: grub menu to ssh login, the bootup time was 453s before the patch and 265s
: after the patch - a saving of 188s (42%).
Daniel Blueman said:
: On a 7TB, 1728-core NumaConnect system with 108 NUMA nodes, we're seeing
: stock 4.0 boot in 7136s. This drops to 2159s, or a 70% reduction with
: this patchset. Non-temporal PMD init (https://lkml.org/lkml/2015/4/23/350)
: drops this to 1045s.
This patch (of 13):
As part of initializing struct page's in 2MiB chunks, we noticed that at
the end of free_all_bootmem(), there was nothing which had forced the
reserved/allocated 4KiB pages to be initialized.
This helper function will be used for that expansion.
Signed-off-by: Robin Holt <holt@sgi.com>
Signed-off-by: Nate Zimmer <nzimmer@sgi.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Tested-by: Nate Zimmer <nzimmer@sgi.com>
Tested-by: Waiman Long <waiman.long@hp.com>
Tested-by: Daniel J Blueman <daniel@numascale.com>
Acked-by: Pekka Enberg <penberg@kernel.org>
Cc: Robin Holt <robinmholt@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Waiman Long <waiman.long@hp.com>
Cc: Scott Norton <scott.norton@hp.com>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 14:56:41 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 15:03:33 -07:00
|
|
|
|
memblock: introduce a for_each_reserved_mem_region iterator
Struct page initialisation had been identified as one of the reasons why
large machines take a long time to boot. Patches were posted a long time ago
to defer initialisation until they were first used. This was rejected on
the grounds it should not be necessary to hurt the fast paths. This series
reuses much of the work from that time but defers the initialisation of
memory to kswapd so that one thread per node initialises memory local to
that node.
After applying the series and setting the appropriate Kconfig variable I
see this in the boot log on a 64G machine
[ 7.383764] kswapd 0 initialised deferred memory in 188ms
[ 7.404253] kswapd 1 initialised deferred memory in 208ms
[ 7.411044] kswapd 3 initialised deferred memory in 216ms
[ 7.411551] kswapd 2 initialised deferred memory in 216ms
On a 1TB machine, I see
[ 8.406511] kswapd 3 initialised deferred memory in 1116ms
[ 8.428518] kswapd 1 initialised deferred memory in 1140ms
[ 8.435977] kswapd 0 initialised deferred memory in 1148ms
[ 8.437416] kswapd 2 initialised deferred memory in 1148ms
Once booted the machine appears to work as normal. Boot times were measured
from the time shutdown was called until ssh was available again. In the
64G case, the boot time savings are negligible. On the 1TB machine, the
savings were 16 seconds.
Nate Zimmer said:
: On an older 8 TB box with lots and lots of cpus the boot time, as
: measure from grub to login prompt, the boot time improved from 1484
: seconds to exactly 1000 seconds.
Waiman Long said:
: I ran a bootup timing test on a 12-TB 16-socket IvyBridge-EX system. From
: grub menu to ssh login, the bootup time was 453s before the patch and 265s
: after the patch - a saving of 188s (42%).
Daniel Blueman said:
: On a 7TB, 1728-core NumaConnect system with 108 NUMA nodes, we're seeing
: stock 4.0 boot in 7136s. This drops to 2159s, or a 70% reduction with
: this patchset. Non-temporal PMD init (https://lkml.org/lkml/2015/4/23/350)
: drops this to 1045s.
This patch (of 13):
As part of initializing struct page's in 2MiB chunks, we noticed that at
the end of free_all_bootmem(), there was nothing which had forced the
reserved/allocated 4KiB pages to be initialized.
This helper function will be used for that expansion.
Signed-off-by: Robin Holt <holt@sgi.com>
Signed-off-by: Nate Zimmer <nzimmer@sgi.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Tested-by: Nate Zimmer <nzimmer@sgi.com>
Tested-by: Waiman Long <waiman.long@hp.com>
Tested-by: Daniel J Blueman <daniel@numascale.com>
Acked-by: Pekka Enberg <penberg@kernel.org>
Cc: Robin Holt <robinmholt@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Waiman Long <waiman.long@hp.com>
Cc: Scott Norton <scott.norton@hp.com>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 14:56:41 -07:00
|
|
|
|
2016-05-20 16:58:33 -07:00
|
|
|
|
2015-09-08 15:03:33 -07:00
|
|
|
|
memblock: introduce a for_each_reserved_mem_region iterator
Struct page initialisation had been identified as one of the reasons why
large machines take a long time to boot. Patches were posted a long time ago
to defer initialisation until they were first used. This was rejected on
the grounds it should not be necessary to hurt the fast paths. This series
reuses much of the work from that time but defers the initialisation of
memory to kswapd so that one thread per node initialises memory local to
that node.
After applying the series and setting the appropriate Kconfig variable I
see this in the boot log on a 64G machine
[ 7.383764] kswapd 0 initialised deferred memory in 188ms
[ 7.404253] kswapd 1 initialised deferred memory in 208ms
[ 7.411044] kswapd 3 initialised deferred memory in 216ms
[ 7.411551] kswapd 2 initialised deferred memory in 216ms
On a 1TB machine, I see
[ 8.406511] kswapd 3 initialised deferred memory in 1116ms
[ 8.428518] kswapd 1 initialised deferred memory in 1140ms
[ 8.435977] kswapd 0 initialised deferred memory in 1148ms
[ 8.437416] kswapd 2 initialised deferred memory in 1148ms
Once booted the machine appears to work as normal. Boot times were measured
from the time shutdown was called until ssh was available again. In the
64G case, the boot time savings are negligible. On the 1TB machine, the
savings were 16 seconds.
Nate Zimmer said:
: On an older 8 TB box with lots and lots of cpus the boot time, as
: measure from grub to login prompt, the boot time improved from 1484
: seconds to exactly 1000 seconds.
Waiman Long said:
: I ran a bootup timing test on a 12-TB 16-socket IvyBridge-EX system. From
: grub menu to ssh login, the bootup time was 453s before the patch and 265s
: after the patch - a saving of 188s (42%).
Daniel Blueman said:
: On a 7TB, 1728-core NumaConnect system with 108 NUMA nodes, we're seeing
: stock 4.0 boot in 7136s. This drops to 2159s, or a 70% reduction with
: this patchset. Non-temporal PMD init (https://lkml.org/lkml/2015/4/23/350)
: drops this to 1045s.
This patch (of 13):
As part of initializing struct page's in 2MiB chunks, we noticed that at
the end of free_all_bootmem(), there was nothing which had forced the
reserved/allocated 4KiB pages to be initialized.
This helper function will be used for that expansion.
Signed-off-by: Robin Holt <holt@sgi.com>
Signed-off-by: Nate Zimmer <nzimmer@sgi.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Tested-by: Nate Zimmer <nzimmer@sgi.com>
Tested-by: Waiman Long <waiman.long@hp.com>
Tested-by: Daniel J Blueman <daniel@numascale.com>
Acked-by: Pekka Enberg <penberg@kernel.org>
Cc: Robin Holt <robinmholt@gmail.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Waiman Long <waiman.long@hp.com>
Cc: Scott Norton <scott.norton@hp.com>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2015-06-30 14:56:41 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:01 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:55 -08:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
2014-09-09 14:50:46 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-11-30 13:28:15 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2011-07-12 11:15:59 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2015-09-08 15:04:22 -07:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2012-06-20 12:53:01 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:16 -08:00
|
|
|
|
2014-01-21 15:50:55 -08:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2016-08-04 15:32:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:35 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-11-30 13:28:15 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-07-28 15:48:56 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:44:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-03-09 16:17:17 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:44:53 -08:00
|
|
|
|
|
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:26 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:26 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:49:26 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2012-10-08 16:32:21 -07:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2014-01-21 15:50:12 -08:00
|
|
|
|
|
|
|
|
|
2013-04-29 15:06:15 -07:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2014-06-06 14:38:20 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-10-11 13:55:11 -07:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2014-06-06 14:38:20 -07:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2014-06-04 16:06:53 -07:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:09 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-12 09:58:07 +02:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
2014-01-21 15:50:55 -08:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 14:05:52 -08:00
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
2017-02-22 15:45:07 -08:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:09 -07:00
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
2017-02-22 15:45:07 -08:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-24 16:58:12 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-10-11 13:55:11 -07:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-10-11 13:55:11 -07:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-06-30 14:56:52 -07:00
|
|
|
|
2014-01-21 15:50:19 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-02-05 15:36:19 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2016-10-07 16:59:18 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-01-24 12:20:09 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-04-07 15:37:53 -07:00
|
|
|
|
2013-01-24 12:20:09 -08:00
|
|
|
|
|
|
|
|
|
2011-10-31 17:08:16 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-28 15:43:02 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-08-04 14:06:41 +10:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2016-07-28 15:48:26 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2016-07-28 15:48:26 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2016-07-28 15:48:26 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2017-04-03 11:23:55 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-07-28 15:48:26 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-04-03 11:23:55 +09:00
|
|
|
|
2016-07-28 15:48:26 -07:00
|
|
|
|
|
|
|
|
|
2010-10-11 12:34:09 -07:00
|
|
|
|
2010-08-04 14:38:47 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-01-14 15:18:54 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2010-08-04 14:38:47 +10:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2016-01-14 15:18:54 -08:00
|
|
|
|
2010-08-04 14:38:47 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-11-30 13:28:15 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-09-11 14:22:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-04-07 15:37:53 -07:00
|
|
|
|
2013-09-11 14:22:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-04 16:07:51 -07:00
|
|
|
|
|
|
|
|
|
2013-09-11 14:22:17 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-05-24 00:45:21 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-09-15 13:05:29 -07:00
|
|
|
|
2010-08-04 14:38:47 +10:00
|
|
|
|
2011-01-20 14:44:20 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2010-08-04 14:38:47 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:45:04 -08:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2012-05-24 00:45:21 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-09-08 15:02:00 -07:00
|
|
|
|
2012-05-24 00:45:21 -07:00
|
|
|
|
2015-09-08 15:02:00 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
2011-12-08 10:22:07 -08:00
|
|
|
|
2015-09-08 15:02:00 -07:00
|
|
|
|
2010-07-12 14:36:09 +10:00
|
|
|
|
|
|
|
|
|
2012-10-22 16:35:18 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
2012-10-22 16:35:18 -07:00
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-10-22 16:35:18 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
|
|
|
|
|
2012-10-22 16:35:18 -07:00
|
|
|
|
2014-04-07 15:37:52 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-10-22 16:35:18 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:01 -07:00
|
|
|
|
2010-09-15 13:05:29 -07:00
|
|
|
|
2010-07-06 15:39:01 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-27 01:23:43 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2016-01-14 15:20:42 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
2014-01-21 15:49:20 -08:00
|
|
|
|
2011-07-14 11:43:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-06-02 14:46:49 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:06 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
|
|
|
|
|
2017-02-24 14:55:56 -08:00
|
|
|
|
2017-02-24 14:55:59 -08:00
|
|
|
|
2017-02-24 14:55:56 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
2011-12-08 10:22:08 -08:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
2010-07-06 15:39:13 -07:00
|
|
|
|
2010-07-12 14:36:48 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2011-07-14 11:46:03 +02:00
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
2017-02-22 15:46:42 -08:00
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-01-29 18:16:01 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-07-06 15:39:19 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
