0
Fork 0
mirror of https://codeberg.org/ziglang/zig synced 2026-11-18 07:05:05 +00:00
zig/lib/std
Ryan Liptak cf3572a66b GeneralPurposeAllocator: Considerably improve worst case performance
Before this commit, GeneralPurposeAllocator could run into incredibly degraded performance in scenarios where the bucket count for a particular size class grew to be large. For example, if exactly `slot_count` allocations of a single size class were performed and then all of them were freed except one, then the bucket for those allocations would have to be kept around indefinitely. If that pattern of allocation were done over and over, then the bucket list for that size class could grow incredibly large.

This allocation pattern has been seen in the wild: https://github.com/Vexu/arocc/issues/508#issuecomment-1738275688

In that case, the length of the bucket list for the `128` size class would grow to tens of thousands of buckets and cause Debug runtime to balloon to ~8 minutes whereas with the c_allocator the Debug runtime would be ~3 seconds.

To address this, there are three different changes happening here:

1. std.Treap is used instead of a doubly linked list for the lists of buckets. This takes the time complexity of searchBucket [used in resize and free] from O(n) to O(log n), but increases the time complexity of insert from O(1) to O(log n) [before, all new buckets would get added to the head of the list]. Note: Any data structure with O(log n) or better search/insert/delete would also work for this use-case.
2. If the 'current' bucket for a size class is full, the list of buckets is never traversed and instead a new bucket is allocated. Previously, traversing the bucket list could only find a non-full bucket in specific circumstances, and only because of a separate optimization that is no longer needed (before, after any resize/free, the affected bucket would be moved to the head of the bucket list to allow searchBucket to perform better on average). Now, the current_bucket for each size class only changes when either (1) the current bucket is emptied/freed, or (2) a new bucket is allocated (due to the current bucket being full or null). Because each bucket's alloc_cursor only moves forward (i.e. slots within a bucket are never re-used), we can therefore always know that any bucket besides the current_bucket will be full, so traversing the list in the hopes of finding an existing non-full bucket is entirely pointless.
3. Size + alignment information for small allocations has been moved into the Bucket data instead of keeping it in a separate HashMap. This offers an improvement over the HashMap since whenever we need to get/modify the length/alignment of an allocation it's extremely likely we will already have calculated any bucket-related information necessary to get the data.

The first change is the most relevant and accounts for most of the benefit here. Also note that the overall functionality of GeneralPurposeAllocator is unchanged.

In the degraded `arocc` case, these changes bring Debug performance from ~8 minutes to ~20 seconds.

Benchmark 1: test-master.bat
  Time (mean ± σ):     481.263 s ±  5.440 s    [User: 479.159 s, System: 1.937 s]
  Range (min … max):   477.416 s … 485.109 s    2 runs

Benchmark 2: test-optim-treap.bat
  Time (mean ± σ):     19.639 s ±  0.037 s    [User: 18.183 s, System: 1.452 s]
  Range (min … max):   19.613 s … 19.665 s    2 runs

Summary
  'test-optim-treap.bat' ran
   24.51 ± 0.28 times faster than 'test-master.bat'

Note: Much of the time taken on Windows in this particular case is related to gathering stack traces. With `.stack_trace_frames = 0` the runtime goes down to 6.7 seconds, which is a little more than 2.5x slower compared to when the c_allocator is used.

These changes may or mat not introduce a slight performance regression in the average case:

Here's the standard library tests on Windows in Debug mode:

Benchmark 1 (10 runs): std-tests-master.exe
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          16.0s  ± 30.8ms    15.9s  … 16.1s           1 (10%)        0%
  peak_rss           42.8MB ± 8.24KB    42.8MB … 42.8MB          0 ( 0%)        0%
Benchmark 2 (10 runs): std-tests-optim-treap.exe
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          16.2s  ± 37.6ms    16.1s  … 16.3s           0 ( 0%)        💩+  1.3% ±  0.2%
  peak_rss           42.8MB ± 5.18KB    42.8MB … 42.8MB          0 ( 0%)          +  0.1% ±  0.0%

And on Linux:

Benchmark 1: ./test-master
  Time (mean ± σ):     16.091 s ±  0.088 s    [User: 15.856 s, System: 0.453 s]
  Range (min … max):   15.870 s … 16.166 s    10 runs
 
Benchmark 2: ./test-optim-treap
  Time (mean ± σ):     16.028 s ±  0.325 s    [User: 15.755 s, System: 0.492 s]
  Range (min … max):   15.735 s … 16.709 s    10 runs
 
Summary
  './test-optim-treap' ran
    1.00 ± 0.02 times faster than './test-master'
History 2023-10-03 01:21:51 -07:00
..
atomic write function types consistently with a space before fn keyword 2023-09-19 15:15:05 +03:00
Build Support non zig dependencies 2023-09-24 02:47:21 +01:00
c Added basic support for wasm32-emscripten target 2023-09-22 12:49:03 -07:00
compress all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
crypto std.crypto.aead: Consistent decryption tail and doc fixes (#16781) 2023-08-14 21:39:51 +02:00
dwarf compiler: move unions into InternPool 2023-08-22 13:54:14 -07:00
event all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
fmt re-enable test "fmt.parseFloat nan and inf" after llvm fix 2023-07-25 11:03:11 -07:00
fs fs tests: Skip UNC path types in Dir.rename tests 2023-09-14 16:30:41 +03:00
hash std/hash: perform iterative + smhasher tests at comptime 2023-09-02 16:58:03 +12:00
heap GeneralPurposeAllocator: Considerably improve worst case performance 2023-10-03 01:21:51 -07:00
http std.http.Client: add note about resource management 2023-09-25 12:17:11 +03:00
io Fix type mismatch for Reader.readIntoBoundedBytes (#16416) 2023-07-24 17:32:06 -07:00
json std.json: add fmt function (#17055) 2023-09-13 18:24:59 -04:00
math std.math.log_int: implement integer logarithm without using float math 2023-09-14 19:33:56 +00:00
mem mem: explicit dupe and dupeZ error on Allocator 2023-09-07 21:56:57 +03:00
meta all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
net std.net: Fix IPv6 address parsing for single digit 2023-09-06 11:14:24 +03:00
os Added basic support for wasm32-emscripten target 2023-09-22 12:49:03 -07:00
rand Use builtin inference over @as where possible 2023-07-24 10:23:51 -07:00
sort improve documentation of std.sort.*Context functions (#16145) 2023-06-27 00:51:06 -07:00
target nvptx: omit sm_90a from the CPU target features for now 2023-09-19 09:37:53 -07:00
testing std: enable FailingAllocator to fail on resize 2023-09-06 19:06:32 +03:00
Thread Fix counting in SingleThreadedRwLock's tryLockShared (#16560) 2023-07-26 08:19:52 -05:00
time all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
tz Support legacy TZ format, expose header struct to a potential writer 2022-01-01 12:47:08 +00:00
unicode all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
valgrind all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
zig Merge pull request #17069 from squeek502/resinator 2023-09-22 12:18:50 -07:00
array_hash_map.zig std.MultiArrayList: add test coverage for 0-bit structs 2023-09-24 15:49:56 -07:00
array_list.zig std: enable FailingAllocator to fail on resize 2023-09-06 19:06:32 +03:00
ascii.zig migration: std.math.{min, min3, max, max3} -> @min & @max 2023-06-16 13:44:09 -07:00
atomic.zig std: remove names from incorrectly named tests 2023-04-21 23:08:48 -04:00
base64.zig Optimize std.base64.standard.Encoder.encode 2023-08-29 16:00:31 -04:00
bit_set.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
BitStack.zig std.json: Unify stringify and writeStream (#16405) 2023-07-21 19:56:46 -04:00
bounded_array.zig Smaller memory footprint for BoundedArray (#16299) 2023-07-03 13:58:03 -04:00
buf_map.zig doc: fix minor grammar issues 2023-04-22 13:09:15 +03:00
buf_set.zig update codebase to use @memset and @memcpy 2023-04-28 13:24:43 -07:00
Build.zig Support non zig dependencies 2023-09-24 02:47:21 +01:00
builtin.zig std.os.uefi: reorganize namespaces 2023-08-24 22:38:47 -07:00
c.zig std: correct getcontext for Android bionic libc 2023-09-16 14:22:09 -07:00
child_process.zig Windows: Fix TooManyParentDirs handling for paths that shouldn't be cwd-relative 2023-08-11 18:58:40 -07:00
coff.zig std.coff: Fixed compile error. 2023-09-02 00:18:53 +02:00
compress.zig std.compress.zstd: renamed from std.compress.zstandard 2023-02-22 00:11:20 +11:00
comptime_string_map.zig ComptimeStringMap: Add version that takes an equality function 2023-09-14 11:03:15 -07:00
crypto.zig std.crypto.ff - Alloc-free, constant-time field arithmetic for crypto (#15795) 2023-05-22 16:11:06 +02:00
cstr.zig std.cstr: deprecate namespace 2023-06-25 14:51:03 -07:00
debug.zig compiler: preserve result type information through address-of operator 2023-09-23 22:01:08 +01:00
dwarf.zig std: remove some unused imports (#16710) 2023-08-06 15:18:50 -04:00
dynamic_library.zig Windows: Fix TooManyParentDirs handling for paths that shouldn't be cwd-relative 2023-08-11 18:58:40 -07:00
elf.zig std: remove some unused imports (#16710) 2023-08-06 15:18:50 -04:00
enums.zig write function types consistently with a space before fn keyword 2023-09-19 15:15:05 +03:00
event.zig std, compiler-rt: remove test names where applicable 2021-09-01 17:54:06 -07:00
fifo.zig std: remove some unused imports (#16710) 2023-08-06 15:18:50 -04:00
fmt.zig write function types consistently with a space before fn keyword 2023-09-19 15:15:05 +03:00
fs.zig remove outdated error message in std.fs 2023-09-08 21:47:44 +03:00
hash.zig add std.hash.uint32 2023-06-10 20:42:30 -07:00
hash_map.zig std: remove some unused imports (#16710) 2023-08-06 15:18:50 -04:00
heap.zig std: remove some unused imports (#16710) 2023-08-06 15:18:50 -04:00
http.zig std.http: allow for arbitrary http methods 2023-08-30 13:05:45 -05:00
Ini.zig add std.Ini for basic .ini file parsing 2023-01-11 15:39:48 -08:00
io.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
json.zig std.json: add fmt function (#17055) 2023-09-13 18:24:59 -04:00
leb128.zig Use builtin inference over @as where possible 2023-07-24 10:23:51 -07:00
linked_list.zig std: Move TailQueue alias to correct namespace 2023-08-28 17:41:37 -07:00
log.zig all: zig fmt and rename "@XToY" to "@YFromX" 2023-06-19 12:34:42 -07:00
macho.zig macho: add helper for accessing tools array in LC_BUILD_VERSION cmd 2023-08-18 11:55:29 +02:00
math.zig std: math.absInt: update docs to use fabs for floats 2023-09-18 21:44:56 +03:00
mem.zig skip regressed LLVM 17 std lib test on powerpc 2023-09-19 09:37:53 -07:00
meta.zig write function types consistently with a space before fn keyword 2023-09-19 15:15:05 +03:00
multi_array_list.zig std.MultiArrayList: add test coverage for 0-bit structs 2023-09-24 15:49:56 -07:00
net.zig std.net: Fix IPv6 address parsing for single digit 2023-09-06 11:14:24 +03:00
once.zig update std lib and compiler sources to new for loop syntax 2023-02-18 19:17:21 -07:00
os.zig Added basic support for wasm32-emscripten target 2023-09-22 12:49:03 -07:00
packed_int_array.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
pdb.zig std.pdb: fix incorrect field alignment in ptrCast 2023-06-30 23:23:03 -04:00
priority_dequeue.zig [priority_dequeue] Fix out-of-bounds access 2023-06-22 02:03:53 -07:00
priority_queue.zig [priority_queue] Simplify sifting & fix edge case 2023-07-24 12:58:19 -07:00
process.zig std.process.totalSystemMemory: return correct error type on FreeBSD 2023-08-01 19:27:04 -07:00
Progress.zig std.Progress: add optional unit to progress indicator 2023-07-25 13:51:22 +01:00
rand.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
RingBuffer.zig all: zig fmt and rename "@XToY" to "@YFromX" 2023-06-19 12:34:42 -07:00
segmented_list.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
SemanticVersion.zig std: replace builtin.Version with SemanticVersion 2023-06-17 13:17:34 -07:00
simd.zig Remove len parameter from splat in standard lib 2023-07-12 15:35:57 -07:00
sort.zig improve documentation of std.sort.*Context functions (#16145) 2023-06-27 00:51:06 -07:00
start.zig start: workaround for LLVM 17 regression 2023-09-19 09:37:32 -07:00
start_windows_tls.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
std.zig ComptimeStringMap: Add version that takes an equality function 2023-09-14 11:03:15 -07:00
tar.zig std.tar: add support for file path in pax attributes 2023-09-08 21:55:14 +03:00
target.zig compiler_rt: fix arm hard-float f16 abi 2023-09-19 09:37:53 -07:00
testing.zig std.testing: expectEqualDeep() - support self-referential structs 2023-09-23 20:25:57 +00:00
Thread.zig Merge pull request #16622 from jacobly0/cbe-asm-compat 2023-07-31 15:34:32 -07:00
time.zig implement std.time.sleep for uefi 2023-07-07 11:57:09 -07:00
treap.zig Treap: Add InorderIterator 2023-10-02 21:11:14 -07:00
tz.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
unicode.zig std.json: Fix decoding of UTF-16 surrogate pairs (#16830) 2023-08-15 09:11:59 -04:00
Uri.zig std.Uri: Don't double-escape escaped query parameters (#16043) 2023-07-27 13:18:48 -04:00
valgrind.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00
wasm.zig all: zig fmt and rename "@XToY" to "@YFromX" 2023-06-19 12:34:42 -07:00
zig.zig all: migrate code to new cast builtin syntax 2023-06-24 16:56:39 -07:00