kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-10-26 19:14:18 +00:00
Code Issues Releases Packages Activity
linux/fs
History
Linus Torvalds a8be8af184 vfs: fix bad hashing of dentries 
commit 99d263d4c5 upstream.

Josef Bacik found a performance regression between 3.2 and 3.10 and
narrowed it down to commit bfcfaa77bd ("vfs: use 'unsigned long'
accesses for dcache name comparison and hashing"). He reports:

 "The test case is essentially

      for (i = 0; i < 1000000; i++)
              mkdir("a$i");

  On xfs on a fio card this goes at about 20k dir/sec with 3.2, and 12k
  dir/sec with 3.10.  This is because we spend waaaaay more time in
  __d_lookup on 3.10 than in 3.2.

  The new hashing function for strings is suboptimal for <
  sizeof(unsigned long) string names (and hell even > sizeof(unsigned
  long) string names that I've tested).  I broke out the old hashing
  function and the new one into a userspace helper to get real numbers
  and this is what I'm getting:

      Old hash table had 1000000 entries, 0 dupes, 0 max dupes
      New hash table had 12628 entries, 987372 dupes, 900 max dupes
      We had 11400 buckets with a p50 of 30 dupes, p90 of 240 dupes, p99 of 567 dupes for the new hash

  My test does the hash, and then does the d_hash into a integer pointer
  array the same size as the dentry hash table on my system, and then
  just increments the value at the address we got to see how many
  entries we overlap with.

  As you can see the old hash function ended up with all 1 million
  entries in their own bucket, whereas the new one they are only
  distributed among ~12.5k buckets, which is why we're using so much
  more CPU in __d_lookup".

The reason for this hash regression is two-fold:

 - On 64-bit architectures the down-mixing of the original 64-bit
   word-at-a-time hash into the final 32-bit hash value is very
   simplistic and suboptimal, and just adds the two 32-bit parts
   together.

   In particular, because there is no bit shuffling and the mixing
   boundary is also a byte boundary, similar character patterns in the
   low and high word easily end up just canceling each other out.

 - the old byte-at-a-time hash mixed each byte into the final hash as it
   hashed the path component name, resulting in the low bits of the hash
   generally being a good source of hash data.  That is not true for the
   word-at-a-time case, and the hash data is distributed among all the
   bits.

The fix is the same in both cases: do a better job of mixing the bits up
and using as much of the hash data as possible.  We already have the
"hash_32|64()" functions to do that.

Reported-by: Josef Bacik <jbacik@fb.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chris Mason <clm@fb.com>
Cc: linux-fsdevel@vger.kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
History 2014-09-17 09:19:29 -07:00
..
9p Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-01-28 08:38:04 -08:00
adfs adfs: delayed freeing of sbi 2013-10-24 23:43:27 -04:00
affs fs/affs/super.c: bugfix / double free 2014-06-07 10:28:16 -07:00
afs afs: proc cells and rootcell are writeable 2014-02-01 10:59:39 -08:00
autofs4 autofs: fix lockref lookup 2014-06-07 10:28:20 -07:00
befs befs: iget_locked() doesn't return an ERR_PTR 2014-01-25 03:14:38 -05:00
bfs truncate: drop 'oldsize' truncate_pagecache() parameter 2013-09-12 15:38:02 -07:00
btrfs Btrfs: fix crash on endio of reading corrupted block 2014-09-05 16:34:16 -07:00
cachefiles Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-11-13 15:34:18 +09:00
ceph ceph: clear directory's completeness when creating file 2014-06-07 10:28:20 -07:00
cifs CIFS: Fix wrong restart readdir for SMB1 2014-09-17 09:19:28 -07:00
coda coda_revalidate_inode(): switch to passing inode... 2013-11-09 00:16:21 -05:00
configfs configfs: fix race between dentry put and lookup 2013-11-21 16:42:27 -08:00
cramfs cramfs: take headers to fs/cramfs 2014-01-25 03:13:02 -05:00
debugfs debugfs: Fix corrupted loop in debugfs_remove_recursive 2014-09-05 16:34:15 -07:00
devpts devpts: plug the memory leak in kill_sb 2013-11-13 12:09:36 +09:00
dlm Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2014-01-25 11:17:34 -08:00
ecryptfs ecryptfs: fix failure handling in ->readlink() 2014-01-25 03:13:00 -05:00
efivarfs consolidate simple ->d_delete() instances 2013-11-15 22:04:17 -05:00
efs efs: get rid of ->put_super() 2014-01-25 03:13:02 -05:00
exofs exofs: Print less in r4w 2014-01-23 18:54:14 +02:00
exportfs exportfs: fix quadratic behavior in filehandle lookup 2013-11-09 00:16:38 -05:00
ext2 ext2/3/4: use generic posix ACL infrastructure 2014-01-25 23:58:19 -05:00
ext3 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-01-28 08:38:04 -08:00
ext4 jbd2: fix descriptor block size handling errors with journal_csum 2014-09-05 16:34:17 -07:00
f2fs Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
fat fat: rcu-delay unloading nls and freeing sbi 2013-10-24 23:43:28 -04:00
freevxfs [readdir] convert freevxfs 2013-06-29 12:56:53 +04:00
fscache FS-Cache: Handle removal of unadded object to the fscache_object_list rb tree 2014-02-17 13:47:35 -08:00
fuse fuse: ignore entry-timeout on LOOKUP_REVAL 2014-07-28 08:05:57 -07:00
gfs2 Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
hfs fs/hfs/btree.h: remove duplicate defines 2013-11-13 12:09:32 +09:00
hfsplus hfsplus: add HFSX subfolder count support 2014-03-10 17:26:21 -07:00
hostfs um: hostfs: make functions static 2014-01-26 11:51:09 +01:00
hpfs hpfs: optimize quad buffer loading 2014-02-02 16:24:07 -08:00
hppfs clean up scary strncpy(dst, src, strlen(src)) uses 2013-07-03 16:07:41 -07:00
hugetlbfs cope with potentially long ->d_dname() output for shmem/hugetlb 2013-08-24 12:10:17 -04:00
isofs isofs: Fix unbounded recursion when processing relocated directories 2014-09-05 16:34:12 -07:00
jbd jbd: Revise KERN_EMERG error messages 2013-12-04 12:27:46 +01:00
jbd2 jbd2: fix descriptor block size handling errors with journal_csum 2014-09-05 16:34:17 -07:00
jffs2 jffs2: remove from wait queue after schedule() 2014-04-26 17:19:06 -07:00
jfs jfs: set i_ctime when setting ACL 2014-02-13 15:56:05 -06:00
kernfs kernfs: add back missing error check in kernfs_fop_mmap() 2014-06-07 10:28:08 -07:00
lockd lockd: ensure we tear down any live sockets when socket creation fails during lockd_up 2014-05-13 13:32:56 +02:00
logfs Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
minix fs/minix: Drop dependency on H8300 2013-09-16 18:20:25 -07:00
ncpfs ncpfs: rcu-delay unload_nls() and freeing ncp_server 2013-10-24 23:43:28 -04:00
nfs NFSv4: Fix problems with close in the presence of a delegation 2014-09-05 16:34:18 -07:00
nfs_common nfs_common: Update the translation between nfsv3 acls linux posix acls 2013-02-13 06:15:14 -08:00
nfsd svcrdma: Select NFSv4.1 backchannel transport based on forward channel 2014-09-05 16:34:18 -07:00
nilfs2 Merge branch 'for-3.14/core' of git://git.kernel.dk/linux-block 2014-01-30 11:19:05 -08:00
nls nls: have register_nls() set ->owner 2014-01-25 03:14:05 -05:00
notify fsnotify: Allocate overflow events with proper type 2014-02-25 11:18:06 +01:00
ntfs fix O_SYNC|O_APPEND syncing the wrong range on write() 2014-02-09 15:18:09 -05:00
ocfs2 ocfs2: fix panic on kfree(xattr->name) 2014-05-06 07:59:36 -07:00
omfs truncate: drop 'oldsize' truncate_pagecache() parameter 2013-09-12 15:38:02 -07:00
openpromfs [readdir] convert openpromfs 2013-06-29 12:56:32 +04:00
proc CAPABILITIES: remove undefined caps from all processes 2014-09-17 09:19:09 -07:00
pstore pstore: Don't allow high traffic options on fragile devices 2013-12-20 13:12:01 -08:00
qnx4 qnx4: clean qnx4_fill_super() up 2014-01-25 03:13:03 -05:00
qnx6 [readdir] convert qnx6 2013-06-29 12:56:39 +04:00
quota quota: missing lock in dqcache_shrink_scan() 2014-07-28 08:05:58 -07:00
ramfs fs/ramfs: move ramfs_aops to inode.c 2014-01-23 16:36:58 -08:00
reiserfs reiserfs: call truncate_setsize under tailpack mutex 2014-07-06 18:57:29 -07:00
romfs romfs: fix returm err while getting inode in fill_super 2014-01-23 16:37:04 -08:00
squashfs Squashfs: fix failure to unlock pages on decompress error 2013-11-24 01:02:50 +00:00
sysfs sysfs: make sure read buffer is zeroed 2014-06-07 10:28:24 -07:00
sysv sysv: Add forgotten superblock lock init for v7 fs 2013-09-29 22:02:02 -04:00
ubifs UBIFS: Remove incorrect assertion in shrink_tnc() 2014-07-06 18:57:26 -07:00
udf udf: Fix data corruption on file type conversion 2014-02-20 21:56:00 +01:00
ufs truncate: drop 'oldsize' truncate_pagecache() parameter 2013-09-12 15:38:02 -07:00
xfs xfs: don't zero partial page cache pages during O_DIRECT writes 2014-09-17 09:19:25 -07:00
aio.c aio: protect reqs_available updates from changes in interrupt handlers 2014-07-28 08:06:03 -07:00
anon_inodes.c vfs: Allocate anon_inode_inode in anon_inode_init() 2014-03-27 09:52:54 -07:00
attr.c fs,userns: Change inode_capable to capable_wrt_inode_uidgid 2014-06-16 13:40:32 -07:00
bad_inode.c [readdir] ->readdir() is gone 2013-06-29 12:57:04 +04:00
binfmt_aout.c dump_skip(): dump_seek() replacement taking coredump_params 2013-11-09 00:16:26 -05:00
binfmt_elf.c fs: binfmt_elf: remove unused defines INTERPRETER_NONE and INTERPRETER_ELF 2014-01-23 16:36:58 -08:00
binfmt_elf_fdpic.c elf{,_fdpic} coredump: get rid of pointless if (siginfo->si_signo) 2013-11-09 00:16:30 -05:00
binfmt_em86.c file->f_op is never NULL... 2013-10-24 23:34:54 -04:00
binfmt_flat.c new helper: read_code() 2013-04-29 15:40:23 -04:00
binfmt_misc.c binfmt_misc: reuse string_unescape_inplace() 2013-04-30 17:04:03 -07:00
binfmt_script.c exec: do not leave bprm->interp on stack 2012-12-20 17:40:19 -08:00
binfmt_som.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
bio-integrity.c bio-integrity: Drop bio_integrity_verify BUG_ON in post bip->bip_iter world 2014-02-21 15:56:36 -08:00
bio.c block: Fix cloning of discard/write same bios 2014-02-11 08:40:45 -07:00
block_dev.c a trivial writeback fix 2013-09-13 23:06:40 -04:00
buffer.c mm: __set_page_dirty uses spin_lock_irqsave instead of spin_lock_irq 2014-02-06 13:48:51 -08:00
char_dev.c Merge branch 'for-3.13/core' of git://git.kernel.dk/linux-block 2013-11-14 12:08:14 +09:00
compat.c [readdir] constify ->actor 2013-06-29 12:57:05 +04:00
compat_binfmt_elf.c coredump: extend core dump note section to contain file names of mapped files 2012-10-06 03:05:17 +09:00
compat_ioctl.c fs/compat_ioctl.c: fix an underflow issue (harmless) 2014-01-21 16:19:42 -08:00
coredump.c coredump: fix the setting of PF_DUMPCORE 2014-07-31 12:52:56 -07:00
dcache.c vfs: fix bad hashing of dentries 2014-09-17 09:19:29 -07:00
dcookies.c fs/compat: fix lookup_dcookie() parameter handling 2014-01-29 16:22:40 -08:00
direct-io.c block: Abstract out bvec iterator 2013-11-23 22:33:47 -08:00
drop_caches.c shrinker: add node awareness 2013-09-10 18:56:31 -04:00
eventfd.c eventfd_ctx_fdget(): use fdget() instead of fget() 2014-01-25 03:13:04 -05:00
eventpoll.c epoll: fix use-after-free in eventpoll_release_file 2014-06-30 20:12:02 -07:00
exec.c metag: Reduce maximum stack size to 256MB 2014-06-07 10:28:23 -07:00
fcntl.c file->f_op is never NULL... 2013-10-24 23:34:54 -04:00
fhandle.c Merge branch 'for-3.8' of git://linux-nfs.org/~bfields/linux 2012-12-20 14:04:11 -08:00
file.c vfs: Don't let __fdget_pos() get FMODE_PATH files 2014-03-23 00:03:12 -04:00
file_table.c don't bother with {get,put}_write_access() on non-regular files 2014-05-31 13:20:29 -07:00
filesystems.c fs: Limit sys_mount to only request filesystem modules. 2013-03-03 19:36:31 -08:00
fs-writeback.c bdi: avoid oops on device removal 2014-04-26 17:19:05 -07:00
fs_struct.c seqcount: Add lockdep functionality to seqcount/seqlock structures 2013-11-06 12:40:26 +01:00
inode.c fs,userns: Change inode_capable to capable_wrt_inode_uidgid 2014-06-16 13:40:32 -07:00
internal.h get rid of s_files and files_lock 2013-11-09 00:16:20 -05:00
ioctl.c file->f_op is never NULL... 2013-10-24 23:34:54 -04:00
ioprio.c Merge branch 'for-3.5/core' of git://git.kernel.dk/linux-block 2012-05-30 08:52:42 -07:00
Kconfig fs: remove generic_acl 2014-01-26 08:26:40 -05:00
Kconfig.binfmt fs: make binfmt support for #! scripts modular and removable 2013-04-30 17:04:04 -07:00
libfs.c consolidate simple ->d_delete() instances 2013-11-15 22:04:17 -05:00
locks.c locks: allow __break_lease to sleep even when break_time is 0 2014-05-13 13:32:53 +02:00
Makefile Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2014-01-28 08:38:04 -08:00
mbcache.c fs: convert fs shrinkers to new scan/count API 2013-09-10 18:56:31 -04:00
mount.h switch mnt_hash to hlist 2014-03-30 19:18:51 -04:00
mpage.c block: Abstract out bvec iterator 2013-11-23 22:33:47 -08:00
namei.c vfs: fix bad hashing of dentries 2014-09-17 09:19:29 -07:00
namespace.c fix copy_tree() regression 2014-09-17 09:19:23 -07:00
no-block.c llseek: automatically add .llseek fop 2010-10-15 15:53:27 +02:00
open.c don't bother with {get,put}_write_access() on non-regular files 2014-05-31 13:20:29 -07:00
pipe.c fs/pipe.c: skip file_update_time on frozen fs 2014-01-23 16:37:00 -08:00
pnode.c get rid of propagate_umount() mistakenly treating slaves as busy. 2014-09-17 09:19:22 -07:00
pnode.h smarter propagate_mnt() 2014-05-06 07:59:36 -07:00
posix_acl.c posix_acl: handle NULL ACL in posix_acl_equiv_mode 2014-06-07 10:28:16 -07:00
proc_namespace.c fs/proc_namespace.c: simplify testing nsp and nsp->mnt_ns 2014-01-23 16:37:02 -08:00
read_write.c vfs: atomic f_pos access in llseek() 2014-03-23 00:03:12 -04:00
readdir.c file->f_op is never NULL... 2013-10-24 23:34:54 -04:00
select.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2013-11-13 15:34:18 +09:00
seq_file.c seq_file: always clear m->count when we free m->buf 2013-11-18 19:07:53 -08:00
signalfd.c switch signalfd{,4}() to COMPAT_SYSCALL_DEFINE 2013-03-03 22:58:46 -05:00
splice.c fuse: fix pipe_buf_operations 2014-01-22 19:36:57 +01:00
stack.c fs: reduce the use of module.h wherever possible 2012-02-28 19:31:58 -05:00
stat.c vfs: split out vfs_getattr_nosec 2013-11-09 00:16:31 -05:00
statfs.c vfs: allow O_PATH file descriptors for fstatfs() 2013-10-12 13:12:31 -07:00
super.c fs: Don't return 0 from get_anon_bdev 2014-05-31 13:20:31 -07:00
sync.c Revert "writeback: do not sync data dirtied after sync start" 2014-02-22 02:02:28 +01:00
timerfd.c timerfd: Add alarm timers 2013-05-29 12:57:34 -07:00
utimes.c locks: break delegations on any attribute modification 2013-11-09 00:16:44 -05:00
xattr.c vfs: make lremovexattr retry once on ESTALE error 2012-12-20 18:50:11 -05:00