linux/fs/ubifs/file.c

1594 lines
45 KiB

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h percpu.h is included by sched.h and module.h and thus ends up being included when building most .c files. percpu.h includes slab.h which in turn includes gfp.h making everything defined by the two files universally available and complicating inclusion dependencies. percpu.h -> slab.h dependency is about to be removed. Prepare for this change by updating users of gfp and slab facilities include those headers directly instead of assuming availability. As this conversion needs to touch large number of source files, the following script is used as the basis of conversion. http://userweb.kernel.org/~tj/misc/slabh-sweep.py The script does the followings. * Scan files for gfp and slab usages and update includes such that only the necessary includes are there. ie. if only gfp is used, gfp.h, if slab is used, slab.h. * When the script inserts a new include, it looks at the include blocks and try to put the new include such that its order conforms to its surrounding. It's put in the include block which contains core kernel includes, in the same order that the rest are ordered - alphabetical, Christmas tree, rev-Xmas-tree or at the end if there doesn't seem to be any matching order. * If the script can't find a place to put a new include (mostly because the file doesn't have fitting include block), it prints out an error message indicating which .h file needs to be added to the file. The conversion was done in the following steps. 1. The initial automatic conversion of all .c files updated slightly over 4000 files, deleting around 700 includes and adding ~480 gfp.h and ~3000 slab.h inclusions. The script emitted errors for ~400 files. 2. Each error was manually checked. Some didn't need the inclusion, some needed manual addition while adding it to implementation .h or embedding .c file was more appropriate for others. This step added inclusions to around 150 files. 3. The script was run again and the output was compared to the edits from #2 to make sure no file was left behind. 4. Several build tests were done and a couple of problems were fixed. e.g. lib/decompress_*.c used malloc/free() wrappers around slab APIs requiring slab.h to be added manually. 5. The script was run on all .h files but without automatically editing them as sprinkling gfp.h and slab.h inclusions around .h files could easily lead to inclusion dependency hell. Most gfp.h inclusion directives were ignored as stuff from gfp.h was usually wildly available and often used in preprocessor macros. Each slab.h inclusion directive was examined and added manually as necessary. 6. percpu.h was updated not to include slab.h. 7. Build test were done on the following configurations and failures were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my distributed build env didn't work with gcov compiles) and a few more options had to be turned off depending on archs to make things build (like ipr on powerpc/64 which failed due to missing writeq). * x86 and x86_64 UP and SMP allmodconfig and a custom test config. * powerpc and powerpc64 SMP allmodconfig * sparc and sparc64 SMP allmodconfig * ia64 SMP allmodconfig * s390 SMP allmodconfig * alpha SMP allmodconfig * um on x86_64 SMP allmodconfig 8. percpu.h modifications were reverted so that it could be applied as a separate patch and serve as bisection point. Given the fact that I had only a couple of failures from tests on step 6, I'm fairly confident about the coverage of this conversion patch. If there is a breakage, it's likely to be something in one of the arch headers which should be easily discoverable easily on most builds of the specific arch. Signed-off-by: Tejun Heo <tj@kernel.org> Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 17:04:11 +09:00
fs: symlink write_begin allocation context fix With the write_begin/write_end aops, page_symlink was broken because it could no longer pass a GFP_NOFS type mask into the point where the allocations happened. They are done in write_begin, which would always assume that the filesystem can be entered from reclaim. This bug could cause filesystem deadlocks. The funny thing with having a gfp_t mask there is that it doesn't really allow the caller to arbitrarily tinker with the context in which it can be called. It couldn't ever be GFP_ATOMIC, for example, because it needs to take the page lock. The only thing any callers care about is __GFP_FS anyway, so turn that into a single flag. Add a new flag for write_begin, AOP_FLAG_NOFS. Filesystems can now act on this flag in their write_begin function. Change __grab_cache_page to accept a nofs argument as well, to honour that flag (while we're there, change the name to grab_cache_page_write_begin which is more instructive and does away with random leading underscores). This is really a more flexible way to go in the end anyway -- if a filesystem happens to want any extra allocations aside from the pagecache ones in ints write_begin function, it may now use GFP_KERNEL (rather than GFP_NOFS) for common case allocations (eg. ocfs2_alloc_write_ctxt, for a random example). [kosaki.motohiro@jp.fujitsu.com: fix ubifs] [kosaki.motohiro@jp.fujitsu.com: fix fuse] Signed-off-by: Nick Piggin <npiggin@suse.de> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: <stable@kernel.org> [2.6.28.x] Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> [ Cleaned up the calling convention: just pass in the AOP flags untouched to the grab_cache_page_write_begin() function. That just simplifies everybody, and may even allow future expansion of the logic. - Linus ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-01-04 12:00:53 -08:00
fs: symlink write_begin allocation context fix With the write_begin/write_end aops, page_symlink was broken because it could no longer pass a GFP_NOFS type mask into the point where the allocations happened. They are done in write_begin, which would always assume that the filesystem can be entered from reclaim. This bug could cause filesystem deadlocks. The funny thing with having a gfp_t mask there is that it doesn't really allow the caller to arbitrarily tinker with the context in which it can be called. It couldn't ever be GFP_ATOMIC, for example, because it needs to take the page lock. The only thing any callers care about is __GFP_FS anyway, so turn that into a single flag. Add a new flag for write_begin, AOP_FLAG_NOFS. Filesystems can now act on this flag in their write_begin function. Change __grab_cache_page to accept a nofs argument as well, to honour that flag (while we're there, change the name to grab_cache_page_write_begin which is more instructive and does away with random leading underscores). This is really a more flexible way to go in the end anyway -- if a filesystem happens to want any extra allocations aside from the pagecache ones in ints write_begin function, it may now use GFP_KERNEL (rather than GFP_NOFS) for common case allocations (eg. ocfs2_alloc_write_ctxt, for a random example). [kosaki.motohiro@jp.fujitsu.com: fix ubifs] [kosaki.motohiro@jp.fujitsu.com: fix fuse] Signed-off-by: Nick Piggin <npiggin@suse.de> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: <stable@kernel.org> [2.6.28.x] Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> [ Cleaned up the calling convention: just pass in the AOP flags untouched to the grab_cache_page_write_begin() function. That just simplifies everybody, and may even allow future expansion of the logic. - Linus ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-01-04 12:00:53 -08:00
fs: symlink write_begin allocation context fix With the write_begin/write_end aops, page_symlink was broken because it could no longer pass a GFP_NOFS type mask into the point where the allocations happened. They are done in write_begin, which would always assume that the filesystem can be entered from reclaim. This bug could cause filesystem deadlocks. The funny thing with having a gfp_t mask there is that it doesn't really allow the caller to arbitrarily tinker with the context in which it can be called. It couldn't ever be GFP_ATOMIC, for example, because it needs to take the page lock. The only thing any callers care about is __GFP_FS anyway, so turn that into a single flag. Add a new flag for write_begin, AOP_FLAG_NOFS. Filesystems can now act on this flag in their write_begin function. Change __grab_cache_page to accept a nofs argument as well, to honour that flag (while we're there, change the name to grab_cache_page_write_begin which is more instructive and does away with random leading underscores). This is really a more flexible way to go in the end anyway -- if a filesystem happens to want any extra allocations aside from the pagecache ones in ints write_begin function, it may now use GFP_KERNEL (rather than GFP_NOFS) for common case allocations (eg. ocfs2_alloc_write_ctxt, for a random example). [kosaki.motohiro@jp.fujitsu.com: fix ubifs] [kosaki.motohiro@jp.fujitsu.com: fix fuse] Signed-off-by: Nick Piggin <npiggin@suse.de> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: <stable@kernel.org> [2.6.28.x] Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> [ Cleaned up the calling convention: just pass in the AOP flags untouched to the grab_cache_page_write_begin() function. That just simplifies everybody, and may even allow future expansion of the logic. - Linus ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-01-04 12:00:53 -08:00
UBIFS: fix assertion warning and refine comments This patch fixes the following UBIFS assertion warning: UBIFS assert failed in do_readpage at 115 (pid 199) [<b00321b8>] (unwind_backtrace+0x0/0xdc) from [<af025118>] (do_readpage+0x108/0x594 [ubifs]) [<af025118>] (do_readpage+0x108/0x594 [ubifs]) from [<af025764>] (ubifs_write_end+0x1c0/0x2e8 [ubifs]) [<af025764>] (ubifs_write_end+0x1c0/0x2e8 [ubifs]) from [<b00a0164>] (generic_file_buffered_write+0x18c/0x270) [<b00a0164>] (generic_file_buffered_write+0x18c/0x270) from [<b00a08d4>] (__generic_file_aio_write+0x478/0x4c0) [<b00a08d4>] (__generic_file_aio_write+0x478/0x4c0) from [<b00a0984>] (generic_file_aio_write+0x68/0xc8) [<b00a0984>] (generic_file_aio_write+0x68/0xc8) from [<af024a78>] (ubifs_aio_write+0x178/0x1d8 [ubifs]) [<af024a78>] (ubifs_aio_write+0x178/0x1d8 [ubifs]) from [<b00d104c>] (do_sync_write+0xb0/0x100) [<b00d104c>] (do_sync_write+0xb0/0x100) from [<b00d1abc>] (vfs_write+0xac/0x154) [<b00d1abc>] (vfs_write+0xac/0x154) from [<b00d1c10>] (sys_write+0x3c/0x68) [<b00d1c10>] (sys_write+0x3c/0x68) from [<b002d9a0>] (ret_fast_syscall+0x0/0x2c) The 'PG_checked' flag is used to indicate that the page does not supposedly exist on the media (e.g., a hole or a page beyond the inode size), so it requires slightly bigger budget, because we have to account the indexing size increase. And this flag basically tells that the budget for this page has to be "new page budget". The "new page budget" is slightly bigger than the "existing page budget". The 'do_readpage()' function has the following assertion which sometimes is hit: 'ubifs_assert(!PageChecked(page))'. Obviously, the meaning of this assertion is: "I should not be asked to read a page which does not exist on the media". However, in 'ubifs_write_begin()' we have a small "trick". Notice, that VFS may write pages which were not read yet, so the page data were not loaded from the media to the page cache yet. If VFS tells that it is going to change only some part of the page, we obviously have to load it from the media. However, if VFS tells that it is going to change whole page, we do not read it from the media for optimization purposes. However, since we do not read it, we do not know if it exists on the media or not (a hole, etc). So we set the 'PG_checked' flag to this page to force bigger budget, just in case. So 'ubifs_write_begin()' sets 'PG_checked'. Then we are in 'ubifs_write_end()'. And VFS tells us: "hey, for some reasons I changed my mind and did not change whole page". Frankly, I do not know why this happens, but I hit this somehow on an ARM platform. And this is extremely rare. So in this case UBIFS does the following: 1. Cancels allocated budget. 2. Loads the page from the media by calling 'do_readpage()'. 3. Asks VFS to repeat the whole write operation from the very beginning (call '->write_begin() again, etc). And the assertion warning is hit at the step 2 - remember we have the 'PG_checked' set for this page, and 'do_readpage()' does not like this. So this patch fixes the problem by adding step 1.5 and cleaning the 'PG_checked' before calling 'do_readpage()'. All in all, this patch does not fix any functionality issue, but it silences UBIFS false positive warning which may happen in very very rare cases. And while on it, this patch also improves a commentary which explains the reasons of setting the 'PG_checked' flag for the page. The old commentary was a bit difficult to understand. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2011-03-23 10:32:58 +02:00
fs: symlink write_begin allocation context fix With the write_begin/write_end aops, page_symlink was broken because it could no longer pass a GFP_NOFS type mask into the point where the allocations happened. They are done in write_begin, which would always assume that the filesystem can be entered from reclaim. This bug could cause filesystem deadlocks. The funny thing with having a gfp_t mask there is that it doesn't really allow the caller to arbitrarily tinker with the context in which it can be called. It couldn't ever be GFP_ATOMIC, for example, because it needs to take the page lock. The only thing any callers care about is __GFP_FS anyway, so turn that into a single flag. Add a new flag for write_begin, AOP_FLAG_NOFS. Filesystems can now act on this flag in their write_begin function. Change __grab_cache_page to accept a nofs argument as well, to honour that flag (while we're there, change the name to grab_cache_page_write_begin which is more instructive and does away with random leading underscores). This is really a more flexible way to go in the end anyway -- if a filesystem happens to want any extra allocations aside from the pagecache ones in ints write_begin function, it may now use GFP_KERNEL (rather than GFP_NOFS) for common case allocations (eg. ocfs2_alloc_write_ctxt, for a random example). [kosaki.motohiro@jp.fujitsu.com: fix ubifs] [kosaki.motohiro@jp.fujitsu.com: fix fuse] Signed-off-by: Nick Piggin <npiggin@suse.de> Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Cc: <stable@kernel.org> [2.6.28.x] Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> [ Cleaned up the calling convention: just pass in the AOP flags untouched to the grab_cache_page_write_begin() function. That just simplifies everybody, and may even allow future expansion of the logic. - Linus ] Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-01-04 12:00:53 -08:00
UBIFS: fix assertion warning and refine comments This patch fixes the following UBIFS assertion warning: UBIFS assert failed in do_readpage at 115 (pid 199) [<b00321b8>] (unwind_backtrace+0x0/0xdc) from [<af025118>] (do_readpage+0x108/0x594 [ubifs]) [<af025118>] (do_readpage+0x108/0x594 [ubifs]) from [<af025764>] (ubifs_write_end+0x1c0/0x2e8 [ubifs]) [<af025764>] (ubifs_write_end+0x1c0/0x2e8 [ubifs]) from [<b00a0164>] (generic_file_buffered_write+0x18c/0x270) [<b00a0164>] (generic_file_buffered_write+0x18c/0x270) from [<b00a08d4>] (__generic_file_aio_write+0x478/0x4c0) [<b00a08d4>] (__generic_file_aio_write+0x478/0x4c0) from [<b00a0984>] (generic_file_aio_write+0x68/0xc8) [<b00a0984>] (generic_file_aio_write+0x68/0xc8) from [<af024a78>] (ubifs_aio_write+0x178/0x1d8 [ubifs]) [<af024a78>] (ubifs_aio_write+0x178/0x1d8 [ubifs]) from [<b00d104c>] (do_sync_write+0xb0/0x100) [<b00d104c>] (do_sync_write+0xb0/0x100) from [<b00d1abc>] (vfs_write+0xac/0x154) [<b00d1abc>] (vfs_write+0xac/0x154) from [<b00d1c10>] (sys_write+0x3c/0x68) [<b00d1c10>] (sys_write+0x3c/0x68) from [<b002d9a0>] (ret_fast_syscall+0x0/0x2c) The 'PG_checked' flag is used to indicate that the page does not supposedly exist on the media (e.g., a hole or a page beyond the inode size), so it requires slightly bigger budget, because we have to account the indexing size increase. And this flag basically tells that the budget for this page has to be "new page budget". The "new page budget" is slightly bigger than the "existing page budget". The 'do_readpage()' function has the following assertion which sometimes is hit: 'ubifs_assert(!PageChecked(page))'. Obviously, the meaning of this assertion is: "I should not be asked to read a page which does not exist on the media". However, in 'ubifs_write_begin()' we have a small "trick". Notice, that VFS may write pages which were not read yet, so the page data were not loaded from the media to the page cache yet. If VFS tells that it is going to change only some part of the page, we obviously have to load it from the media. However, if VFS tells that it is going to change whole page, we do not read it from the media for optimization purposes. However, since we do not read it, we do not know if it exists on the media or not (a hole, etc). So we set the 'PG_checked' flag to this page to force bigger budget, just in case. So 'ubifs_write_begin()' sets 'PG_checked'. Then we are in 'ubifs_write_end()'. And VFS tells us: "hey, for some reasons I changed my mind and did not change whole page". Frankly, I do not know why this happens, but I hit this somehow on an ARM platform. And this is extremely rare. So in this case UBIFS does the following: 1. Cancels allocated budget. 2. Loads the page from the media by calling 'do_readpage()'. 3. Asks VFS to repeat the whole write operation from the very beginning (call '->write_begin() again, etc). And the assertion warning is hit at the step 2 - remember we have the 'PG_checked' set for this page, and 'do_readpage()' does not like this. So this patch fixes the problem by adding step 1.5 and cleaning the 'PG_checked' before calling 'do_readpage()'. All in all, this patch does not fix any functionality issue, but it silences UBIFS false positive warning which may happen in very very rare cases. And while on it, this patch also improves a commentary which explains the reasons of setting the 'PG_checked' flag for the page. The old commentary was a bit difficult to understand. Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2011-03-23 10:32:58 +02:00
UBIFS: endian handling fixes and annotations Noticed by sparse: fs/ubifs/file.c:75:2: warning: restricted __le64 degrades to integer fs/ubifs/file.c:629:4: warning: restricted __le64 degrades to integer fs/ubifs/dir.c:431:3: warning: restricted __le64 degrades to integer This should be checked to ensure the ubifs_assert is working as intended, I've done the suggested annotation in this patch. fs/ubifs/sb.c:298:6: warning: incorrect type in assignment (different base types) fs/ubifs/sb.c:298:6: expected int [signed] [assigned] tmp fs/ubifs/sb.c:298:6: got restricted __le64 [usertype] <noident> fs/ubifs/sb.c:299:19: warning: incorrect type in assignment (different base types) fs/ubifs/sb.c:299:19: expected restricted __le64 [usertype] atime_sec fs/ubifs/sb.c:299:19: got int [signed] [assigned] tmp fs/ubifs/sb.c:300:19: warning: incorrect type in assignment (different base types) fs/ubifs/sb.c:300:19: expected restricted __le64 [usertype] ctime_sec fs/ubifs/sb.c:300:19: got int [signed] [assigned] tmp fs/ubifs/sb.c:301:19: warning: incorrect type in assignment (different base types) fs/ubifs/sb.c:301:19: expected restricted __le64 [usertype] mtime_sec fs/ubifs/sb.c:301:19: got int [signed] [assigned] tmp This looks like a bugfix as your tmp was a u32 so there was truncation in the atime, mtime, ctime value, probably not intentional, add a tmp_le64 and use it here. fs/ubifs/key.h:348:9: warning: cast to restricted __le32 fs/ubifs/key.h:348:9: warning: cast to restricted __le32 fs/ubifs/key.h:419:9: warning: cast to restricted __le32 Read from the annotated union member instead. fs/ubifs/recovery.c:175:13: warning: incorrect type in assignment (different base types) fs/ubifs/recovery.c:175:13: expected unsigned int [unsigned] [usertype] save_flags fs/ubifs/recovery.c:175:13: got restricted __le32 [usertype] flags fs/ubifs/recovery.c:186:13: warning: incorrect type in assignment (different base types) fs/ubifs/recovery.c:186:13: expected restricted __le32 [usertype] flags fs/ubifs/recovery.c:186:13: got unsigned int [unsigned] [usertype] save_flags Do byteshifting at compile time of the flag value. Annotate the saved_flags as le32. fs/ubifs/debug.c:368:10: warning: cast to restricted __le32 fs/ubifs/debug.c:368:10: warning: cast from restricted __le64 Should be checked if the truncation was intentional, I've changed the printk to print the full width. Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com> Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
2008-10-24 10:52:57 -07:00