kernel/linux
0
Fork 0
mirror of https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ synced 2026-09-28 19:20:30 +00:00
Code Issues Releases Packages Activity
linux/lib/iov_iter.c

851 lines
20 KiB
Raw Permalink Normal View History

take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
new helper: iov_iter_get_pages_alloc() same as iov_iter_get_pages(), except that pages array is allocated (kmalloc if possible, vmalloc if that fails) and left for caller to free. Lustre and NFS ->direct_IO() switched to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-03-21 04:58:33 -04:00
csum_and_copy_..._iter() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-24 01:08:00 -05:00
take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: iterate_and_advance same as iterate_all_kinds, but iterator is moved to the position past the last byte we'd handled. iov_iter_advance() converted to it Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:59:45 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: iterate_and_advance same as iterate_all_kinds, but iterator is moved to the position past the last byte we'd handled. iov_iter_advance() converted to it Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:59:45 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
optimize copy_page_{to,from}_iter() if we'd ended up in the end of a segment, jump to the beginning of the next one (iov_offset = 0, iov++), rather than having the next primitive deal with that. Ought to be folded back... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 19:23:46 -04:00
take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
new helper: copy_page_from_iter() parallel to copy_page_to_iter(). pipe_write() switched to it (and became ->write_iter()). Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-03 15:05:18 -04:00
optimize copy_page_{to,from}_iter() if we'd ended up in the end of a segment, jump to the beginning of the next one (iov_offset = 0, iov++), rather than having the next primitive deal with that. Ought to be folded back... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 19:23:46 -04:00
new helper: copy_page_from_iter() parallel to copy_page_to_iter(). pipe_write() switched to it (and became ->write_iter()). Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-03 15:05:18 -04:00
take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
take iov_iter stuff to mm/iov_iter.c Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-02-05 19:11:33 -05:00
VFS: Add iov_iter_fault_in_multipages_readable() simillar to iov_iter_fault_in_readable() but differs in that it is not limited to faulting in the first iovec and instead faults in "bytes" bytes iterating over the iovecs as necessary. Also, instead of only faulting in the first and last page of the range, all pages are faulted in. This function is needed by NTFS when it does multi page file writes. Signed-off-by: Anton Altaparmakov <anton@tuxera.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-11 10:43:31 -04:00
start adding the tag to iov_iter For now, just use the same thing we pass to ->direct_IO() - it's all iovec-based at the moment. Pass it explicitly to iov_iter_init() and account for kvec vs. iovec in there, by the same kludge NFS ->direct_IO() uses. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-03-05 19:28:09 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
start adding the tag to iov_iter For now, just use the same thing we pass to ->direct_IO() - it's all iovec-based at the moment. Pass it explicitly to iov_iter_init() and account for kvec vs. iovec in there, by the same kludge NFS ->direct_IO() uses. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-03-05 19:28:09 -05:00
new helper: iov_iter_get_pages() iov_iter_get_pages(iter, pages, maxsize, &start) grabs references pinning the pages of up to maxsize of (contiguous) data from iter. Returns the amount of memory grabbed or -error. In case of success, the requested area begins at offset start in pages[0] and runs through pages[1], etc. Less than requested amount might be returned - either because the contiguous area in the beginning of iterator is smaller than requested, or because the kernel failed to pin that many pages. direct-io.c switched to using iov_iter_get_pages() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-03-15 04:05:57 -04:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: convert copy_to_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:28:06 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert copy_to_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:28:06 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert copy_to_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:28:06 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert copy_to_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:28:06 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert copy_to_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:28:06 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: get rid of bvec_copy_page_{to,from}_iter() Just have copy_page_{to,from}_iter() fall back to kmap_atomic + copy_{to,from}_iter() + kunmap_atomic() in ITER_BVEC case. As the matter of fact, that's what we want to do for any iov_iter kind that isn't blocking - e.g. ITER_KVEC will also go that way once we recognize it on iov_iter.c primitives level Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:22:37 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: get rid of bvec_copy_page_{to,from}_iter() Just have copy_page_{to,from}_iter() fall back to kmap_atomic + copy_{to,from}_iter() + kunmap_atomic() in ITER_BVEC case. As the matter of fact, that's what we want to do for any iov_iter kind that isn't blocking - e.g. ITER_KVEC will also go that way once we recognize it on iov_iter.c primitives level Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:22:37 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: convert copy_from_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:26:43 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert copy_from_iter() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:26:43 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: get rid of bvec_copy_page_{to,from}_iter() Just have copy_page_{to,from}_iter() fall back to kmap_atomic + copy_{to,from}_iter() + kunmap_atomic() in ITER_BVEC case. As the matter of fact, that's what we want to do for any iov_iter kind that isn't blocking - e.g. ITER_KVEC will also go that way once we recognize it on iov_iter.c primitives level Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:22:37 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
copy_from_iter_nocache() BTW, do we want memcpy_nocache()? Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 20:27:08 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: get rid of bvec_copy_page_{to,from}_iter() Just have copy_page_{to,from}_iter() fall back to kmap_atomic + copy_{to,from}_iter() + kunmap_atomic() in ITER_BVEC case. As the matter of fact, that's what we want to do for any iov_iter kind that isn't blocking - e.g. ITER_KVEC will also go that way once we recognize it on iov_iter.c primitives level Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:22:37 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: get rid of bvec_copy_page_{to,from}_iter() Just have copy_page_{to,from}_iter() fall back to kmap_atomic + copy_{to,from}_iter() + kunmap_atomic() in ITER_BVEC case. As the matter of fact, that's what we want to do for any iov_iter kind that isn't blocking - e.g. ITER_KVEC will also go that way once we recognize it on iov_iter.c primitives level Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:22:37 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
iov_iter.c: convert iov_iter_zero() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:18:54 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert iov_iter_zero() to iterate_and_advance Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:18:54 -05:00
Add copy_to_iter(), copy_from_iter() and iov_iter_zero() For DAX, we want to be able to copy between iovecs and kernel addresses that don't necessarily have a struct page. This is a fairly simple rearrangement for bvec iters to kmap the pages outside and pass them in, but for user iovecs it gets more complicated because we might try various different ways to kmap the memory. Duplicating the existing logic works out best in this case. We need to be able to write zeroes to an iovec for reads from unwritten ranges in a file. This is performed by the new iov_iter_zero() function, again patterned after the existing code that handles iovec iterators. [AV: and export the buggers...] Signed-off-by: Matthew Wilcox <willy@linux.intel.com> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-08-01 09:27:22 -04:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
Fix thinko in iov_iter_single_seg_count The branches of the if (i->type & ITER_BVEC) statement in iov_iter_single_seg_count() are the wrong way around; if ITER_BVEC is clear then we use i->bvec, when we should be using i->iov. This fixes it. In my case, the symptom that this caused was that a KVM guest doing filesystem operations on a virtual disk would result in one of qemu's threads on the host going into an infinite loop in generic_perform_write(). The loop would hit the copied == 0 case and call iov_iter_single_seg_count() to reduce the number of bytes to try to process, but because of the error, iov_iter_single_seg_count() would just return i->count and the loop made no progress and continued forever. Cc: stable@vger.kernel.org # 3.16+ Signed-off-by: Paul Mackerras <paulus@samba.org> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-13 20:15:23 +11:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
new helper: iov_iter_kvec() initialization of kvec-backed iov_iter Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-24 14:46:11 -05:00
new helper: iov_iter_bvec() similar to iov_iter_kvec(), for ITER_BVEC ones Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-01-23 01:08:07 -05:00
new helper: iov_iter_kvec() initialization of kvec-backed iov_iter Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-24 14:46:11 -05:00
new helper: iov_iter_bvec() similar to iov_iter_kvec(), for ITER_BVEC ones Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-01-23 01:08:07 -05:00
new helper: iov_iter_kvec() initialization of kvec-backed iov_iter Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-24 14:46:11 -05:00
new helper: iov_iter_bvec() similar to iov_iter_kvec(), for ITER_BVEC ones Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-01-23 01:08:07 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: macros for iterating over iov_iter iterate_all_kinds(iter, size, ident, step_iovec, step_bvec) iterates through the ranges covered by iter (up to size bytes total), repeating step_iovec or step_bvec for each of those. ident is declared in expansion of that thing, either as struct iovec or struct bvec, and it contains the range we are currently looking at. step_bvec should be a void expression, step_iovec - a size_t one, with non-zero meaning "stop here, that many bytes from this range left". In the end, the amount actually handled is stored in size. iov_iter_copy_from_user_atomic() and iov_iter_alignment() converted to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 13:51:41 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
fuse: honour max_read and max_write in direct_io mode The third argument of fuse_get_user_pages() "nbytesp" refers to the number of bytes a caller asked to pack into fuse request. This value may be lesser than capacity of fuse request or iov_iter. So fuse_get_user_pages() must ensure that *nbytesp won't grow. Now, when helper iov_iter_get_pages() performs all hard work of extracting pages from iov_iter, it can be done by passing properly calculated "maxsize" to the helper. The other caller of iov_iter_get_pages() (dio_refill_pages()) doesn't need this capability, so pass LONG_MAX as the maxsize argument here. Fixes: c9c37e2e6378 ("fuse: switch to iov_iter_get_pages()") Reported-by: Werner Baumann <werner.baumann@onlinehome.de> Tested-by: Maxim Patlasov <mpatlasov@parallels.com> Signed-off-by: Miklos Szeredi <mszeredi@suse.cz> Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-09-24 17:09:11 +02:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert iov_iter_get_pages() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:12:09 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert iov_iter_get_pages() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:12:09 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert iov_iter_get_pages_alloc() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:14:31 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert iov_iter_get_pages_alloc() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:14:31 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert iov_iter_get_pages_alloc() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:14:31 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
csum_and_copy_..._iter() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-24 01:08:00 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
iov_iter.c: convert iov_iter_npages() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:09:46 -05:00
iov_iter.c: handle ITER_KVEC directly ... without bothering with copy_..._user() Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:48:42 -05:00
iov_iter.c: convert iov_iter_npages() to iterate_all_kinds Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-11-27 14:09:46 -05:00
bio_vec-backed iov_iter New variant of iov_iter - ITER_BVEC in iter->type, backed with bio_vec array instead of iovec one. Primitives taught to deal with such beasts, __swap_write() switched to using that kind of iov_iter. Note that bio_vec is just a <page, offset, length> triple - there's nothing block-specific about it. I've left the definition where it was, but took it from under ifdef CONFIG_BLOCK. Next target: ->splice_write()... Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-04-04 23:12:29 -04:00
new helper: iov_iter_npages() counts the pages covered by iov_iter, up to given limit. do_block_direct_io() and fuse_iter_npages() switched to it. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2014-03-19 01:16:16 -04:00
new helper: dup_iter() Copy iter and kmemdup the underlying array for the copy. Returns a pointer to result of kmemdup() to be kfree()'d later. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-01-31 20:08:47 -05:00
saner iov_iter initialization primitives iovec-backed iov_iter instances are assumed to satisfy several properties: * no more than UIO_MAXIOV elements in iovec array * total size of all ranges is no more than MAX_RW_COUNT * all ranges pass access_ok(). The problem is, invariants of data structures should be established in the primitives creating those data structures, not in the code using those primitives. And iov_iter_init() violates that principle. For a while we managed to get away with that, but once the use of iov_iter started to spread, it didn't take long for shit to hit the fan - missed check in sys_sendto() had introduced a roothole. We _do_ have primitives for importing and validating iovecs (both native and compat ones) and those primitives are almost always followed by shoving the resulting iovec into iov_iter. Life would be considerably simpler (and safer) if we combined those primitives with initializing iov_iter. That gives us two new primitives - import_iovec() and compat_import_iovec(). Calling conventions: iovec = iov_array; err = import_iovec(direction, uvec, nr_segs, ARRAY_SIZE(iov_array), &iovec, &iter); imports user vector into kernel space (into iov_array if it fits, allocated if it doesn't fit or if iovec was NULL), validates it and sets iter up to refer to it. On success 0 is returned and allocated kernel copy (or NULL if the array had fit into caller-supplied one) is returned via iovec. On failure all allocations are undone and -E... is returned. If the total size of ranges exceeds MAX_RW_COUNT, the excess is silently truncated. compat_import_iovec() expects uvec to be a pointer to user array of compat_iovec; otherwise it's identical to import_iovec(). Finally, import_single_range() sets iov_iter backed by single-element iovec covering a user-supplied range - err = import_single_range(direction, address, size, iovec, &iter); does validation and sets iter up. Again, size in excess of MAX_RW_COUNT gets silently truncated. Next commits will be switching the things up to use of those and reducing the amount of iov_iter_init() instances. Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
2015-03-21 17:45:43 -04:00