Currently, there exists a race between delayed allocated writes and
the writeback when bigalloc feature is in use. The race was because we
wanted to determine what blocks in a cluster are under delayed
allocation and we were using buffer_delayed(bh) check for it. But, the
writeback codepath clears this bit without any synchronization which
resulted in a race and an ext4 warning similar to:
EXT4-fs (ram1): ext4_da_update_reserve_space: ino 13, used 1 with only 0
reserved data blocks
The race existed in two places.
(1) between ext4_find_delalloc_range() and ext4_map_blocks() when called from
writeback code path.
(2) between ext4_find_delalloc_range() and ext4_da_get_block_prep() (where
buffer_delayed(bh) is set.
To fix (1), this patch introduces a new buffer_head state bit -
BH_Da_Mapped. This bit is set under the protection of
EXT4_I(inode)->i_data_sem when we have actually mapped the delayed
allocated blocks during the writeout time. We can now reliably check
for this bit inside ext4_find_delalloc_range() to determine whether
the reservation for the blocks have already been claimed or not.
To fix (2), it was necessary to set buffer_delay(bh) under the
protection of i_data_sem. So, I extracted the very beginning of
ext4_map_blocks into a new function - ext4_da_map_blocks() - and
performed the required setting of bh_delay bit and the quota
reservation under the protection of i_data_sem. These two fixes makes
the checking of buffer_delay(bh) and buffer_da_mapped(bh) consistent,
thus removing the race.
Tested: I was able to reproduce the problem by running 'dd' and
'fsync' in parallel. Also, xfstests sometimes used to reproduce this
race. After the fix both my test and xfstests were successful and no
race (warning message) was observed.
Google-Bug-Id: 4997027
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch adds some tracepoints in ext4/extents.c and updates a tracepoint in
ext4/inode.c.
Tested: Built and ran the kernel and verified that these tracepoints work.
Also ran xfstests.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Rename the function so it is more clear what is going on. Also rename
the various variables so it's clearer what's happening.
Also fix a missing blocks to cluster conversion when reading the
number of reserved blocks for root.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This function really claims a number of free clusters, not blocks, so
rename it so it's clearer what's going on.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This function really returns the number of clusters after initializing
an uninitalized block bitmap has been initialized.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This function really counts the free clusters reported in the block
group descriptors, so rename it to reduce confusion.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fixes sparse warning:
zram_drv.c:666:6: warning: symbol 'zram_slot_free_notify' was not
declared. Should it be static?
Also, max_zpage_size is now size_t just to be consistent with data-type
of other variables maintaining sizes of various kinds.
Signed-off-by: Nitin Gupta <ngupta@vflare.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The field bg_free_blocks_count_{lo,high} in the block group
descriptor has been repurposed to hold the number of free clusters for
bigalloc functions. So rename the functions so it makes it easier to
read and audit the block allocation and block freeing code.
Note: at this point in bigalloc development we doesn't support
online resize, so this also makes it really obvious all of the places
we need to fix up to add support for online resize.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
In tomoyo_get_mode() since 2.6.36, CONFIG::file::execute was by error used in
place of CONFIG::file if CONFIG::file::execute was set to other than default.
As a result, enforcing mode was not applied in a way documentation says.
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: James Morris <jmorris@namei.org>
With bigalloc changes, the i_blocks value was not correctly set (it was still
set to number of blocks being used, but in case of bigalloc, we want i_blocks
to represent the number of clusters being used). Since the quota subsystem sets
the i_blocks value, this patch fixes the quota accounting and makes sure that
the i_blocks value is set correctly.
Signed-off-by: Aditya Kali <adityakali@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The default group preallocation size had been previously set to 512
blocks/clusters, regardless of the block/cluster size. This is
probably too big for large cluster sizes. So adjust the default so
that it is 2 megabytes or 32 clusters, whichever is larger.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
remove the following two paragraphs as they are not needed:
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
License for more details.
You should have received a copy of the GNU General Public License along with
this program; if not, write to the Free Software Foundation, Inc.,59
Temple Place - Suite 330, Boston, MA 02111-1307, USA.
Signed-off-by: Klaus Schwarzkopf <schwarzkopf@sensortherm.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Convert the free_blocks to be free_clusters to make the final revised
bigalloc changes easier to read/understand.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
There is a multi-year old bug in the MUSB hardware which is not documented.
It causes spurious interrupts and have various symptoms, like endless
"SetupEnd came in a wrong ep0stage" messages. The fix is taken from the
FreeBSD's musb driver.
How to reproduce:
For example issue clear-stall on a couple of endpoints very fast,
like one request per 125us. After a while the bug triggers and the
musb-chip becomes unusable until next re-enumeration.
Signed-off-by: Hans Petter Selasky <hps@bitfrost.no>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
A return value of -EINPROGRESS from pm_runtime_get indicates that
the device is already resuming due to a previous call. Internally,
usb_autopm_get_interface_async doesn't treat this as an error and
increments the usage count, but passes the error status along
to the caller. The logical assumption of the caller is that
any negative return value reflects the device not resuming
and the pm_usage_cnt not being incremented. Since the usage count
is being incremented and the device is resuming, return success (0)
instead.
Signed-off-by: James Wylder <james.wylder@motorola.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Convert the percpu counters s_dirtyblocks_counter and
s_freeblocks_counter in struct ext4_super_info to be
s_dirtyclusters_counter and s_freeclusters_counter.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch solves two things:
1) Enables autosense emulation code to correctly
interpret descriptor format sense data, and
2) Fixes a bug whereby the autosense emulation
code would overwrite descriptor format sense data
with SENSE KEY HARDWARE ERROR in fixed format, to
incorrectly look like this:
Oct 21 14:11:07 localhost kernel: sd 7:0:0:0: [sdc] Sense Key : Recovered Error [current] [descriptor]
Oct 21 14:11:07 localhost kernel: Descriptor sense data with sense descriptors (in hex):
Oct 21 14:11:07 localhost kernel: 72 01 04 1d 00 00 00 0e 09 0c 00 00 00 00 00 00
Oct 21 14:11:07 localhost kernel: 00 4f 00 c2 00 50
Oct 21 14:11:07 localhost kernel: sd 7:0:0:0: [sdc] ASC=0x4 ASCQ=0x1d
Signed-off-by: Luben Tuikov <ltuikov@yahoo.com>
Acked-by: Alan Stern <stern@rowland.harvard.edu>
Acked-by: Matthew Dharm <mdharm-usb@one-eyed-alien.net>
Cc: stable <stable@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
When we are truncating (as opposed unlinking) a file, we need to worry
about partial truncates of a file, especially in the light of sparse
files. The changes here make sure that arbitrary truncates of sparse
files works correctly. Yeah, it's messy.
Note that these functions will need to be revisted when the punch
ioctl is integrated --- in fact this commit will probably have merge
conflicts with the punch changes which Allison Henders and the IBM LTC
have been working on. I will need to fix this up when either patch
hits mainline.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The xhci_hcd->devs is an array of pointers rather than pointer to pointer.
Hence this check is not required.
Signed-off-by: Sifram Rajas <Sifram Rajas sifram.rajas@gmail.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In xhci_urb_enqueue(), allocate a block of memory for all the TDs instead
of allocating memory for each of them separately. This reduces the number
of kzalloc calling when an isochronous usb is submitted.
Signed-off-by: Andiry Xu <andiry.xu@amd.com>
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Now that the xHCI driver always return a status value of zero for isochronous
URBs, when the last TD of an isochronous URB is short, the local variable
"status" stays set to -EINPROGRESS. When xHCI driver debugging is turned on,
this causes the log file to fill with messages like this:
[ 38.859282] xhci_hcd 0000:00:14.0: Giveback URB ffff88013ad47800, len = 1408, expected = 580, status = -115
Don't print out the status of an URB for isochronous URBs.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The xHCI host controller in the Intel Panther Point chipset needs to have
software check whether new devices will fit in the available bus
bandwidth. Activate the software bandwidth checking quirk when we find
the right PCI device.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Now that we have a bandwidth interval table per root port or TT that
describes the endpoint bandwidth information, we can finally use it to
check whether the bus bandwidth is oversubscribed for a new device
configuration/alternate interface setting.
The complication for this algorithm is that the bit of hardware logic that
creates the bus schedule is only 12-bit logic. In order to make sure it
can represent the maximum bus bandwidth in 12 bits, it has to convert the
endpoint max packet size and max esit payload into "blocks" (basically a
less-precise representation). The block size for each speed of device is
different, aside from low speed and full speed. In order to make sure we
don't allow a setup where the scheduler might fail, we also have to do the
bandwidth checking in blocks.
After checking that the endpoints fit in the schedule, we store the
bandwidth used for this root port or TT. If this is a FS/LS device under
an external HS hub, we also update the TT bandwidth and the root port
bandwidth (if this is a newly activated or deactivated TT).
I won't go into the details of the algorithm, as it's pretty well
documented in the comments.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In order to update the root port or TT's bandwidth interval table, we will
need to keep track of a list of endpoints, per interval. That way we can
easily know the new largest max packet size when we have to remove an
endpoint.
Add an endpoint list for each root port or TT structure, sorted by
endpoint max packet size. Insert new endpoints into the list such that
the head of the list always has the endpoint with the greatest max packet
size. Only insert endpoints and update the interval table with new
information when those endpoints are periodic.
Make sure to update the number of active TTs when we add or drop periodic
endpoints. A TT is only considered active if it has one or more periodic
endpoints attached (control and bulk are best effort, and counted in the
20% reserved on the high speed bus). If the number of active endpoints
for a TT was zero, and it's now non-zero, increment the number of active
TTs for the rootport. If the number of active endpoints was non-zero, and
it's now zero, decrement the number of active TTs.
We have to be careful when we're checking the bandwidth for a new
configuration/alt setting. If we don't have enough bandwidth, we need to
be able to "roll back" the bandwidth information stored in the endpoint
and the root port/TT interval bandwidth table. We can't just create a
copy of the interval bandwidth table, modify it, and check the bandwidth
with the copy because we have lists of endpoints and entries can't be on
more than one list. Instead, we copy the old endpoint bandwidth
information, and use it to revert the interval table when the bandwidth
check fails.
We don't check the bandwidth after endpoints are dropped from the interval
table when a device is reset or freed after a disconnect, because having
endpoints use less bandwidth should not push the bandwidth usage over the
limits. Besides which, we can't fail a device disconnect.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
In the upcoming patches, we'll use some stored endpoint information to
make software keep track of the worst-case bandwidth schedule. We need to
store several variables associated with each periodic endpoint:
- the type of endpoint
- Max Packet Size
- Mult
- Max ESIT payload
- Max Burst Size (aka number of packets, stored in one-based form)
- the endpoint interval (normalized to powers of 2 microframes)
All this information is available to the hardware, and stored in its
device output context. However, we need to ensure that the new
information is stored before the xHCI driver drops the xhci->lock to wait
on the Configure Endpoint command, so that another driver requesting a
configuration or alt setting change will see the update. The Configure
Endpoint command will never fail on the hardware that needs this software
bandwidth checking (assuming the slot is enabled and the flags are set
properly), so updating the endpoint info before the command completes
should be fine.
Until we add in the bandwidth checking code, just update the endpoint
information after the Configure Endpoint command completes, and after a
Reset Device command completes. Don't bother to clear the endpoint
bandwidth info when a device is being freed, since the xhci_virt_ep is
just going to be freed anyway.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
For upcoming patches, we need to keep information about the bandwidth
domains under the xHCI host. Each root port is a separate primary
bandwidth domain, and each high speed hub's TT (and potentially each port
on a multi-TT hub) is a secondary bandwidth domain.
If the table were in text form, it would look a bit like this:
EP Interval Sum of Number Largest Max Max Packet
of Packets Packet Size Overhead
0 N mps overhead
...
15 N mps overhead
Overhead is the maximum packet overhead (for bit stuffing, CRC, protocol
overhead, etc) for all the endpoints in this interval. Devices with
different speeds have different max packet overhead. For example, if
there is a low speed and a full speed endpoint that both have an interval
of 3, we would use the higher overhead (the low speed overhead). Interval
0 is a bit special, since we really just want to know the sum of the max
ESIT payloads instead of the largest max packet size. That's stored in
the interval0_esit_payload variable. For root ports, we also need to keep
track of the number of active TTs.
For each root port, and each TT under a root port, store some information
about the bandwidth consumption. Dynamically allocate an array of root
port bandwidth information for the number of root ports on the xHCI host.
Each root port stores a list of TTs under the root port. A single TT hub
only has one entry in the list, but a multi-TT hub will have an entry per
port.
When the USB core says that a USB device is a hub, create one or more
entries in the root port TT list for the hub. When a device is deleted,
and it is a hub, search through the root port TT list and delete all
TT entries for the hub. Keep track of which TT entry is associated with a
device under a TT.
LS/FS devices attached directly to the root port will have usb_device->tt
set to the roothub. Ignore that, and treat it like a primary bandwidth
domain, since there isn't really a high speed bus between the roothub and
the host.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Since the xHCI driver now has split USB2/USB3 roothubs, devices under each
roothub can have duplicate "fake" port numbers. For the next set of
patches, we need to keep track of the "real" port number that the xHCI
host uses to index into the port status arrays.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Move the code to check whether we've reached the host controller's limit
on the number of endpoints out of the two conditional statements, to
remove duplicate code.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The "port" field in xhci_virt_dev stores the port number associated with
one of the two xHCI split roothubs, not the unique port number the xHCI
hardware uses. Since we'll need to store the real hardware port number in
future patches, rename this field to "fake_port".
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
Some alternate interface settings have no endpoints associated with them.
This shows up in some USB webcams, particularly the Logitech HD 1080p,
which uses the uvcvideo driver. If a driver switches between two alt
settings with no endpoints, there is no need to issue a configure endpoint
command, because there is no endpoint information to update.
The only time a configure endpoint command with just the add slot flag set
makes sense is when the driver is updating hub characteristics in the slot
context. However, that code never calls xhci_check_bandwidth, so we
should be safe not issuing a command if only the slot context add flag is
set.
Signed-off-by: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
If we need to allocate a new block in ext4_ext_map_blocks(), the
function needs to see if the cluster has already been allocated.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext4_free_blocks() function now has two new flags that indicate
whether a partial cluster at the beginning or the end of the block
extents should be freed or not. That will be up the caller (i.e.,
truncate), who can figure out whether partial clusters at the
beginning or the end of a block range can be freed.
We also have to update the ext4_mb_free_metadata() and
release_blocks_on_commit() machinery to be cluster-based, since it is
used by ext4_free_blocks().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
* 'for-linus' of git://dev.laptop.org/users/cjb/mmc:
mmc: sdhci-s3c: Fix mmc card I/O problem
mmc: sd: UHS-I bus speed should be set last in UHS initialization
mmc: sdhi: initialise mmc_data->flags before use
mmc: core: use non-reentrant workqueue for clock gating
mmc: core: prevent aggressive clock gating racing with ios updates
mmc: rename mmc_host_clk_{ungate|gate} to mmc_host_clk_{hold|release}
mmc: sdhci-esdhc-imx: add missing inclusion of linux/module.h
In most of mballoc.c, we do everything in units of clusters, since the
block allocation bitmaps and buddy bitmaps are all denominated in
clusters. The one place where we do deal with absolute block numbers
is in the code that handles the preallocation regions, since in the
case of inode-based preallocation regions, the start of the
preallocation region can't be relative to the beginning of the group.
So this adds a bit of complexity, where pa_pstart and pa_lstart are
block numbers, while pa_free, pa_len, and fe_len are denominated in
units of clusters.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Certain parts of the ext4 code base, primarily in mballoc.c, use a
block group number and offset from the beginning of the block group.
This offset is invariably used to index into the allocation bitmap, so
change the offset to be denominated in units of clusters.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The function ext4_free_blocks_after_init() used to be a #define of
ext4_init_block_bitmap(). This actually made it difficult to
understand how the function worked, and made it hard make changes to
support clusters. So as an initial cleanup, I've separated out the
functionality of initializing block bitmap from calculating the number
of free blocks in the new block group.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Prior to 2.6.38 automount would not trigger on either stat(2) or
lstat(2) on the automount point.
After 2.6.38, with the introduction of the ->d_automount()
infrastructure, stat(2) and others would start triggering automount
while lstat(2), etc. still would not. This is a regression and a
userspace ABI change.
Problem originally reported here:
http://thread.gmane.org/gmane.linux.kernel.autofs/6098
It appears that there was an attempt at fixing various userspace tools
to not trigger the automount. But since the stat system call is
rather common it is impossible to "fix" all userspace.
This patch reverts the original behavior, which is to not trigger on
stat(2) and other symlink following syscalls.
[ It's not really clear what the right behavior is. Apparently Solaris
does the "automount on stat, leave alone on lstat". And some programs
can get unhappy when "stat+open+fstat" ends up giving a different
result from the fstat than from the initial stat.
But the change in 2.6.38 resulted in problems for some people, so
we're going back to old behavior. Maybe we can re-visit this
discussion at some future date - Linus ]
Reported-by: Leonardo Chiquitto <leonardo.lists@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@suse.cz>
Acked-by: Ian Kent <raven@themaw.net>
Cc: David Howells <dhowells@redhat.com>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This makes it easier to understand how ext4_init_block_bitmap() works,
and it will assist when we split out ext4_free_blocks_after_init() in
the next commit.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Change the places in fs/ext4/mballoc.c where EXT4_BLOCKS_PER_GROUP are
used to indicate the number of bits in a block bitmap (which is really
a cluster allocation bitmap in bigalloc file systems). There are
still some places in the ext4 codebase where usage of
EXT4_BLOCKS_PER_GROUP needs to be audited/fixed, in code paths that
aren't used given the initial restricted assumptions for bigalloc.
These will need to be fixed before we can relax those restrictions.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
At least initially if the bigalloc feature is enabled, we will not
support non-extent mapped inodes, online resizing, online defrag, or
the FITRIM ioctl. This simplifies the initial implementation.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This adds supports for bigalloc file systems. It teaches the mount
code just enough about bigalloc superblock fields that it will mount
the file system without freaking out that the number of blocks per
group is too big.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The del_gendisk() function uninitializes the disk-specific data
structures, including the bdi structure, without telling anyone
else. Once this happens, any attempt to call mark_buffer_dirty()
(for example, by ext4_commit_super), will cause a kernel OOPS.
Fix this for now until we can fix things in an architecturally correct
way.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
V3 is a straightforward forward port to teh current tree of V2.
Simple fix is to just not cache REG_TEST (offset 8).
Cache doesn't help REG_TEST anyway since we write all 8 bits exactly once
(at resume/init time).
Also fix an "off-by-one" allocation of reg_cache[] array size that
was in the original code before I touched it.
Reported-by: Dan Carpenter <error27@gmail.com>
Signed-off-by: Grant Grundler <grundler@chromium.org>
Signed-off-by: Jonathan Cameron <jic23@cam.ac.uk>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
The vmbus devices are NOT child devices; rename vmbus_child_device_unregister
to reflect this.
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>
