2020-06-05 22:05:19 +02:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-02-01 15:02:46 +02:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-01-29 18:52:50 +08:00
|
|
|
|
2021-10-18 11:04:51 -07:00
|
|
|
|
2022-04-19 15:58:08 -07:00
|
|
|
|
|
|
|
|
|
2023-11-18 21:44:43 +09:00
|
|
|
|
2024-08-13 15:12:35 +03:00
|
|
|
|
2022-12-14 19:06:21 -08:00
|
|
|
|
2022-04-19 15:58:08 -07:00
|
|
|
|
2023-01-08 23:40:56 +01:00
|
|
|
|
2022-04-19 15:58:08 -07:00
|
|
|
|
2023-07-27 12:41:17 -07:00
|
|
|
|
2022-05-11 14:25:52 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-09-21 12:22:47 -07:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:58 +05:30
|
|
|
|
2022-12-01 15:04:37 +01:00
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
|
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-04-23 17:20:16 -07:00
|
|
|
|
|
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-11-02 22:24:40 -08:00
|
|
|
|
2013-06-26 22:39:29 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2024-07-08 14:15:57 -07:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2025-10-14 13:00:56 -07:00
|
|
|
|
2013-06-26 22:39:29 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scsi: ufs: Fix concurrency of error handler and other error recovery paths
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6fc ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-09 05:15:51 -07:00
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scsi: ufs: Fix concurrency of error handler and other error recovery paths
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6fc ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-09 05:15:51 -07:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scsi: ufs: Fix concurrency of error handler and other error recovery paths
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6fc ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-09 05:15:51 -07:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
2020-03-27 17:58:35 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-11-03 16:14:02 +02:00
|
|
|
|
|
|
|
|
|
2020-03-27 17:58:35 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-11-03 16:14:02 +02:00
|
|
|
|
|
|
|
|
|
2020-03-27 17:58:35 +08:00
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
2020-11-03 16:14:02 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
2021-05-19 13:20:58 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2025-08-19 08:43:49 -07:00
|
|
|
|
2022-08-04 06:50:34 +10:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-07-06 20:04:14 +00:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:29 +03:00
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2025-08-19 08:43:49 -07:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2022-08-04 06:50:34 +10:00
|
|
|
|
2017-09-27 10:06:06 +08:00
|
|
|
|
2022-08-04 06:50:34 +10:00
|
|
|
|
2020-07-06 20:04:14 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2023-07-27 12:41:22 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:58 +05:30
|
|
|
|
2016-02-09 10:25:41 +02:00
|
|
|
|
2013-07-30 00:35:58 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:58 +05:30
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2014-09-25 15:32:23 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:33 +03:00
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
2020-11-25 18:01:00 -08:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2014-09-25 15:32:23 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:33 +03:00
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
2020-11-25 18:01:00 -08:00
|
|
|
|
2014-09-25 15:32:23 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-28 13:15:51 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:31 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-05-30 17:25:08 +03:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
2024-10-18 12:47:39 -07:00
|
|
|
|
|
|
|
|
|
2015-05-17 18:55:05 +03:00
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:31 +03:00
|
|
|
|
|
|
|
|
|
2024-11-22 10:49:43 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-11-10 21:14:36 +09:00
|
|
|
|
|
|
|
|
|
2016-11-10 21:16:15 +09:00
|
|
|
|
|
|
|
|
|
2016-11-10 21:17:43 +09:00
|
|
|
|
2016-12-05 19:25:32 -08:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
2015-10-28 13:15:50 +02:00
|
|
|
|
2016-05-11 12:21:30 +01:00
|
|
|
|
2019-08-28 12:17:54 -07:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2024-07-08 16:53:28 -07:00
|
|
|
|
2020-12-05 19:59:00 +08:00
|
|
|
|
2023-01-13 12:48:42 -08:00
|
|
|
|
2024-07-08 14:16:05 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2023-01-13 12:48:50 -08:00
|
|
|
|
2022-12-14 19:06:20 -08:00
|
|
|
|
2024-03-01 11:46:10 +08:00
|
|
|
|
2025-02-13 16:00:03 +08:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-05-30 17:25:08 +03:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
2015-05-17 18:55:05 +03:00
|
|
|
|
2024-10-18 12:47:39 -07:00
|
|
|
|
2025-02-13 16:00:01 +08:00
|
|
|
|
|
|
|
|
|
2016-10-06 21:48:22 -07:00
|
|
|
|
|
|
|
|
|
2015-10-28 13:15:51 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:31 +03:00
|
|
|
|
2025-02-12 13:38:02 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-12-03 15:19:35 -08:00
|
|
|
|
|
|
|
|
|
2016-11-10 21:16:15 +09:00
|
|
|
|
2016-11-10 21:17:43 +09:00
|
|
|
|
2016-12-05 19:25:32 -08:00
|
|
|
|
2020-01-20 14:08:14 +01:00
|
|
|
|
2020-05-08 16:01:09 +08:00
|
|
|
|
2021-10-06 13:47:05 +08:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
2015-10-28 13:15:50 +02:00
|
|
|
|
2016-05-11 12:21:30 +01:00
|
|
|
|
2020-11-03 16:14:03 +02:00
|
|
|
|
2020-03-25 11:29:01 -07:00
|
|
|
|
2022-04-19 15:57:57 -07:00
|
|
|
|
|
|
|
|
|
2024-07-08 16:53:28 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-12-05 19:59:00 +08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:42 -08:00
|
|
|
|
2023-01-13 12:48:43 -08:00
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2023-01-13 12:48:50 -08:00
|
|
|
|
|
|
|
|
|
2022-12-14 19:06:20 -08:00
|
|
|
|
2024-10-08 14:59:42 +08:00
|
|
|
|
2025-02-13 16:00:03 +08:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-11-24 09:08:07 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-22 18:40:39 -08:00
|
|
|
|
|
|
|
|
|
2021-01-20 02:04:22 -08:00
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-11-24 09:08:07 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-22 18:40:39 -08:00
|
|
|
|
|
|
|
|
|
2021-01-20 02:04:22 -08:00
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
|
|
|
|
|
2024-11-24 09:08:08 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-10-12 22:51:26 +05:30
|
|
|
|
2020-11-26 17:58:48 -08:00
|
|
|
|
2025-02-13 16:00:07 +08:00
|
|
|
|
|
|
|
|
|
2021-01-20 02:04:21 -08:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2021-01-20 02:04:21 -08:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2021-01-20 02:04:22 -08:00
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
2024-11-24 09:08:08 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
|
|
|
|
|
2020-06-11 18:10:43 +08:00
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
2016-12-22 18:40:50 -08:00
|
|
|
|
2023-03-30 09:29:18 +08:00
|
|
|
|
2023-10-12 22:51:26 +05:30
|
|
|
|
2020-11-26 17:58:48 -08:00
|
|
|
|
2025-02-13 16:00:07 +08:00
|
|
|
|
2021-01-20 02:04:21 -08:00
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
2021-01-20 02:04:22 -08:00
|
|
|
|
2017-02-03 16:57:39 -08:00
|
|
|
|
|
|
|
|
|
2024-06-27 14:07:55 +05:30
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
|
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2021-01-07 09:25:38 +02:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2022-08-04 06:50:34 +10:00
|
|
|
|
2021-01-07 09:25:38 +02:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-08-04 06:50:34 +10:00
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
|
|
|
|
|
2021-07-21 20:34:29 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-02-21 06:08:12 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-28 06:46:49 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-28 06:46:50 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-28 06:46:51 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-28 06:46:52 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-28 06:46:53 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-08-25 10:43:15 +09:00
|
|
|
|
2020-08-10 17:10:24 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-09-15 11:24:32 -04:00
|
|
|
|
2020-08-25 10:43:15 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-09-15 11:24:32 -04:00
|
|
|
|
|
|
|
|
|
2020-12-21 10:24:40 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-18 21:42:02 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-10-18 21:42:03 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-06-03 20:05:19 +09:00
|
|
|
|
2022-06-03 20:05:20 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-12-22 19:39:57 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-12 16:58:09 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-06-12 16:58:10 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-08 16:53:25 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-08 16:53:27 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-07-08 16:53:29 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-08-16 11:55:10 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-10-24 11:59:16 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-02-21 06:08:12 -08:00
|
|
|
|
|
|
|
|
|
2020-03-18 18:40:11 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-04-22 14:41:42 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-07-06 20:04:12 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-10-27 12:10:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-11-03 16:14:02 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-09-15 09:04:06 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-08-04 10:54:22 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-03-18 18:40:11 +08:00
|
|
|
|
|
|
|
|
|
2020-05-09 17:37:13 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-09 17:37:15 +08:00
|
|
|
|
2020-05-09 17:37:13 +08:00
|
|
|
|
|
|
|
|
|
2021-04-21 19:28:39 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:57:59 -07:00
|
|
|
|
2025-11-08 00:05:18 +01:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-13 13:30:45 +05:30
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2021-07-21 20:34:34 -07:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:38 -08:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
scsi: ufs: core: Allow RTT negotiation
The rtt-upiu packets precede any data-out upiu packets, thus synchronizing
the data input to the device: this mostly applies to write operations, but
there are other operations that requires rtt as well.
There are several rules binding this rtt - data-out dialog, specifically
There can be at most outstanding bMaxNumOfRTT such packets. This might
have an effect on write performance (sequential write in particular), as
each data-out upiu must wait for its rtt sibling.
UFSHCI expects bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT). However, as of
today, there does not appears to be no-one who sets it: not the host
controller nor the driver. It wasn't an issue up to now: bMaxNumOfRTT is
set to 2 after manufacturing, and wasn't limiting the write performance.
UFS4.0, and specifically gear 5 changes this, and requires the device to be
more attentive. This doesn't come free - the device has to allocate more
resources to that end, but the sequential write performance improvement is
significant. Early measurements shows 25% gain when moving from rtt 2 to
9. Therefore, set bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT) as UFSHCI
expects.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240530142510.734-2-avri.altman@wdc.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2024-05-30 17:25:07 +03:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
2022-12-08 15:43:58 -08:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-12-09 10:13:09 -08:00
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2024-09-12 15:30:02 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-07-21 20:34:29 -07:00
|
|
|
|
2014-05-26 10:59:14 +05:30
|
|
|
|
2013-06-26 22:39:27 +05:30
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:26 +03:00
|
|
|
|
2021-01-13 19:13:28 -08:00
|
|
|
|
|
|
|
|
|
scsi: ufs: core: Revert "scsi: ufs: Synchronize SCSI and UFS error handling"
This reverts commit a113eaaf86373362b053279049907ff82b5df6c8.
There are a couple of issues with the commit:
1. It causes deadlocks.
2. It causes the shost->eh_cmd_q list of failed requests not to be
processed, ever.
So revert it.
1. Deadlocks
The SCSI error handler runs with requests blocked beginning when
scsi_schedule_eh() sets SHOST_RECOVERY state, continuing through
scsi_error_handler() callback ->eh_strategy_handler() until
scsi_restart_operations() is called. By setting eh_strategy_handler to
ufshcd_err_handler, the patch changed the UFS error handler to run with
requests blocked, including PM requests, for the entire run of the error
handler.
That conflicts with UFS error handler existing synchronization with UFS
device PM operations. The UFS error handler synchronizes with runtime PM
by doing pm_runtime_get_sync() prior to blocking requests itself. It
synchronizes with system PM by use of hba->host_sem, again before blocking
requests itself. However, if requests are already blocked, then PM
operations will block. So:
the UFS error handler blocks waiting on PM
+ PM blocks waiting on SCSI PM requests to process or fail
+ PM requests are blocked waiting on error handling to finish
= deadlock
This happens both for runtime PM and system PM.
Prior to the patch, these deadlocks could not happen even if SCSI error
handling was running, because the presence of requests in shost->eh_cmd_q
would mean the queues could not be suspended, which would mean that, should
the UFS error handler run at the same time, it would not need to wait for
PM or vice versa.
Please note these scenarios are not just theoretical, they were found
during testing on a Samsung Galaxy Book S.
2. ->eh_strategy_handler() must process shost->eh_cmd_q list of failed
requests, as all other eh_strategy_handler's do except UFS error handler.
Refer for example: scsi_unjam_host(), ata_scsi_error() and
sas_scsi_recover_host().
Link: https://lore.kernel.org/r/20210917144349.14058-1-adrian.hunter@intel.com
Fixes: a113eaaf8637 ("scsi: ufs: Synchronize SCSI and UFS error handling")
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-17 17:43:49 +03:00
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2014-05-26 10:59:15 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
scsi: ufs: Fix concurrency of error handler and other error recovery paths
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6fc ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-09 05:15:51 -07:00
|
|
|
|
2020-08-24 19:07:06 -07:00
|
|
|
|
scsi: ufs: Complete pending requests in host reset and restore path
In UFS host reset and restore path, before probe, we stop and start the
host controller once. After host controller is stopped, the pending
requests, if any, are cleared from the doorbell, but no completion IRQ
would be raised due to the hba is stopped. These pending requests shall be
completed along with the first NOP_OUT command (as it is the first command
which can raise a transfer completion IRQ) sent during probe. Since the
OCSs of these pending requests are not SUCCESS (because they are not yet
literally finished), their UPIUs shall be dumped. When there are multiple
pending requests, the UPIU dump can be overwhelming and may lead to
stability issues because it is in atomic context. Therefore, before probe,
complete these pending requests right after host controller is stopped and
silence the UPIU dump from them.
Link: https://lore.kernel.org/r/1574751214-8321-5-git-send-email-cang@qti.qualcomm.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Tested-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-11-25 22:53:33 -08:00
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
2015-03-31 17:37:14 +03:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2014-09-25 15:32:22 +03:00
|
|
|
|
2014-09-25 15:32:23 +03:00
|
|
|
|
2023-10-12 22:51:26 +05:30
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:31 +03:00
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-10-18 13:29:56 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-03-10 17:37:15 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-16 17:12:01 +01:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-07-06 20:04:13 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
blk-crypto: rename blk_keyslot_manager to blk_crypto_profile
blk_keyslot_manager is misnamed because it doesn't necessarily manage
keyslots. It actually does several different things:
- Contains the crypto capabilities of the device.
- Provides functions to control the inline encryption hardware.
Originally these were just for programming/evicting keyslots;
however, new functionality (hardware-wrapped keys) will require new
functions here which are unrelated to keyslots. Moreover,
device-mapper devices already (ab)use "keyslot_evict" to pass key
eviction requests to their underlying devices even though
device-mapper devices don't have any keyslots themselves (so it
really should be "evict_key", not "keyslot_evict").
- Sometimes (but not always!) it manages keyslots. Originally it
always did, but device-mapper devices don't have keyslots
themselves, so they use a "passthrough keyslot manager" which
doesn't actually manage keyslots. This hack works, but the
terminology is unnatural. Also, some hardware doesn't have keyslots
and thus also uses a "passthrough keyslot manager" (support for such
hardware is yet to be upstreamed, but it will happen eventually).
Let's stop having keyslot managers which don't actually manage keyslots.
Instead, rename blk_keyslot_manager to blk_crypto_profile.
This is a fairly big change, since for consistency it also has to update
keyslot manager-related function names, variable names, and comments --
not just the actual struct name. However it's still a fairly
straightforward change, as it doesn't change any actual functionality.
Acked-by: Ulf Hansson <ulf.hansson@linaro.org> # For MMC
Reviewed-by: Mike Snitzer <snitzer@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Link: https://lore.kernel.org/r/20211018180453.40441-4-ebiggers@kernel.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-10-18 11:04:52 -07:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:41 -08:00
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:39 -08:00
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2025-04-07 12:17:04 +02:00
|
|
|
|
2023-01-13 12:48:42 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
2023-12-12 23:08:24 +01:00
|
|
|
|
|
|
|
|
|
2023-12-19 18:07:05 +05:30
|
|
|
|
|
|
|
|
|
2025-09-17 17:41:43 +08:00
|
|
|
|
scsi: ufs: core: Critical health condition
Martin hi,
The UFS4.1 standard, released on January 8, 2025, added a new exception
event: HEALTH_CRITICAL, which notifies the host of a device's critical
health condition. This notification implies that the device is approaching
the end of its lifetime based on the amount of performed program/erase
cycles.
Once an EOL (End-of-Life) exception event is received, we increment a
designated member, which is exposed via a sysfs entry. This new entry, will
report the number of times a critical health event has been reported by a
UFS device.
To handle this new sysfs entry, userspace applications can use select(),
poll(), or epoll() to monitor changes in the critical_health attribute. The
kernel will call sysfs_notify() to signal changes, allowing the userspace
application to detect and respond to these changes efficiently.
The host can gain further insight into the specific issue by reading one of
the following attributes: bPreEOLInfo, bDeviceLifeTimeEstA,
bDeviceLifeTimeEstB, bWriteBoosterBufferLifeTimeEst, and
bRPMBLifeTimeEst. All those are available for reading via the driver's
sysfs entries or through an applicable utility. It is up to userspace to
read these attributes if needed.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20250211065813.58091-1-avri.altman@wdc.com
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2025-02-11 08:58:13 +02:00
|
|
|
|
2025-03-28 14:46:13 -07:00
|
|
|
|
2025-11-08 00:05:18 +01:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:57:59 -07:00
|
|
|
|
2025-11-08 00:05:18 +01:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2021-09-15 09:04:06 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2026-01-13 13:30:45 +05:30
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
2018-03-20 15:07:38 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2021-07-21 20:34:34 -07:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
scsi: ufs: core: Allow RTT negotiation
The rtt-upiu packets precede any data-out upiu packets, thus synchronizing
the data input to the device: this mostly applies to write operations, but
there are other operations that requires rtt as well.
There are several rules binding this rtt - data-out dialog, specifically
There can be at most outstanding bMaxNumOfRTT such packets. This might
have an effect on write performance (sequential write in particular), as
each data-out upiu must wait for its rtt sibling.
UFSHCI expects bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT). However, as of
today, there does not appears to be no-one who sets it: not the host
controller nor the driver. It wasn't an issue up to now: bMaxNumOfRTT is
set to 2 after manufacturing, and wasn't limiting the write performance.
UFS4.0, and specifically gear 5 changes this, and requires the device to be
more attentive. This doesn't come free - the device has to allocate more
resources to that end, but the sequential write performance improvement is
significant. Early measurements shows 25% gain when moving from rtt 2 to
9. Therefore, set bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT) as UFSHCI
expects.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20240530142510.734-2-avri.altman@wdc.com
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2024-05-30 17:25:07 +03:00
|
|
|
|
2023-01-13 12:48:38 -08:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2019-03-04 20:39:11 +01:00
|
|
|
|
2020-05-09 17:37:13 +08:00
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
2022-12-08 15:43:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
2018-10-16 14:29:41 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2015-03-31 17:37:14 +03:00
|
|
|
|
2013-06-26 22:39:29 +05:30
|
|
|
|
2016-03-10 17:37:10 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-12-09 10:13:09 -08:00
|
|
|
|
|
|
|
|
|
2021-09-22 12:10:59 +03:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-08-31 21:40:22 +05:30
|
|
|
|
2021-07-21 20:34:29 -07:00
|
|
|
|
2014-05-26 10:59:14 +05:30
|
|
|
|
2013-06-26 22:39:27 +05:30
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-02-09 08:24:36 +02:00
|
|
|
|
2014-09-25 15:32:26 +03:00
|
|
|
|
2021-01-13 19:13:28 -08:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
scsi: ufs: core: Revert "scsi: ufs: Synchronize SCSI and UFS error handling"
This reverts commit a113eaaf86373362b053279049907ff82b5df6c8.
There are a couple of issues with the commit:
1. It causes deadlocks.
2. It causes the shost->eh_cmd_q list of failed requests not to be
processed, ever.
So revert it.
1. Deadlocks
The SCSI error handler runs with requests blocked beginning when
scsi_schedule_eh() sets SHOST_RECOVERY state, continuing through
scsi_error_handler() callback ->eh_strategy_handler() until
scsi_restart_operations() is called. By setting eh_strategy_handler to
ufshcd_err_handler, the patch changed the UFS error handler to run with
requests blocked, including PM requests, for the entire run of the error
handler.
That conflicts with UFS error handler existing synchronization with UFS
device PM operations. The UFS error handler synchronizes with runtime PM
by doing pm_runtime_get_sync() prior to blocking requests itself. It
synchronizes with system PM by use of hba->host_sem, again before blocking
requests itself. However, if requests are already blocked, then PM
operations will block. So:
the UFS error handler blocks waiting on PM
+ PM blocks waiting on SCSI PM requests to process or fail
+ PM requests are blocked waiting on error handling to finish
= deadlock
This happens both for runtime PM and system PM.
Prior to the patch, these deadlocks could not happen even if SCSI error
handling was running, because the presence of requests in shost->eh_cmd_q
would mean the queues could not be suspended, which would mean that, should
the UFS error handler run at the same time, it would not need to wait for
PM or vice versa.
Please note these scenarios are not just theoretical, they were found
during testing on a Samsung Galaxy Book S.
2. ->eh_strategy_handler() must process shost->eh_cmd_q list of failed
requests, as all other eh_strategy_handler's do except UFS error handler.
Refer for example: scsi_unjam_host(), ata_scsi_error() and
sas_scsi_recover_host().
Link: https://lore.kernel.org/r/20210917144349.14058-1-adrian.hunter@intel.com
Fixes: a113eaaf8637 ("scsi: ufs: Synchronize SCSI and UFS error handling")
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-09-17 17:43:49 +03:00
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-05-26 10:59:15 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-22 18:42:18 -08:00
|
|
|
|
scsi: ufs: Fix concurrency of error handler and other error recovery paths
Error recovery can be invoked from multiple code paths, including hibern8
enter/exit (from ufshcd_link_recovery), ufshcd_eh_host_reset_handler() and
eh_work scheduled from IRQ context. Ultimately, these paths are all trying
to invoke ufshcd_reset_and_restore() in either a synchronous or
asynchronous manner. This causes problems:
- If link recovery happens during ungate work, ufshcd_hold() would be
called recursively. Although commit 53c12d0ef6fc ("scsi: ufs: fix error
recovery after the hibern8 exit failure") fixed a deadlock due to
recursive calls of ufshcd_hold() by adding a check of eh_in_progress
into ufshcd_hold, this check allows eh_work to run in parallel while
link recovery is running.
- Similar concurrency can also happen when error recovery is invoked from
ufshcd_eh_host_reset_handler and ufshcd_link_recovery.
- Concurrency can even happen between eh_works. eh_work, currently queued
on system_wq, is allowed to have multiple instances running in parallel,
but we don't have proper protection for that.
If any of above concurrency scenarios happen, error recovery would fail and
lead ufs device and host into bad states. To fix the concurrency problem,
this change queues eh_work on a single threaded workqueue and removes link
recovery calls from the hibern8 enter/exit path. In addition, make use of
eh_work in eh_host_reset_handler instead of calling
ufshcd_reset_and_restore. This unifies the UFS error recovery mechanism.
According to the UFSHCI JEDEC spec, hibern8 enter/exit error occurs when
the link is broken. This essentially applies to any power mode change
operations (since they all use PACP_PWR cmds in UniPro layer). So, if a
power mode change operation (including AH8 enter/exit) fails, mark link
state as UIC_LINK_BROKEN_STATE and schedule the eh_work. In this case,
error handler needs to do a full reset and restore to recover the link back
to active. Before the link state is recovered to active,
ufshcd_uic_pwr_ctrl simply returns -ENOLINK to avoid more errors.
Link: https://lore.kernel.org/r/1596975355-39813-6-git-send-email-cang@codeaurora.org
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Asutosh Das <asutoshd@codeaurora.org>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-08-09 05:15:51 -07:00
|
|
|
|
2020-08-24 19:07:06 -07:00
|
|
|
|
scsi: ufs: Complete pending requests in host reset and restore path
In UFS host reset and restore path, before probe, we stop and start the
host controller once. After host controller is stopped, the pending
requests, if any, are cleared from the doorbell, but no completion IRQ
would be raised due to the hba is stopped. These pending requests shall be
completed along with the first NOP_OUT command (as it is the first command
which can raise a transfer completion IRQ) sent during probe. Since the
OCSs of these pending requests are not SUCCESS (because they are not yet
literally finished), their UPIUs shall be dumped. When there are multiple
pending requests, the UPIU dump can be overwhelming and may lead to
stability issues because it is in atomic context. Therefore, before probe,
complete these pending requests right after host controller is stopped and
silence the UPIU dump from them.
Link: https://lore.kernel.org/r/1574751214-8321-5-git-send-email-cang@qti.qualcomm.com
Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Tested-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <cang@codeaurora.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2019-11-25 22:53:33 -08:00
|
|
|
|
2013-07-30 00:35:57 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-03-31 17:37:14 +03:00
|
|
|
|
2021-08-31 17:53:17 +03:00
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
|
|
|
|
|
2013-07-30 00:35:59 +05:30
|
|
|
|
2014-09-25 15:32:22 +03:00
|
|
|
|
2014-09-25 15:32:23 +03:00
|
|
|
|
2023-10-12 22:51:26 +05:30
|
|
|
|
2014-09-25 15:32:30 +03:00
|
|
|
|
2017-02-03 16:56:50 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-03-10 17:37:05 +02:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:31 +03:00
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-10-18 13:29:56 -07:00
|
|
|
|
2014-09-25 15:32:36 +03:00
|
|
|
|
2016-03-10 17:37:15 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2017-02-03 16:57:02 -08:00
|
|
|
|
2023-01-16 17:12:01 +01:00
|
|
|
|
2017-02-03 16:57:02 -08:00
|
|
|
|
scsi: ufs: Add a bsg endpoint that supports UPIUs
For now, just provide an API to allocate and remove ufs-bsg node. We
will use this framework to manage ufs devices by sending UPIU
transactions.
For the time being, implements an empty bsg_request() - will add some
more functionality in coming patches.
Nonetheless, we reveal here the protocol we are planning to use: UFS
Transport Protocol Transactions. UFS transactions consist of packets
called UFS Protocol Information Units (UPIU).
There are UPIU’s defined for UFS SCSI commands, responses, data in and
data out, task management, utility functions, vendor functions,
transaction synchronization and control, and more.
By using UPIUs, we get access to the most fine-grained internals of this
protocol, and able to communicate with the device in ways, that are
sometimes beyond the capacity of the ufs driver.
Moreover and as a result, our core structure - ufs_bsg_node has a pretty
lean structure: using upiu transactions that contains the outmost
detailed info, so we don't really need complex constructs to support it.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <Bart.VanAssche@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2018-10-07 17:30:35 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-05-22 16:32:12 +08:00
|
|
|
|
2020-07-06 20:04:13 +00:00
|
|
|
|
2021-04-21 19:28:39 -07:00
|
|
|
|
|
|
|
|
|
2020-07-06 20:04:13 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
blk-crypto: rename blk_keyslot_manager to blk_crypto_profile
blk_keyslot_manager is misnamed because it doesn't necessarily manage
keyslots. It actually does several different things:
- Contains the crypto capabilities of the device.
- Provides functions to control the inline encryption hardware.
Originally these were just for programming/evicting keyslots;
however, new functionality (hardware-wrapped keys) will require new
functions here which are unrelated to keyslots. Moreover,
device-mapper devices already (ab)use "keyslot_evict" to pass key
eviction requests to their underlying devices even though
device-mapper devices don't have any keyslots themselves (so it
really should be "evict_key", not "keyslot_evict").
- Sometimes (but not always!) it manages keyslots. Originally it
always did, but device-mapper devices don't have keyslots
themselves, so they use a "passthrough keyslot manager" which
doesn't actually manage keyslots. This hack works, but the
terminology is unnatural. Also, some hardware doesn't have keyslots
and thus also uses a "passthrough keyslot manager" (support for such
hardware is yet to be upstreamed, but it will happen eventually).
Let's stop having keyslot managers which don't actually manage keyslots.
Instead, rename blk_keyslot_manager to blk_crypto_profile.
This is a fairly big change, since for consistency it also has to update
keyslot manager-related function names, variable names, and comments --
not just the actual struct name. However it's still a fairly
straightforward change, as it doesn't change any actual functionality.
Acked-by: Ulf Hansson <ulf.hansson@linaro.org> # For MMC
Reviewed-by: Mike Snitzer <snitzer@redhat.com>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Link: https://lore.kernel.org/r/20211018180453.40441-4-ebiggers@kernel.org
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-10-18 11:04:52 -07:00
|
|
|
|
2020-07-06 20:04:13 +00:00
|
|
|
|
2021-01-07 09:25:38 +02:00
|
|
|
|
|
|
|
|
|
2021-02-09 08:24:37 +02:00
|
|
|
|
|
|
|
|
|
2023-11-18 21:44:43 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-01-07 09:25:38 +02:00
|
|
|
|
2021-04-23 17:20:16 -07:00
|
|
|
|
2023-01-13 12:48:41 -08:00
|
|
|
|
|
|
|
|
|
2021-04-23 17:20:16 -07:00
|
|
|
|
2023-01-13 12:48:40 -08:00
|
|
|
|
2023-01-13 12:48:39 -08:00
|
|
|
|
2024-07-10 08:25:20 +09:00
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2025-04-07 12:17:04 +02:00
|
|
|
|
2023-01-13 12:48:42 -08:00
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2023-12-12 23:08:24 +01:00
|
|
|
|
|
|
|
|
|
2023-12-19 18:07:05 +05:30
|
|
|
|
|
|
|
|
|
2025-09-17 17:41:43 +08:00
|
|
|
|
|
|
|
|
|
scsi: ufs: core: Critical health condition
Martin hi,
The UFS4.1 standard, released on January 8, 2025, added a new exception
event: HEALTH_CRITICAL, which notifies the host of a device's critical
health condition. This notification implies that the device is approaching
the end of its lifetime based on the amount of performed program/erase
cycles.
Once an EOL (End-of-Life) exception event is received, we increment a
designated member, which is exposed via a sysfs entry. This new entry, will
report the number of times a critical health event has been reported by a
UFS device.
To handle this new sysfs entry, userspace applications can use select(),
poll(), or epoll() to monitor changes in the critical_health attribute. The
kernel will call sysfs_notify() to signal changes, allowing the userspace
application to detect and respond to these changes efficiently.
The host can gain further insight into the specific issue by reading one of
the following attributes: bPreEOLInfo, bDeviceLifeTimeEstA,
bDeviceLifeTimeEstB, bWriteBoosterBufferLifeTimeEst, and
bRPMBLifeTimeEst. All those are available for reading via the driver's
sysfs entries or through an applicable utility. It is up to userspace to
read these attributes if needed.
Signed-off-by: Avri Altman <avri.altman@wdc.com>
Link: https://lore.kernel.org/r/20250211065813.58091-1-avri.altman@wdc.com
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2025-02-11 08:58:13 +02:00
|
|
|
|
|
|
|
|
|
2025-03-28 14:46:13 -07:00
|
|
|
|
|
|
|
|
|
2025-10-13 12:38:16 -07:00
|
|
|
|
2025-11-08 00:05:18 +01:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2023-01-13 12:48:47 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:50 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:51 -08:00
|
|
|
|
2023-05-29 15:12:22 -07:00
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-03-29 13:13:03 +03:00
|
|
|
|
2023-01-13 12:48:44 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
2023-01-13 12:48:47 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:50 -08:00
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:51 -08:00
|
|
|
|
2023-05-29 15:12:22 -07:00
|
|
|
|
|
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
|
|
|
|
|
2024-05-20 07:14:57 +09:00
|
|
|
|
|
|
|
|
|
2024-05-20 07:14:56 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-05-20 07:14:57 +09:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-12-08 15:43:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2024-12-12 20:19:45 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-05-04 23:44:50 +08:00
|
|
|
|
2022-12-08 15:43:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2016-12-22 18:40:50 -08:00
|
|
|
|
2014-09-25 15:32:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:35 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-09-16 23:56:50 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:35 +03:00
|
|
|
|
2015-05-17 18:54:57 +03:00
|
|
|
|
|
|
|
|
|
2021-06-28 14:58:01 +09:00
|
|
|
|
|
|
|
|
|
2015-05-17 18:54:57 +03:00
|
|
|
|
|
|
|
|
|
2020-10-27 12:10:36 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-05-21 14:44:52 +08:00
|
|
|
|
|
|
|
|
|
2020-08-10 17:10:24 +03:00
|
|
|
|
|
|
|
|
|
2019-05-21 14:44:52 +08:00
|
|
|
|
|
|
|
|
|
2020-01-29 18:52:50 +08:00
|
|
|
|
|
|
|
|
|
2022-04-19 15:57:46 -07:00
|
|
|
|
2020-01-29 18:52:50 +08:00
|
|
|
|
|
|
|
|
|
2020-04-22 14:41:42 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-08-04 10:54:22 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-01-13 12:48:45 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-26 22:39:26 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:36 +03:00
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:36 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-12-08 12:28:51 +05:30
|
|
|
|
|
|
|
|
|
2014-09-25 15:32:21 +03:00
|
|
|
|
2020-01-17 11:51:07 +08:00
|
|
|
|
2021-05-18 21:12:17 +09:00
|
|
|
|
2020-03-27 17:53:28 +08:00
|
|
|
|
2020-01-17 11:51:07 +08:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2021-09-28 02:06:12 -07:00
|
|
|
|
2020-01-17 11:51:07 +08:00
|
|
|
|
2020-03-18 18:40:12 +08:00
|
|
|
|
2018-10-16 14:29:41 +05:30
|
|
|
|
2020-12-05 19:58:59 +08:00
|
|
|
|
2021-05-28 11:36:21 +08:00
|
|
|
|
2021-10-20 14:40:19 -07:00
|
|
|
|
2023-07-01 20:44:40 +08:00
|
|
|
|
2024-06-01 06:22:43 +09:00
|
|
|
|
2023-07-01 20:44:40 +08:00
|
|
|
|
2022-12-14 19:06:21 -08:00
|
|
|
|
2023-05-29 15:12:25 -07:00
|
|
|
|
2022-12-14 19:06:21 -08:00
|
|
|
|
2023-07-01 20:44:40 +08:00
|
|
|
|
2024-01-02 10:42:22 +09:00
|
|
|
|
2024-10-22 12:30:57 -07:00
|
|
|
|
2022-12-14 19:06:21 -08:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|
2023-10-12 22:51:27 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-28 13:15:47 +02:00
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
|
|
|
|
|
2015-10-28 13:15:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-04-19 15:58:07 -07:00
|
|
|
|
2015-10-28 13:15:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-09-15 09:04:06 +03:00
|
|
|
|
2021-07-21 20:34:23 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2023-02-02 21:40:45 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-05 14:15:54 +08:00
|
|
|
|
|
|
|
|
|
2020-11-16 14:50:52 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-08-31 21:40:21 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2018-05-06 15:44:18 +05:30
|
|
|
|
|
|
|
|
|
2022-06-16 13:37:15 +08:00
|
|
|
|
2013-08-31 21:40:21 +05:30
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2025-02-12 13:38:02 -08:00
|
|
|
|
2016-03-10 17:37:20 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-02-07 15:03:57 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2019-12-30 13:32:28 +08:00
|
|
|
|
2022-04-19 15:57:53 -07:00
|
|
|
|
|
|
|
|
|
2018-02-15 14:14:07 +02:00
|
|
|
|
2023-05-29 13:26:40 -07:00
|
|
|
|
2014-09-25 15:32:32 +03:00
|
|
|
|
2017-02-23 09:05:30 +00:00
|
|
|
|
2022-04-19 15:58:04 -07:00
|
|
|
|
|
|
|
|
|
2022-06-16 13:37:23 +08:00
|
|
|
|
|
|
|
|
|
2018-10-07 17:30:39 +03:00
|
|
|
|
|
|
|
|
|
2022-12-01 15:04:37 +01:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-03-18 17:55:36 +08:00
|
|
|
|
2022-08-04 16:53:54 +09:00
|
|
|
|
2025-04-11 17:29:24 +08:00
|
|
|
|
2021-04-23 17:20:16 -07:00
|
|
|
|
2021-10-27 16:06:14 +03:00
|
|
|
|
2021-04-23 17:20:16 -07:00
|
|
|
|
2023-07-26 19:11:39 +05:30
|
|
|
|
2023-12-19 18:07:05 +05:30
|
|
|
|
|
|
|
|
|
2025-07-14 13:23:35 +05:30
|
|
|
|
2021-01-19 17:38:42 +01:00
|
|
|
|
2015-10-28 13:15:48 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2020-12-05 20:00:39 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2022-06-23 12:24:32 +02:00
|
|
|
|
2018-02-15 14:14:01 +02:00
|
|
|
|
2018-06-14 11:14:09 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2021-02-09 08:24:37 +02:00
|
|
|
|
|
|
|
|
|
2022-06-23 12:24:32 +02:00
|
|
|
|
|
|
|
|
|
2025-09-03 10:44:37 +08:00
|
|
|
|
2025-09-24 17:43:23 +08:00
|
|
|
|
2025-09-24 17:43:28 +08:00
|
|
|
|
2021-02-09 08:24:36 +02:00
|
|
|
|
2013-02-25 21:44:33 +05:30
|
|
|
|