2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-09-05 15:29:12 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-09-05 15:29:12 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-28 11:35:41 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-05-08 10:11:12 +02:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-27 13:13:12 -08:00
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-27 13:13:12 -08:00
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-02 03:50:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2013-03-27 17:23:41 -07:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-09-09 22:24:56 +03:00
|
|
|
|
2013-03-27 17:23:41 -07:00
|
|
|
|
|
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
|
|
|
|
|
2013-04-28 15:38:52 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2013-04-28 15:38:52 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 13:10:51 -08:00
|
|
|
|
2013-01-22 11:20:25 +08:00
|
|
|
|
2015-01-28 13:10:51 -08:00
|
|
|
|
|
|
|
|
|
2013-01-22 11:20:25 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2013-06-06 02:20:41 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-09-17 22:54:31 +03:00
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-06 02:20:41 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-07-03 11:22:17 -04:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2014-03-22 14:55:56 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-06-06 01:44:48 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-06-06 02:20:41 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-06-06 02:20:41 -07:00
|
|
|
|
|
|
|
|
|
2015-04-27 13:52:36 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-05 15:20:27 +03:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
|
|
|
|
|
2015-01-27 13:13:12 -08:00
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-01-06 14:36:13 +08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2015-01-27 13:13:12 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-01-19 08:26:37 +00:00
|
|
|
|
2013-09-23 11:42:28 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-09-23 11:42:28 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-28 13:10:51 -08:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2014-02-22 18:34:08 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 13:10:51 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 13:10:51 -08:00
|
|
|
|
2013-01-22 11:20:27 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
2013-01-22 11:20:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-01-22 11:20:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-01-22 11:20:27 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:08 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:08 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-28 00:15:00 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:34:43 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-12-21 10:42:08 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-04-14 13:26:44 +02:00
|
|
|
|
2012-10-01 18:40:55 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
|
|
|
|
|
2014-12-21 10:42:08 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2012-10-01 18:40:55 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-10 15:06:15 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-07 14:54:35 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-07 14:54:35 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-05-07 14:54:35 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-05 15:20:27 +03:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-02-24 14:13:32 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2014-02-24 14:13:32 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2013-04-10 15:06:15 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-11-23 18:01:34 +02:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
|
|
|
|
|
2014-06-10 01:19:38 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
|
|
|
|
|
2015-01-27 13:13:12 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-01-29 17:21:13 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:29 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-25 21:14:58 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-07 14:54:35 +08:00
|
|
|
|
2013-04-10 15:06:14 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-02-05 10:37:33 +03:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
tcm_vhost: Multi-queue support
This adds virtio-scsi multi-queue support to tcm_vhost. In order to use
multi-queue, guest side multi-queue support is need. It can
be found here:
https://lkml.org/lkml/2012/12/18/166
Currently, only one thread is created by vhost core code for each
vhost_scsi instance. Even if there are multi-queues, all the handling of
guest kick (vhost_scsi_handle_kick) are processed in one thread. This is
not optimal. Luckily, most of the work is offloaded to the tcm_vhost
workqueue.
Some initial perf numbers:
1 queue, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 127K/127k IOPS
4 queues, 4 targets, 1 lun per target
4K request size, 50% randread + 50% randwrite: 181K/181k IOPS
Signed-off-by: Asias He <asias@redhat.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-02-06 13:20:59 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-28 15:38:52 +03:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-10-08 06:19:20 +00:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2014-10-08 06:19:20 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-03 08:50:52 +02:00
|
|
|
|
2014-10-08 06:19:20 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-03-17 18:06:10 +01:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-05-07 14:54:36 +08:00
|
|
|
|
2013-04-03 14:17:38 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2014-10-08 06:19:20 +00:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2013-03-15 09:14:05 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:20 +08:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2013-03-15 09:14:05 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
|
|
|
|
|
2013-03-15 09:14:05 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2014-10-08 06:19:20 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-03 08:50:52 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
2013-05-07 14:54:36 +08:00
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2013-03-15 09:14:05 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2013-03-15 09:14:05 +08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
2014-06-05 15:20:23 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-06-05 15:20:23 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-03 14:17:37 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-09-17 09:30:34 +03:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-09-17 09:30:34 +03:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-09-17 09:30:34 +03:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-12-07 04:13:03 +08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-27 11:16:49 +08:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-09-17 09:30:34 +03:00
|
|
|
|
|
|
|
|
|
2014-06-12 19:00:01 +03:00
|
|
|
|
2013-09-17 09:30:34 +03:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
2013-02-05 12:31:57 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-05-06 16:38:26 +08:00
|
|
|
|
|
|
|
|
|
2014-06-12 19:00:01 +03:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:22 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-08-18 15:44:09 -07:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-08-18 15:44:09 -07:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:22 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-03-27 17:23:41 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-12-06 14:03:34 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-07-30 15:59:57 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-07-30 15:59:57 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-27 11:16:48 +08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2014-06-05 15:20:23 +03:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2014-06-05 15:20:23 +03:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-04-25 15:35:21 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-06-07 17:47:46 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2014-02-22 18:22:31 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2014-02-22 18:08:24 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-02-12 11:48:49 +01:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-02-12 11:48:49 +01:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-06-21 14:32:04 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:28 +08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-02-05 10:37:33 +03:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-02-05 10:37:33 +03:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-05-20 21:48:03 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2013-05-06 16:38:27 +08:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-10-03 15:32:55 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2015-04-08 20:01:35 +02:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-03-28 00:03:51 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2013-06-06 02:20:41 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-04-08 20:01:35 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2015-04-08 20:01:35 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-07-30 13:30:00 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-04-08 20:01:35 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-04-08 20:01:35 +02:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
|
|
|
|
|
2013-05-02 03:52:59 +03:00
|
|
|
|
|
|
|
|
|
2012-07-18 14:31:32 -07:00
|
|
|
|
2015-01-31 23:56:53 -08:00
|
|
|
|
|
|
|
|
|