选择通过

Question

组相比明显快我想运行以下搜索：

schema->resultset('Entity')->search({ 
     -or => { "me.user_id" => $user_id, 'set_to_user.user_id' => $user_id } 
    }, { 
     'distinct' => 1, 
     'join' => {'entity_to_set' => {'entity_set' => 'set_to_user'}}, 
     'order_by' => {'-desc' => 'modified'}, 
     'page' => 1,'rows' => 100 
    }); 

在有表的数据库，如下图所示。

CREATE TABLE entity (
    id varchar(500) NOT NULL, 
    user_id varchar(100) NOT NULL, 
    modified timestamp NOT NULL, 
    PRIMARY KEY (id, user_id), 
    FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE ON UPDATE CASCADE 
); 

CREATE TABLE entity_to_set (
    set_id varchar(100) NOT NULL, 
    user_id varchar(100) NOT NULL, 
    entity_id varchar(500) NOT NULL, 
    PRIMARY KEY (set_id, user_id, entity_id), 
    FOREIGN KEY (entity_id, user_id) REFERENCES entity(id, user_id) ON DELETE CASCADE ON UPDATE CASCADE, 
    FOREIGN KEY (set_id) REFERENCES entity_set(id) ON DELETE CASCADE ON UPDATE CASCADE 
); 

CREATE TABLE entity_set (
    id varchar(100) NOT NULL, 
    PRIMARY KEY (id) 
); 

CREATE TABLE set_to_user (
    set_id varchar(100) NOT NULL, 
    user_id varchar(100) NOT NULL, 
    PRIMARY KEY (set_id, user_id), 
    FOREIGN KEY (user_id) REFERENCES user(id) ON DELETE CASCADE ON UPDATE CASCADE, 
    FOREIGN KEY (set_id) REFERENCES entity_set(id) ON DELETE CASCADE ON UPDATE CASCADE 
); 

CREATE TABLE user (
    id varchar(100) NOT NULL, 
    PRIMARY KEY (id) 
); 

我有大约6000 entity，6000 entity_to_set，10 entity_set和50 set_to_user。

现在，这个查询需要一些时间，（一两秒钟），这是不幸的。在仅对实体表进行查询时，包括ORDER BY，结果几乎是即时的。至于调试这第一步，我发现DBIC代码变成实际的SQL查询：

SELECT me.id, me.user_id, me.modified FROM entity me 
LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) 
LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id 
LEFT JOIN set_to_user set_to_user ON set_to_user.set_id = entity_set.id 
WHERE ((set_to_user.user_id = 'Craigy' OR me.user_id = 'Craigy')) 
GROUP BY me.id, me.user_id, me.modified ORDER BY modified DESC LIMIT 100; 

这里是EXPLAIN QUERY PLAN

0|0|0|SCAN TABLE entity AS me USING INDEX sqlite_autoindex_entity_1 (~1000000 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_to_set_idx_cover (entity_id=? AND user_id=?) (~9 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
0|3|3|SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoindex_set_to_user_1 (set_id=?) (~5 rows) 
0|0|0|USE TEMP B-TREE FOR ORDER BY 

结果，其中entity_to_set_idx_cover是

CREATE INDEX entity_to_set_idx_cover ON entity_to_set (entity_id, user_id, set_id); 

现在，问题是用于排序的b-tree，而不是在我没有进行联接时使用的索引。

我注意到DBIx :: Class将'distinct' => 1转换为GROUP BY语句（I believe the documentation says they are equivalent here）。我删除了GROUP BY声明和使用SELECT DISTINCT代替，用下面的查询

SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me 
LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) 
LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id 
LEFT JOIN set_to_user set_to_user ON set_to_user.set_id = entity_set.id 
WHERE ((set_to_user.user_id = 'Craigy' OR me.user_id = 'Craigy')) 
ORDER BY modified DESC LIMIT 100; 

我相信的结果相同。该EXPLAIN QUERY PLAN此查询是

0|0|0|SCAN TABLE entity AS me USING COVERING INDEX entity_sort_modified_user_id (~1000000 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_to_set_idx_cover (entity_id=? AND user_id=?) (~9 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
0|3|3|SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoindex_set_to_user_1 (set_id=?) (~5 rows) 

其中entity_sort_modified_user_id是使用

CREATE INDEX entity_sort_modified_user_id ON entity (modified, user_id, id); 

创建的索引这将运行几乎瞬间（无b树）。

编辑：为了演示当ORDER BY按升序排列时问题仍然存在，以及索引对这些查询的影响，这里是对相同表格的类似查询。前两个查询没有索引，分别使用SELECT DISTINCT和GROUP BY，后两个查询和索引相同。

sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE (me.user_id = 'Craigy' AND entity_set.id = 'SetID') ORDER BY modified LIMIT 100; 
0|0|0|SCAN TABLE entity AS me (~100000 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
0|0|0|USE TEMP B-TREE FOR DISTINCT 
0|0|0|USE TEMP B-TREE FOR ORDER BY 
sqlite> EXPLAIN QUERY PLAN SELECT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE (me.user_id = 'Craigy' AND entity_set.id = 'SetID') GROUP BY me.id, me.user_id, me.modified ORDER BY modified LIMIT 100; 
0|0|0|SCAN TABLE entity AS me USING INDEX sqlite_autoindex_entity_1 (~100000 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
0|0|0|USE TEMP B-TREE FOR ORDER BY 
sqlite> CREATE INDEX entity_idx_user_id_modified_id ON entity (user_id, modified, id); 
sqlite> EXPLAIN QUERY PLAN SELECT DISTINCT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE (me.user_id = 'Craigy' AND entity_set.id = 'SetID') ORDER BY modified LIMIT 100; 
0|0|0|SEARCH TABLE entity AS me USING COVERING INDEX entity_idx_user_id_modified_id (user_id=?) (~10 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
sqlite> EXPLAIN QUERY PLAN SELECT me.id, me.user_id, me.modified FROM entity me LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id WHERE (me.user_id = 'Craigy' AND entity_set.id = 'SetID') GROUP BY me.id, me.user_id, me.modified ORDER BY modified LIMIT 100; 
0|0|0|SEARCH TABLE entity AS me USING COVERING INDEX entity_idx_user_id_modified_id (user_id=?) (~10 rows) 
0|1|1|SEARCH TABLE entity_to_set AS entity_to_set USING AUTOMATIC COVERING INDEX (entity_id=? AND user_id=?) (~7 rows) 
0|2|2|SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoindex_entity_set_1 (id=?) (~1 rows) 
0|0|0|USE TEMP B-TREE FOR GROUP BY 
0|0|0|USE TEMP B-TREE FOR ORDER BY 

我的问题是：我怎么解决我DBIx ::类代码，以便它执行还有SELECT DISTINCT查询。或者，如何添加索引以使其工作得很好？还是需要一些其他类型的修复？

Answer 1

注意：这不是这个问题的完整答案。它只显示如何在排序升序顺序时避免临时B树。当排序降序需要订单，目前有AFAIK（版本3.8.1）没有办法（没有调整sqlite）来避免GROUP BY版本的临时B树。

使用表定义和指标从问题：

sqlite> select sqlite_version(); 
sqlite_version() 
---------------- 
3.8.1 

运行查询后，没有在（a）您ORDER BY升序和（b）GROUP BY子句顺序相匹配温度B树BY子句逐列。

查询不变除了GROUP BY和ORDER BY子句：

/* table definitions as shown in the question */ 
sqlite> CREATE INDEX entity_to_set_idx_cover ON entity_to_set (entity_id, user_id, set_id); 
sqlite> CREATE INDEX entity_sort_modified_user_id ON entity (modified, user_id, id); 

sqlite> EXPLAIN QUERY PLAN 
    ...> SELECT me.id, me.user_id, me.modified FROM entity me 
    ...> LEFT JOIN entity_to_set entity_to_set ON (entity_to_set.entity_id = me.id AND entity_to_set.user_id = me.user_id) 
    ...> LEFT JOIN entity_set entity_set ON entity_set.id = entity_to_set.set_id 
    ...> LEFT JOIN set_to_user set_to_user ON set_to_user.set_id = entity_set.id 
    ...> WHERE ((set_to_user.user_id = 'Craigy' OR me.user_id = 'Craigy')) 
    ...> GROUP BY me.modified, me.user_id, me.id 
    ...> ORDER BY me.modified, me.user_id, me.id ASC LIMIT 100; 

selectid order  from  detail 
---------- ---------- ---------- ------------------------------------------------------------------------- 
0   0   0   SCAN TABLE entity AS me USING COVERING INDEX entity_sort_modified_user_id 
0   1   1   SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_t 
0   2   2   SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoind 
0   3   3   SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoi 

然而，当你按降序排列顺序由你得到一个临时的B-tree：

...> ... 
    ...> GROUP BY me.modified, me.user_id, me.id 
    ...> ORDER BY me.modified, me.user_id, me.id DESC LIMIT 100; 
selectid order  from  detail 
---------- ---------- ---------- ------------------------------------------------------------------------- 
0   0   0   SCAN TABLE entity AS me USING COVERING INDEX entity_sort_modified_user_id 
0   1   1   SEARCH TABLE entity_to_set AS entity_to_set USING COVERING INDEX entity_t 
0   2   2   SEARCH TABLE entity_set AS entity_set USING COVERING INDEX sqlite_autoind 
0   3   3   SEARCH TABLE set_to_user AS set_to_user USING COVERING INDEX sqlite_autoi 
0   0   0   USE TEMP B-TREE FOR ORDER BY 

原因是sqlite（高达当前版本的3.8.1）不承认它可以按降序进行分组。因此，您将始终获得单独的步骤。即使索引被声明为DESC，也无法避免这种情况。请参阅关于此的sqlite mailing list的讨论。

结论 如果你想要查询ORDER BY DESC没有临时B树，你必须调整你的SQL代以使用DISTINCT。