优化django查询

Question

在网站上实现了一项新功能，就好像它是“社交网络”一样。

用户进行的各种操作，接收消息和内容。他们的朋友可以看到所有的动作。

class userMuralEntry(models.Model): 
    user_src = models.ForeignKey(userInfo, related_name="user_src") 
    user = models.ManyToManyField(userInfo, related_name="user_dest") 
    user_follow = models.ManyToManyField(userInfo, related_name="user_follow") 
    private = models.BooleanField(default=False) 
    content = models.TextField() 
    date = models.DateTimeField(default=datetime.now) 
    last_update = models.DateTimeField() 

    object_type = models.ForeignKey(muralType) 

    content_type = models.ForeignKey(ContentType) 
    object_id = models.PositiveIntegerField() 
    content_object = generic.GenericForeignKey('content_type', 'object_id') 

每当我访问我的首页，我得到我所有的朋友行动，除了私人的行为。 所以，我做了这个Django模块查询

mural_list = db.userMuralEntry.objects.filter(Q(user__pk__in=friend_list)|Q(user_src__pk__in=friend_list)).order_by('-last_update') 
    m = mural_list.exclude(
    Q(private=True), 
    ~Q(user_src=me_db) & ~Q(user=me_db)) 

    m = m.distinct() 

所有这些代码生成这个大查询（这里解释输出）

SELECT DISTINCT myps3t_usermuralentry.id, 
       myps3t_usermuralentry.user_src_id, 
       myps3t_usermuralentry.private, 
       myps3t_usermuralentry.content, 
       myps3t_usermuralentry.DATE, 
       myps3t_usermuralentry.last_update, 
       myps3t_usermuralentry.object_type_id, 
       myps3t_usermuralentry.content_type_id, 
       myps3t_usermuralentry.object_id 
FROM myps3t_usermuralentry 
     LEFT OUTER JOIN myps3t_usermuralentry_user 
     ON (myps3t_usermuralentry.id = 
       myps3t_usermuralentry_user.usermuralentry_id) 
     LEFT OUTER JOIN myps3t_userinfo t4 
     ON (myps3t_usermuralentry.user_src_id = t4.id) 
WHERE ((myps3t_usermuralentry_user.userinfo_id = 20877 
      OR myps3t_usermuralentry.user_src_id = 20877) 
     AND NOT (myps3t_usermuralentry.private = 1 
        AND (NOT ((t4.id = 21095 
           AND NOT (t4.id IS NULL))) 
         AND NOT (myps3t_usermuralentry.id IN 
            (SELECT u1.usermuralentry_id 
            FROM 
            myps3t_usermuralentry_user u1 
                   WHERE ( 
            u1.userinfo_id = 
            21095 
            AND 
            u1.usermuralentry_id 
            IS 
            NOT 
            NULL 
                     ))))) 
     ) 
ORDER BY myps3t_usermuralentry.last_update; 



mysql> explain SELECT DISTINCT `myps3t_usermuralentry`.`id`, `myps3t_usermuralentry`.`user_src_id`, `myps3t_usermuralentry`.`private`, `myps3t_usermuralentry`.`content`, `myps3t_usermuralentry`.`date`, `myps3t_usermuralentry`.`last_update`, `myps3t_usermuralentry`.`object_type_id`, `myps3t_usermuralentry`.`content_type_id`, `myps3t_usermuralentry`.`object_id` FROM `myps3t_usermuralentry` LEFT OUTER JOIN `myps3t_usermuralentry_user` ON (`myps3t_usermuralentry`.`id` = `myps3t_usermuralentry_user`.`usermuralentry_id`) LEFT OUTER JOIN `myps3t_userinfo` T4 ON (`myps3t_usermuralentry`.`user_src_id` = T4.`id`) WHERE ((`myps3t_usermuralentry_user`.`userinfo_id` = 20877 OR `myps3t_usermuralentry`.`user_src_id` = 20877) AND NOT (`myps3t_usermuralentry`.`private` = 1 AND (NOT ((T4.`id` = 21095 AND NOT (T4.`id` IS NULL))) AND NOT (`myps3t_usermuralentry`.`id` IN (SELECT U1.`usermuralentry_id` FROM `myps3t_usermuralentry_user` U1 WHERE (U1.`userinfo_id` = 21095 AND U1.`usermuralentry_id` IS NOT NULL)))))) ORDER BY `myps3t_usermuralentry`.`last_update` DESC LIMIT 20; 
+----+--------------------+----------------------------+-----------------+-------------------------------------------------------------------------------------------+-------------------+---------+---------------------------------------------------+--------+------------------------------------+ 
| id | select_type  | table      | type   | possible_keys                    | key    | key_len | ref            | rows | Extra        | 
+----+--------------------+----------------------------+-----------------+-------------------------------------------------------------------------------------------+-------------------+---------+---------------------------------------------------+--------+------------------------------------+ 
| 1 | PRIMARY   | myps3t_usermuralentry  | ALL    | myps3t_usermuralentry_99bd10ae               | NULL    | NULL | NULL            | 665410 | Using temporary; Using filesort | 
| 1 | PRIMARY   | myps3t_usermuralentry_user | ref    | usermuralentry_id,myps3t_usermuralentry_user_bcd7114e          | usermuralentry_id | 4  | fabriciols_ps3t.myps3t_usermuralentry.id   |  2 | Using where; Using index; Distinct | 
| 1 | PRIMARY   | T4       | eq_ref   | PRIMARY                     | PRIMARY   | 4  | fabriciols_ps3t.myps3t_usermuralentry.user_src_id |  1 | Using where; Using index; Distinct | 
| 2 | DEPENDENT SUBQUERY | U1       | unique_subquery | usermuralentry_id,myps3t_usermuralentry_user_bcd7114e,myps3t_usermuralentry_user_6b192ca7 | usermuralentry_id | 8  | func,const          |  1 | Using index; Using where   | 
+----+--------------------+----------------------------+-----------------+-------------------------------------------------------------------------------------------+-------------------+---------+---------------------------------------------------+--------+------------------------------------+ 
4 rows in set (0.00 sec) 

mysql> show indexes from myps3t_usermuralentry ; 
+-----------------------+------------+--------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+ 
| Table     | Non_unique | Key_name      | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | 
+-----------------------+------------+--------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+ 
| myps3t_usermuralentry |   0 | PRIMARY      |   1 | id    | A   |  665410 |  NULL | NULL |  | BTREE  |   | 
| myps3t_usermuralentry |   1 | myps3t_usermuralentry_99bd10ae |   1 | user_src_id  | A   |  22180 |  NULL | NULL |  | BTREE  |   | 
| myps3t_usermuralentry |   1 | myps3t_usermuralentry_ae71a55b |   1 | object_type_id | A   |   8 |  NULL | NULL |  | BTREE  |   | 
| myps3t_usermuralentry |   1 | myps3t_usermuralentry_e4470c6e |   1 | content_type_id | A   |   13 |  NULL | NULL |  | BTREE  |   | 
+-----------------------+------------+--------------------------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+ 
6 rows in set (0.00 sec) 

我做了一些的调整对指数，但其过于缓慢。拥有3-4位朋友的用户需要5秒钟的时间！

此表格现在有500,000行。

改进的一些想法？我的Django代码是否杂乱？最好是在我自己的手中进行查询？

Answer 1

您需要再次阅读：https://docs.djangoproject.com/en/dev/topics/db/queries/

有很多，你可以简化代码的方式。例如：

mural_list = db.userMuralEntry.objects.filter(Q(user__pk__in=friend_list)|Q(user_src__pk__in=friend_list)) 

等同于：

mural_list = db.userMuralEntry.objects.filter(user__pk__in=friend_list)|db.userMuralEntry.objects.filter(user_src__pk__in=friend_list) 

和

m = mural_list.exclude(
    Q(private=True), 
    ~Q(user_src=me_db) & ~Q(user=me_db)) 

相当于

m = mural_list.exclude(private=True).exclude(user_src=me_db).exclude(user=me_db) 

注意，它可能是更好的地方ORDER_BY条款在最后，避免外部连接在ordered_by查询。

最终优化的SQL是好心由@fabriciols这里分享：http://chat.stackoverflow.com/transcript/message/2795524#2795524