MySQL缓慢查询和memcached

Question

我缓慢查询缓存与memcached，它很好，但有时我们在mysql中有很多缓慢的查询，就像memcached已停止，然后我们的网站关闭。

慢查询图：http://i.imgur.com/ReyWe.png

在这个时刻，我们在30秒内得到约100慢查询。它可能是什么？ 我们查询：

# Query_time: 5.942602 Lock_time: 0.010214 Rows_sent: 10000 Rows_examined: 493139 

SET timestamp=1335194149; 
SELECT story_id FROM dug_stories d WHERE d.story_is_permanent=0 AND 
         ( 
         (d.story_time>1335190543 AND (d.story_pluses-d.story_minuses) > 5) 
         OR 
         (((d.story_pluses-d.story_minuses) > 12 OR (d.story_pluses >= 10 AND d.story_minuses<=0)) AND d.story_cat!=48 AND d.st 
ory_cat!=131 AND d.story_cat!=55 AND d.story_cat!=44 AND d.story_cat!=126 AND d.story_cat!=53 AND d.story_cat!=370 AND d.story_cat!=381 AND d.stor 
y_cat!=304 AND d.story_cat!=497) OR (d.story_cat=48 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_cat=131 AND (d.story_pluses-d.story_min 
uses) > 8) OR (d.story_cat=55 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=44 AND (d.story_pluses-d.story_minuses) > 9) OR (d.story_c 
at=126 AND (d.story_pluses-d.story_minuses) > 13) OR (d.story_cat=53 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=370 AND (d.story_pl 
uses-d.story_minuses) > 8) OR (d.story_cat=381 AND (d.story_pluses-d.story_minuses) > 8) OR (d.story_cat=304 AND (d.story_pluses-d.story_minuses) 
> 8) OR (d.story_cat=497 AND (d.story_pluses-d.story_minuses) > 9) 
         ) 
         ORDER BY d.story_rating DESC, d.story_time DESC LIMIT 0, 10000; 

小幅调整查询（而不是全部=类别，我更改为不类别中（要排除的类别）的名单，但格式，可读性更强！ ......另外，还要注意的SET TIMESTAMP变量甚至没有被查询中使用的...这是一个硬编码值。

SET timestamp=1335194149; 
SELECT 
     story_id 
    FROM 
     dug_stories d 
    WHERE 
      d.story_is_permanent = 0 
     AND ( (d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5) 
      OR ( ( d.story_pluses - d.story_minuses > 12 
         OR ( d.story_pluses >= 10 AND d.story_minuses <= 0) 
        ) 
        AND NOT d.story_cat IN (44, 48, 53, 55, 126, 131, 304, 370, 381, 497) 
       ) 
      OR (d.story_cat = 44 AND d.story_pluses - d.story_minuses > 9) 
      OR (d.story_cat = 48 AND d.story_pluses - d.story_minuses > 9) 
      OR (d.story_cat = 53 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 55 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13) 
      OR (d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8) 
      OR (d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9) 
     ) 
    ORDER BY 
     d.story_rating DESC, 
     d.story_time DESC 
    LIMIT 
     0, 10000; 

非常感谢您的帮助！ 对不起，我的英文不好:)

Answer 1

如果rows_examined大于结果集，缓慢的日志可能表明索引没有被充分利用。

理想情况下，MySQL应该能够在WHERE子句中为所有过滤器表达式使用单个索引。由于所有嵌套的AND和OR以及对多个列的引用，您的WHERE子句非常复杂。 MySQL可能无法利用索引来优化此查询，即使您有一个。

通常，可以通过将这种类型的查询分成几个较小的查询来改进，每个查询都能够使用索引，并使用UNION将结果组合成单个查询。

举例来说，如果我有FIRST_NAME的索引和姓氏索引，下面的查询：

SELECT * 
FROM Users 
WHERE active = true 
    AND ((first_name = 'Marcus') OR (last_name = 'Adams')) 

可以这样改进：

SELECT * 
FROM Users 
WHERE active = true 
    AND first_name = 'Marcus' 
UNION 
SELECT * 
FROM Users 
WHERE active = true 
    AND last_name = 'Adams' 

事实上，有更多的在您的LIMIT子句中检查的行数超过10000也表示ORDER BY子句可能正在使用文件排序，而不是查询已按顺序返回行。这使得MySQL可以完成所有的工作，即使你只需要一部分记录。您很可能需要利用覆盖索引来处理您的所有条件，这可能是不可能的，或者您需要删除ORDER BY子句，或者如您所述，依靠缓存来节省一天。

对于高负载的服务器，查询时间是一个糟糕的性能测量，因为任何查询在繁忙的服务器上可能会很慢。您应该使用EXPLAIN和SHOW PROFILE来分析您的查询并对其进行改进，以便它们不会对服务器造成太大负担。

允许太多线程在服务器上运行可能会导致级联效应，导致服务器停止。如果您使用的是InnoDB，请确保您的thread concurrency不是太高。最好让每个人一次等待并提供8个线程（非常快），而不是一次尝试提供100个或无限的线程（非常慢）。

Answer 2

除了调整查询的可读性以外，我会尝试重新构建查询。
我会确保你有一个索引（Story_Is_Permanent，Story_Cat，Story_Time）

我会假设你有一个“故事类别”表的文字描述呈现给用户。由于您无论如何都在查询所有类别，但仅基于某些您想要的INCLUDE或EXCLUDE ...，如果包含，则仅基于该类别的PLUSES和MINUSES的净计数。我想补充到列该类别表...像

DefaultToInclude integer as 1 or 0 
PlusMinusNetCount integer 

然后，做了一个连接以类别表，并简化您的查询，像

SELECT 
     story_id 
    FROM 
     dug_stories d 
     JOIN YourCategories YC 
      on d.story_cat = YC.categoryID 
    WHERE 
      d.story_is_permanent = 0 
     AND ( (d.story_time > 1335190543 AND d.story_pluses - d.story_minuses > 5) 
      OR ( YC.DefaultToInclude = 0 
        AND ( d.story_pluses - d.story_minuses > 12 
         OR (d.story_pluses >= 10 AND d.story_minuses <= 0) 
        ) 
       ) 
      OR ( YC.DefaultToInclude = 1 
        AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount) 
     ) 
ORDER BY 
     d.story_rating DESC, 
     d.story_time DESC 
    LIMIT 
     0, 10000; 

The

( YC.DefaultToInclude = 0 
    AND ( d.story_pluses - d.story_minuses > 12 
     OR (d.story_pluses >= 10 AND d.story_minuses <= 0) 
    ) 
) 

最终将取代

OR ( ( d.story_pluses - d.story_minuses > 12 
      OR ( d.story_pluses >= 10 AND d.story_minuses <= 0) 
     ) 
    AND d.story_cat!=48 
    AND d.story_cat!=131 
    AND d.story_cat!=55 
    AND d.story_cat!=44 
    AND d.story_cat!=126 
    AND d.story_cat!=53 
    AND d.story_cat!=370 
    AND d.story_cat!=381 
    AND d.story_cat!=304 
    AND d.story_cat!=497 
    ) 

的

OR ( YC.DefaultToInclude = 1 
     AND d.story_pluses-d.story_minuses > YC.PlusMinusNetCount) 

随后将最终替换

(OR (d.story_cat = 44 AND d.story_pluses - d.story_minuses > 9) 
    OR (d.story_cat = 48 AND d.story_pluses - d.story_minuses > 9) 
    OR (d.story_cat = 53 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 55 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 126 AND d.story_pluses - d.story_minuses > 13) 
    OR (d.story_cat = 131 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 304 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 370 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 381 AND d.story_pluses - d.story_minuses > 8) 
    OR (d.story_cat = 497 AND d.story_pluses - d.story_minuses > 9) 
) 

这可以灵活地简单地包含/排除其他类别与其各自的“净”数量，而无需为您已经冗长的查询添加更多的OR/AND条件。毫无疑问，它看起来总是需要经历整个“dug_Stories”表，因为您正在查看每个类别的一个条件，以及每个其他类别的另一个类别。您的时间戳记仅适用于NOT EQUAL类别，但您的其他条件没有任何时间戳记限制（除非您的查询中存在错误）。