PHP PDO巨大的数据库查询速度优化

Question

我正在构建一个网站geekwiz.com。测试加载它，LONG。

我是新来的PHP，只是从mysql_query更改为PDO。为了在第一页显示产品，我必须做多个查询，并且该表有超过150万条记录。

我该如何让这个更快？

这里是我创建的代码：

#connection 
$DBH = new PDO("mysql:host=$host;dbname=$dbname", $user, $pass); 
$DBH->setAttribute(PDO::ATTR_EMULATE_PREPARES, 0); 

单独的文件

#fetch data 
require_once('db_config.php'); 
$STH = $DBH->query(" 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('notebook pc - other') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Desktop PC') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Hard Drives - External') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Camcorders - Analog/Digital') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Monitors - LCD Flat Panel') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Software - PC Games') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('third party accessories') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE advertisercategory LIKE ('LED TV') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 3; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Motherboards - %') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE advertisercategory LIKE ('LED TV') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Hard Drives - External') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('CPU Cooling') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('CPU Thermal Paste/Grease') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Keyboards') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (USB Flash Drive)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (Desktop Memory)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Memory (Notebook Memory)') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Mouse') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Printer - Inkjet Printers') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Projectors') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 
SELECT name, buyurl, imageurl, price, manufacturer, manufacturerid FROM catalog WHERE MATCH advertisercategory AGAINST ('Processors - Desktops') AND special = 'YES' AND instock = 'YES' AND standardshippingcost LIKE '0' ORDER BY lastupdated DESC limit 9; 

"); 

$STH->setFetchMode(PDO::FETCH_OBJ); 

while ($row = $STH->fetch()) { 
echo "<ul>"; 
echo "<li>" .$row->name ."</li>"; 
echo "<li>" .$row->buyurl ."</li>"; 
echo "<li>" .$row->imageurl ."</li>"; 
echo "<li>" .$row->price ."</li>"; 
echo "<li>" .$row->manufacturer ."</li>"; 
echo "<li>" .$row->manufacturerid ."</li>"; 
echo "</ul>"; 
} 

Answer 1

嗯，起动机，PDO有绝对无关，与大型数据库的速度优化。
为了让它更清楚一点，想象一下，你正在问一个类似于“如何驾驶我的福特Escort从纽约布鲁克林到佛罗里达州坦帕”的问题。汽车制造在这里是无关紧要的。数据库驱动程序也一样。

现在到您的数据库。
您有很多工作要做。

首先，绝对没有办法使用Mysql FULLTEXT进行分类。它应该只是一个数字索引，链接到具有分类结构的表格。

因此，查询变成这样一个

SELECT * FROM catalog WHERE category=1 AND special = 'YES' AND instock = 'YES' 
AND standardshippingcost=0 ORDER BY lastupdated DESC limit 3; 

而且，如果适当的索引，它可以非常快。比方说，如果特殊优惠的数量是有限的，那么无论数据库中有多少条记录，该字段上的索引都会使其更快。

但是，整体数据库体系结构可能会有所不同。以及可以有其他方式进行优化。但是我们不太了解你的项目是否可以肯定，而且说实话，这不是简单的格式问题。

Answer 2

编写一个名为saveCacheData.php的php脚本，执行该组sql查询，该查询也将数据转储到名为cacheData.php之类的文件中。

在主页上，用require_once('cacheData.php')替换sql查询。

现在，当曾经在mysql表catalog数据更新，请确保您还执行file_get_contents('saveCacheData.php')（或执行某种cron作业的将缓存中的数据你）。

顺便说一句，我认为你的数据库设计可能会好很多，但将数据缓存到磁盘通常也是一个好主意。

Answer 3

有几个问题，你可能不会喜欢我要说的。

通过文本/ varchar排序非常缓慢，并且更新将持续进行。你应该做的是将所有文本转换为整数。所以advertisercategory不应该有其中的实际文字。相反，它应该是固定长度INT，链接到存储实际文本的表格。 special和instock也是如此。standardshippingcost看起来像一个varchar，因为它是用单引号引起来的，如果它看起来像这样，那真的不应该是这种情况。它应该是数字，如果它是数字。我不太清楚你的模式，所以standardshippingcost可能需要在它自己的表中由FK链接到catalog。在unix时间，lastupdated应该是INT。

之所以你的数据应该是INT是因为：1）它需要更少的空间（从而在排序时内存小于）notebook pc - other和2）更新会更快一些（我假设你正在使用InnoDB的）。

所以，当你排序，SELECT所有id S的（这应该是一个自动增加PRIMARY），advertisercategory，instock，standardshippingcost，standardshippingcost，并且lastupdated到一个PHP数组。找到最有效的方式来获得你想要的东西（你会发现捷径）。最后，SELECT的具体数据按PRIMARY键。

我会倒退解释。这种方法会产生最高性能的原因是因为如果您在没有任何其他索引的InnoDB表上关闭SELECTSELECT，表的大小无关紧要，它将立即返回。正如你所经历的那样，使用巨大的，深奥的查询是很慢的。这是你应该如何检索数据。

PHP比具有逻辑的MySQL更快。这是它的目的。堆栈中有很多Q显示了这一点。这就是为什么你想对PHP进行排序的原因。此外，如果您在数据中看到某种统计或其他类型的模式，则实际上可以编写更好的算法来使分类运行更快。

除了unix日期之外，其他列不应该太长。 advertisercategory可能会变大一点，但与notebook pc - other相比，它已经是19的长度了。我在您的Q中看到约30 advertisercategory s，因此这只是一个INTLENGTH2因为这是100个潜在值。这将在分类时节省空间和内存。

此外，更新时使用固定长度整数会更快，因为群集表将始终保持自己的紧凑和碎片整理。如果你的UPDATE在集群InnoDB中间的一行从'Memory (USB Flash Drive)'到'Projectors'，你只是导致你的数据库整理你的整个数据库，以消除你刚才的差距。如果在固定长度的int列中从一个1更改为10，则不会发生这种情况，因为它们都占用相同的空间量，不会造成间隙，因此不需要对群集进行碎片整理。

当你这样做时，你会对性能的大幅提升感到震惊，但是，这将涉及更多的代码。那么你的优先级，性能或开发时间是什么？

---

呐喊！忘记unix时间的解释。您希望使用此功能，因为您可以将其存储在INT秒，DECIMAL为微秒。这样做可以在PHP中进行最简单的操作，因为php在unix中。