我想查询大量数据到服务器,这里是我的代码:MySQL查询执行失败
$queryString = "";
$connect = mysqli_connect("localhost", "username", "password", "database");
$loopLength = 20;
$currentGroup = 1;
$currentLoopAmount = 0;
$queryAmount = 5;
for($v = 0; $v < ceil($loopLength/$queryAmount); $v++){
//echo "Looping Main: " . $v . "<br>";
//echo $loopLength - (($currentGroup - 1) * 10) . "<br>";
if($loopLength - (($currentGroup - 1) * $queryAmount) >= $queryAmount){
$currentLoopAmount = $queryAmount;
}
else{
$currentLoopAmount = $loopLength - (($currentGroup - 1) * $queryAmount);
}
//echo $currentLoopAmount;
$queryString = "";
for($q = (($currentGroup - 1) * $queryAmount); $q < $currentLoopAmount + (($currentGroup - 1) * $queryAmount); $q++){
//echo " Looping Sub: " . $q . "<br>";
$tempVariable = grabPageData($URLs[$q], $q);
$queryString .= $tempVariable;
if($q < $loopLength-1){
$queryString .= ",";
}
else{
$queryString .= ";";
}
}
echo $queryString;
$query = "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES " . $queryString;
$result = mysqli_query($connect, $query);
if($result){
echo "Success";
}
else{
echo "Failed : " . mysqli_error($connect) . "<br>";
}
$currentGroup += 1;
}
的$loopLength
变量是动态的,可以在数千或潜在十万。我设计了这个功能,将这个庞大的数字划分为一批较小的查询,因为我无法一次性通过GoDaddy在我的共享托管服务上传所有数据。变量$queryAmount
代表较小的查询有多大。
下面是插入到表中的其中一个值集的示例: 这是公告中的数据,表示我的代码在grabPageData()
函数中检索到。
('http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=37/7292017_24266919.htm','Pima','Green Valley News and Sun','2017/07/30',' ___________________________ARIZONA SUPERIOR COURT, PIMA COUNTYIn the Matter of the Estate of:JOSEPH T, DILLARD, SR.,Deceased.DOB: 10/09/1931No. PB20170865NOTICE TO CREDITORS(FOR PUBLICATION)NOTICE IS HEREBY GIVEN that DANA ANN DILLARD CALL has been appointed Personal Representative of this Estate. All persons having claims against the Estate are required to present their claimswithin four months after the date of the firat publication of this notice or the claims will be forever barred. Claims must be presented by delivering or mailing a written statement of the claim to the Personal Representative at the Law Offices of Michael W. Murray, 257 North Stone Avenue, Tucson, Arizona 85701.DATED this 17th day of July, 2017./S/ Micahel W. MurrayAttorney for the Personal RepresentativePub: Green Valley News & SunDate: July 23, 30, August 6, 2017 Public Notice ID: 24266919',' 24266919'),
为了获得这些数据,我通过抓取页面并抓取它的函数来运行它。然后我通过这个功能把网页html代码:
function cleanData($data){
$data = strip_tags($data);
//$data = preg_replace("/[^a-zA-Z0-9]/", "", $data);
//$data = mysql_real_escape_string($data);
return $data;
}
这给了我没有标签的内容,如你所见。这是问题。
该函数执行,一切似乎只是丹迪。然后(我不保持在10题的缘故取决于$queryAmount
变量)的输出,你可以看到它会在功能类似的功能...
Failed : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
怪异的是,当我有大量的数据,比如说$loopLength
变量就像116.结果会输出,“失败:(错误)失败:(错误)辉...(错误)成功所以它只是实际查询最后一组数据???不确定
我不知道如何解决这个问题,我想要一个新鲜的眼睛,有人可以帮助我,我一直在努力解决这个问题...几个小时,试图找到解决方案
对不起,让这个问题的一个痛苦的对接:(
编辑:
我改变了代码从以前用mysql准备语句并没有什么...见下文:
$grabDataResults = [
"url" => "",
"county" => "",
"paperco" => "",
"date" => "",
"notice" => "",
"id" => "",
];
$connect = mysqli_connect("localhost", "bwt_admin", "Thebeast1398", "NewCoDatabase");
if($stmt = mysqli_prepare($connect, "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES (?, ?, ?, ?, ?, ?)")){
mysqli_stmt_bind_param($stmt, 'ssssss', $grabDataResults["url"], $grabDataResults["county"], $grabDataResults["paperco"], $grabDataResults["date"], $grabDataResults["notice"], $grabDataResults["id"]);
$loopLength = 1;
for($v = 0; $v < $loopLength; $v++){
$grabDataResults = grabPageData($URLs[$v], $v);
mysqli_stmt_execute($stmt);
printf("%d Row inserted.\n", mysqli_stmt_affected_rows($stmt));
printf("Error:\n", mysqli_stmt_error($stmt));
echo "(" . $grabDataResults["url"] . "," . $grabDataResults["county"] . "," . $grabDataResults["paperco"] . "," . $grabDataResults["date"] . "," . $grabDataResults["notice"] . "," . $grabDataResults["id"] . ")";
}
mysqli_stmt_close($stmt);
mysqli_close($connect);
}
不幸的是,这是我从输出获得:
1 Row inserted. 0 Error:
没有错误实际打印出来,该行是在serted。但是,当我导航到我的数据库,并查看已存储的值..他们都是空的。 echo语句输出这样的:
(http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=31/7292017_24266963.htm,Yuma,Sun (Yuma), The,2017/07/30,, 24266963)
所以我知道所有变量包含除$notice
变量,它得到由我cleanData()
功能出于某种原因被破坏的东西。
FRO对目前为止的问题的评论 - 我肯定会使用准备好的语句并绑定这些值。如果您的数据来自网页,那么如果直接插入数据会导致您的SQL发生中断的可能性非常高,https://stackoverflow.com/questions/9629328/how-to-use-mysqli-prepared-statements-in -php可能会有帮助。 –
我做了一些关于创建预处理语句的研究,并为了不直接插入而对值进行了绑定......看起来像是我的代码试图动态生成查询语句和东西将是一个巨大的混乱。我想我可以尝试一下,但就像我说过的,我可能会产生成千上万的数据行,不知道绑定语句会是什么样子,更不用说查询本身了。 – Pixelknight1398
您可以使用1行插入来准备语句,然后每行数据只需执行准备好的语句和一行数据。 –