登录后用cURL从网站抓取数据？

Question

我试图做的是登录到一个网站，然后去抓取表中的数据，因为它们没有导出功能。到目前为止，我已设法登录，并向我显示用户主页。不过，我需要导航到不同的页面或以某种方式抓取该页面，同时仍然使用curl登录。

到目前为止我的代码：

$username="email"; 
$password="password"; 
$url="https://jiltapp.com/sessions"; 
$cookie="cookie.txt"; 
$url2 = "https://jiltapp.com/shops/shopname/orders"; 

$postdata = "email=".$username."&password=".$password; 

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_REFERER, $url); 

curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch); 

echo $result; 
curl_close($ch); 

正如我所说，我得到访问主要用户页面，但我需要抢$ URL2变量，而不是$ URL的内容。我怎么能做到这样的事情？

谢谢！

Answer 1

登录后，再次请求包含您之后数据的页面。

对于后续的请求，您必须设置指向与CURLOPT_COOKIEJAR相同的文件的选项CURLOPT_COOKIEFILE。 cURL将从该文件中读取cookie并将其发送给请求。

$username="email"; 
$password="password"; 
$url="https://jiltapp.com/sessions"; 
$cookie="cookie.txt"; 
$url2 = "https://jiltapp.com/shops/shopname/orders"; 

$postdata = "email=".$username."&password=".$password; 

$ch = curl_init(); 
curl_setopt ($ch, CURLOPT_URL, $url); 
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); 
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6"); 
curl_setopt ($ch, CURLOPT_TIMEOUT, 60); 
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie); 
curl_setopt ($ch, CURLOPT_COOKIEFILE, $cookie); // <-- add this line 
curl_setopt ($ch, CURLOPT_REFERER, $url); 

curl_setopt ($ch, CURLOPT_POSTFIELDS, $postdata); 
curl_setopt ($ch, CURLOPT_POST, 1); 
$result = curl_exec ($ch); 

echo $result; 

// make second request 

$url = 'page you want to get data from'; 
curl_setopt($ch, CURLOPT_URL, $url); 
curl_setopt($ch, CURLOPT_POST, 0); 

$data = curl_exec($ch);