2013-07-25 34 views
0

多年来一直在这里读书,喜欢它。我自己的小问题的时间。Wget Cookie会话设置为DUMMY,302重定向问题?

我们曾使用wget SSH从供应商的后端检索产品信息(价格,可用性)。不幸的是,他们改变了他们的方式,瘫痪了我的数据流:/

我已经修复了一些,但我无法获取所有数据,因为我的cookie不能很好地存储了。阅读所有Google结果,尝试所有结果,但我的知识欠缺。我会坚持饼干设置的一部分,我敢打赌,如果这是固定的,我可以再次检索所有文件的顺序也很好。

这是我使用,我没有信誉点,所以我不能添加多个链接,我会用“URL-这里”作为替补我https://www.supplier.com

该Cookie给出了这样的:

# HTTP cookie file. 
# Generated by Wget on 2013-07-25 06:03:44. 
# Edit at your own risk. 

www.supplier.com FALSE / FALSE 0 PHPSESSID DUMMY 

通过浏览器一个cookie显示会话的值,这部分似乎没有再被写为:/

也试过这个,都具有相同的RESU LT:

/usr/local/bin/wget -O /dev/null --cookies=on --keep-session-cookies --save-cookies=cookiepp.txt --referer='url-here/auth/login.php' --post-data='login=123654&password=123654' 'url-here/auth/login.php' 

而且也试过这种直接与手动保存cookie来retireve:

wget --no-check-certificate --span-hosts --header "Cookie:PHPSESSID=h771tqr0spe8ufbq6fash2msf6" "url-here/sortment/s/?group="$i"&excel=1" 

最后,SSH命令行流量:

--2013-07-25 06:03:43-- https://www.supplier.com/auth/login.php 
Resolving www.supplier.com (www.supplier.com)... 00.000.000.0 
Connecting to www.supplier.com (www.supplier.com)|00.000.000.0|:443... connected. 
HTTP request sent, awaiting response... 302 Found 
Location: https://www.supplier.com/auth/login.php [following] 
--2013-07-25 06:03:44-- https://www.supplier.com/auth/login.php 
Reusing existing connection to www.supplier.com:443. 
HTTP request sent, awaiting response... 200 OK 
Length: 6884 (6.7K) [text/html] 
Saving to: '/dev/null' 

最后的调试信息:

Setting --output-document (outputdocument) to /dev/null 
Setting --cookies (cookies) to on 
Setting --keep-session-cookies (keepsessioncookies) to 1 
Setting --save-cookies (savecookies) to cookiepp.txt 
Setting --referer (referer) to https://www.supplier.com/auth/login.php 
Setting --post-data (postdata) to login=123654&password=123654 
DEBUG output created by Wget 1.14 on freebsd9.1. 

URI encoding = 'US-ASCII' 
--2013-07-25 07:01:30-- https://www.supplier.com/auth/login.php 
Resolving www.supplier.com (www.supplier.com)... 89.105.214.133 
Caching www.supplier.com => 89.105.214.133 
Connecting to www.supplier.com (www.supplier.com)|89.105.214.133|:443... connected. 
Created socket 5. 
Releasing 0x00000008021baa40 (new refcount 1). 
Initiating SSL handshake. 
Handshake successful; connected socket 5 to SSL handle 0x000000080204c400 
certificate: 
    subject: /C=NL/postalCode=7271LB/ST=Gelderland/L=Borculo/street=Hesselinks Es 11/O=Varuvo bv/OU= /OU=Hosted by WideXS/OU=InstantSSL/CN=www.supplier.com 
    issuer: /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO High-Assurance Secure Server CA 
X509 certificate successfully verified and matches host www.supplier.com 

---request begin--- 
POST /auth/login.php HTTP/1.1 
Referer: https://www.supplier.com/auth/login.php 
User-Agent: Wget/1.14 (freebsd9.1) 
Accept: */* 
Host: www.supplier.com 
Connection: Keep-Alive 
Content-Type: application/x-www-form-urlencoded 
Content-Length: 32 

---request end--- 
[POST data: login=123654&password=123654] 
HTTP request sent, awaiting response... 
---response begin--- 
HTTP/1.1 302 Found 
Date: Thu, 25 Jul 2013 14:01:32 GMT 
Server: Apache 
P3P: CP="NOI DSP COR NID CURa OUR NOR STA" 
Set-Cookie: PHPSESSID=DUMMY; path=/ 
Status: 302 Found 
Location: https://www.supplier.com/auth/login.php 
Vary: Accept-Encoding 
Content-Length: 0 
Keep-Alive: timeout=15, max=100 
Connection: Keep-Alive 
Content-Type: text/html; charset=Windows-1252 

---response end--- 
302 Found 

Stored cookie www.supplier.com -1 (ANY)/<session> <insecure> [expiry none] PHPSESSID DUMMY 
Registered socket 5 for persistent reuse. 
URI content encoding = 'Windows-1252' 
Location: https://www.supplier.com/auth/login.php [following] 
] done. 
URI content encoding = None 
--2013-07-25 07:01:31-- https://www.supplier.com/auth/login.php 
Reusing existing connection to www.supplier.com:443. 
Reusing fd 5. 

---request begin--- 
GET /auth/login.php HTTP/1.1 
Referer: https://www.supplier.com/auth/login.php 
User-Agent: Wget/1.14 (freebsd9.1) 
Accept: */* 
Host: www.supplier.com 
Connection: Keep-Alive 
Cookie: PHPSESSID=DUMMY 

---request end--- 
HTTP request sent, awaiting response... 
---response begin--- 
HTTP/1.1 200 OK 
Date: Thu, 25 Jul 2013 14:01:32 GMT 
Server: Apache 
P3P: CP="NOI DSP COR NID CURa OUR NOR STA" 
Set-Cookie: PHPSESSID=DUMMY; path=/ 
Vary: Accept-Encoding 
Content-Length: 6884 
Keep-Alive: timeout=15, max=99 
Connection: Keep-Alive 
Content-Type: text/html; charset=Windows-1252 

---response end--- 
200 OK 
Deleted old cookie (to be replaced.) 

Stored cookie www.supplier.com -1 (ANY)/<session> <insecure> [expiry none] PHPSESSID DUMMY 
URI content encoding = 'Windows-1252' 
Length: 6884 (6.7K) [text/html] 
Saving to: '/dev/null' 

100%[===================================================================================================================>] 6,884  43.1KB/s in 0.2s 

2013-07-25 07:01:32 (43.1 KB/s) - '/dev/null' saved [6884/6884] 

Saving cookies to cookiepp.txt. 
Done saving cookies. 

我真的是我失去了这一点,在这一点上我希望得到别人的帮助。

回答

1

固定它使用卷曲:虽然

/usr/local/bin/curl -k -c cookie.txt -d "login_alias=464" -d "password=6446" -d "from=%2F" https://www.supplier.com/auth/login.php 
paste theCodesOnePerLine theNamesOnePerLine | 
while read i j 
do 
/usr/local/bin/curl -k -L -b cookie.txt -o /usr/home/$j.xls 'https://www.supplier.com/?group='$i'&excel=1' 
done 

感谢,对页面空间:P