我试图解析大Apache日志,但import_logs.py失败,此错误:Import_logs.py失败(大于900 MB)
2017年1月23日18: 30:39245:达到尝试[INFO]最大数,服务器无法访问_
Fatal error: HTTP Error 500 Internal Server Error, response:
{"status":"error","tracked":0,"invalid":0,"invalidindices":[]}_ You
can restart the import of
"/awdata/piwik/cosmote/geratgweb04/www.cosmote.grwith_ssl-access.log-20170123" from the point it failed by specifying --skip=78454 on the command line.
我使用的是虚拟机与8个CPU和8 GB MEM!
我的OS版本RHEL 7.3,
与Apache版本的Apache/2.4.6(红帽企业Linux),
PHP是PHP 7.0.14和MariaDB的版本是52年5月5日 Piwik 3.0.1版
我使用的命令是:
/var/www/html/zak/piwik/misc/log-analytics/import_logs.py --url=http://middlinf.ote.gr:81/piwik/ --idsite=6 --recorders=8 --enable-http-errors --enable-http-redirects --enable-static /awdata/piwik/cosmote/geratgweb04/www.cosmote.gr_with_ssl-access.log-2017--retry-max-attempts=20
Apache的错误日志中显示:
[Tue Jan 24 13:44:33.855134 2017] [:error] [pid 17456:tid 140544525190912] [client 172.18.20.26:16610] Piwik(tracker)中的错误:错误查询:SQLSTATE [40001]:序列化失败:1213尝试锁定时发现死锁;尝试重新启动事务在查询:
UPDATE piwik_log_visit SET idvisitor = ?, user_id = ?,
visit_last_action_time = ?, visit_exit_idaction_url = ?,
visit_total_actions = visit_total_actions + 1 ,
visit_total_interactions = visit_total_interactions + 1 ,
visit_total_time = ?
WHERE idsite = ? AND idvisit = ?
参数:阵列(0 => '\ xa6 \ X96 \ XBC \ XEF \ xb9 \ XDE \ XF5',1 => ' “”',2 =>“2017 -01-23 8时14分43' 秒,3 => 47298,4 => 0,5 => 4,6 => 11224,)
和:
[星期二年01月24 13点51:错误查询:SQLSTATE [HY000]:常规错误:1205超出锁定超时等待时间;错误代码:SQLSTATE [HY000]:常规错误:尝试重新启动事务在查询:
UPDATE piwik_log_visit SET idvisitor = ?, user_id = ?,
visit_last_action_time = ?, visit_exit_idaction_url = ?,
visit_total_actions = visit_total_actions + 1 ,
visit_total_interactions = visit_total_interactions + 1 ,
visit_total_time = ?
WHERE idsite = ? AND idvisit = ?
参数:阵列(0 => '\ xa6 \ X96 \ XBC \ XEF \ xb9 \ XDE \ XF5',1 => ' “”',2 =>“2017 -01-23 8时42分33' 秒,3 => 49791,4 => 242,5 => 4,6 => 11371)>
and mariadb log is showing: Time: 17:00:46 [email protected]: root[root] @ localhost [127.0.0.1] Thread_id: 1691 Schema: piwik_db QC_hit: No Query_time: 3.858223 Lock_time: 0.000060 Rows_sent: 1 Rows_examined: 1
SET timestamp=1485187246;
SELECT visit_last_action_time, visit_first_action_time, idvisitor,
idvisit, user_id, visit_exit_idaction_url, visit_exit_idaction_name,
visitor_returning, visitor_days_since_first, visitor_days_since_order,
visitor_count_visits, visit_goal_buyer, location_country,
location_region, location_city, location_latitude, location_longitude,
referer_name, referer_keyword, referer_type, idsite,
visit_entry_idaction_url, visit_total_actions,
visit_total_interactions, visit_total_searches, config_device_brand,
config_device_model, config_device_type, visit_total_events,
visit_total_time, location_ip, location_browser_lang, custom_var_k1,
custom_var_v1, custom_var_k2, custom_var_v2, custom_var_k3,
custom_var_v3, custom_var_k4, custom_var_v4, custom_var_k5,
custom_var_v5 FROM piwik_log_visit
WHERE visit_last_action_time >= '2017-01-22 05:14:21'
AND visit_last_action_time <= '2017-01-22 06:14:21'
AND idsite = '6' AND idvisitor = '
ORDER BY visit_last_action_time DESC
我做了一些研究在论坛中的但没有发现任何有趣的事情。 你有什么建议吗?
预先感谢您。
-Thanassis
非常感谢您的回复。这是SHOW CREATE TABLE piwik_log_visit的输出。 https://docs.google.com/document/d/1po0mmh9oxXXmCebns9392fxRpzPHWEYGd02eyYO6HjY/edit确实没有这样的索引。我将尝试创建它并重新运行日志导入。我会尽快更新你的。关于间隔模式,我不知道该怎么做,因为它是从piwik命令import_logs.py中手工完成的。这不是一个自定义查询。 –
抱怨piwik。 –