Logstash无法添加字段？

我一直在使用logstash来读取一些数据库恢复日志。以下是一些样本记录。Logstash无法添加字段？

07/08/2016 6:33:22.50: START restore database      
SQL2540W Restore is successful, however a warning "2539" was encountered 
during Database Restore while processing in No Interrupt mode. 
07/08/2016 6:33:28.93: END restore database       
SQL4406W The DB2 Administration Server was started successfully. 
07/08/2016 6:35:35.29: END restart server       
connect reset 
DB20000I The SQL command completed successfully. 
07/08/2016 6:35:38.48: END p:\s6\source\system\CMD\res_uw.cmd

这是我的conf文件的过滤器部分。

if ([message] =~ /Backup successful/){ 
    grok{ 
     match => {"message" => ['%{GREEDYDATA:Message}'] } 
    } 
    mutate { 
     add_tag => "send_to_es" 
     add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"} 
    } 
} 
if ([message] =~ /warning "2539"/){ 
    grok{ 
     match => {"message" => ['%{GREEDYDATA:Message}'] } 
    } 
    mutate { 
     add_tag => "send_to_es" 
     add_field => {"Timestamp" => "%{GREEDYDATA:DATETIME}"} 
    } 
} 
if ([message] =~ /(END p:|END P:)/){ 
    grok{ 
     match => {"message" => ['%{GREEDYDATA:DATETIME}:%{SPACE}END%{SPACE}%{GREEDYDATA:Mis}'] } 
     remove_field => "%{GREEDYDATA:Mis}" 
    } 
    mutate { 
     add_tag => "send_to_es" 
    } 
}

我希望将数据“DATETIME”从我的纪录的最后一行提取到的消息在同一时间加入到其他邮件索引。但是，它无法成功添加该字段。输出将变成

 "message": "SQL2540W Restore is successful, however a warning \"2539\" was encountered \r\r", 
     "@version": "1", 
     "@timestamp": "2016-07-12T02:28:52.337Z", 
     "path": "C:/CIGNA/hkiapp67_db_restore/res_uw.log", 
     "host": "SIMSPad", 
     "type": "txt", 
     "Message": "SQL2540W Restore is successful, however a warning \"2539\" was encountered \r\r", 
     "Timestamp": "%{GREEDYDATA:DATETIME}", 
     "tags": [ 
     "send_to_es" 
     ]

我怎么能解决这个问题？

来源

2016-07-12 Kennedy Kan

当接收到一条线时，Logstash不知道任何其他线。您将不得不使用多行编解码器/过滤器来重新组合使用日期行的所有行。然后，您使用Grok过滤器提取日期并将其添加到文档中。

多行编解码器/过滤器的结构将是这样的：

multiline { 
    pattern => "%{DATE}" 
    negate => "true" 
    what => "next" 
}

借助于此，不与图案DATE开头的所有线将与下一行接合。

来源

2016-07-12 11:44:38 baudsp

这是聚合函数也可以这样做吗？ –

@KennedyKan从我在文档中读到的内容，它应该能够做到。但我从来没有使用过滤器，所以我可以帮助你 – baudsp

Logstash无法添加字段？

回答

相关问题