logstash和elasticsearch geo_point

我正在使用logstash将地理空间数据从csv输入到elasticsearch中作为geo_points。logstash和elasticsearch geo_point

的CSV如下所示：

$ head -5 geo_data.csv 
"lon","lat","lon2","lat2","d","i","approx_bearing" 
-1.7841,50.7408,-1.7841,50.7408,0.982654,1,256.307 
-1.7841,50.7408,-1.78411,50.7408,0.982654,1,256.307 
-1.78411,50.7408,-1.78412,50.7408,0.982654,1,256.307 
-1.78412,50.7408,-1.78413,50.7408,0.982654,1,256.307

我必须创建一个看起来像下面这样的映射模板：

$ cat map_template.json 

{ 
    "template": "base_map_template", 
    "order": 1, 
    "settings": { 
    "number_of_shards": 1 
    }, 
    { 
     "mappings": { 
     "base_map": { 
      "properties": { 
      "lon2": { "type" : "float" }, 
      "lat2": { "type" : "float" }, 
      "d": { "type" : "float" }, 
      "appox_bearing": { "type" : "float" }, 
      "location": { "type" : "geo_point" } 
      } 
     } 
     } 
    } 
}

为logstash我的配置文件已被设置如下：

$ cat map.conf 

input { 
    stdin {} 
} 

filter { 
    csv { 
     columns => [ 
     "lon","lat","lon2","lat2","d","i","approx_bearing" 
     ] 
    } 

    if [lon] == "lon" { 
     drop { } 
    } else { 
     mutate { 
      remove_field => [ "message", "host", "@timestamp", "@version" ] 
     } 

     mutate { 
      convert => { "lon" => "float" } 
      convert => { "lat" => "float" } 
      convert => { "lon2" => "float" } 
      convert => { "lat2" => "float" } 
      convert => { "d" => "float" } 
      convert => { "i" => "integer"} 
      convert => { "approx_bearing" => "float"} 
     } 

     mutate { 
      rename => { 
       "lon" => "[location][lon]" 
       "lat" => "[location][lat]" 
      } 
     } 
    } 
} 

output { 
# stdout { codec => rubydebug } 
    stdout { codec => dots } 
    elasticsearch { 
     index => "base_map" 
     template => "map_template.json" 
     document_type => "node_points" 
     document_id => "%{i}" 
    } 
}

然后我尝试使用logstash将csv数据输入到elasticsearch中作为geo_点使用以下命令：

$ cat geo_data.csv | logstash-2.1.3/bin/logstash -f map.conf

我得到以下错误：

Settings: Default filter workers: 16 
Unexpected character ('{' (code 123)): was expecting double-quote to start field name 
at [Source: [[email protected]; line: 7, column: 3]{:class=>"LogStash::Json::ParserError", :level=>:error} 
Logstash startup completed 
....Logstash shutdown completed

我缺少什么？

来源

2016-03-06 h.l.m

你碰巧在'geo_data.csv'中有'{'字符吗？或者可能是一个记录，其中'i'单元格是空白的？ – Val

我不这么认为，对我来说都很好......即使我运行'head -5 geo_data.csv | logstash-2.1.3/bin/logstash -f map.conf' –

您的CSV文件开头看起来有空行。如果你删除它会发生什么？ – Val

任性“{”在你的模板文件的第7行

来源

2016-03-07 11:10:56 darktachyon

logstash和elasticsearch geo_point

回答

相关问题