2016-11-30 42 views
1

我目前有两个大型数据集(大约200GB),我想将其存储到ContextBroker中,以便对其执行数据分析。到目前为止,我已经能够设置ContextBrokerMongoDB。我还安装了Cygnus订阅ONCHANGE为了坚持数据从ContextBroker。是否可以使用CKAN将我的外部数据集存储到FIWARE平台中,还是必须使用其他内容?将数据集导入ContextBroker

EDIT1:这些数据集是一组包含一系列事件的对象,是timestampedgeo-locatedjson文件。

EDIT2:由于请求的数据的一个小样本我指的是:

{"track":[ 
{"time":"2015-11-16T00:45:29.016","midpoint":{"x":70.66,"y":188.90},"realworld":{"x":-7.49,"y":22.96},"RDCoordinate":{"x":161582.09,"y":383031.34},"UTM-WGS84":{"Latitude":51.4363489,"Longitude":5.4818640}}, 
{"time":"2015-11-16T00:45:29.022","midpoint":{"x":66.85,"y":189.61},"realworld":{"x":-7.53,"y":22.92},"RDCoordinate":{"x":161582.16,"y":383031.34},"UTM-WGS84":{"Latitude":51.4363489,"Longitude":5.4818649}}, 
{"time":"2015-11-16T00:45:29.029","midpoint":{"x":61.66,"y":189.92},"realworld":{"x":-7.60,"y":22.90},"RDCoordinate":{"x":161582.22,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818658}}, 
{"time":"2015-11-16T00:45:29.082","midpoint":{"x":56.16,"y":190.47},"realworld":{"x":-7.66,"y":22.87},"RDCoordinate":{"x":161582.28,"y":383031.28},"UTM-WGS84":{"Latitude":51.4363483,"Longitude":5.4818667}}, 
{"time":"2015-11-16T00:45:29.090","midpoint":{"x":50.74,"y":191.64},"realworld":{"x":-7.71,"y":22.80},"RDCoordinate":{"x":161582.38,"y":383031.28},"UTM-WGS84":{"Latitude":51.4363483,"Longitude":5.4818681}}, 
{"time":"2015-11-16T00:45:29.112","midpoint":{"x":45.58,"y":192.07},"realworld":{"x":-7.78,"y":22.78},"RDCoordinate":{"x":161582.44,"y":383031.25},"UTM-WGS84":{"Latitude":51.4363480,"Longitude":5.4818690}}, 
{"time":"2015-11-16T00:45:29.151","midpoint":{"x":41.80,"y":193.42},"realworld":{"x":-7.80,"y":22.69},"RDCoordinate":{"x":161582.52,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818701}}, 
{"time":"2015-11-16T00:45:29.197","midpoint":{"x":36.27,"y":194.43},"realworld":{"x":-7.86,"y":22.63},"RDCoordinate":{"x":161582.59,"y":383031.31},"UTM-WGS84":{"Latitude":51.4363486,"Longitude":5.4818712}} 
]} 
+0

你可以编辑你的问题,以阐述你的数据集是怎样的?我的意思是,它们对应于上下文信息的“拍摄”,事件的历史序列等。 – fgalan

+0

(查看Edit1之后)您可以编辑添加一个或两个您引用的JSON对象吗? – fgalan

回答

0

甲posibility可能是每个项目建模为在猎户Conext下列方式的Event类型的实体经纪人:

{ 
    "id": "Item1", 
    "type": "Event", 
    "time": { 
     "value": "2015-11-16T00:45:29.016", 
     "type": "DateTime" 
    }, 
    "midpoint": { 
     "value": { 
      "x": 70.66, 
      "y": 188.90 
     }, 
     "type": "Object" 
    }, 
    "realworld": { 
     "value": { 
      "x": -7.49, 
      "y": 22.96 
     }, 
     "type": "Object" 
    }, 
    "RDCoordinate": { 
     "value": { 
      "x": 161582.09, 
      "y": 383031.34 
     }, 
     "type": "Object" 
    }, 
    "UTM-WGS84": { 
     "value": "51.4363489, 5.4818640", 
     "type": "geo:point" 
    } 
} 

(以上可作为有效载荷为POST /v2/entities操作来创建实体)。

在这个模型中,timemidpointrealworldRDCoordinateUTM-WG84是实体属性。请注意,特殊的属性类型timeUTM-WG84

  • time使用DateTime,这是NGSIv2标准的方式来指定日期(见NSGIv2 specification document“特殊属性类型”)。我建议加入时区,以获得更高的精度(例如2015-11-16T00:45:29.016Z)。

  • UTM-WG84使用geo:point,这是NGSIv2标准方式来指定实体的点位置(请参阅NSGIv2 specification document中的“实体的地理空间属性”)。

+0

嗨fgalan。谢谢您的回复!这看起来确实是我正在寻找的东西,但是这需要很多工作来插入所有数据。是否有可能自动完成这项工作? –

+0

处理JSON文件中的每行数据并向Orion发送相应的创建实体命令的脚本应该不是很困难(即使您可以考虑使用POST/v2/op /更新'与APPEND actionType)。 – fgalan