我试图通过使用python 2.7和Apache Avro(python客户端)的kafka代理交换序列化的消息。我想知道在没有创建模式之前是否有交换消息的方法。Python - 无模式Apache Avro数据序列化
这是代码(使用模式,sensor.avsc,我想避免的东西):
from kafka import SimpleProducer, KafkaClient
import avro.schema
import io, random
from avro.io import DatumWriter
# To send messages synchronously
kafka = KafkaClient('localhost:9092')
producer = SimpleProducer(kafka, async = False)
# Kafka topic
topic = "sensor_network_01"
# Path to user.avsc avro schema that i don't want
schema_path="sensor.avsc"
schema = avro.schema.parse(open(schema_path).read())
for i in xrange(100):
writer = avro.io.DatumWriter(schema)
bytes_writer = io.BytesIO()
encoder = avro.io.BinaryEncoder(bytes_writer)
# creation of random data
writer.write({"sensor_network_name": "Sensor_1", "value": random.randint(0,10), "threshold_value":10 }, encoder)
raw_bytes = bytes_writer.getvalue()
producer.send_messages(topic, raw_bytes)
这是sensor.avsc文件:
{
"namespace": "sensors.avro",
"type": "record",
"name": "Sensor",
"fields": [
{"name": "sensor_network_name", "type": "string"},
{"name": "value", "type": ["int", "null"]},
{"name": "threshold_value", "type": ["int", "null"]}
]
}