0
我有一个在CDH 5.8.3中运行的flume代理程序。如果发送超过3个有效文件，它在写入hdfs时会创建多个.tmp文件。有一个拦截器可以在hdfs接收器之前将有效的xmls传递给适当的主题。此代理正在使用flafka。拦截器和卡夫卡正在正常工作。连续发送数据时生成多个.tmp文件的Flume代理程序
agent.sinks.hdfs_valid.channel=valid_channel
agent.sinks.hdfs_valid.type=hdfs
agent.sinks.hdfs_valid.writeFormat=Text
agent.sinks.hdfs_valid.hdfs.fileType=DataStream
agent.sinks.hdfs_valid.hdfs.filePrefix=event
agent.sinks.hdfs_valid.hdfs.fileSuffix=.xml
agent.sinks.hdfs_valid.hdfs.path=locationoffile/%{time}
agent.sinks.hdfs_valid.hdfs.idleTimeout=900
agent.sinks.hdfs_valid.hdfs.rollInterval=3600
[email protected]mple.com
agent.sinks.hdfs_valid.hdfs.kerberosKeytab=locationofkeytab
agent.sinks.hdfs_valid.hdfs.rollSize=0
agent.sinks.hdfs_valid.hdfs.rollCount=0
agent.sinks.hdfs_valid.hdfs.callTimeout=100000