需求:獲取oracle表增量信息,發(fā)送至udp514端口,支持ip配置
步驟:
(1)需要的jar oracle的 odbc5.jar(oracle安裝目錄 /jdbc/lib下查找)
(2)flume的開(kāi)源包flume-ng-sql-source-1.4.3.jar 最新的好像是1.5的 小版本記不住了 這個(gè)下載地址直接csdn上就有
這兩個(gè)jar 都拷貝到flume的lib下
(3)flume配置文件
a1.sources = r1a1.sinks = k1a1.channels = c1#接收syslog配置#a1.sources.r1.type = syslogudp#a1.sources.r1.port = 514#a1.sources.r1.host = 0.0.0.0###########sql source#################a1.sources.r1.type = org.keedio.flume.source.SQLSourcea1.sources.r1.hibernate.connection.url = jdbc:oracle:thin:@10.9.224.36:1521/soc# Hibernate Database connection propertiesa1.sources.r1.hibernate.connection.user = soc_appa1.sources.r1.hibernate.connection.password = soc_appa1.sources.r1.hibernate.connection.autocommit = truea1.sources.r1.hibernate.dialect = org.hibernate.dialect.Oracle10gDialecta1.sources.r1.hibernate.connection.driver_class = oracle.jdbc.driver.OracleDrivera1.sources.r1.run.query.delay=1a1.sources.r1.status.file.path = /usr/local/flumea1.sources.r1.status.file.name = a1.sqlSource.status# Custom querya1.sources.r1.start.from = '2017-07-31 07:06:20'a1.sources.r1.custom.query = SELECT CHR(39)||TO_CHAR(CREATETIME,'YYYY-MM-DD HH24:MI:SS')||CHR(39),MSG FROM FLUME_NG_SQL_SOURCE WHERE CREATETIME > TO_DATE($@$,'YYYY-MM-DD HH24:MI:SS') ORDER BY CREATETIME ASCa1.sources.r1.batch.size = 6000a1.sources.r1.max.rows = 1000a1.sources.r1.hibernate.connection.provider_class = org.hibernate.connection.C3P0ConnectionProvidera1.sources.r1.hibernate.c3p0.min_size=1a1.sources.r1.hibernate.c3p0.max_size=10###########sql source##################自定義類(lèi)型a1.sinks.k1.type = com.neusoft.utils.SyslogSink#配置發(fā)送ipa1.sinks.k1.destination = 10.9.224.36a1.channels.c1.type = memorya1.channels.c1.capacity = 10000a1.channels.c1.transactionCapacity = 10000a1.sources.r1.channels = c1a1.sinks.k1.channel = c1#啟動(dòng)命令 ./flume-ng agent -c ../conf -f ../conf/flume-sql.conf -n a1 -Dflume.root.logger=INFO,console &
由于是自己定義的sink 在我的博客中也有提到,所以配置了自定義的sink。
總結(jié)
以上所述是小編給大家介紹的Flume監(jiān)聽(tīng)oracle表增量的步驟詳解,希望對(duì)大家有所幫助,如果大家有任何疑問(wèn)請(qǐng)給我留言,小編會(huì)及時(shí)回復(fù)大家的。在此也非常感謝大家對(duì)VeVb武林網(wǎng)網(wǎng)站的支持!
新聞熱點(diǎn)
疑難解答
圖片精選