【需求背景】

有時候我們要對比兩份配置文件是不是一樣，或者比較兩個文本是否異樣，可以使用linux命令行工具diff a_file b_file，但是輸出的結(jié)果讀起來不是很友好。這時候使用python的標(biāo)準(zhǔn)庫difflib就能滿足我們的需求。

下面這個腳本使用了difflib和argparse，argparse用于解析我們給此腳本傳入的兩個參數(shù)(即兩份待比較的文件)，由difflib執(zhí)行比較，比較的結(jié)果放到了一個html里面，只要找個瀏覽器打開此html文件，就能直觀地看到比較結(jié)果，兩份文件有差異的地方會高亮顯示出來。

【程序正文】

以python2.7為例，compare_two_files.py程序正文：

#!/bin/env python# -*- coding: utf-8 -*-# 20180430import difflibimport sysimport argparse# 讀取建表語句或配置文件def read_file(file_name): try:  file_desc = open(file_name, 'r')  # 讀取后按行分割  text = file_desc.read().splitlines()  file_desc.close()  return text except IOError as error:  print 'Read input file Error: {0}'.format(error)  sys.exit()# 比較兩個文件并把結(jié)果生成一份html文本def compare_file(file1, file2): if file1 == "" or file2 == "":  print '文件路徑不能為空：第一個文件的路徑：{0}, 第二個文件的路徑：{1} .'.format(file1, file2)  sys.exit() else:  print "正在比較文件{0} 和 {1}".format(file1, file2) text1_lines = read_file(file1) text2_lines = read_file(file2) diff = difflib.HtmlDiff() # 創(chuàng)建HtmlDiff 對象 result = diff.make_file(text1_lines, text2_lines) # 通過make_file 方法輸出 html 格式的對比結(jié)果 # 將結(jié)果寫入到result_comparation.html文件中 try:  with open('result_comparation.html', 'w') as result_file:   result_file.write(result)   print "0==}==========> Successfully Finished/n" except IOError as error:  print '寫入html文件錯誤：{0}'.format(error)if __name__ == "__main__": # To define two arguments should be passed in, and usage: -f1 fname1 -f2 fname2 my_parser = argparse.ArgumentParser(description="傳入兩個文件參數(shù)") my_parser.add_argument('-f1', action='store', dest='fname1', required=True) my_parser.add_argument('-f2', action='store', dest='fname2', required=True) # retrieve all input arguments given_args = my_parser.parse_args() file1 = given_args.fname1 file2 = given_args.fname2 compare_file(file1, file2)

【待比較的文件】

兩份文件分別是old_ddl_file和new_ddl_file，內(nèi)容分別是—— old_ddl_file文件內(nèi)容 CREATE EXTERNAL TABLE raw_tags( p0 string COMMENT ‘uid', p3 string COMMENT ‘tag name, e.g. news, games, fairs, shoopingURL', p4 string COMMENT ‘e.g. 0, Games', p11 int COMMENT ‘gender', dt string COMMENT ‘date, like 26/6/2017', action string COMMENT ‘clickmodule, click_taghead_link, clicklink') CLUSTERED BY ( dt) INTO 4 BUCKETS ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,' STORED AS INPUTFORMAT ‘org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT ‘org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION ‘hdfs://hdfs-ha/apps/hive/warehouse/ksai.db/raw_tags' TBLPROPERTIES ( ‘numFiles'='1', ‘numRows'='0', ‘rawDataSize'='0', ‘totalSize'='70575510', ‘transient_lastDdlTime'='1500469448')new_ddl_file文件內(nèi)容 CREATE EXTERNAL TABLE raw_tags( p0 string COMMENT ‘uid', p3 string COMMENT ‘tag name, e.g. news, games, fairs, shoopingURL', p4 string COMMENT ‘e.g. 0, Games', p11 int COMMENT ‘gender', dt string COMMENT ‘date, like 26/6/2017', action string COMMENT ‘clickmodule, click_taghead_link, clicklink') ROW FORMAT DELIMITED FIELDS TERMINATED BY ‘,' STORED AS INPUTFORMAT ‘org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT ‘org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION ‘hdfs://hdfs-ha/apps/hive/warehouse/ksai.db/raw_tags' TBLPROPERTIES ( ‘COLUMN_STATS_ACCURATE'='{/”BASIC_STATS/”:/”true/”}', ‘numFiles'='0', ‘numRows'='0', ‘rawDataSize'='0', ‘totalSize'='0', ‘transient_lastDdlTime'='1521546069')