在用爬蟲(chóng)爬取網(wǎng)站數(shù)據(jù)時(shí)，有些站點(diǎn)的一些關(guān)鍵數(shù)據(jù)的獲取需要使用賬號(hào)登錄，這里可以使用requests發(fā)送登錄請(qǐng)求，并用Session對(duì)象來(lái)自動(dòng)處理相關(guān)Cookie。

另外在登錄時(shí)，有些網(wǎng)站有時(shí)會(huì)要求輸入驗(yàn)證碼，比較簡(jiǎn)單的驗(yàn)證碼可以直接用pytesser來(lái)識(shí)別，復(fù)雜的驗(yàn)證碼可以依據(jù)相應(yīng)的特征自己采集數(shù)據(jù)訓(xùn)練分類(lèi)器。

以CSDN網(wǎng)站的登錄為例，這里用Python的requests庫(kù)與pytesser庫(kù)寫(xiě)了一個(gè)登錄函數(shù)。如果需要輸入驗(yàn)證碼，函數(shù)會(huì)首先下載驗(yàn)證碼到本地，然后用pytesser識(shí)別驗(yàn)證碼后登錄，對(duì)于CSDN登錄驗(yàn)證碼，pytesser的識(shí)別率很高。

其中的pytesser的下載地址為: pytesser下載

具體代碼如下：

#coding:utf-8import sysimport timeimport urllibimport shutilimport pytesserimport requestsfrom lxml import etreeconfig = {'gid': 1}def parse(s, html, idx): result = {} tree = etree.HTML(html) try:  result['lt'] = tree.xpath('//input[@name="lt"]/@value')[0]  result['execution'] = tree.xpath('//input[@name="execution"]/@value')[0]  result['path'] = tree.xpath('//form[@id="fm1"]/@action')[0] except IndexError, e:  return None valimg = None valimgs = tree.xpath('//img[@id="yanzheng"]/@src') if len(valimgs) > 0:  valimg = valimgs[0] validateCode = None if valimg:  fname = 'img/' + str(idx) + '_' + str(config['gid']) + '.jpg'  config['gid'] = config['gid'] + 1  ri = s.get("https://passport.csdn.net" + valimg)  with open(fname, 'wb') as f:   for chk in ri:    f.write(chk)   f.close()  validateCode = pytesser.image_file_to_string(fname)  validateCode = validateCode.strip()  validateCode = validateCode.replace(' ', '')  validateCode = validateCode.replace('/n', '')  result['validateCode'] = validateCode return resultdef login(usr, pwd, idx): s = requests.Session() r = s.get('https://passport.csdn.net/account/login', headers={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0', 'Host': 'passport.csdn.net', }) while True:  res = parse(s, r.text, idx)  if res == None:   return False  url = 'https://passport.csdn.net' + res['path']  form = {'username': usr, 'password':pwd, '_eventId':'submit', 'execution':res['execution'], 'lt':res['lt'],}  if res.has_key('validateCode'):   form['validateCode'] = res['validateCode']  s.headers.update({   'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0',   'Accept-Language': 'zh-CN,zh;q=0.8,en-US;q=0.6,en;q=0.4',   'Content-Type': 'application/x-www-form-urlencoded',   'Host': 'passport.csdn.net',   'Origin': 'https://passport.csdn.net',   'Referer': 'https://passport.csdn.net/account/login',   'Upgrade-Insecure-Requests': 1,   })  r = s.post(url, data=form)  tree = etree.HTML(r.text)  err_strs = tree.xpath('//span[@id="error-message"]/text()')  if len(err_strs) == 0:   return True  err_str = err_strs[0]  print err_str  err = err_str.encode('utf8')  validate_code_err = '驗(yàn)證碼錯(cuò)誤'  usr_pass_err = '帳戶名或登錄密碼不正確，請(qǐng)重新輸入'  try_later_err = '登錄失敗連續(xù)超過(guò)5次，請(qǐng)10分鐘后再試'  if err[:5] == validate_code_err[:5]:   pass  elif err[:5] == usr_pass_err[:5]:   return False  elif err[:5] == try_later_err[:5]:   return False  else:   return Trueif __name__ == '__main__': main(sys.argv[1], sys.argv[2], 0)