抓取（爬取）網(wǎng)上信息的腳本程序，俗稱(chēng)網(wǎng)絡(luò)蜘蛛。
powershell中自帶了這樣的兩個(gè)命令，【Invoke-WebRequest】和【Invoke-RestMethod】，但這兩個(gè)命令有時(shí)候會(huì)亂碼。
現(xiàn)在轉(zhuǎn)帖分享， 某個(gè)【歪果仁】寫(xiě)的腳本。來(lái)源于 墻外出處： https://gist.github.com/angel-vladov/9482676
核心代碼
function Read-HtmlPage {param ([Parameter(Mandatory=$true, Position=0, ValueFromPipeline=$true)][String] $Uri)# Invoke-WebRequest and Invoke-RestMethod can't work properly with UTF-8 Response so we need to do things this way.[Net.HttpWebRequest]$WebRequest = [Net.WebRequest]::Create($Uri)[Net.HttpWebResponse]$WebResponse = $WebRequest.GetResponse()$Reader = New-Object IO.StreamReader($WebResponse.GetResponseStream())$Response = $Reader.ReadToEnd()$Reader.Close()# Create the document class[mshtml.HTMLDocumentClass] $Doc = New-Object -com "HTMLFILE"$Doc.IHTMLDocument2_write($Response)# Returns a HTMLDocumentClass instance just like Invoke-WebRequest ParsedHtml$Doc#powershell 傳教士 轉(zhuǎn)帖并修改的文章 2016-01-01, 允許再次轉(zhuǎn)載，但必須保留名字和出處，否則追究法律責(zé)任}原文函數(shù)
function Read-HtmlPage { param ([Parameter(Mandatory=$true, Position=0, ValueFromPipeline=$true)][String] $Uri) # Invoke-WebRequest and Invoke-RestMethod can't work properly with UTF-8 Response so we need to do things this way. [Net.HttpWebRequest]$WebRequest = [Net.WebRequest]::Create($Uri) [Net.HttpWebResponse]$WebResponse = $WebRequest.GetResponse() $Reader = New-Object IO.StreamReader($WebResponse.GetResponseStream()) $Response = $Reader.ReadToEnd() $Reader.Close() # Create the document class [mshtml.HTMLDocumentClass] $Doc = New-Object -com "HTMLFILE" $Doc.IHTMLDocument2_write($Response) # Returns a HTMLDocumentClass instance just like Invoke-WebRequest ParsedHtml $Doc}PowerShell function you can use for reading UTF8 encoded HTML pages content. The built in Invoke-WebRequest and Invoke-RestMethod fail miserably.
新聞熱點(diǎn)
疑難解答
圖片精選
網(wǎng)友關(guān)注
