从HTML页面读取固定宽度格式的文本表

我试图从类似于以下的表格中读取数据http://www.fec.gov/pubrec/fe1996/hraz.htm使用R但是一直无法取得进展。我意识到要做到这一点，我需要使用XML和RCurl，但尽管网络上有许多关于类似问题的其他示例，但我无法解决这个问题。从HTML页面读取固定宽度格式的文本表

第一个问题是，该表只是一个表格查看时，但没有编码。把它作为一个XML文档处理我可以访问表中的“数据”，但是因为有几个表我想得到我不认为这是最优雅的解决方案。

将它作为一个html文档处理可能会更好，但我对xpathApply相对不熟悉，不知道如何获取表中的实际“数据”，因为它没有被任何东西括起来（例如，i-/i或b-/b）。

我在过去使用xml文件取得了一些成功，但这是我第一次尝试做类似于html文件的事情。特别是这些文件的结构似乎比其他我见过的例子少。

任何帮助，非常感谢。

来源

2013-04-17 johnson-shuffle

查看该页面的来源，这不是一个表。 – A5C1D2H2I1M1N2O1R2T1

我想你想要xml阅读器，而不是csv（表格）阅读器 – Nishanth

再次查看页面的源代码，他们*尝试*以固定宽度格式显示数据，所以你可能只能“复制和粘贴“并使用'read.fwf'。然而。我提到他们已*尝试*将数据固定为宽度，但您可以看到“区域3”与其他区域的对齐方式不同，这将导入后需要进一步清理。 – A5C1D2H2I1M1N2O1R2T1

假设你可以阅读html输出到一个文本文件（复制+粘贴的等价形式Web浏览器），这应该让你的存在方式的好大块：

# x is the output from the website 


library(stringr) 
library(data.table) 

# First, remove commas from numbers (easiest to do at beginning) 
x <- gsub(",([0-9])", "\\1", x) 

# split the data by District 
districts <- strsplit(x, "DISTRICT *")[[1]] 

# separate out the header info 
headerInfo <- districts[[1]] 
districts <- tail(districts, -1) 


# grab the straggling district number, use it as a name and remove it 

    # end of first line 
    eofl <- str_locate(districts, "\n")[,2] 

    # trim white space and assign as name 
    names(districts) <- str_trim(substr(districts, 1, eofl)) 

    # remove first line 
    districts <- substr(districts, eofl+1, nchar(districts)) 

# replace the ending '-------' and trime white space 
    districts <- str_trim(str_replace_all(districts, "---*", "")) 

# Adjust delimeter (this is the tricky part) 

    ## more than two spaces are a spearator 
    districts <- str_replace_all(districts, " +", "\t") 

    ## lines that are total tallies are missing two columns. 
    ## thus, need to add two extra delims. After the first and third columns 

     # this function will 
     padDelims <- function(section, splton) { 
      # split into lines 
      section <- strsplit(section, splton)[[1]] 
      # identify lines starting with totals 
      LinesToFix <- str_detect(section, "^Total") 
      # pad appropriate columns 
      section[LinesToFix] <- sub("(.+)\t(.+)\t(.*)?", "\\1\t\t\\2\t\t\\3", section[LinesToFix]) 

      # any rows missing delims, pad at end 
      counts <- str_count(section, "\t") 
      toadd <- max(counts) - counts 
      section[ ] <- mapply(function(s, p) if (p==0) return (s) else paste0(s, paste0(rep("\t", p), collapse="")), section, toadd) 

      # paste it back together and return 
      paste(section, collapse=splton) 
     } 

    districts <- lapply(districts, padDelims, splton="\n") 

    # reading the table and simultaneously addding the district column 
    districtTables <- 
     lapply(names(districts), function(d) 
     data.table(read.table(text=districts[[d]], sep="\t"), district=d)) 
    # ... or without adding district number: 
    ##  lapply(districts, function(d) data.table(read.table(text=d, sep="\t"))) 

    # flatten it 
    votes <- do.call(rbind, districtTables) 
    setnames(votes, c("Candidate", "Party", "PrimVotes.Abs", "PrimVotes.Perc", "GeneralVotes.Abs", "GeneralVotes.Perc", "District"))

样品表：

votes 

         Candidate  Party PrimVotes.Abs PrimVotes.Perc GeneralVotes.Abs GeneralVotes.Perc District 
1:     Salmon, Matt   R   33672   100.00  135634.00    60.18  1 
2:   Total Party Votes:     33672    NA    NA    NA  1 
3:              NA    NA    NA    NA  1 
4:      Cox, John  W(D)/D   1942   100.00   89738.00    39.82  1 
5:   Total Party Votes:      1942    NA    NA    NA  1 
6:              NA    NA    NA    NA  1 
7:   Total District Votes:     35614    NA  225372.00    NA  1 
8:     Pastor, Ed   D   29969   100.00   81982.00    65.01  2 
9:   Total Party Votes:     29969    NA    NA    NA  2 
10:              NA    NA    NA    NA  2 
... 
51:    Hayworth, J.D.   R   32554   100.00  121431.00    47.57  6 
52:   Total Party Votes:     32554    NA    NA    NA  6 
53:              NA    NA    NA    NA  6 
54:     Owens, Steve   D   35137   100.00  118957.00    46.60  6 
55:   Total Party Votes:     35137    NA    NA    NA  6 
56:              NA    NA    NA    NA  6 
57:    Anderson, Robert  LBT   148   100.00   14899.00    5.84  6 
58:              NA    NA    NA    NA  6 
59:   Total District Votes:     67839    NA  255287.00    NA  6 
60:              NA    NA    NA    NA  6 
61:   Total State Votes:     368185    NA  1356446.00    NA  6 
         Candidate  Party PrimVotes.Abs PrimVotes.Perc GeneralVotes.Abs GeneralVotes.Perc District

来源

2013-04-17 06:58:39

从HTML页面读取固定宽度格式的文本表

回答

相关问题