Разбор XML-файла в R с помощью rentrez

Я не XML эксперт. У меня возникла проблема при разборе файла XML с помощью rentrez. Я пытаюсь получить автора и принадлежность к каждому pmid (идентификатор статьи в базе данных PubMed) в качестве вывода. У меня есть код, который работает хорошо, за исключением случаев, когда автор имеет несколько связей. Когда автор имеет несколько аффилиаций, длина столбцов first_names, last_names и affiliation становится разной, и возвращается ошибка. У меня действительно нет опыта в разборе xml, чтобы справиться с этим. Я строго ожидаю результата, как показано ниже:

pmid         first_names  last_names              affiliation
27869504     Luca           Villa         Division of Experimental Oncology/Unit of Urology, URI , IRCCS Ospedale San Raffaele, Milan, Italy 
27869504     Luca           Villa         Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France
27869504     Tarik Emre     Şener         Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France
27869504     Tarik Emre     Şener         Department of Urology, Marmara University School of Medicine, Istanbul, Turkey

Ниже приведена структура моего образца XML-файла, возвращаемого entrez_fetch:

 <?xml version="1.0"?>
<!DOCTYPE PubmedArticleSet PUBLIC "-//NLM//DTD PubMedArticle, 1st January 2017//EN" "https://dtd.nlm.nih.gov/ncbi/pubmed/out/pubmed_170101.dtd">
<PubmedArticleSet>
  <PubmedArticle>
   <MedlineCitation Status="In-Data-Review" Owner="NLM">
    <PMID Version="1">27869504</PMID>
    <DateCreated>
     <Year>2016</Year>
     <Month>11</Month>
      <Day>21</Day>
    </DateCreated>
  <DateRevised>
    <Year>2017</Year>
    <Month>01</Month>
    <Day>06</Day>
  </DateRevised>
  <Article PubModel="Print-Electronic">
    <Journal>
      <ISSN IssnType="Electronic">1557-900X</ISSN>
      <JournalIssue CitedMedium="Internet">
        <Volume>31</Volume>
        <Issue>1</Issue>
        <PubDate>
          <Year>2017</Year>
          <Month>Jan</Month>
        </PubDate>
      </JournalIssue>
      <Title>Journal of endourology</Title>
      <ISOAbbreviation>J. Endourol.</ISOAbbreviation>
    </Journal>
    <ArticleTitle>Initial Content Validation Results of a New Simulation Model for Flexible Ureteroscopy: The Key-Box.</ArticleTitle>
    <Pagination>
      <MedlinePgn>72-77</MedlinePgn>
    </Pagination>
    <ELocationID EIdType="doi" ValidYN="Y">10.1089/end.2016.0677</ELocationID>
    <Abstract>
      <AbstractText Label="PURPOSE" NlmCategory="OBJECTIVE">We sought to test the content validity of a new training model for flexible ureteroscopy: the Key-Box.</AbstractText>
      <AbstractText Label="MATERIAL AND METHODS" NlmCategory="METHODS">Sixteen medical students were randomized to undergo a 10-day training consisting of performing 10 different exercises aimed at learning specific movements with the flexible ureteroscope, and how to catch and release stones with a nitinol basket using the Key-Box (n&#x2009;=&#x2009;8 students in the training group, n&#x2009;=&#x2009;8 students in the nontraining control group). Subsequently, an expert endourologist (O.T.) blindly assessed skills acquired by the whole cohort of students through two exercises on ureteroscope manipulation and one exercise on stone capture selected among those used for the training. A performance scale (1-5) assessing different steps of the procedure was used to evaluate each student. Time to complete the exercises was measured. Mann-Whitney Rank Sum test was used for comparisons between the two groups.</AbstractText>
      <AbstractText Label="RESULTS" NlmCategory="RESULTS">Mean scores obtained by trained students were significantly higher compared with those obtained by nontrained students (all p&#x2009;&lt;&#x2009;0.001). All trained students were able to complete the two exercises on ureteroscope manipulation within 3 minutes, whereas two students (25%) were not able to finish the exercise on stone capture. Conversely, four (50%) and six (75%) nontrained students were not able to finish one out of the two exercises on ureteroscope manipulation and the exercise on stone capture, respectively. The mean time to complete the three exercises was 76.3, 69.9, and 107 and 172.5, 137.9, and 168 seconds in the trained and nontrained groups, respectively (all p&#x2009;&lt;&#x2009;0.001).</AbstractText>
      <AbstractText Label="CONCLUSIONS" NlmCategory="CONCLUSIONS">The K-Box(&#xAE;) seems to be a valid easy-to-use training model for initiating novel endoscopists to flexible ureteroscopy.</AbstractText>
    </Abstract>
    <AuthorList CompleteYN="Y">
      <Author ValidYN="Y">
        <LastName>Villa</LastName>
        <ForeName>Luca</ForeName>
        <Initials>L</Initials>
        <AffiliationInfo>
          <Affiliation>1 Division of Experimental Oncology/Unit of Urology, URI , IRCCS Ospedale San Raffaele, Milan, Italy .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>&#x15E;ener</LastName>
        <ForeName>Tarik Emre</ForeName>
        <Initials>TE</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>3 Department of Urology, Marmara University School of Medicine , Istanbul, Turkey .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Somani</LastName>
        <ForeName>Bhaskar K</ForeName>
        <Initials>BK</Initials>
        <AffiliationInfo>
          <Affiliation>4 Department of Urology, University Hospital Southampton NHS Trust , Southampton, United Kingdom .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Cloutier</LastName>
        <ForeName>Jonathan</ForeName>
        <Initials>J</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>5 Department of Urology, University Hospital Centre of Quebec City , Quebec, Canada .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Buttic&#xE8;</LastName>
        <ForeName>Salvatore</ForeName>
        <Initials>S</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>6 Department of Urology, University of Messina , Messina, Italy .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Marson</LastName>
        <ForeName>Francesco</ForeName>
        <Initials>F</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>7 Department of Urology, Citt&#xE0; della Salute e della Scienza, Turin, Italy .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Doizi</LastName>
        <ForeName>Steeve</ForeName>
        <Initials>S</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Proietti</LastName>
        <ForeName>Silvia</ForeName>
        <Initials>S</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
        <AffiliationInfo>
          <Affiliation>8 Department of Urology, IRCCS San Raffaele Scientific Institute , Ville Turro Division, Milan, Italy .</Affiliation>
        </AffiliationInfo>
      </Author>
      <Author ValidYN="Y">
        <LastName>Traxer</LastName>
        <ForeName>Olivier</ForeName>
        <Initials>O</Initials>
        <AffiliationInfo>
          <Affiliation>2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .</Affiliation>
        </AffiliationInfo>
      </Author>
    </AuthorList>
    <Language>eng</Language>
    <PublicationTypeList>
      <PublicationType UI="D016428">Journal Article</PublicationType>
    </PublicationTypeList>
    <ArticleDate DateType="Electronic">
      <Year>2016</Year>
      <Month>12</Month>
      <Day>16</Day>
    </ArticleDate>
  </Article>
  <MedlineJournalInfo>
    <Country>United States</Country>
    <MedlineTA>J Endourol</MedlineTA>
    <NlmUniqueID>8807503</NlmUniqueID>
    <ISSNLinking>0892-7790</ISSNLinking>
  </MedlineJournalInfo>
  <KeywordList Owner="NOTNLM">
    <Keyword MajorTopicYN="N">flexible ureteroscopy</Keyword>
    <Keyword MajorTopicYN="N">learning curve</Keyword>
    <Keyword MajorTopicYN="N">training model</Keyword>
    <Keyword MajorTopicYN="N">ureteroscopy curriculum</Keyword>
  </KeywordList>
</MedlineCitation>
<PubmedData>
  <History>
    <PubMedPubDate PubStatus="pubmed">
      <Year>2016</Year>
      <Month>11</Month>
      <Day>22</Day>
      <Hour>6</Hour>
      <Minute>0</Minute>
    </PubMedPubDate>
    <PubMedPubDate PubStatus="medline">
      <Year>2016</Year>
      <Month>11</Month>
      <Day>22</Day>
      <Hour>6</Hour>
      <Minute>0</Minute>
    </PubMedPubDate>
    <PubMedPubDate PubStatus="entrez">
      <Year>2016</Year>
      <Month>11</Month>
      <Day>22</Day>
      <Hour>6</Hour>
      <Minute>0</Minute>
    </PubMedPubDate>
  </History>
  <PublicationStatus>ppublish</PublicationStatus>
  <ArticleIdList>
    <ArticleId IdType="pubmed">27869504</ArticleId>
    <ArticleId IdType="doi">10.1089/end.2016.0677</ArticleId>
  </ArticleIdList>
 </PubmedData>
</PubmedArticle>
</PubmedArticleSet>

Ниже приведен код, который я использую, который работает хорошо, за исключением случаев, когда в базе данных PubMed есть несколько аффилиаций для авторов статьи:

 library(rentrez)
 library(XML)

 pubmedSearch <- entrez_search("pubmed", term = "flexible ureteroscope Simulation Model", 
                          retmax = 10)
 SearchResults <- entrez_fetch(db="pubmed", pubmedSearch$ids, rettype="xml", 
                          parsed=TRUE)

 xmlGetValue <- function(x, node){
   a <- xpathSApply(x, node, xmlValue)
   if(length(a) == 0) {a <- NA} else {a}
 }

 parse_paper <- function(paper){
    pmid <- xmlGetValue(paper, ".//ArticleId[@IdType='pubmed']")
    first_names <- xmlGetValue(paper, ".//Author/ForeName")
    last_names <- xmlGetValue(paper, ".//Author/LastName")
    affiliation <- xmlGetValue(paper, ".//AffiliationInfo/Affiliation")
    data.frame(pmid=pmid, first_names=first_names, last_names=last_names,
         affiliation=affiliation)
 }  

parse_multiple_papers <- function(papers){
  res <- xpathApply(papers, "/PubmedArticleSet/*", parse_paper)
  do.call(rbind.data.frame, res)
}

test_df <- parse_multiple_papers(SearchResults)

Любая помощь и поддержка очень ценятся.


r xml xpath ropensci pubmed
person Santosh M.    schedule 04.03.2017    source источник

Ответы (2)


arrow_upward
3
arrow_downward

Этот вопрос также возник как ошибка в репозитории Rentrez, детали одного из возможных решений приведены ниже. дано там. Я также включу этот код здесь

parse_author <- function(author){
  fn  <- xmlValue(author[["ForeName"]])
  ln  <- xmlValue(author[["LastName"]])
  aff <-paste(xpathApply(author, "AffiliationInfo/Affiliation", xmlValue), collapse="; ")
  list(forname=fn, lastname=ln, affiliation=aff)
}

parse_paper <- function(paper){
  author_info <- xpathApply(paper, ".//AuthorList/Author", parse_author)
  res <- do.call(rbind.data.frame, author_info)
  res$pmid <-xpathSApply(paper, ".//ArticleId[@IdType='pubmed']", xmlValue)
  res
}

parse_multiple_papers <- function(papers){
 res <- xpathApply(papers, "/PubmedArticleSet/*", parse_paper)
 do.call(rbind.data.frame, res)
}

head(parse_multiple_papers(SearchResults))
person david w    schedule 04.03.2017

arrow_upward
4
arrow_downward

Вы можете сделать это следующим образом, используя xml2 и purrr

require(xml2)
require(purrr)

doc <- read_xml(doc)
scope <- doc %>% xml_find_all("//author")
scope %>% map_df(~data.frame(
  first_names = xml_find_first(.x, "./forename") %>% xml_text,
  last_names = xml_find_first(.x, "./lastname") %>% xml_text,
  affiliation = xml_find_all(.x, ".//affiliation") %>% xml_text,
  stringsAsFactors = FALSE
))

Что дает вам:

   first_names last_names                                                                                             affiliation
1         Luca      Villa  1 Division of Experimental Oncology/Unit of Urology, URI , IRCCS Ospedale San Raffaele, Milan, Italy .
2         Luca      Villa            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
3   Tarik Emre      Şener            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
4   Tarik Emre      Şener                     3 Department of Urology, Marmara University School of Medicine , Istanbul, Turkey .
5    Bhaskar K     Somani      4 Department of Urology, University Hospital Southampton NHS Trust , Southampton, United Kingdom .
6     Jonathan   Cloutier            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
7     Jonathan   Cloutier                   5 Department of Urology, University Hospital Centre of Quebec City , Quebec, Canada .
8    Salvatore    Butticè            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
9    Salvatore    Butticè                                       6 Department of Urology, University of Messina , Messina, Italy .
10   Francesco     Marson            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
11   Francesco     Marson                             7 Department of Urology, Città della Salute e della Scienza, Turin, Italy .
12      Steeve      Doizi            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
13      Silvia   Proietti            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
14      Silvia   Proietti 8 Department of Urology, IRCCS San Raffaele Scientific Institute , Ville Turro Division, Milan, Italy .
15     Olivier     Traxer            2 Department of Urology, Tenon Hospital, Pierre and Marie Curie University , Paris, France .
person Rentrop    schedule 04.03.2017
comment
Хороший ответ. Поскольку это вопрос о rentrez, я просто оставляю примечание, чтобы указать, что для использования xml2::read_xml в опубликованных записях пользователи должны будут установить parsed=FALSE (по умолчанию) при получении записей с entrez_fetch. - person david w; 04.03.2017