获取MarkLogic中的所有文档URI使用Java客户端API

Question

我试图从数据库中获取所有文档而不知道确切的URL。我有一个查询

DocumentPage documents =docMgr.read(); 
while (documents.hasNext()) { 
    DocumentRecord document = documents.next(); 
    System.out.println(document.getUri()); 
} 

但是我没有特定的网址，我想所有的文件

Answer 1

的第一步是使数据库的URI的词汇。

你可以EVAL一些XQuery和运行CTS：URI的（）（或服务器端JS和运行cts.uris（））：

ServerEvaluationCall call = client.newServerEval() 
     .xquery("cts:uris()"); 
    for (EvalResult result : call.eval()) { 
     String uri = result.getString(); 
     System.out.println(uri); 
    } 

两个缺点是：（1）你需要一个用户privileges和（2）没有分页。

如果您有少量文件，则不需要分页。但是对于大量的文档分页建议。下面是使用搜索API和分页一些代码：如果您需要添加/删除在这个过程中发生的同时分离出一种保证“快照”列表

// do the next eight lines just once 
    String options = 
     "<options xmlns='http://marklogic.com/appservices/search'>" + 
     " <values name='uris'>" + 
     " <uri/>" + 
     " </values>" + 
     "</options>"; 
    QueryOptionsManager optionsMgr = client.newServerConfigManager().newQueryOptionsManager(); 
    optionsMgr.writeOptions("uriOptions", new StringHandle(options)); 

    // run the following each time you need to list all uris 
    QueryManager queryMgr = client.newQueryManager(); 
    long pageLength = 10000; 
    queryMgr.setPageLength(pageLength); 
    ValuesDefinition query = queryMgr.newValuesDefinition("uris", "uriOptions"); 
    // the following "and" query just matches all documents 
    query.setQueryDefinition(new StructuredQueryBuilder().and()); 
    int start = 1; 
    boolean hasMore = true; 
    Transaction transaction = client.openTransaction(); 
    try { 
     while (hasMore) { 
      CountedDistinctValue[] uriValues = 
       queryMgr.values(query, new ValuesHandle(), start, transaction).getValues(); 
      for (CountedDistinctValue uriValue : uriValues) { 
       String uri = uriValue.get("string", String.class); 
       //System.out.println(uri); 
      } 
      start += uriValues.length; 
      // this is the last page if uriValues is smaller than pageLength 
      hasMore = uriValues.length == pageLength; 
     } 
    } finally { 
     transaction.commit(); 
    } 

交易才是必需的。由于它增加了一些开销，如果你不需要这样的精确度，可以随意删除它。

Answer 2

找出页面长度并在queryMgr中指定要访问的起始点。继续增加所有URL的起点和循环。我能够获取所有的URI。这可能不是那么好的方法，但工作。

List<String> uriList = new ArrayList<>();  
     QueryManager queryMgr = client.newQueryManager(); 
     StructuredQueryBuilder qb = new StructuredQueryBuilder(); 
     StructuredQueryDefinition querydef = qb.and(qb.collection("xxxx"), qb.collection("whatever"), qb.collection("whatever"));//outputs 241152 
     SearchHandle results = queryMgr.search(querydef, new SearchHandle(), 10); 
     long pageLength = results.getPageLength(); 
     long totalResults = results.getTotalResults(); 
     System.out.println("Total Reuslts: " + totalResults); 
     long timesToLoop = totalResults/pageLength; 
     for (int i = 0; i < timesToLoop; i = (int) (i + pageLength)) { 
      System.out.println("Printing Results from: " + (i) + " to: " + (i + pageLength)); 
      results = queryMgr.search(querydef, new SearchHandle(), i); 
      MatchDocumentSummary[] summaries = results.getMatchResults();//10 results because page length is 10 
      for (MatchDocumentSummary summary : summaries) { 
//    System.out.println("Extracted friom URI-> " + summary.getUri()); 
       uriList.add(summary.getUri()); 
      } 
      if (i >= 1000) {//number of URI to store/retreive. plus 10 
       break; 
      } 
     } 
     uriList= uriList.stream().distinct().collect(Collectors.toList()); 
     return uriList;