通过抓取其内容类型不是文本/ html的URL获取URL

Question

我可以获取其内容/类型为text/html的所有url，但是如果我想要那些内容/类型不是text/html的url。那我们怎么检查一下。而对于字符串，我们可以使用contains方法，但它并没有像notcontains东西..任何建议，可以理解的。而也

The key variable contains: 

Content-Type=[text/html; charset=ISO-8859-1] 

这是下面的代码检查text/html的我也尝试了不是text/html的内容类型，但它也打印出内容类型也是text/html的内容类型。

try { 
      URL url1 = new URL(url); 
      System.out.println("URL:- " +url1); 
      URLConnection connection = url1.openConnection(); 

      Map responseMap = connection.getHeaderFields(); 
      Iterator iterator = responseMap.entrySet().iterator(); 
      while (iterator.hasNext()) 
      { 
       String key = iterator.next().toString(); 

       if (key.contains("text/html") || key.contains("text/xhtml")) 
       { 
        System.out.println(key); 
        // Content-Type=[text/html; charset=ISO-8859-1] 
        if (filters.matcher(key) != null){ 
         System.out.println(url1); 
         try { 
          final File parentDir = new File("crawl_html"); 
          parentDir.mkdir(); 
          final String hash = MD5Util.md5Hex(url1.toString()); 
          final String fileName = hash + ".txt"; 
          final File file = new File(parentDir, fileName); 
          boolean success =file.createNewFile(); // Creates file crawl_html/abc.txt 


          System.out.println("hash:-" + hash); 

            System.out.println(file); 
          // Create file if it does not exist 



           // File did not exist and was created 
           FileOutputStream fos = new FileOutputStream(file, true); 

           PrintWriter out = new PrintWriter(fos); 

           // Also could be written as follows on one line 
           // Printwriter out = new PrintWriter(new FileWriter(args[0])); 

              // Write text to file 
           Tika t = new Tika(); 
           String content= t.parseToString(new URL(url1.toString())); 


           out.println("==============================================================="); 
           out.println(url1); 
           out.println(key); 
           out.println(success); 
           out.println(content); 

           out.println("==============================================================="); 
           out.close(); 
           fos.flush(); 
           fos.close(); 



         } catch (FileNotFoundException e) { 
          // TODO Auto-generated catch block 
          e.printStackTrace(); 
         } catch (IOException e) { 
          // TODO Auto-generated catch block 

          e.printStackTrace(); 
         } catch (TikaException e) { 
          // TODO Auto-generated catch block 
          e.printStackTrace(); 
         } 


         // http://google.com 
        } 
       } 
    else if (!connection.getContentType().startsWith("text/html"))//print duplicate records of each url 
       //else if (!key.contains("text/html")) 
       { 
        if (filters.matcher(key) != null){ 
        try { 
         final File parentDir = new File("crawl_media"); 
         parentDir.mkdir(); 
         final String hash = MD5Util.md5Hex(url1.toString()); 
         final String fileName = hash + ".txt"; 
         final File file = new File(parentDir, fileName); 
        // Create file if it does not exist 
         boolean success =file.createNewFile(); // Creates file crawl_html/abc.txt 


         System.out.println("hash:-" + hash); 

         Tika t = new Tika(); 
         String content_media= t.parseToString(new URL(url1.toString())); 



          // File did not exist and was created 
          FileOutputStream fos = new FileOutputStream(file, true); 

          PrintWriter out = new PrintWriter(fos); 

          // Also could be written as follows on one line 
          // Printwriter out = new PrintWriter(new FileWriter(args[0])); 

             // Write text to file 
          out.println("==============================================================="); 
          out.println(url1); 
          out.println(key); 
          out.println(success); 
          out.println(content_media); 
          //out.println("==============================================================="); 
          out.close(); 
          fos.flush(); 
          fos.close(); 




        } catch (FileNotFoundException e) { 
         // TODO Auto-generated catch block 
         e.printStackTrace(); 
        } catch (IOException e) { 
         // TODO Auto-generated catch block 

         e.printStackTrace(); 
        } catch (TikaException e) { 
         // TODO Auto-generated catch block 
         e.printStackTrace(); 
        } 
        } 

       } 



      } 
     } catch (MalformedURLException e) { 
      e.printStackTrace(); 
     } catch (IOException e) { 
      e.printStackTrace(); 
     } 



     System.out.println("============="); 
    } 
} 

一种方法是逐个检查每个内容类型像PDF它是应用程序/ PDF

if (key.contains("application/pdf") 

和XML的同样的方式......但任何其他方法比这其他...

Answer 1

这会有帮助吗？

if (!connection.getContentType.startsWith("text/html")) 

Answer 2

什么是错的使用：

if (key.contains("text/html") || key.contains("text/xhtml")) { 
    //Do stuff 
} else if (key.contains("application/pdf") { 
    //Do other stuff 
} else { 
    //All other cases 
} 

由于对其他格式的内容类型可以从每个类型而有所改变，你可能需要为每个内容类型明确的情况下。如果遇到通用内容类型，那么通用方法（else）应该足够吗？ Strategy Pattern可能对您有用。

我很抱歉，如果我误解了你的问题。您能否提供一个示例打印输出key的不同值是通过测试运行的吗？ （你的代码的第10行）