Dropwizard和hive-jdbc之间的不兼容

我正在尝试使用Dropwizard创建一个WS REST来获取查询的Hive结果并以XML格式进行呈现。这可以完成每个分离的部分没有错误。Dropwizard和hive-jdbc之间的不兼容

当我加入所有内容时，我遇到了Dropwizard和hive-jdbc之间的不兼容问题。显然hive-jdbc有球衣1作为依赖，Dropwizard有球衣2.

我不确定这是否是问题。我试图在pom.xml中排除依赖关系，但没有解决。

我试图制作一个应用程序，只需将hive-jdbc作为依赖项放置在pom.xml文件中，而不用在代码的任何部分使用或导入，并且错误仍在发生。

的pom.xml

<?xml version="1.0" encoding="UTF-8"?> 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> 

    <prerequisites> 
     <maven>3.0.0</maven> 
    </prerequisites> 

    <modelVersion>4.0.0</modelVersion> 
    <groupId>test</groupId> 
    <artifactId>ws-test</artifactId> 
    <version>1.0.0</version> 
    <name>ws-test</name> 

    <properties> 
     <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> 
     <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> 
     <dropwizard.version>1.0.5</dropwizard.version> 
     <mainClass>test.TestApp</mainClass> 
    </properties> 

    <dependencyManagement> 
     <dependencies> 
      <dependency> 
       <groupId>io.dropwizard</groupId> 
       <artifactId>dropwizard-bom</artifactId> 
       <version>${dropwizard.version}</version> 
       <type>pom</type> 
       <scope>import</scope> 
      </dependency> 
     </dependencies> 
    </dependencyManagement> 

    <dependencies> 
     <dependency> 
      <groupId>io.dropwizard</groupId> 
      <artifactId>dropwizard-core</artifactId> 
     </dependency> 
     <dependency> 
      <groupId>io.dropwizard</groupId> 
      <artifactId>dropwizard-forms</artifactId> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hive</groupId> 
      <artifactId>hive-jdbc</artifactId> 
      <version>1.1.0</version> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.hadoop</groupId> 
      <artifactId>hadoop-core</artifactId> 
      <version>1.2.1</version> 
     </dependency> 
    </dependencies> 

    <build> 
     <plugins> 
      <plugin> 
       <artifactId>maven-shade-plugin</artifactId> 
       <version>2.4.1</version> 
       <configuration> 
        <createDependencyReducedPom>true</createDependencyReducedPom> 
        <transformers> 
         <transformer 
          implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" /> 
         <transformer 
          implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> 
          <mainClass>${mainClass}</mainClass> 
         </transformer> 
        </transformers> 
        <!-- exclude signed Manifests --> 
        <filters> 
         <filter> 
          <artifact>*:*</artifact> 
          <excludes> 
           <exclude>META-INF/*.SF</exclude> 
           <exclude>META-INF/*.DSA</exclude> 
           <exclude>META-INF/*.RSA</exclude> 
          </excludes> 
         </filter> 
        </filters> 
       </configuration> 
       <executions> 
        <execution> 
         <phase>package</phase> 
         <goals> 
          <goal>shade</goal> 
         </goals> 
        </execution> 
       </executions> 
      </plugin> 
      <plugin> 
       <artifactId>maven-jar-plugin</artifactId> 
       <version>2.6</version> 
       <configuration> 
        <archive> 
         <manifest> 
          <addClasspath>true</addClasspath> 
          <mainClass>${mainClass}</mainClass> 
          <addDefaultImplementationEntries>true</addDefaultImplementationEntries> 
         </manifest> 
        </archive> 
       </configuration> 
      </plugin> 
     </plugins> 
    </build> 
</project>

TestApp.java

package test; 

import io.dropwizard.Application; 
import io.dropwizard.forms.MultiPartBundle; 
import io.dropwizard.setup.Bootstrap; 
import io.dropwizard.setup.Environment; 

public class TestApp extends Application<TestConf> { 

    public static void main(String[] args){ 
     try { 
      new TestApp().run(args); 
     } 
     catch(Exception e) { 
      e.printStackTrace(); 
     } 
    } 

    @Override 
    public String getName() { 
     return "ID Generator Server"; 
    } 

    @Override 
    public void initialize(final Bootstrap<TestConf> bootstrap) { 
     bootstrap.addBundle(new MultiPartBundle()); 
    } 

    @Override 
    public void run(final TestConf configuration, 
        final Environment environment) { 
     final TestRes resource = new TestRes(
       configuration.getTemplate(), 
       configuration.getDefaultName() 
      ); 
      environment.jersey().register(resource); 
    } 

}

TestConf.java

package test; 

import io.dropwizard.Configuration; 
import com.fasterxml.jackson.annotation.JsonProperty; 
import org.hibernate.validator.constraints.*; 

public class TestConf extends Configuration { 

    @NotEmpty 
    private String template; 

    @NotEmpty 
    private String defaultName = "MarineTraffic"; 

    @JsonProperty 
    public String getTemplate() { 
     return template; 
    } 

    @JsonProperty 
    public void setTemplate(String template) { 
     this.template = template; 
    } 

    @JsonProperty 
    public String getDefaultName() { 
     return defaultName; 
    } 

    @JsonProperty 
    public void setDefaultName(String name) { 
     this.defaultName = name; 
    } 
}

TestRes.java

package test; 

import javax.ws.rs.GET; 
import javax.ws.rs.Path; 
import javax.ws.rs.Produces; 
import javax.ws.rs.core.MediaType; 

import org.slf4j.Logger; 
import org.slf4j.LoggerFactory; 

@Path("/hive") 
public class TestRes { 

    protected final static Logger LOGGER = LoggerFactory.getLogger(TestRes.class); 
    protected final static String HEADER = "GroupService::"; 

    public TestRes(String template, String defaultName) { 
     LOGGER.info(HEADER + "init: PostConstruct"); 
    } 

    @GET 
    @Path("/test") 
    @Produces(MediaType.TEXT_PLAIN) 
    public String getNewID() { 

     return "works"; 
    } 
}

我得到的错误是这样的：

WARN [2017-08-10 21:50:21,444] org.glassfish.jersey.internal.Errors: The following warnings have been detected: WARNING: Unknown HK2 failure detected: 
MultiException stack 1 of 2 
java.lang.NullPointerException 
    at com.sun.jersey.core.provider.jaxb.AbstractJAXBProvider.setConfiguration(AbstractJAXBProvider.java:107) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
... 
    at test.TestApp.main(TestApp.java:12) 
MultiException stack 2 of 2 
java.lang.IllegalStateException: Unable to perform operation: method inject on com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App 
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:392) 
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:471) 
... 

WARNING: Unknown HK2 failure detected: 
MultiException stack 1 of 3 
java.lang.NullPointerException 
    at com.sun.jersey.core.provider.jaxb.AbstractJAXBProvider.setConfiguration(AbstractJAXBProvider.java:107) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
... 
    at test.TestApp.main(TestApp.java:12) 
MultiException stack 2 of 3 
java.lang.IllegalStateException: Unable to perform operation: method inject on com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App 
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:392) 
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:471) 
... 
    at test.TestApp.main(TestApp.java:12) 
MultiException stack 3 of 3 
java.lang.IllegalStateException: Unable to perform operation: create on org.glassfish.jersey.message.internal.MessageBodyFactory 
    at org.jvnet.hk2.internal.ClazzCreator.create(ClazzCreator.java:386) 
    at org.jvnet.hk2.internal.SystemDescriptor.create(SystemDescriptor.java:471) 
...

我发现，一些用户遇到类似的错误（如dropwizard and hive/hiverunner integration with maven和http://thread.gmane.org/gmane.comp.java.dropwizard.devel/461）

难道这（泽西不兼容）导致错误的问题？有谁知道如何解决它？

来源

2017-08-10 Caio

看看maven shade插件。 –

我试图在maven-shade-plugin配置中添加排除子句来删除球衣服务器，maven抱怨不兼容。然而，没有什么变化。 – Caio

Shade允许你重写包名和对它们的引用 - 这样你就可以重写jersey1中的包，并且可以在hive-jdbc中引用它们，这可以帮助你解决你的问题。请参阅：https://stackoverflow.com/questions/13620281/what-is-the-maven-shade-plugin-used-for-and-why-would-you-want-to-relocate-java。 Shade旨在让您解决不得不依赖不兼容库的问题。如果这是你的问题，那么这可能会解决它。但是，这看起来很像反射，这意味着ymmv。 –

是的，它的球衣冲突问题，从hive-jdbc和hadoop-core中排除球衣1依赖关系（com.sun.jersey）。已经运行了您的测试应用程序，它可以处理以下排除事项。

<dependency> 
     <groupId>org.apache.hive</groupId> 
     <artifactId>hive-jdbc</artifactId> 
     <version>1.1.0</version> 
     <exclusions> 
      <exclusion> 
       <groupId>com.sun.jersey</groupId> 
       <artifactId>*</artifactId> 
      </exclusion> 
     </exclusions> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.hadoop</groupId> 
     <artifactId>hadoop-core</artifactId> 
     <version>1.2.1</version> 
     <exclusions> 
      <exclusion> 
       <groupId>com.sun.jersey</groupId> 
       <artifactId>*</artifactId> 
      </exclusion> 
     </exclusions> 
    </dependency>

来源

2017-08-11 14:38:27 Ravi

大声笑。非常感谢。我没有意识到hadoop-core也使用球衣。那是我错过的。现在正在完美工作。我仍然不明白为什么hive驱动程序和hadoop核心需要使用球衣服务器。 – Caio

Dropwizard和hive-jdbc之间的不兼容

回答

相关问题