Nutch 2.2.1 + hBase

我试图运行新版本的Apache Nutch进行爬网。当我启动script/bin/crawl时，它失败，hadoop.log说：Nutch 2.2.1 + hBase

java.lang.Exception：java.lang.NoSuchMethodError：org.apache.gora.persistent.Persistent.getSchema（）Lorg/apache /阿夫罗/模式;在org.apache.hadoop.mapred.LocalJobRunner $ Job.run（LocalJobRunner.java:354）引起的：java.lang.NoSuchMethodError： /模式; 在org.apache.gora.hbase.store.HBaseStore.put（HBaseStore.java:177）

这里是日志：

2013-07-04 16:12:05,069 WARN mapred.LocalJobRunner - job_local1522971864_0001 
java.lang.Exception: java.lang.NoSuchMethodError:  org.apache.gora.persistency.Persistent.getSchema()Lorg/apache/avro/Schema; 
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354) 
Caused by: java.lang.NoSuchMethodError:  org.apache.gora.persistency.Persistent.getSchema()Lorg/apache/avro/Schema; 
at org.apache.gora.hbase.store.HBaseStore.put(HBaseStore.java:177) 
at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:65) 
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638) 
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80) 
at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:191) 
at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:88) 
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) 
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) 
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364) 
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223) 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
at java.util.concurrent.FutureTask.run(FutureTask.java:166) 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:722) 

2013-07-04 16:12:05,720 ERROR crawl.InjectorJob - InjectorJob: java.lang.RuntimeException: job failed: name=[new]inject /opt/ir/nutch2/urls, jobid=job_local1522971864_0001 
at org.apache.nutch.util.NutchJob.waitForCompletion(NutchJob.java:54) 
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:233) 
at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:251) 
at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:273) 
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) 
at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:282)

我应该设置一些文物山里面的ivy.xml或什么？请帮帮我。

来源

2013-07-04 Jan Bouchner

我将如何运行nutch 2.2.1。请提供任何指导 – jackyesind

Nutch有一个教程：http://wiki.apache.org/nutch/NutchTutorial。如果你有任何麻烦，你可以问一下看看:) –

如果你有任何问题，你可以问我... –

已解决。您必须为您的库添加正确版本的gora-hbase。 gora-hbase-0.3.jar

来源

2013-07-04 16:18:03

你为Nutch安装了哪个版本的HBase 2.2.1？ – sunskin

有很多版本的HBase不确定哪一个是稳定的，并且Nutch 2.2.1兼容版本？ http://archive.apache.org/dist/hbase/?C=M;O=D – sunskin

Hbase版本0.90.4 –

Nutch 2.2.1 + hBase

回答

相关问题