我试图导出250MB的数据(75 chararray列)从hdfs到sqlserver。它没有与下面的错误,sqoop导出到sql server失败
引起:产生java.io.IOException:com.microsoft.sqlserver.jdbc.SQLServerException:传入的表格格式的数据流(TDS)远程过程调用(RPC)协议流不正确。此RPC请求中提供的参数太多。最大值是2100.
然后我通过"-D sqoop.export.records.per.statement=10"
这个声明以及sqoop导出它的工作,但它非常缓慢。花了15分钟来加载250MB的数据。
无论如何,我们可以提高性能。
下面是实际的sqoop CMD:
sqoop出口-D sqoop.export.records.per.statement = 10 --connect“JDBC:SQLSERVER://199.198.165.191:1433;用户名= ;密码=;数据库=数据库” --table Facttable --columns DimDateID,DimQHourID,ETLMergedFileQHourlyNortelID,DimSWVersionID,DimFreqCellRelationID,OSSC_RC,SubNetwork1,SubNetwork2,MeContext,ENodeBFunction,EUtranCellFDD,EUtranFreqRelation,EUtranCellRelation,时间,GmtOffset,FFV ,SN,ST,VN,CBT,TS,NeuN的,nedn,NESW,MTS,GP,SF,pmHoExeAttLteInterF,pmHoExeAttLteIntraF,pmHoExeSuccLteInterF,pmHoExeSuccLteIntraF,pmHoPrepAttLteInterF,pmHoPrepAttLteIntr AF,pmHoPrepSuccLteInterF,pmHoPrepSuccLteIntraF,Count_Null,Count_Negative,Count_Threshold,pmHoExeAttLteInterFLb,pmHoExeSuccLteInterFLb,pmHoOscInterF,pmHoOscIntraF,pmHoPrepAttLteInterFLb,pmHoPrepSuccLteInterFLb,pmHoPrepTNotAllowedLteInterF,pmHoPrepTNotAllowedLteIntraF,pmHoTooEarlyHoInterF,pmHoTooEarlyHoIntraF,pmHoTooLateHoInterF,pmHoTooLateHoIntraF,pmHoWrongCellInterF,pmHoWrongCellIntraF,pmHoWrongCellReestInterF,pmHoWrongCellReestIntraF,pmLbQualifiedUe,pmZtemporary36,pmHoExeAttLteIntraFTuneOut, pmHoExeSuccLteIntraFTuneOut --export-dir/Fact_Peg --direct -m 8 --input-fields-terminated-by“,”--input-lines-terminated-by“\ n”;
“