sqoop匯出hive資料到mysql錯誤ava.util.NoSuchElementException解決辦法

wangmm0218發表於2014-05-08
原文網址 : https://blog.csdn.net/wangmuming/article/details/25296697
OOPHiveMySqlException

sqoop錯誤日誌：

14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000000_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more


14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000001_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more


sqoop匯出語句：

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\001' --export-dir '/user/hive/warehouse/wordcount'


這個錯誤的原因是指定Hive中表欄位之間使用的分隔符錯誤，供Sqoop讀取解析不正確，由於此處hive中的資料是由hdfs中匯入的，其分隔符是'\t',故修改從下列的匯出語句即可：

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\t' --export-dir '/user/hive/warehouse/wordcount'


注：如果是由hive執行mapreduce操作彙總的結果，預設的分隔符是 '\001'.

相關文章