Tuesday, July 11, 2017

线上spark和sqoop问题修复


线上sqoop日志错误如下,https://stackoverflow.com/questions/38649071/getting-an-file-exists-error-while-import-into-hive-using-sqoop
https://stackoverflow.com/questions/38649071/getting-an-file-exists-error-while-import-into-hive-using-sqoop

参见这两个,大概意思是说sqoop导数据的时候,会将数据先放到一个临时路径下,然后在挪过去。如果这个临时路径存在,就会报错,哪怕选线是overwrite也不行;

spark错误

http://blog.csdn.net/a921122/article/details/45095845
https://stackoverflow.com/questions/32820087/spark-multiple-spark-submit-in-parallel

No comments:

Post a Comment