All Forums Hadoop
nandkarthik 2 posts Joined 01/13
28 Apr 2015
Sqoop export fails with untranslatable character error

HI, we have a sqoop job that exports data to Teradata hourly. Sometimes job fails with untranslatable character error. We would like to know, if there is any Java Function to check if the string is translatable by teradata. This is of higher priority. Please let us know, if there are any questions.
2015-04-19 22:59:56,734 WARN org.apache.hadoop.mapred.Child: Error running child
com.teradata.hadoop.exception.TeradataHadoopSQLExcepti on: java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 14.00.00.01] [Error 1338] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request. Details of the failure can be found in the exception chain that is accessible with getNextException.
   at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeBatchUp dateException(ErrorFactory.java:147)
 at com.teradata.jdbc.jdbc_4.util.ErrorFactory.makeBatchUp dateException(ErrorFactory.java:136)
       at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBa tchDMLArray(TDPreparedStatement.java:239)
        at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBa tch(TDPreparedStatement.java:1951)
        at com.teradata.hadoop.mapreduce.TeradataRecordWriter.wri te(TeradataRecordWriter.java:60)
  at com.teradata.hadoop.mapreduce.TeradataRecordWriter.wri te(TeradataRecordWriter.java:23)
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollec tor.write(MapTask.java:558)
   at org.apache.hadoop.mapreduce.task.TaskInputOutputContex tImpl.write(TaskInputOutputContextImpl.java:85)
      at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Cont ext.write(WrappedMapper.java:106)
       at com.teradata.hadoop.mapreduce.TeradataTextFileExportMa pper.map(TeradataTextFileExportMapper.java:32)
   at com.teradata.hadoop.mapreduce.TeradataTextFileExportMa pper.map(TeradataTextFileExportMapper.java:12)
       at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java: 140)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask. java:672)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
   at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
       at java.security.AccessController.doPrivileged(Native Method)
       at javax.security.auth.Subject.doAs(Subject.java:415)
     at org.apache.hadoop.security.UserGroupInformation.doAs(U serGroupInformation.java:1642)
 at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: com.teradata.jdbc.jdbc_4.util.JDBCException: [Teradata Database] [TeraJDBC 14.00.00.01] [Error 6706] [SQLState HY000] The string contains an untranslatable character.
     at com.teradata.jdbc.jdbc_4.util.ErrorFactory.cloneJDBCEx ception(ErrorFactory.java:169)
 at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchSta tementController.handleRunException(PreparedBatchState mentController.java:93)
      at com.teradata.jdbc.jdbc_4.statemachine.StatementControl ler.runBody(StatementController.java:126)
       at com.teradata.jdbc.jdbc_4.statemachine.PreparedBatchSta tementController.run(PreparedBatchStatementController. java:56)
   at com.teradata.jdbc.jdbc_4.TDStatement.executeStatement( TDStatement.java:372)
        at com.teradata.jdbc.jdbc_4.TDPreparedStatement.executeBa tchDMLArray(TDPreparedStatement.java:219)
 ... 16 more

   at com.teradata.hadoop.mapreduce.TeradataRecordWriter.wri te(TeradataRecordWriter.java:64)
     at com.teradata.hadoop.mapreduce.TeradataRecordWriter.wri te(TeradataRecordWriter.java:23)
       at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollec tor.write(MapTask.java:558)
      at org.apache.hadoop.mapreduce.task.TaskInputOutputContex tImpl.write(TaskInputOutputContextImpl.java:85)
 at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Cont ext.write(WrappedMapper.java:106)
  at com.teradata.hadoop.mapreduce.TeradataTextFileExportMa pper.map(TeradataTextFileExportMapper.java:32)
      at com.teradata.hadoop.mapreduce.TeradataTextFileExportMa pper.map(TeradataTextFileExportMapper.java:12)
  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java: 140)
   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask. java:672)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
      at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:415)
        at org.apache.hadoop.security.UserGroupInformation.doAs(U serGroupInformation.java:1642)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
2015-04-19 22:59:56,737 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task

TDThrottle 51 posts Joined 11/11
08 May 2015

Karthik,
Assume here you Export data from Teradata using Sqoop utility.
Check if any UNICODE column is part of export column list in Teradata. Use TRANSLATE_CHK on unicode data to verify translate conversion can be performed without production errors.
Cheers!!

sravan_meda 1 post Joined 02/15
12 Jun 2015

Hi nandkarthik,
I am also facing same issue while exporting data from Hdfs to Teradata using sqoop export (internally uses teradata connector for hadoop).
are you able to find any solution for the above issue?
 

27 Jun 2015

Hi,
I am also facing similar issue. I am able to load data with 1K bytes but data is more than 1MB it throws following error 
Error: com.teradata.hadoop.exception.TeradataHadoopSQLException: java.sql.BatchUpdateException: [Teradata JDBC Driver] [TeraJDBC 14.10.00.26] [Error 1338] [SQLState HY000] A failure occurred while executing a PreparedStatement batch request. Details of the failure can be found in the exception chain that is accessible with getNextException.
 
Anybody found solution for this issue.
 
Thanks

nandkarthik 2 posts Joined 01/13
28 Aug 2015

Hi,
We haven't found the solution. We restricted characters only to ASCII solved this issue on our side.
-Karthik

joshuawinter 5 posts Joined 03/11
22 Sep 2015

I have a similar issue.  The data is already ASCII and per the customers request we are using ASCII \037 as a seperator in our tdch cli scripts.  Upon execution we error out because the data is being stuffed into the  first column and is not understanding the ASCII seperator.  It does work when we translate the seperator to a \t.  Any suggestions if we can support ASCII seperators in TDCH because it works in pure, non-TD sqoop.
 
--Josh

joshuawinter 5 posts Joined 03/11
23 Sep 2015

We found a solution to this problem.  TDCH will take an ASCII seperator in the form of unicode.  '\037' = '\u001F'
 
--Josh

You must sign in to leave a comment.