All Forums Connectivity
jimirwin 1 post Joined 03/14
26 Jun 2014
Problem with Connector for Hadoop 1.3 and HCatalog

I'm trying to use TDCH 1.3 command line edition to import from TD to HCatalog.  I consistently get an exception.  I've tried various versions of Hive and HCatalog with no success: CDH4.5 with Hive 0.11, CDH5.0 with Hive 0.12, and HDP 2.1 with Hive 0.13.  All throw the same exception.
 
Here is my job setup:

cat > _td.hql <<EOF
create database if not exists td_gnis;
drop table if exists td_gnis.lakes;
create table td_gnis.lakes (
  Feature_ID       STRING,
  Feature_name     STRING,
  Primary_lat_dec  DOUBLE,
  Primary_lon_dec  DOUBLE
)
STORED AS TEXTFILE;
EOF

hive -f _td.hql

hadoop jar $TDCH_JAR com.teradata.connector.common.tool.ConnectorImportTool \
-libjars $LIB_JARS \
-url jdbc:teradata://192.168.11.200/database=vmtest \
-username vmtest \
-password vmtest \
-classname com.teradata.jdbc.TeraDriver \
-fileformat textfile \
-jobtype hcat \
-method split.by.amp \
-sourcetable gnis \
-sourcefieldnames "Feature_ID,Feature_name,Primary_lat_dec,Primary_lon_d ec" \
-targetdatabase td_gnis \
-targettable lakes \
-targetfieldnames "Feature_ID,Feature_name,Primary_lat_dec,Primary_lon_d ec" \
-nummappers 2

 
And this is the exception that gets thrown:

14/06/25 15:18:30 INFO hive.metastore: Trying to connect to metastore with URI thrift://hdp2.jri.revelytix.com:9083
14/06/25 15:18:30 INFO hive.metastore: Connected to metastore.
14/06/25 15:18:30 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
14/06/25 15:18:30 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/06/25 15:18:31 INFO processor.TeradataInputProcessor: input postprocessor com.teradata.connector.teradata.processor.TeradataSpli tByAmpProcessor starts at:  1403723911542
14/06/25 15:18:31 INFO processor.TeradataInputProcessor: input postprocessor com.teradata.connector.teradata.processor.TeradataSpli tByAmpProcessor ends at:  1403723911542
14/06/25 15:18:31 INFO processor.TeradataInputProcessor: the total elapsed time of input postprocessor com.teradata.connector.teradata.processor.TeradataSpli tByAmpProcessor is: 0s
14/06/25 15:18:31 INFO tool.ConnectorImportTool: com.teradata.connector.common.exception.ConnectorExcep tion: java.lang.NullPointerException
	at org.apache.hcatalog.data.schema.HCatSchema.get(HCatSch ema.java:99)
	at com.teradata.connector.hcat.utils.HCatSchemaUtils.getT argetFieldsTypeName(HCatSchemaUtils.java:37)
	at com.teradata.connector.hcat.processor.HCatOutputProces sor.outputPreProcessor(HCatOutputProcessor.java:70)
	at com.teradata.connector.common.tool.ConnectorJobRunner. runJob(ConnectorJobRunner.java:88)
	at com.teradata.connector.common.tool.ConnectorJobRunner. runJob(ConnectorJobRunner.java:48)
	at com.teradata.connector.common.tool.ConnectorImportTool .run(ConnectorImportTool.java:57)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java: 70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java: 84)
	at com.teradata.connector.common.tool.ConnectorImportTool .main(ConnectorImportTool.java:694)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth odAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delega tingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

	at com.teradata.connector.common.tool.ConnectorJobRunner. runJob(ConnectorJobRunner.java:103)
	at com.teradata.connector.common.tool.ConnectorJobRunner. runJob(ConnectorJobRunner.java:48)
	at com.teradata.connector.common.tool.ConnectorImportTool .run(ConnectorImportTool.java:57)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java: 70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java: 84)
	at com.teradata.connector.common.tool.ConnectorImportTool .main(ConnectorImportTool.java:694)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMeth odAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delega tingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

 

You must sign in to leave a comment.