Storing a local CSV file into table of ORC Format;

 Storing a local CSV file into table of ORC Format(hands on explanation)

--------------------------------------------------------
Step 1:
Create a stage table of  storage format textfile

hive>create table product_stage
(prod_name string, description varchar(20),category string,qty_onhand int, prod_num bigint, packaged_with Array<string> )
row format delimited
 fields terminated by ','
 collection items terminated by ':'
stored as textfile;

Step 2: load data into table stage

hive>Load data local inpath '/home/hadoop/Desktop/Product.csv'
 overwrite into table product_stage ;

Step3 :
Write a CTAS command

hive>Create table product_orc
stored as orc as
select * from product_stage;

Hint:When loading data from a csv file into orc table format using Load statement it doesn't produce any error but while executing a separate statement select * from table table_name it will produce an error

" Failed with exception java.io.IOException:java.io.IOException: Malformed ORC file hdfs://localhost:8020/user/hive/warehouse/customersales.db/product_stage_orc/Product.csv. Invalid postscript".

 

No comments:

Post a Comment