HIVE nested ARRAY in MAP data type

http://stackoverflow.com/questions/18812025/hive-nested-array-in-map-data-type

Hive’s default delimiters are:

  • Row Delimiter => Control-A (‘\001’)
  • Collection Item Delimiter => Control-B (‘\002’)
  • Map Key Delimiter => Control-C (‘\003’)

If you override these delimiters then overridden delimiters are used during parsing. The preceding description of delimiters is correct for the usual case of flat data structures, where the complex types only contain primitive types. For nested types the level of the nesting determines the delimiter.

For an array of arrays, for example, the delimiters for the outer array are Control-B (‘\002’) characters, as expected, but for the inner array they are Control-C (‘\003’) characters, the next delimiter in the list.

Hive actually supports eight levels of delimiters, corresponding to ASCII codes 1, 2, … 8, but you can only override the first three.

For your case delimiter for items in nested Array of Map datatype field sabu will be ‘\004’ as Map Key Delimiter is ‘\003’ (Overridden as ‘:’).

So you can write your input file as following format:

1|JOHN|abu1/abu2|key1:1'\004'2'\004'3/key2:6'\004'7'\004'8

Output of SELECT * FROM test_stg; will be:

1       JOHN     ["abu1","abu2"]     {"key1":[1,2,3],"key2":[6,7,8]}

Reference: Hadoop The Definitive Guide – Chapter 12: Hive, Page No: 433, 434

 

Input ^A ^B … in vim:

Ctrl + v, Ctrl + A/B/….

 

http://bbs.csdn.net/topics/390481438

1、先把TXT导Hive表tbtxt ,再执行

INSERT OVERWRITE table tb_RC SELECT * from tbtxt ;

2、据我所知,RCFile不能直接导入RC格式的Hive表

 

显示表格式:

desc formatted table_name;

3 thoughts on “HIVE nested ARRAY in MAP data type

  1. Yesterday, while I was at work, my cousin stole my iPad and tested to see if it can survive a forty foot drop, just so she can be a youtube sensation. My apple ipad is now destroyed and she has 83 views. I know this is entirely off topic but I had to share it with someone!

Leave a Reply

Your email address will not be published. Required fields are marked *