AWS, BIG DATA, JAVA

Ping is highly used command to see if the machine is responding or not or if the machine is reachable or not. There is one thing which we forget to realize that Ping uses ICMP protocol and does not use any particular port. So Ping tells that machine is reachable and up  but it does not guarantee that machine is reachable on a particular TCP port which many utilities like SFTP , SSH etc. use.  See a nice article at  https://www.rootusers.com/how-to-ping-a-port/

The "lateral view explode" construct in HIVE allows optimizing  storage for data where only one column values differ for multiple records.  It allows collapsing all such records into a single record, with the column having different values in a array or comma separated list.  By using lateral view explode, you can always generate a exploded view of the data. HIVE supports "lateral view explode" construct but Impala  does not. In Impala you have to achieve the same using complex types like ARRAY. http://www.ericlin.me/2013/09/how-to-use-hive-lateral-view-in-your-query/ http://linehrr.blogspot.com/2013/09/how-to-use-lateral-view-explode.html

* HDFS Commnads   Running file system commands in HDFS:   hdfs dfs -du  <hdfs_path>   hdfs dfs -ls -R   <hdfs_path>  : To list all files in the HDFS path       you can also use wildcard for pattenmatching. like       hdfs dfs -ls  dir1/dir2*/file1   hdfs dfs -rm -R   <hdfs_path> : To remove all files under the hdfs path      hdfs dfs -rm -R -skipTrash  dir1/dir2*/file1  ( wildcard)   hdfs dfs is same as now deprecated "hadoop dfs".  "hdfs dfs"  works only for HDFS , while "hadoop fs .." works all kinds of file systems. * To temporarily enable Kerberos  authentication for a user kinit -k -t  /home/username/username.keytab   username@ doamin