AWS, BIG DATA, JAVA

* Good article on MapReduce http://www.bigsynapse.com/mapreduce-internals *  org.apache.hadoop.mapreduce.Mapper<KEYIN,VALUEIN,KEYOUT,VALUEOUT>  is generic class. There is an Mapper generic  interface too but it is in different package   org.apache.hadoop.mapre.Mapper<K1,V1,K2,V2> and has nothing to do with the Mapper class. The Reducer is also a generic class similar to Mapper. * Hadoop also has two special annotations to indicate  the audience and stability of any interface or class        -InterfaceStability        -InterfaceAudience * All the key value types in Hadoop Map Reduce programming must implement Writable interface, which results in type implementing efficient serialization using DataInput/DataOutput. * hadoop fs -put   input/sample.txt   hdfs://quickstart.cloudera:8020/sample.txt  returned "ConnectException"  because namenode service was not running.  Used f...

"Verifying  jdk......" I just got a new macbook pro OS X EL Capitan and I was not able to run applications needing java. I tried installing java but it always got stuck at "verifying jdk.......".  Finally I saw a post on google where it mentioned that issues was due to SIP ( System Integrity Protection) being enabled. I found an article how to disable it. http://osxdaily.com/2015/10/05/disable-rootless-system-integrity-protection-mac-os-x/ Once I disable , I was able to install jdk.

1. To get sum for all the values from column B1 to B6 where value in column C1 to C6 is "S1"  all  from Sheet1. =SUMIF(Sheet1!C1:C6,"S2",Sheet1!B1:B6) See "S2" in double quotes because without that, it will represent the value in cell S2. 2. In a formula, a single string can only 255 char long. So if you need to have longer string, you will have represent it by concatenation of multiple smaller strings like  for example   "abcdef" => "abc" & "def" 3.  If you are editing a long formula and getting errors, one thing you might want to look for is new lines in it esp. if you copied the text and pasted it from some place. 4. LEFT(txt , num_chars) => Returns  left num_chars characters from   txt. 5. How to escape double quote in string  as part of formula ? by double quote itself .   If you want to have "abc" as text where double quotes are included in it ...you have to put it  like ""...

1. How to enable secure cookies in an application.  Include below config in web.xml at the top level.   <session-config>     <cookie-config>       <http-only>true</http-only>       <secure>true</secure>     </cookie-config>   </session-config> Once you do that , you will have request server to set secure cookies or http only cookies and it will set those . Without this config , even if you request server to set secure cookies or http only cookies , it wont do it.  This is true at least for Tomcat. 2. Some versions of  Tomcat remove trailing =  in cookies.  To stop Tomcat from removing the trailing "="  signs, start Tomcat with  "-Dorg.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true"  option.eds

1. To find the list of IPs of any particular attributes of the nodes belonging to a particular stack  aws ec2  describe-instances --filters "Name=tag:aws:cloudformation:stack-name,Values=yourstack"  --output text --query 'Reservations[*].Instances[*].PrivateIpAddress' 2.  If you multiple VPCs , make sure your access keys which used during aws  configure point to the VPC you are searching the instances in.  If you are looking for a stack which returns zero result and stack is there , most likely your keys belong to different VPC. 3.  To get only the required attributes as key value pair from AWS CLI output.  aws ec2  describe-instances --filters "Name=tag:aws:cloudformation:stack-name,Values=yourstack" --output text --query 'Reservations[*].Instances[*].{IP:PrivateIpAddress,ID:InstanceId}' 4. To stop multiple instances aws ec2 stop-instances --instance-ids  "i-06eea80" "i-00eww42a86" 5. To upload all *.sh files from a ...

ssoadm/openam/bin/ssoadm get-attr-defs  -u amadmin -f passwd -s iPlanetAMPlatformService -t global -a "iplanet-am-platform-cookie-domains" 1.  At times when you create a new module instance under a newly created realm , it does not show up the in list of module instances.  But if I delete it using ssoadm and recreate it , it shows up.

1. Concatenating the fields SELECT CONCAT('http://www.abc.com?parentCategoryId=' ,CATEGORY_ID,'&categoryId=', RELATED_CATEGORY_ID ) FROM RELATED_CATEGORY WHERE CATEGORY_ID IN (SELECT CATEGORY_ID FROM CATALOG_TO_CATEGORY WHERE CATALOG_ID = 1 AND ACTIVE_FLAG='Y') 2.  ERROR 1148 (42000): The used command is not allowed with this MySQL version .  LOAD DATA LOCAL INFILE 'users.txt' INTO TABLE  users FIELDS TERMINATED BY  ',' ENCLOSED BY '"'  LINES TERMINATED BY '\n' (userid,fistname); Throws an error something like " ERROR 1148 (42000): The used command is not allowed with this MySQL version ."  For me the mysql server was an RDS and I was trying to load data from another ec2 instance.  When I google around I got suggestions to restart mysqld as "mysql -u user -p --local-infile menagerie" , which was not possible as it was an RDS service. To get it working I had to add following in /etc/mysq...