AWS, BIG DATA, JAVA

Spring REST API App Deployment in Azure App Services  Within Azure App Services , Spring REST API application is deployed slightly different than a simple J2EE web application. A Sprint REST application can simply run inside JRE and don't need a container like tomcat. If you are planning to use azure-webapp-maven-plugin , it will need to be configured slightly differently. For a Spring REST API  application  , the plugin  configuration will look like following.     <plugin>         <groupId>com.microsoft.azure</groupId>         <artifactId>azure-webapp-maven-plugin</artifactId>         <version>1.7.0</version>         <configuration>             <!-- Specify v2 schema -->             <schemaVersion>v2</schemaVersion> ...

I learnt following lesson while trying to submit a job using livy  API which was part of azure HDInsight cluster. By default when you use following command. curl  -k --user "admin:<password> " -v  -d  '{"name":"gopal", "address":"1512 edwina ct" }'    https://<clustername>.azurehdinsight.net/livy/batches -H  "X-Requested-By: admin" It will treat the payload as  application/x-www-form-urlencoded. So you need to specify Content-Type header. curl  -k --user "admin:<password> " -v  -H "Content-Type: application/json"   -d  '{"name":"gopal", "address":"1512 edwina ct" }'    https://<clustername>.azurehdinsight.net/livy/batches  -H  "X-Requested-By: admin" This may work fine on linux but on window even this is going to be an issue. So you need put json payload into a file. Let us call the file "payl...

You can quickly launch Azure HDInsight cluster using following link. https://azure.microsoft.com/en-in/resources/templates/101-hdinsight-spark-linux/ A few things I learned while using above link. First thing is it lands you on a page where you need to enter cluster name, cluster admin name and  password , sshuser name and password.  When I entered required info and clicked "Purchase" , it failed. When I looked into the log, I found that issue was related to password. The message coming back is  not always correct. It says the password should be between  6-xx characters. In reality it needs the password to be at least 10 characters with at least one upper case , on small case and 1 number.  I had to spend 15-20 minutes to figure this out and get it working. Once you launch the cluster , you can access the Ambari UI at   <clustername>.azurehdinsight.net. You can access spark cluster via commands like <clustername>.azurehdinsight.net/l...