AWS, BIG DATA, JAVA

Spring REST API App Deployment in Azure App Services  Within Azure App Services , Spring REST API application is deployed slightly different than a simple J2EE web application. A Sprint REST application can simply run inside JRE and don't need a container like tomcat. If you are planning to use azure-webapp-maven-plugin , it will need to be configured slightly differently. For a Spring REST API  application  , the plugin  configuration will look like following.     <plugin>         <groupId>com.microsoft.azure</groupId>         <artifactId>azure-webapp-maven-plugin</artifactId>         <version>1.7.0</version>         <configuration>             <!-- Specify v2 schema -->             <schemaVersion>v2</schemaVersion> ...

I learnt following lesson while trying to submit a job using livy  API which was part of azure HDInsight cluster. By default when you use following command. curl  -k --user "admin:<password> " -v  -d  '{"name":"gopal", "address":"1512 edwina ct" }'    https://<clustername>.azurehdinsight.net/livy/batches -H  "X-Requested-By: admin" It will treat the payload as  application/x-www-form-urlencoded. So you need to specify Content-Type header. curl  -k --user "admin:<password> " -v  -H "Content-Type: application/json"   -d  '{"name":"gopal", "address":"1512 edwina ct" }'    https://<clustername>.azurehdinsight.net/livy/batches  -H  "X-Requested-By: admin" This may work fine on linux but on window even this is going to be an issue. So you need put json payload into a file. Let us call the file "payl...

You can quickly launch Azure HDInsight cluster using following link. https://azure.microsoft.com/en-in/resources/templates/101-hdinsight-spark-linux/ A few things I learned while using above link. First thing is it lands you on a page where you need to enter cluster name, cluster admin name and  password , sshuser name and password.  When I entered required info and clicked "Purchase" , it failed. When I looked into the log, I found that issue was related to password. The message coming back is  not always correct. It says the password should be between  6-xx characters. In reality it needs the password to be at least 10 characters with at least one upper case , on small case and 1 number.  I had to spend 15-20 minutes to figure this out and get it working. Once you launch the cluster , you can access the Ambari UI at   <clustername>.azurehdinsight.net. You can access spark cluster via commands like <clustername>.azurehdinsight.net/l...

Few nice links: https://css-tricks.com/html5-progress-element/ https://stackoverflow.com/questions/7988962/continuous-css-rotation-animation-on-hover-animated-back-to-0deg-on-hover-out Remember even <br/> can impact the positioning of elements following it. 

1. Two very useful links if you are trying to setup Angular environment on windows 10 in a corporate network. https://www.zeolearn.com/magazine/setup-angular-windows https://yakucoding.com/npm-and-corporate-proxy-pac/ https://yakovfain.com/2016/11/06/angular-cli-speed-up-installing-dependencies-with-yarn/ https://coryrylan.com/blog/angular-ng-for-syntax 2. Very useful free courses/links for Angular: https://scrimba.com/g/gyourfirstangularapp https://nehalist.io/working-with-models-in-angular/ 3. You can change the style  dynamically using property binding. [style.background-color]="expression" 4. Be careful searching for help on Internet about Angular , as there is lots of documentation around old versions which no longer apply. 5. At times when you are making an  call in following manner and let us say getUser() is actually making an http call to a REST service. this . statusService . getUser (). subscribe (( value : Body ) => { this . resp = value...

Amazon Aurora allows creating up 15 read replicas in a region. Read replicas share the source storage. Storage scales automatically into 10GB increments up to 64 TB. There is option for Global Database which allows single database to span across multiple regions.  It uses storage based replication. By default Aurora database is replicated six ways across three AZs. It can handle loss of two copies without affecting writes and three copies without affecting reads. Aurora also has a backtrack feature, it can go back to point in time without restore from a backup. 

For every domain which you want to manage, you need to create a hosted zone. Within Hosted zone , you can configure various record sets which specify how traffic within that domain will be routed. There are various types of records.  Typically A record points to an IP address or is an alias to another endpoint or record within same hosted zone.    CNAME record is slightly different. CNAME can point to another  domain or sub-domain ( not IP address ) . You can not create  CNAME for zone -apex (or  hosted zone itself).  A CNAME record can redirect DNS queries to any DNS record. A few important differences between CNAME and aliases are listed on below link. https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-choosing-alias-non-alias.html

1. How to get the list of all the tables , views and stored procedures etc from a  Teradata database. select TableKind , TableName from DBC.TABLES where DatabaseName='db_name' TableKind  can be T, O, Q, P, V ....

HTTP/2.0 modifies  how data is formatted and transported between client and server, in doing so , it encapsulates all this  additional complexity in a new framing layer both on the client and server side.  So  applications built on previous versions of protocol continue to work except with added benefits of being faster and with some new capabilities at their disposal  like request prioritization , flow control and server push. 

It is two way ( full-duplex - in both direction at same time ) communication protocol between server & client using long running TCP connection.  Client  & server both  can send messages to one another in asynchronous manner.  The messages are sent with very little overhead and hence results in very low latency. Websockets remove the need for long polling in the HTTP based applications needing real time updates from server.  Long-polling is done so that client application does not have to keep polling/requesting  server for updates as it results in  frequent HTTP connection overheads.  So client makes a request and server waits as long as it can, until either there is update for client or it times out. This is called long polling.  How websockets work ? 

Kinesis Data Analytics can use standard SQL queries to process kinesis data streams and can ingest data from Kinesis Data Streams and kinesis Firehose.  Kinesis Firehose can be used for running SQL queries. ( Udemy ). DynamoDB has two pricing models; on-demand ( unpredictable traffic ) and provisioned (predictable). With API Gateway, you can maintain multiple versions and multiple stages of each version of an API. Access Policy vs. ACL IAM Policy vs. Password Policy You can not assign deny rules to a security group. ALB only features: - Path Based Routing - Host Based Routing - HTTP Method Based Routing - HTTP Header Based Routing -Query String Parameter Based Routing -Source IP Address-CIDR Based Routing -Native HTTP/2  NLB  only features: -Static IP -Elastic IP -Source IP Preservation ( at some places it talks that all ALB , GLB also support this )  CLB only features -Custom security policies SNI , secure listener Amazon Cognito lets ...