MiNiFi

Apache NiFi/MiNiFi - A subproject to collect data where it originates(source).

Comments

Post a Comment

Popular posts from this blog

Unit of Storage

Unit of Storage Unit Description Kilobyte 1024 Bytes Megabyte 1024 Kilobytes Gigabyte 1024 Megabytes Terabyte 1024 Gigabytes Petabyte 1024 Terabytes Exabyte 1024 Petabytes Zettabyte 1024 Exabytes Yottabyte 1024 Zettabytes Brontobyte 1024 Yottabytes
Read more

Statistics & Machine Learning

Useful Resources  Statistics http://www.statisticshowto.com/probability-and-statistics/statistics-definitions/ http://tutorials.istudy.psu.edu/basicstatistics/basicstatistics2.html http://www.reading.ac.uk/ssc/resources/Docs/Statistical_Glossary.pdf http://blog.minitab.com/blog/adventures-in-statistics-2/understanding-analysis-of-variance-anova-and-the-f-test Random Forest - http://blog.yhat.com/posts/random-forests-in-python.html https://www.analyticsvidhya.com/blog/2016/04/complete-tutorial-tree-based-modeling-scratch-in-python/#one http://www.ling.upenn.edu/~clight/chisquared.htm http://analyticstrainings.com/?p=151 Feature Selection - Machine Learning http://blog.datadive.net/selecting-good-features-part-i-univariate-selection/ http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importances.html
Read more

Kafka

Kafka is streaming ( can also be used as messaging, message storage) platform which provide best of both traditional messaging models:  message queues and publish-subscribe. At the same time it serves as durable distributed storage and provides in order guarantee of messages and above all it is scalable. Kafka APIs :  Producer, Consumer , Stream ( Allow creating data processing pipelines ), Connector. For more details see  Kafka Introduction Kafka clusters can span multiple data centers ( durable, distributed storage ) . Consider each topic like a folder in the file-system and each event as a file. Because each Kafka topic ( stream of records ) is stored in partitioned logs ( number is configurable) for configurable retention period, multiple clients can read those in the ways they want. Hence Kafka topics are multi-subscriber.  Also there is very less overhead per consumer.  The only metadata needed to be kept is offset ( in the partition ).  T...
Read more