Google Colab Session and keeping that alive

Google colab takes 90 minutes if we close the browser to disconnect the session. 12 hours if we keep the browser open. Additionally, if we close our browser with a code cell is running, if that same cell has not finished, when we reopen the browser it will still be running (the current executing cell keeps running even after browser is closed).

some times we have to keep out loaded data for long time. Usually google colab keeps disconnecting after 30 mins automatically if we do not respond. And our data is lost.

We can prevent this just by running the following code in the console and it will prevent  from disconnecting. Ctrl+ Shift + i to open inspector view . Then go to console.

function ClickConnect(){
console.log("Working"); 
document.querySelector("colab-toolbar-button#connect").click() 
}
setInterval(ClickConnect,60000)

It would keep on clicking the page and prevent it from disconnecting. It will solve the issue for us.

AI vs ML vs DL vs DS

The most fundamental question of data science is what the basic difference among AI(Artificial Intelligence), ML(Machine Learning), DL(Deep Learning) and DS(Data Science). Many of us have confusion regarding this.

AI(Artificial Intelligence) enables the machine to think so that without human intervention the machine is capable of making its own decision. This is the final goal of an AI application.

Machine Learning is the sub-set of AI. It provides us with a statistical tool to explore and analyze the data for understanding. It has 3 different approaches:

   

Supervised ML:	Using this, Prediction is done on basis of past labeled data
Unsupervised ML:	It provides us the clustering technique . Base on similarities of data, it tries to group the data together. Hierarchical clustering,K-means clustering,K-NN (k nearest neighbors),Principal Component Analysis,Singular Value Decomposition,Independent Component Analysis
Reinforcement/ Semi Supervised ML:	This approach is used when some part of data is labeled and some are not labeled.

Deep learning: Sub-set of Machine Learning: 

It makes the machine learn to think like how the human brain does.

Multi Neural Network Architecture is used for the purpose. Its main target is to mimic the human brain.  Different techniques sued for this are:

ANN	If Data Input is in numeric form.
Convolutional Neural Networks (CNN)	If input is im images, videos .. It generally used in the field of computer vision.
RNN	If input is in form of time series form of data

Data Science: Its a technique tries to apply all the above processes. Statistics, Probability, Linear Algebra, Differential Calculus. All the above things will have to be learnt for becoming a data scientist.  Data science is an inter-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from structured and unstructured data.[1][2] Data science is related to data mining and big data.

Python Libraries needed for scientific computations and data analysis

To import a library, we can do that in below ways:

import math as m

from math import *

In the first manner, we have defined an alias m to library math. We can now use various functions from math library (e.g. factorial) by referencing it using the alias m.factorial().

In the second manner, you have imported the entire name space in math i.e. you can directly use factorial() without referring to math.

Tip: Google recommends that you use first style of importing libraries, as you will know where the functions have come from.

NumPy stands for Numerical Python. The most powerful feature of NumPy is n-dimensional array. This library also contains basic linear algebra functions, Fourier transforms,  advanced random number capabilities and tools for integration with other low level languages like Fortran, C and C++

SciPy stands for Scientific Python. SciPy is built on NumPy. It is one of the most useful library for variety of high level science and engineering modules like discrete Fourier transform, Linear Algebra, Optimization and Sparse matrices.

Matplotlib for plotting vast variety of graphs, starting from histograms to line plots to heat plots.. You can use Pylab feature in ipython notebook (ipython notebook –pylab = inline) to use these plotting features inline. If you ignore the inline option, then pylab converts ipython environment to an environment, very similar to Matlab. You can also use Latex commands to add math to your plot.

Pandas for structured data operations and manipulations. It is extensively used for data munging and preparation. Pandas were added relatively recently to Python and have been instrumental in boosting Python’s usage in data scientist community.

Scikit Learn for machine learning. Built on NumPy, SciPy and matplotlib, this library contains a lot of effiecient tools for machine learning and statistical modeling including classification, regression, clustering and dimensionality reduction.

Statsmodels for statistical modeling. Statsmodels is a Python module that allows users to explore data, estimate statistical models, and perform statistical tests. An extensive list of descriptive statistics, statistical tests, plotting functions, and result statistics are available for different types of data and each estimator.

Seaborn for statistical data visualization. Seaborn is a library for making attractive and informative statistical graphics in Python. It is based on matplotlib. Seaborn aims to make visualization a central part of exploring and understanding data.

Bokeh for creating interactive plots, dashboards and data applications on modern web-browsers. It empowers the user to generate elegant and concise graphics in the style of D3.js. Moreover, it has the capability of high-performance interactivity over very large or streaming datasets.

Blaze for extending the capability of Numpy and Pandas to distributed and streaming datasets. It can be used to access data from a multitude of sources including Bcolz, MongoDB, SQLAlchemy, Apache Spark, PyTables, etc. Together with Bokeh, Blaze can act as a very powerful tool for creating effective visualizations and dashboards on huge chunks of data.

SymPy for symbolic computation. It has wide-ranging capabilities from basic symbolic arithmetic to calculus, algebra, discrete mathematics and quantum physics. Another useful feature is the capability of formatting the result of the computations as LaTeX code.

Scrapy for web crawling. It is a very useful framework for getting specific patterns of data. It has the capability to start at a website home url and then dig through web-pages within the website to gather information.

Requests for accessing the web. It works similar to the the standard python library urllib2 but is much easier to code. You will find subtle differences with urllib2 but for beginners, Requests might be more convenient.

jupyter notebook commands

To start jupyter:

jupyter-navigator

To get the list of running jupiter instance:

jupyter notebook list

To set password for the jupiter instead of temporary token id:

jupyter notebook password

Installing the latest Version of R in Ubuntu

By executing the below commands we can install R in ubuntu. 

sudo apt-get --purge remove r-base
sudo apt-get --purge remove r-base-dev

This install the default version of R. If we want to install the latest version of R, we can follow as:
Uninstalling the previous version of R


sudo apt-get --purge remove r-base
sudo apt-get --purge remove r-base-dev
sudo apt-get --purge remove r-base-core


After that to install R:

sudo add-apt-repository "deb http://cran.rstudio.com/bin/linux/ubuntu $(lsb_release -sc)/"


Then we will have to execute the below commands:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9
sudo add-apt-repository ppa:marutter/rdev
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install r-base

Reinstalling R in Ubuntu

For the case if we want to reinstall R on our ubuntu, we have to uninstall R at first. To do so, we can use the bwlow commands:


sudo apt-get --purge remove r-base
sudo apt-get --purge remove r-base-dev

As r-base package is just a metapackage that also installs r-base-core internally. We will have to remove r-base-core to remove fully.

sudo apt-get --purge remove r-base-core

After removing all the packge we will have to use below command to install the R again:


sudo apt-get install r-base
sudo apt-get install r-base-dev