[Must Read] Popular Handpicked best Books for Data science

I started learning data science about a years ago..  This is mostly geared towards people who are in the same position I was in.

A lot of advice around learning data science starts with "first learn python", or "first take a linear algebra course".  This advice is fine, but if I followed it, I never would have learned any data science. Being data scientist requires a solid foundation typically in computer science and applications,  modelling, statistics, analytics and math. 

What sets the data scientist  apart is strong business acumen, coupled with the ability to communicate  findings to both business and IT leaders in a way that can influence  how an organization approaches a business challenge. Good data  scientists will not just address business problems, they will pick the  right problems that have the most value to the organization.

Here is a list of books on doing machine learning / data science in R and Python which I’ve come across in last one year. Since, reading helps to keep close to the topic it will also works and reference guide.

Disclosure: The amazon links in this article are affiliate links. If you buy a book through this link, we would get paid through Amazon. This is one of the ways for us to cover our costs while we continue to create these awesome articles. Further, the list reflects our recommendation based on content of book and is no way influenced by the commission.

Python and R

1) Python for Data Analysis

I was new to Python and to Data Analysis when I started working through this book. I found the chapters on numpy particularly useful: what was also helpful was to have everything on one place, rather than having to scratch around for it online.

The book covers the basics of Python, as well as IPython, Numpy and Pandas. I still use it now as a reference. If you’re well versed in python and data analysis, it’s probably worth the purchase price; but if you’re new to it all, I would definitely recommend it.

2)Think Python

However, Think Python is a book I'd recommend over  again and again to anyone who seeks a gentle introduction to the good parts of the Python language.

The book, as I've found is often recommended by professionals everywhere - for example, right on Quora, stack overflow. Personally, it's one of those few books I've managed to go through cover-to-cover .

3)R for Data Science: Import, Tidy, Transform, Visualize, and Model Data

If you've used `R` in the past but mainly use base functions then this will be a great refresher for you. If you're new to the world of `R` then this book will give you a solid foundation of how to get started. It is a collection of R packages designed to work together to make data science fast and fluent.

Statistical and Math for data analysis

1) Think stat (2nd edition)

This book covers more aspect of statistics required to get your hands dirty by learning to do practical work. This book works completely fine for  beginner as well .

2) Introduction to Probability

An intuitive, yet precise introduction to probability theory, stochastic processes, statistical inference, and probabilistic models used in science, engineering, economics, and related fields. This is the currently used textbook for "Probabilistic Systems Analysis," an introductory probability course at the Massachusetts Institute of Technology, attended by a large number of undergraduate and graduate students.

3) An Introduction to Statistical Learning with Applications in R

“An Introduction to Statistical Learning With Applications in R” gives you an overview of analyzing, organizing, and leveraging data using the powerful and popular R programming language. Written by Gareth James, professor of data sciences at USC; Daniela Witten, professor of biostatistics at University of Washington; and Robert Tibshirani and Trevor Hastie, professors of statistics at Stanford, it is ideal for both statisticians and non-technical professionals who are looking to understand data management, analysis, and presentation techniques.

Big data

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die

In “Predictive Analytics“, Eric Siegel, a renowned expert in data analytics and former professor at Columbia University, explains how scientists use big data to help predict, well, anything – from what you will buy, to where you will travel, to when you will quit your job, and more. The Seattle Post-Intelligencer called the book “mesmerizing,” and also praised its relevance to multiple business departments.

Hadoop, the Definitive Guide

Apache Hadoop is a framework used to process large amounts of data. Tom White is an expert Hadoop consultant, trainer, and member of the Apache Software Foundation. His guide, “Hadoop: The Definitive Guide: Storage and Analysis at Internet Scale,” will help you understand how to build and manage scalable systems using Hadoop. It’s a good reference for programmers, and for IT managers tasked with running Hadoop clusters.

Big Data: A Revolution That Will Transform How We Live, Work, and Think

This book is written by Kenneth Cukier and Viktor Mayer Schonberger. This book takes you on a world tour of values added by big data across all industries. This book will help you to stay ahead of the key trends defining businesses in coming years. Jeff Jonas, Chief Scientist, IBM Entity Analytics said, ‘The book teems with great insights on the new ways of harnessing information, and offers a convincing vision of the future. It is essential reading for anyone who uses — or is affected by — big data.’

We will keep on updating the list with few more resource and books  .Feel free to share your views and suggestion which will be helpful to refer to become a better data scientist.

Pandas in Python - Dataframe Tutorial(With examples)

Creating Data Frames¶

In the previous post we looked the tutorial on basic of Series in Pandas.In this part of tutorial we will be looking building one of the important data structure in pandas "The DataFrame" .Pandas has an abundance of functionality, far too much for me to cover in this introduction.We will cover more functions of dataframe in the example sections which will be coming in next post!
Hope you are enjoying by learning, our Suggestion would be to practice by writing and calling the functions and understanding it.

Lets get Started!¶

DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it
like a spreadsheet or SQL table, or a dict of Series objects.
The Important
You can create a data frame using:

• Dict of 1D ndarrays, lists, dicts, or Series
• 2-D numpy.ndarray
• Structured or record ndarray
• A Series
• Another DataFrame

Data Frame attributes¶

Importing numpy and pandas librabry¶

In [61]:

import pandas as pd
import numpy as np

How to create data frame from Python dictionary ?¶

In [62]:

my_dictionary = {'a' : 45.1, 'b' : -19.52, 'c' : 4444}
print(my_dictionary.keys())
print(my_dictionary.values())

['a', 'c', 'b']
[45.1, 4444, -19.52]

We'll call the dictionary My_Dictionary. It will have three values, 45.1, minus 19.52 and 4,444. Its keys will be A, B, and C.
We can print both the keys and the values. And we see that the keys are C, D, A, and the values are the same as those that we used in the input. Now, let's use the dictionary in a data frames constructor. The column headers for the data frame are derived from the keys in the dictionary.

In [63]:

my_dictionary_df = pd.DataFrame(my_dictionary, index=['first', 'again'])
my_dictionary_df

Out[63]:

	a	b	c
first	45.1	-19.52	4444
again	45.1	-19.52	4444

How to use constructor without explicit index¶

The values in the dictionary are replicated one time, that is one row, for each of the values in the index. In this case, first and again. In this example from the Pandas cookbook, the constructor has three labels and three lists. Each of the lists has four values. Since an index is not included within the constructor, the integers zero through 3 are used as an index and displayed as row labels. In this example, we create a dictionary whose labels, that is, whose keys, are 1 and 2, and the values associated with each of these keys are a series.

In [64]:

cookbook_df = pd.DataFrame({'AAA' : [4,5,6,7], 'BBB' : [10,20,30,40],'CCC' : [100,50,-30,-50]})
cookbook_df

Out[64]:

	AAA	BBB	CCC
0	4	10	100
1	5	20	50
2	6	30	-30
3	7	40	-50

How to use constructor contains dictionary with Series as values¶

The series has a list of values and its own index. When we display this by pressing shift + enter, we see the column headers again are derived from the keys, and the values are derived from the indices. Note that since the key 1 has an index with A, B, and C, but does not include D, the value NaN, or Not A Number, is displayed for this result. In this example, we create a dictionary whose keys and values are strings.

In [65]:

series_dict = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']),
               'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])}
series_df = pd.DataFrame(series_dict)
series_df

Out[65]:

	one	two
a	1.0	1.0
b	2.0	2.0
c	3.0	3.0
d	NaN	4.0

How to create df using dictionary of lists¶

In [66]:

produce_dict = {'veggies': ['potatoes', 'onions', 'peppers', 'carrots'],
                'fruits': ['apples', 'bananas', 'pineapple', 'berries']}
produce_dict

Out[66]:

{'fruits': ['apples', 'bananas', 'pineapple', 'berries'],
 'veggies': ['potatoes', 'onions', 'peppers', 'carrots']}

In [67]:

pd.DataFrame(produce_dict)

Out[67]:

	fruits	veggies
0	apples	potatoes
1	bananas	onions
2	pineapple	peppers
3	berries	carrots

list of dictionaries¶

In [68]:

data2 = [{'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20}]
pd.DataFrame(data2)

Out[68]:

	a	b	c
0	1	2	NaN
1	5	10	20.0

dictionary of tuples, with multi index¶

we use a dictionary of Tuples to create a data frame that has a multi index. Here, you can see the multiple levels of column headers.

In [69]:

pd.DataFrame({('a', 'b'): {('A', 'B'): 1, ('A', 'C'): 2},
('a', 'a'): {('A', 'C'): 3, ('A', 'B'): 4},
('a', 'c'): {('A', 'B'): 5, ('A', 'C'): 6},
('b', 'a'): {('A', 'C'): 7, ('A', 'B'): 8},
('b', 'b'): {('A', 'D'): 9, ('A', 'B'): 10}})

Out[69]:

		a			b
		a	b	c	a	b
A	B	4.0	1.0	5.0	8.0	10.0
	C	3.0	2.0	6.0	7.0	NaN
	D	NaN	NaN	NaN	NaN	9.0

How to Select, Add, Delete, Columns in df¶

we'll examine how to select, add and delete columns from a data frame. The select file in your exercises files folders is prepopulated with import statements for pandas and num pi. Execute the cell by pressing shift+enter. If we create a data frame from the pandas cookbook we can reference columns in the data frame using a dictionary like syntax. In this cell we reference the second column with the string BBB.

dictionary like operations¶

dictionary selection with string index¶

In [70]:

cookbook_df = pd.DataFrame({'AAA' : [4,5,6,7], 'BBB' : [10,20,30,40],'CCC' : [100,50,-30,-50]})
cookbook_df['BBB']

Out[70]:

0    10
1    20
2    30
3    40
Name: BBB, dtype: int64

arithmetic vectorized operation using string indices¶

In [71]:

cookbook_df['BBB'] * cookbook_df['CCC']

Out[71]:

0    1000
1    1000
2    -900
3   -2000
dtype: int64

column deletion¶

In [72]:

del cookbook_df['BBB']
cookbook_df

Out[72]:

	AAA	CCC
0	4	100
1	5	50
2	6	-30
3	7	-50

We can use these string references or these string selections in arithmetic vectorized operations. Copying from the final version of your file, we can take the column BBB and multiply every value in the column by every column in the column CCC. Here we see we have ten times a hundred, twenty times fifty, thirty times -30 and forty times -50. There are two ways that we can release columns from a data frame, the DEL or delete operator and the pop function.

In [73]:

last_column = cookbook_df.pop('CCC')
last_column

Out[73]:

0    100
1     50
2    -30
3    -50
Name: CCC, dtype: int64

In [24]:

cookbook_df

Out[24]:

	AAA
0	4
1	5
2	6
3	7

add a new column using a Python list¶

In [25]:

cookbook_df['DDD'] = [32, 21, 43, 'hike']
cookbook_df

Out[25]:

	AAA	DDD
0	4	32
1	5	21
2	6	43
3	7	hike

insert function¶

documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.insert.html

In [26]:

cookbook_df.insert(1, "new column", [3,4,5,6])
cookbook_df

Out[26]:

	AAA	new column	DDD
0	4	3	32
1	5	4	21
2	6	5	43
3	7	6	hike

Indexing and Selection¶

Operation	Syntax	Result
Select column	df[col]	Series
Select row by label	df.loc[label]	Series
Select row by integer	df.iloc[loc]	Series
Select rows	df[start:stop]	DataFrame
Select rows with boolean mask	df[mask]	DataFrame

documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html

Note the double square brackets. We can select a row from a data frame using an integer index, using the I location, or the I L-O-C function. In this case, we're selecting the row whose index is equal to two. The fruit associated with this row is pineapple, and the veggies associated with this row is peppers. We can select a range of rows by using an integer slice. In this case, we obtain the rows zero through one, up to, but not including, two.
We can also use a slice to count backwards using negative numbers within the slice. In this example, we see that the plus symbol is overloaded as a concatenation operator when dealing with data frames. When we execute this cell, we see that apples is concatenated to each of the previous values in the fruit column.

In [32]:

nutrient_dict = {'veggies': ['potatoes', 'carrot', 'beans', 'leafy'],'fruits': ['apples', 'mango', 'pineapple', 'banana']}
nutrient_df = pd.DataFrame(produce_dict)
nutrient_df

Out[32]:

	fruits	veggies
0	apples	potatoes
1	mango	carrot
2	pineapple	beans
3	banana	leafy

How to select using dectionary-like String¶

In [33]:

nutrient_df['fruits']

Out[33]:

0       apples
1        mango
2    pineapple
3       banana
Name: fruits, dtype: object

How to Select row using integer index¶

In [74]:

nutrient_df.iloc[2:]

Out[74]:

	fruits	veggies
2	pineapple	beans
3	banana	leafy

Slicing the row¶

In [43]:

nutrient_df.iloc[3:4]

Out[43]:

	fruits	veggies
3	banana	leafy

+ is over-loaded as concatenation operator¶

In [46]:

nutrient_df + nutrient_df.iloc[0]

Out[46]:

	fruits	veggies
0	applesapples	potatoespotatoes
1	mangoapples	carrotpotatoes
2	pineappleapples	beanspotatoes
3	bananaapples	leafypotatoes

Data alignment and arithmetic¶

Data alignment between DataFrame objects automatically align on both the columns and the index (row labels).
Note locations for 'NaN'

In [75]:

df = pd.DataFrame(np.random.randn(10, 4), columns=['A', 'B', 'C', 'D'])
df2 = pd.DataFrame(np.random.randn(7, 3), columns=['A', 'B', 'C'])
sum_df = df + df2
sum_df

Out[75]:

	A	B	C	D
0	2.796434	0.681719	1.249369	NaN
1	-1.920570	-0.748472	-0.455429	NaN
2	-0.335982	-2.323809	0.365608	NaN
3	-0.565566	0.885914	-1.261485	NaN
4	-0.315269	0.300453	0.582013	NaN
5	-0.076879	0.762971	-1.182593	NaN
6	0.460198	-0.533756	-1.903300	NaN
7	NaN	NaN	NaN	NaN
8	NaN	NaN	NaN	NaN
9	NaN	NaN	NaN	NaN

Boolean Indexing¶

In [76]:

sum_df>0

Out[76]:

	A	B	C	D
0	True	True	True	False
1	False	False	False	False
2	False	False	True	False
3	False	True	False	False
4	False	True	True	False
5	False	True	False	False
6	True	False	False	False
7	False	False	False	False
8	False	False	False	False
9	False	False	False	False

In [77]:

sum_df[sum_df>0]

Out[77]:

	A	B	C	D
0	2.796434	0.681719	1.249369	NaN
1	NaN	NaN	NaN	NaN
2	NaN	NaN	0.365608	NaN
3	NaN	0.885914	NaN	NaN
4	NaN	0.300453	0.582013	NaN
5	NaN	0.762971	NaN	NaN
6	0.460198	NaN	NaN	NaN
7	NaN	NaN	NaN	NaN
8	NaN	NaN	NaN	NaN
9	NaN	NaN	NaN	NaN

first select rows in column B whose values are less than zero
then, include information for all columns in that row in the resulting data set

One more on using where function¶

In [51]:

nutrient_df.where(nutrient_df > 'k')

Out[51]:

	fruits	veggies
0	NaN	potatoes
1	mango	NaN
2	pineapple	NaN
3	NaN	leafy

Important links data frames from various data types¶

documentation: http://pandas.pydata.org/pandas-docs/stable/dsintro.html
cookbook: http://pandas.pydata.org/pandas-docs/stable/cookbook.html
documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html

Great! we have learnt about lots of function in pandas to deal with dataframe. Feel free to fork this long notebook on github and try it .Feel free to share with other learners and on social media.
In the next post we are going to start with Plotting which is again most important as visualisation will help to understand the data better!
See you soon with next post! Happy coding.

In [ ]:

Pandas in Python for Data Analysis with Example(Step-by-Step guide)

Beginners Pandas Getting Started¶

Pandas is a high-level data manipulation tool developed by Wes McKinney. It is built on the Numpy package and its key data structure is called the DataFrame. DataFrames allow you to store and manipulate tabular data in rows of observations and columns of variables.

• If you're new to this first get the enviroment Setup in our previous post
  • Getting Started with Jupyter [Part -1] http://www.androidxu.com/2017/04/guide-On-Jupyter-Notebook.html
  • Getting Started with Jupyter [Part -2] http://www.androidxu.com/2017/04/the-ultimate-guide-on-jupyter-ipython-mardown.html#.WPJOBYVOL4g

pandas is well suited for:

• Tabular data with heterogeneously-typed columns, as in an SQL table or Excel spreadsheet
• Ordered and unordered (not necessarily fixed-frequency) time series data.
• Arbitrary matrix data (homogeneously typed or heterogeneous) with row and column labels
• Any other form of observational / statistical data sets. The data actually need not be labeled at all to be placed into a pandas data structure

Key features:

• Easy handling of missing data
• Size mutability: columns can be inserted and deleted from DataFrame and higher dimensional objects
• Automatic and explicit data alignment: objects can be explicitly aligned to a set of labels, or the data can be aligned automatically
• Powerful, flexible group by functionality to perform split-apply-combine operations on data sets
• Intelligent label-based slicing, fancy indexing, and subsetting of large data sets
• Intuitive merging and joining data sets
• Flexible reshaping and pivoting of data sets
• Hierarchical labeling of axes
• Robust IO tools for loading data from flat files, Excel files, databases, and HDF5
• Time series functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging, etc.

We’ll start with a quick, non-comprehensive overview of the fundamental data structures in pandas to get you started. The fundamental behavior about data types, indexing, and axis labeling / alignment apply across all of the objects. To get started, import numpy and load pandas into your namespace:
documentation: http://pandas.pydata.org/pandas-docs/stable/10min.html

Series¶

Series is a one-dimensional labeled array capable of holding any data type (integers, strings, floating point numbers,
Python objects, etc.). The axis labels are collectively referred to as the index.
documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.html

In [38]:

#importing numpy and pandas library
import pandas as pd
import numpy as np

Create series from NumPy array¶

Creating a basic series from NumpPy array.
Number of labels in 'index' must be the same as the number of elements in array

In [39]:

my_simple_series = pd.Series(np.random.randn(7), index=['a', 'b', 'c', 'd', 'e','f','g'])
my_simple_series

Out[39]:

a    0.623720
b    0.397227
c    0.470759
d    0.323920
e   -1.186631
f   -1.175695
g    0.744503
dtype: float64

In [40]:

my_simple_series.index

Out[40]:

Index([u'a', u'b', u'c', u'd', u'e', u'f', u'g'], dtype='object')

Create series from NumPy array, without explicit index¶

In [41]:

my_simple_series = pd.Series(np.random.randn(5))
my_simple_series

Out[41]:

0    1.285379
1   -0.672387
2   -0.720461
3   -0.263968
4    0.547311
dtype: float64

Access a series like a NumPy array

In [42]:

my_simple_series[:3]

Out[42]:

0    1.285379
1   -0.672387
2   -0.720461
dtype: float64

Create series from Python dictionary¶

In [43]:

my_dictionary = {'a' : 45., 'b' : -19.5, 'c' : 4444}
my_second_series = pd.Series(my_dictionary)
my_second_series

Out[43]:

a      45.0
b     -19.5
c    4444.0
dtype: float64

Access a series like a dictionary

In [44]:

my_second_series['b']

Out[44]:

-19.5

note order in display; same as order in "index"
note NaN

In [45]:

pd.Series(my_dictionary, index=['b', 'c', 'd', 'a'])

Out[45]:

b     -19.5
c    4444.0
d       NaN
a      45.0
dtype: float64

In [46]:

my_second_series.get('a')

Out[46]:

45.0

In [47]:

unknown = my_second_series.get('f')
type(unknown)

Out[47]:

NoneType

Create series from scalar¶

If data is a scalar value, an index must be provided. The value will be repeated to match the length of index

In [48]:

pd.Series(5., index=['a', 'b', 'c', 'd', 'e'])

Out[48]:

a    5.0
b    5.0
c    5.0
d    5.0
e    5.0
dtype: float64

Vectorized Operations¶

• not necessary to write loops for element-by-element operations
• pandas' Series objects can be passed to MOST NumPy functions

documentation: http://pandas.pydata.org/pandas-docs/stable/basics.html

In [49]:

my_dictionary = {'a' : 45., 'b' : -19.5, 'c' : 4444}
my_series = pd.Series(my_dictionary)
my_series

Out[49]:

a      45.0
b     -19.5
c    4444.0
dtype: float64

Add Series without loop¶

In [50]:

my_series + my_series

Out[50]:

a      90.0
b     -39.0
c    8888.0
dtype: float64

In [51]:

my_series

Out[51]:

a      45.0
b     -19.5
c    4444.0
dtype: float64

Series within arithmetic expression¶

In [52]:

#adding values into a series
my_series +5

Out[52]:

a      50.0
b     -14.5
c    4449.0
dtype: float64

Series used as argument to NumPy function¶

In [53]:

np.exp(my_series)

Out[53]:

a    3.493427e+19
b    3.398268e-09
c             inf
dtype: float64

A key difference between Series and ndarray is that operations between Series automatically align the data based on
label. Thus, you can write computations without giving consideration to whether the Series involved have the same labels.

In [54]:

my_series[1:]

Out[54]:

b     -19.5
c    4444.0
dtype: float64

In [55]:

my_series[:-1]

Out[55]:

a    45.0
b   -19.5
dtype: float64

In [56]:

my_series[1:] + my_series[:-1]

Out[56]:

a     NaN
b   -39.0
c     NaN
dtype: float64

Apply Python functions on an element-by-element basis¶

In [57]:

def multiply_by_ten (input_element):
    return input_element * 10.0

In [58]:

my_series.map(multiply_by_ten)

Out[58]:

a      450.0
b     -195.0
c    44440.0
dtype: float64

Vectorized string methods¶

Series is equipped with a set of string processing methods that make it easy to operate on each element of the array. Perhaps most importantly, these methods exclude missing/NA values automatically.

In [59]:

series_of_strings = pd.Series(['A', 'B', 'C', 'Aaba', 'Baca', np.nan, 'CABA', 'dog', 'cat'])

In [60]:

series_of_strings.str.lower()

Out[60]:

0       a
1       b
2       c
3    aaba
4    baca
5     NaN
6    caba
7     dog
8     cat
dtype: object

In the next post we will continue seeing the arithmetic Operations, So Subscribe it and Stay tuned!¶

Please Subscribe and Share with fellow developer!

• Reference resource :
  • documentation: http://pandas.pydata.org/pandas-docs/stable/10min.html
  • documentation: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.Series.html