Python Notes-12

In continuation from Python Notes-11 Python Notes-10 Python Notes-9 Python Notes-8 Python Notes-7 Python Notes-6 Python Notes-5 Python Notes-4 Python Notes-3 Python Notes-2 Python Notes-1

pandas

import pandas as varanasi_panda 🙂 Each time I import this library this is what comes to my mind and then the panda (cute Chinese animal) every time. FYI I am atheist.


# Import pandas as pd

import pandas as pd

# Import the cars.csv data: cars

cars = pd.read_csv(‘cars.csv’)

# Print out cars

print(cars)

dict = {“country”: [“Brazil”, “Russia”, “India”, “China”, “South Africa”],
“capital”: [“Brasilia”, “Moscow”, “New Dehli”, “Beijing”, “Pretoria”],
“area”: [8.516, 17.10, 3.286, 9.597, 1.221],
“population”: [200.4, 143.5, 1252, 1357, 52.98] }

import pandas as pd
brics = pd.DataFrame(dict)
print(brics)

# Set the index for brics
brics.index = [“BR”, “RU”, “IN”, “CH”, “SA”]

# Print out brics with new index values
print(brics)

Install pandas, lxml .. etc
pip3 install pandas lxml html5lib BeautifulSoup4

Let us read tata motors historical data from yahoo finance

In yahoo finance in sidebar you can find the link for historical prices for any stock.
For instance go to this link

https://in.finance.yahoo.com/q/hp?s=TATAMTRDVR.BO

Now in pandas you can directly read this html file and get the tabular data.

url = 'https://in.finance.yahoo.com/q/hp?s=TATAMTRDVR.BO'
dfs = pd.read_html(url)

dfs #check what is there in the data

das[10] for me gave the actual table I want. Anyway this is not the proper way to do this as this reads many tables from the web page. Anyway for the sake of understanding we do some analysis on das[10]

df = dfs[10]

df.columns = df.iloc[0]
df.reindex(df.index.drop(0))

df.describe()

numpy

Numpy arrays are great alternatives to Python Lists. Some of the key advantages of Numpy arrays are that they are fast, easy to work with, and give users the opportunity to perform calculations across entire arrays.

In the following example, you will first create two Python lists. Then, you will import the numpy package and create numpy arrays out of the newly created lists.

# Create 2 new lists height and weight
height = [1.87, 1.87, 1.82, 1.91, 1.90, 1.85]
weight = [81.65, 97.52, 95.25, 92.98, 86.18, 88.45]

# Import the numpy package as np
import numpy as np

# Create 2 numpy arrays from height and weight
np_height = np.array(height)
np_weight = np.array(weight)

# Calculate bmi
bmi = np_weight / np_height ** 2

# Print the result
print(bmi)

# For a boolean response
bmi > 23

# Print only those observations above 23
bmi[bmi > 23]

Some code samples for you to start

https://github.com/ndubey/algorithms/tree/master/python

I guess it is good enough python to start tacking problems. Hence

Not To be continued …

 

 

 

Python Notes-11

In continuation from
Python Notes-10
Python Notes-9
Python Notes-8
Python Notes-7
Python Notes-6
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

Exception


def spam(divideBy):
try:
return 42 / divideBy
except ZeroDivisionError:
print('Error: Invalid argument.')

try:
    ## Either of these two lines could throw an IOError, say
    ## if the file does not exist or the read() encounters a low level error.
    f = open(filename, 'rU')
    text = f.read()
    f.close()
  except IOError:
    ## Control jumps directly to here if any of the above lines throws IOError.
    sys.stderr.write('problem reading:' + filename)
  ## In any case, the code then continues with the line after the try/except

#!/usr/bin/python
import time; # This is required to include time module.

ticks = time.time()

#!/usr/bin/python
import time;

localtime = time.localtime(time.time())
print("Local current time :", localtime)

This would produce the following result, which could be formatted in any other presentable form −

Local current time : time.struct_time(tm_year=2013, tm_mon=7,
tm_mday=17, tm_hour=21, tm_min=26, tm_sec=3, tm_wday=2, tm_yday=198, tm_isds


localtime = time.asctime( time.localtime(time.time()) )

#!/usr/bin/python
import calendar

cal = calendar.month(2008, 1)
print ("Here is the calendar:")
print (cal)
This would produce the following result −

Here is the calendar:
January 2008
Mo Tu We Th Fr Sa Su
1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31


Too much useless information above 🙂 I will compensate it with next post which will introduce pandas and numpy(Two great libraries).

To be continued …

Python Notes-10

In continuation from
Python Notes-9
Python Notes-8
Python Notes-7
Python Notes-6
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

Let us solve a real life problem. I want to send email from my gmail account using python. Here is a simple example:


import smtplib
fromaddr = 'user_me@gmail.com'
toaddrs = 'user_you@gmail.com'
msg = 'Stop sending me spams!'
username = 'user_me@gmail.com'
password = 'pwd'
server = smtplib.SMTP('smtp.gmail.com:587')
server.ehlo()
server.starttls()

server.login(username,password)
server.sendmail(fromaddr, toaddrs, msg)
server.quit()

If you use gmail two way authentication you can generate application password from gmail and use that. Later remember to delete the application.

Regular expression

To use Python’s regular expression library you ned to import re


import re
match = re.search('abc','abc: "Always be coding" Title of some blog I read today')

match.group() # prints 'abc'

Let us write find method to make our life easier

def find(pat, data):
  match = re.search(pat, data)
  if match:
    print(match.group())
  else:
    print('Not found!')

find('and','hjkl') #print 'Not found!'

find(r'word:\w\w\w','an example word:cat!!')

The code match = re.search(pat, data) stores the search result in a variable named “match”. Then the if-statement tests the match — if true the search succeeded and match.group() is the matching text. Otherwise if the match is false (None to be more specific), then the search did not succeed, and there is no matching text.

Generally people prefer r in pattern that way you can use special characters in pattern without escaping it so you can use:

find(r':...','rat:.cat') #Looks confusing but first '.' will match actual '.' rest two '.' will match next any two characters.


Justifying Text with rjust(), ljust(), and center()
The rjust() and ljust() string methods return a padded version of the string they are called on, with spaces inserted to justify the text. The first argument to both methods is an integer length for the justified string. Enter the following into the interactive shell:

>>> 'Hello'.rjust(10)
' Hello'
>>> 'Hello'.rjust(20)
' Hello'
>>> 'Hello World'.rjust(20)
' Hello World'
>>> 'Hello'.ljust(10)
'Hello '

An optional second argument to rjust() and ljust() will specify a fill character other than a space character. Enter the following into the interactive shell:

>>> 'Hello'.rjust(20, '*')
'***************Hello'
>>> 'Hello'.ljust(20, '-')
'Hello---------------'

The center() string method works like ljust() and rjust() but centers the text rather than justifying it to the left or right. Enter the following into the interactive shell:

>>> 'Hello'.center(20)
' Hello '
>>> 'Hello'.center(20, '=')
'=======Hello========'

Removing Whitespace with strip(), rstrip(), and lstrip()
Sometimes you may want to strip off whitespace characters (space, tab, and newline) from the left side, right side, or both sides of a string. The strip() string method will return a new string without any whitespace characters at the beginning or end. The lstrip() and rstrip() methods will remove whitespace characters from the left and right ends, respectively. Enter the following into the interactive shell:

>>> spam = ' Hello World '
>>> spam.strip()
'Hello World'
>>> spam.lstrip()
'Hello World '

sentence = “the quick brown fox jumps over the lazy dog”
words = sentence.split()


>>> list('hello')
['h', 'e', 'l', 'l', 'o']

Converting a tuple to a list is handy if you need a mutable version of a tuple value.

To be continued …

Python Notes-9

In continuation from
Python Notes-8
Python Notes-7
Python Notes-6
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

Some important points to review:

Converting ascii to char and vice-versa. Python gives a very good way to do this using chr() and ord() functions.


>>> chr(0)
'\x00'
>>> ord('0')
48

reverse a string:

str[::-1]


python ‘is’ operator compares references

reverse a list in place: (So string and tuple doesn’t have this method)

list.reverse()


join a list to create a string.

‘ABC'.join(['My', 'name', 'is', 'Simon'])
'MyABCnameABCisABCSimon'

Notice that the string join() calls on is inserted between each string of the list argument


'MyABCnameABCisABCSimon'.split('ABC')
['My', 'name', 'is', 'Simon']


Creating a package

Each package in Python is a directory which MUST contain a special file called __init__.py. This file can be empty, and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported.

If we create a directory called foo, which marks the package name, we can then create a module inside that package called bar. We also must not forget to add the __init__.py file inside the foo directory.

To use the module bar, we can import it in two ways:


import foo.bar

from foo import bar

The __init__.py file can also decide which modules the package exports as the API, while keeping other modules internal, by overriding the __all__ variable, like so:

__init__.py:

__all__ = ["bar"]

Global variables

It is recommended to avoid global variable as much as possible because it make the code cryptic and hard to debug but sometime you want to get things done fast and easy.

In python you can access global variable wherever you want but when you assign something into it becomes slightly tricky. The assignment creates a new local variable in that scope(weird huh). It may be made like this to make life tough as changing global variable here and there in the code will result in problems hard to track down.
So whenever you want to change global variable in different scope you have to specifically tell python that you are doing this knowingly by declaring the variable first with global keyword.

Example:

var = 9
def test():
  print(var)  # will print 9
  var = 10    # this will not change the global var
              # but will create a new local var named variable.
  print(var)  #10

def func():
  global var
  var = 10    # Now global var is modified 
test()
print(var)   # will print 9

func()
print(var)   # will print modified value 10

To be continued …

Python Notes-8

In continuation from
Python Notes-7
Python Notes-6
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

dict

phonebook = {}
phonebook["John"] = 938477566
phonebook["Jack"] = 938377264
phonebook["Jill"] = 947662781
print(phonebook)




sentence = "the quick brown fox jumps over the lazy dog"
words = sentence.split()
word_lengths = []
for word in words:
      if word != "the":
          word_lengths.append(len(word))
print(words)

Pythonic way?

sentence = "the quick brown fox jumps over the lazy dog"
words = sentence.split()
word_lengths = [len(word) for word in words if word != "the"]
print(words)

dict.clear()

str(dict)

Produces a printable string representation of a dictionary

Exploring built-in modules
Two very important functions come in handy when exploring modules in Python – the dir and help functions.

We can look for which functions are implemented in each module by using the dir function:


>>> import urllib
>>> dir(urllib)

['ContentTooShortError', 'FancyURLopener', 'MAXFTPCACHE', 'URLopener', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__version__', '_ftperrors', '_get_proxies', '_get_proxy_settings', '_have_ssl', '_hexdig', '_hextochr', '_hostprog', '_is_unicode', '_localhost', '_noheaders', '_nportprog', '_passwdprog', '_portprog', '_queryprog', '_safe_map', '_safe_quoters', '_tagprog', '_thishost', '_typeprog', '_urlopener', '_userprog', '_valueprog', 'addbase', 'addclosehook', 'addinfo', 'addinfourl', 'always_safe', 'basejoin', 'c', 'ftpcache', 'ftperrors', 'ftpwrapper', 'getproxies', 'getproxies_environment', 'getproxies_macosx_sysconf', 'i', 'localhost', 'main', 'noheaders', 'os', 'pathname2url', 'proxy_bypass', 'proxy_bypass_environment', 'proxy_bypass_macosx_sysconf', 'quote', 'quote_plus', 'reporthook', 'socket', 'splitattr', 'splithost', 'splitnport', 'splitpasswd', 'splitport', 'splitquery', 'splittag', 'splittype', 'splituser', 'splitvalue', 'ssl', 'string', 'sys', 'test', 'test1', 'thishost', 'time', 'toBytes', 'unquote', 'unquote_plus', 'unwrap', 'url2pathname', 'urlcleanup', 'urlencode', 'urlopen', 'urlretrieve']

import sys

sys.exit()

name = ''
while not name: #(1)
    print('Enter your name:')
    name = input()

To be continued …

Python Notes-7

In continuation from
Python Notes-6
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1


assert

There are three parts to an assert statement: the assert keyword, an expression which, if False, results in crashing the program. The third part (after the comma after the expression) is a string that appears if the program crashes because of the assertion.

assert (BOARDWIDTH * BOARDHEIGHT) % 2 == 0, ‘Board needs to have an even number of boxes for pairs of matches.’


import random

random.randint(1,9);

As python is strongly type there is no concept of function overload

Python is great for interviews see how easy reversing a string is even without using any function


astring = "Hello world!"
print(astring[::-1])

astring = "Hello world!"
print(astring.upper())
print(astring.lower())

astring = "Hello world!"
print(astring.startswith("Hello"))
print(astring.endswith("asdfasdfasdf"))

astring = "Hello world!"
afewwords = astring.split(" ")

x = [1,2,3]
y = [1,2,3]
print(x == y) # Prints out True
print(x is y) # Prints out False

Reference comparison

is is reference compare operator

The ‘is’ operator
Unlike the double equals operator “==”, the “is” operator does not match the values of the variables, but the instances themselves.

Using else in loops

Unlike languages like C,CPP.. we can use else for loops. When the loop condition of “for” or “while” statement fails then code part in “else” is executed. If break statement is executed inside for loop then the “else” part is skipped. Note that “else” part is executed even if there is a continue statement.

Here are a few examples:


# Prints out 0,1,2,3,4 and then it prints "count value reached 5"

count=0
while(count<5):
    print(count)
    count +=1
else:
    print("count value reached %d" %(count))

# Prints out 1,2,3,4
for i in range(1, 10):
    if(i%5==0):
        break
    print(i)
else:
    print("this is not printed because for loop is terminated because of break but not due to fail in condition")


list1 = ['physics', 'chemistry', 1997, 2000];

print list1
del list1[2];
print "After deleting value at index 2 : "
print list1

Python Expression Results Description
len([1, 2, 3]) 3 Length
[1, 2, 3] + [4, 5, 6] [1, 2, 3, 4, 5, 6] Concatenation
['Hi!'] * 4 ['Hi!', 'Hi!', 'Hi!', 'Hi!'] Repetition
3 in [1, 2, 3] True Membership
for x in [1, 2, 3]: print x, 1 2 3 Iteration


max(list)


Returns item from the list with max value.


min(list)


Returns item from the list with min value.

list.append(obj)


Appends object obj to list

list.count(obj)


Returns count of how many times obj occurs in list


list.extend(seq)


Appends the contents of seq to list


list.index(obj)


Returns the lowest index in list that obj appears


list.insert(index, obj)


Inserts object obj into list at offset index


list.pop(obj=list[-1])


Removes and returns last object or obj from list


list.remove(obj)


Removes object obj from list


list.reverse()


Reverses objects of list in place

To be continued …

Python Notes-6

In continuation from
Python Notes-5
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

Dictionary

dict

d = {1:2, 'q':2}

key:value pair

Unlike lists, items in dictionaries are unordered.

So access is only via key not by index. There is no first item or last item in a python dictionary data structure.

The order of items matters for determining whether two lists are the same, it does not matter in what order the key-value pairs are typed in a dictionary. Enter the following into the interactive shell:


>>> spam = ['cats', 'dogs', 'moose']
>>> bacon = ['dogs', 'moose', 'cats']
>>> spam == bacon
False
>>> eggs = {'name': 'Zophie', 'species': 'cat', 'age': '8'}
>>> ham = {'species': 'cat', 'age': '8', 'name': 'Zophie'}
>>> eggs == ham
True


for key, value in d.items()

for key in d.keys() 

or just for key in d

Loop

for i in range(15):


for i in range(10):
for i in range(0,10):
for i in range(0,10,1):


all equals

you can make for loop count down

for i in range(5, -1, -1):
    print(i)

while True:

while a < 5:

Conditional statementif else elif

if name == 'Alice':
    print('Hi, Alice.')
elif name == ‘Bob’
   print(‘hi Bob’);
else:
    print('Hello, stranger.')


mystring = "hello"
myfloat = 10.0
myint = 20

# testing code
if mystring == "hello":
    print("String: %s" % mystring)
if isinstance(myfloat, float) and myfloat == 10.0:
    print("Float: %f" % myfloat)
if isinstance(myint, int) and myint == 20:
    print("Integer: %d" % myint)

When used in conditions, 0, 0.0, and ” (the empty string) are considered False,

When you assign a list to a variable, you are actually assigning a list reference to the variable. A reference is a value that points to some bit of data, and a list reference is a value that points to a list. e.g.

❶ >>> spam = [0, 1, 2, 3, 4, 5]
❷ >>> cheese = spam
❸ >>> cheese[1] = ‘Hello!’
>>> spam
[0, ‘Hello!’, 2, 3, 4, 5]
>>> cheese
[0, ‘Hello!’, 2, 3, 4, 5]
The code changed only the cheese list, but it seems that both the cheese and spam lists have changed.

This is not true for c++ STL vector

To be continued …

Python Notes-5

In continuation from
Python Notes-4
Python Notes-3
Python Notes-2
Python Notes-1

Let us change the topic for a while from list, string, tuple etc.

Functions in python are also very user friendly for example:

def func(a, b):
  return a+b,a*b

Above function will return a tuple so func(2,3) will give (5,6)

Note: It is never a good practice to do multiple thing in a single function. You should always try to make your function serve single purpose that will make it more usable.

Let us go back to list there is so much to cover there so much power in this data structure.


del li[indx]


you can also delete a variable, kind of unassignment .. after deleting use will give you TypeError

Operators in, not in:

operators, in and not in are used very often


>>>'howdy' in ['hello', 'hi', 'howdy', 'heyas']
>>>True

in is used to Find a value in a list
You can also use index for find but it will give ValueError if the item is not present in the list.
When there are multiple elements index will give you the lowest index:

>>> var = ['hello', 'hi', 'howdy', 'heyas']
>>> var.index('hello')
0
>>> spam.index('heyas')
3

remove()

remove a value from the list

Sorting

var.sort()

var.sort(reverse=True)

sorts the list IN PLACE SO NONE return value

There is None value whenever something is un-assigned or a function doesn’t return anything None is used.

list can store different type of variables but then you can’t use sort on that list. [GOT YOU!! NOPE]


name = 'Zophie'
>>> name[0]
'Z'
>>> name[-2]
'i'

Zo' in name
True
>>> 'z' in name
False

Repeatation of old concept

A lists and strings are different in an important way. A list value is a mutable data type: It can have values added, removed, or changed. However, a string is immutable: It cannot be changed.

The tuple data type is almost identical to the list data type, except in two ways. First, tuples are typed with parentheses, ( and ), instead of square brackets, [ and ]. For example, enter the following into the interactive shell:

>>> eggs = ('hello', 42, 0.5)
>>> eggs[0]
'hello'


But the main way that tuples are different from lists is that tuples, like strings, are immutable. Tuples cannot have their values modified, appended, or removed.

If you have only one value in your tuple, you can indicate this by placing a trailing comma after the value inside the parentheses. Otherwise, Python will think you’ve just typed a value inside regular parentheses. The comma is what lets Python know this is a tuple value. (Unlike some other programming languages, in Python it’s fine to have a trailing comma after the last item in a list or tuple.) Enter the following type() function calls into the interactive shell to see the distinction:

>>> type(('hello',))

Just like how str(42) will return ’42’, the string representation of the integer 42, the functions list() and tuple() will return list and tuple versions of the values passed to them. Enter the following into the interactive shell, and notice that the return value is of a different data type than the value passed:

>>> tuple(['cat', 'dog', 5])
('cat', 'dog', 5)
>>> list(('cat', 'dog', 5))
['cat', 'dog', 5]
>>> list('hello')
['h', 'e', 'l', 'l', 'o']

Converting a tuple to a list is handy if you need a mutable version of a tuple value.

To be continued …

Python Notes-4

In continuation from Python Notes-3
Python Notes-2
Python Notes-1

There are proper ways to format the string in python there is str.format() method then there is % operator like in c


print "My name is %s." % myName
print "I weigh %d k.g." % myWeight


I can do away just with simple print and print with end=”. When I don’t need new line just after a print. Rest I manage with string concatenation and str() function for converting to str.

the str() flota() int() function
“I am “ + str(29) + “ years old.”

There are couple of interview questions where you need to convert ascii to char and vice-versa. Python gives a very good way to do this using chr() and ord() functions.

>>> chr(0)
'\x00'
>>> ord('0')
48


Slicing with step


var[3:7:2] start : stop: step


reverse a string:


str[::-1]


We will come back to ord(), chr() and reversing a str again.


round(9.7)
10


Multiple assignment

var = ['fat', 'orange', 'loud']
>>> size, color, disposition = var

The number of variables and the length of the list must be exactly equal, or Python will give you a ValueError.

The multiple assignment trick can also be used to swap the values in two variables:

>>> a, b = 'Alice', 'Bob'
>>> a, b = b, a


To be continued …

Python Notes-3

In continuation from Python Notes-2
Python Notes-1

We will learn about python string as you know string in python are immutable all function which applies to string will eventually not modify the string but will return a str e.g.


var = 'ab'
var.upper() can’t change str but returns another string

sometime I just forget this and assume that my original variable is changed to all upper, silly mistake.

var = var.lower()


Count number of times a value is appearing in str or tuple or list
var.count('o')
Index of a value
var.index('o')

To include special characters in a string you can escape using \ for “ or ‘ just use another for quotation then you can use one inside without escaping.
If you want to be more specific you can place r before the beginning quotation mark of a string to make it a raw string. A raw string completely ignores all escape characters and prints any backslash that appears in the string. e.g.


>>> print(r'This!! This is me \' " cat.')
This!! This is me \' " cat.


Multiline Strings with Triple Quotes

print('''Dear Apple,

You are so sweet.

Sincerely,
Me”’)

Dear Apple,

You are so sweet.

Sincerely,
Me

After reading about str, list and tuple I wanted to know why use tuple? Why another data structure?

Seems like python uses tuple behind our back when a function has multiple return type. Even when it is not specifically mentioned.
Also since tuple data structure is restrictive it is much faster compared to list. There is one more benefit which relates to code understanding if you use tuple data type in your code for variables which doesn’t need to be changed. It will be known to reader without you telling specifically.

Just like string concatenation list and tuple can also be concatenated. All three cal be multiplied/replicated

t1 = (1,2)
t2 = (3,4)
t3 = t1+t2

Multiplication

t4 = t1*4


To be continued …