Regular expression is a set of characters, called as the pattern, which helps in finding substrings in a given string. The pattern is used to detect the substrings
For example, suppose you have a dataset of customer reviews about your restaurant. Say, you want to extract the emojis from the reviews because they are a good predictor os the sentiment of the review.
Take another example, the artificial assistants such as Siri, Google Now use information retrieval to give you better results. When you ask them for any query or ask them to search for something interesting on the screen, they look for common patterns such as emails, phone numbers, place names, date and time and so on. This is because then the assitant can automatically make a booking or ask you to call the resturant to make a booking.
Regular expressions are very powerful tool in text processing. It will help you to clean and handle your text in a much better way.
import re
Let's do a quick search using a pattern.
re.search('Ravi', 'Ravi is an exceptional student!')
# print output of re.search()
match = re.search('Ravi', 'Ravi is an exceptional student!')
print(match.group())
Let's define a function to match regular expression patterns
def find_pattern(text, patterns):
if re.search(patterns, text):
return re.search(patterns, text)
else:
return 'Not Found!'
# '*': Zero or more
print(find_pattern("acasdasd", "ab*"))
print(find_pattern("abc", "ab*"))
print(find_pattern("abbc", "ab*"))
# '?': Zero or one (tells whether a pattern is absent or present)
#print(find_pattern("ackuamraguru", "ab?"))
#print(find_pattern("abc", "ab?"))
#print(find_pattern("abbbbbc", "ab?"))
print(find_pattern("Nisvartha foundbation", "a.*b"))
# '+': One or more
print(find_pattern("ac", "ab+"))
print(find_pattern("abc", "ab+"))
print(find_pattern("abbc", "ab+"))
print(find_pattern("Nisvartha foundbbatbion", "a.+b"))
# {n}: Matches if a character is present exactly n number of times
print(find_pattern("abbbc", "ab{2}"))
# {m,n}: Matches if a character is present from m to n number of times
print(find_pattern("aabbbbbbc", "ab{3,5}")) # return true if 'b' is present 3-5 times
print(find_pattern("aabbbbbbc", "ab{7,10}")) # return true if 'b' is present 7-10 times
print(find_pattern("abbbbbbbbbbbbbc", "ab{,10}")) # return true if 'b' is present atmost 10 times
print(find_pattern("aabbbbbbc", "ab{10,}")) # return true if 'b' is present from at least 10 times
# '^': Indicates start of a string
# '$': Indicates end of string
print(find_pattern("Jammmmmes", "^Jam{3}")) # return true if string starts with 'J'
print(find_pattern("Pramod", "^J")) # return true if string starts with 'J'
print(find_pattern("India", "a$")) # return true if string ends with 'c'
print(find_pattern("Japan", "a$")) # return true if string ends with 'c'
# '.': Matches any character
print(find_pattern("a234234", ".*"))
print(find_pattern("#", "."))
# Now we will look at '[' and ']'.
# They're used for specifying a character class, which is a set of characters that you wish to match.
# Characters can be listed individually as follows
print(find_pattern("ca", "[bcad]"))
# Or a range of characters can be indicated by giving two characters and separating them by a '-'.
print(find_pattern("b3AF3bD", "[a-fA-F0-2]{3,5}")) # same as above
# '^' is used inside character set to indicate complementary set
print(find_pattern("cba", "[abc]")) # return true if neither of these is present - a,b or c
print(find_pattern("Ubd", "^U[^a|R]{2}")) # return true if neither of these is present - a,b or c
| Pattern | Matches |
|---|---|
| [abc] | Matches either an a, b or c character |
| [abcABC] | Matches either an a, A, b, B, c or C character |
| [a-z] | Matches any characters between a and z, including a and z |
| [A-Z] | Matches any characters between A and Z, including A and Z |
| [a-zA-Z] | Matches any characters between a and z, including a and z ignoring cases of the characters |
| [0-9] | Matches any character which is a number between 0 and 9 |
| Pattern | Equivalent to |
|---|---|
| \s | [ \t\n\r\f\v] |
| \S | [^ \t\n\r\f\v] |
| \d | [0-9] |
| \D | [^0-9] |
| \w | [a-zA-Z0-9_] |
| \W | [^a-zA-Z0-9_] |
print(find_pattern("aabbbbbb", "ab{3,5}")) # return if a is followed by b 3-5 times GREEDY
print(find_pattern("aabbbbbb", "ab{3,5}?")) # return if a is followed by b 3-5 times GREEDY
# Example of HTML code
print(re.search("<.*>","<HTML><TITLE>My Page</TITLE></HTML>"))
# Example of HTML code
print(re.search("<.*?>","<HTML><TITLE>My Page</TITLE></HTML>"))
match() Determine if the RE matches at the beginning of the string
search() Scan through a string, looking for any location where this RE matches
finall() Find all the substrings where the RE matches, and return them as a list
finditer() Find all substrings where RE matches and return them as asn iterator
sub() Find all substrings where the RE matches and substitute them with the given string
# - this function uses the re.match() and let's see how it differs from re.search()
def match_pattern(text, patterns):
if re.match(patterns, text):
return re.match(patterns, text)
else:
return ('Not found!')
print(find_pattern("abbc", "b+"))
print(match_pattern("abbc", "b+"))
## Example usage of the sub() function. Replace Road with rd.
street = '21 Ramakrishna Road'
print(re.sub('Road', 'Rd', street))
print(re.sub('R\w+', 'Rd', street))
## Example usage of finditer(). Find all occurrences of word Festival in given sentence
text = 'Diwali is a festival of lights, Holi is a festival of colors!'
pattern = 'festival'
for match in re.finditer(pattern, text):
print('START -', match.start(), end="")
print('END -', match.end())
# Example usage of findall(). In the given URL find all dates
url = "http://www.telegraph.co.uk/formula-1/2017/10/28/mexican-grand-prix-2017-time-does-start-tv-channel-odds-lewisl/2017/05/12/"
date_regex = '/(\d{4})/(\d{1,2})/(\d{1,2})/'
print(re.findall(date_regex, url))
## Exploring Groups
m1 = re.search(date_regex, url)
print(m1.group()) ## print the matched group
print(m1.group(1)) # - Print first group
print(m1.group(2)) # - Print second group
print(m1.group(3)) # - Print third group
print(m1.group(0)) # - Print zero or the default group
print(m1.group(1))