Now let’s try word2vec on the property details page and see if we can find something interesting.

load libs

1
2
3
4
5
6
7
8
9

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import gensim
import nltk
import string
import re
%matplotlib inline

While we’re waiting for the data, let’s do some data visualization, take one day’s rental data as an example.

load library

1
2
3
4
5
6

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import calendar
%matplotlib inline

If you:

• Followed some tutorials called “how to plot in python in 5 mins” and learned matplotlib.
• Know to add ‘from matplotlib.pyplot import plt’.
• But always need to open the google page when doing any plot.
• Always struggling at adding labels/text/ticks/ to the plots.
• Know some tasks(like adding xlabels) can be done in several ways, but don’t know which is the best way.
• Open matplotlib official doc several times but have no idea what they are talking about.
• Confused by the terms: figure, axes, axis.

Just for some extra fun, Let’s do some plots to explore the ads dataset a bit

1
2
3
4
5
6

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import calendar
%matplotlib inline

I think there is no solid evidence to prove which is better than another. These two algorithms build from different methods with different hyper-parameters to tune. Therefore, I think the right approach is to understand the pros and cons of the two, recall the pros and cons when solving your specific problem.

Gradient Boosting Tree Basic

• start with a weak and simpler learner (e.g. mean()), get a prediction
• use a lost function J to compute the error between y_true and y_predict

When I first started learning Pandas library, I seriously suffered a memory issue:

My brain said:

It’s hard to write the pandas functions into the disk, while your memory is not enough.

Just another day of Data cleaning… this dataset really required lots and lots of cleaning…

Data prepare: continued

From last notebook we’ve get the correct label: next 24 hour exposure rate.

In data science data cleaning stage, we may encounter this situation:

In one column you have multiple features, each feature has multiple values, they all stacked in one column.

In my previous post I summarized the project information and challenges, now let’s take a look at the data.

Data download

First, the raw dataset can be downloaded here.