Pythonで機械学習[2018/07/15版]

データセットの特徴の外観

欠損

data.isnull().sum()

カラムの型

data.dtypes.value_counts()

カテゴリカル変数の種類

data.select_dtypes('object').apply(pd.Series.nunique, axis = 0)

前処理

カテゴリカル変数のエンコード

Label Encoding

from sklearn.preprocessing import LabelEncoder

# Create a label encoder object
le = LabelEncoder()
le_count = 0

# Iterate through the columns
for col in app_train:
    if app_train[col].dtype == 'object':
        # If 2 or fewer unique categories
        if len(list(app_train[col].unique())) <= 2:
            # Train on the training data
            le.fit(app_train[col])
            # Transform both training and testing data
            app_train[col] = le.transform(app_train[col])
            app_test[col] = le.transform(app_test[col])
            
            # Keep track of how many columns were label encoded
            le_count += 1
            
print('%d columns were label encoded.' % le_count)

One Hot Encoding

# one-hot encoding of categorical variables
app_train = pd.get_dummies(app_train)
app_test = pd.get_dummies(app_test)