We have studied lots of things about traditional(old) ML and DL but if we would like to summarize our learning then we can say that ML learning is approach to find correct weight and bias, but biggest difficulty to find correct weight and bias is to find correct feature. Because weight and bias both are dependent of correct feature.
So finally we want a correct feature. Also I have written multiple posts on feature selection approach. But in short if data is small(small data means number of rows can be any number but number of columns are limited) we can use manual approach for feature selection and it can be done using traditional ML. But if we have Big Data(Big Data mean unlimited number of columns) we can't select feature manually and it is impossible. So for this we have new machine learning approach which is known as DL(Deep learning). In deep learning NN(Neural Network) help to select features automatically.
One of great example of Big Data is image/video/voice/signal. What is an image? actually it is just collection of lots of pixel. In python word it is just a numpy array. Where if array is 2D than image is black and white(Grey image) and if array is 3D then it is an color image. So color image is just a 3D array.
There can be two way of feature selection and why do we want less feature? because less feature help us to train a model which takes less time and most important less resources(RAM and CPU) in prediction. One more thing has to be noticed over here is that whatever operation you do either feature elimination/extraction but original information shouldn't loss, like if we convert an image from color to black & white. You don't know but you are removing lots of features but still you can recognize that who is there in that picture.
- Feature Elimination - In this approach we eliminate the columns(features) on which our target y (y = b+wx) is not dependent.
- Feature Extraction - In this approach we combined two or more features so that finally we have less features.
One more thing would like to let you know before we start with CNN practical, as we know we have two type of classification Binary and multi classification. Accordingly we use activation function. Activation functions are mathematical equations that determine the output of a neural network. We use below activation functions-
- Binary classification - Sigmoid activation function
- Multi classification - Softmax activation function
We have a dataset available from MNIST, in this dataset we have lots of hand written images of number(from 0-9). It is very interesting dataset, please google if you want to know more about this dataset and father of NN.
from keras.datasets import mnist
mnist_dataset = mnist.load_data('mnist_dataset.db')
len(mnist_dataset)
2
Mean of this output is that, there are two part of this dataset, one is for training and another is for testing.
train, test = mnist_dataset
len(train), len(test)
(2, 2)
Still the length is 2 of both because there are two variable X.train and y_train
X_train, y_train = train
X_train.shape
(60000, 28, 28)
That means X_train dataset has 60000 images(raw) and size of image 28 X 28 pixels.
X_test, y_test = test
X_test.shape
(10000, 28, 28)
So training data we will use for training and test data for testing
image1 = X_train[0]
image1.shape
(28, 28)
image1_lable = y_train[0]
image1_lable
5
import matplotlib.pyplot as plt
plt.imshow(image1)
Let's take one more image but black & white this time
image2 = X_train[7]
plt.imshow(image2, cmap='gray')
Let's convert this image(2D array) into 1D so that we can proceed for feature selection and this process is known as flattening.
image1.reshape(28*28)
array([ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 18, 18, 18,
126, 136, 175, 26, 166, 255, 247, 127, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 30, 36, 94, 154, 170, 253,
253, 253, 253, 253, 225, 172, 253, 242, 195, 64, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 49, 238, 253, 253, 253,
253, 253, 253, 253, 253, 251, 93, 82, 82, 56, 39, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 219, 253,
253, 253, 253, 253, 198, 182, 247, 241, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
80, 156, 107, 253, 253, 205, 11, 0, 43, 154, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 14, 1, 154, 253, 90, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 139, 253, 190, 2, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 190, 253, 70,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 35,
241, 225, 160, 108, 1, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 81, 240, 253, 253, 119, 25, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 45, 186, 253, 253, 150, 27, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 16, 93, 252, 253, 187,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 249,
253, 249, 64, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 46, 130,
183, 253, 253, 207, 2, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 39, 148,
229, 253, 253, 253, 250, 182, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 24, 114,
221, 253, 253, 253, 253, 201, 78, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 23, 66,
213, 253, 253, 253, 253, 198, 81, 2, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 18, 171,
219, 253, 253, 253, 253, 195, 80, 9, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 55, 172,
226, 253, 253, 253, 253, 244, 133, 11, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
136, 253, 253, 253, 212, 135, 132, 16, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0], dtype=uint8)
image1 = image1.reshape(28*28)
image1.shape
(784,)
X_train_id = X_train.reshape(-1, 28*28)
X_train_id.shape
(60000, 784)
X_train = X_train_id.astype('float32')
Now 2D array has been converted into 1D
from keras.utils.np_utils import to_categorical
y_train_cat = to_categorical(y_train)
y_train_cat
array([[0., 0., 0., ..., 0., 0., 0.],
[1., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 1., 0.]], dtype=float32)
y_train_cat[0]
array([0., 0., 0., 0., 0., 1., 0., 0., 0., 0.], dtype=float32)
from keras.models import Sequential
from keras.layers import Dense
model = Sequential()
model.add(Dense(units=512, input_dim=28*28, activation='relu'))
model.add(Dense(units=256,activation='relu'))
model.add(Dense(units=128,activation='relu'))
model.add(Dense(units=32,activation='relu'))
model.summary()
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_1 (Dense) (None, 512) 401920 _________________________________________________________________ dense_2 (Dense) (None, 256) 131328 _________________________________________________________________ dense_3 (Dense) (None, 128) 32896 _________________________________________________________________ dense_4 (Dense) (None, 32) 4128 ================================================================= Total params: 570,272 Trainable params: 570,272 Non-trainable params: 0
model.add(Dense(units=10,activation='softmax'))
model.summary()
Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_1 (Dense) (None, 512) 401920 _________________________________________________________________ dense_2 (Dense) (None, 256) 131328 _________________________________________________________________ dense_3 (Dense) (None, 128) 32896 _________________________________________________________________ dense_4 (Dense) (None, 32) 4128 _________________________________________________________________ dense_5 (Dense) (None, 10) 330 ================================================================= Total params: 570,602 Trainable params: 570,602 Non-trainable params: 0
from keras.optimizers import RMSprop
model.compile(optimizer=RMSprop(), loss='categorical_crossentropy', metrics=['accuracy'])
h = model.fit(X_train, y_train_cat, epochs=20)
Epoch 1/20 60000/60000 [==============================] - 25s 415us/step - loss: 2.2099 - accuracy: 0.3225 Epoch 2/20 60000/60000 [==============================] - 24s 403us/step - loss: 1.7813 - accuracy: 0.3489 Epoch 3/20 60000/60000 [==============================] - 24s 408us/step - loss: 1.8624 - accuracy: 0.3136 Epoch 4/20 60000/60000 [==============================] - 24s 395us/step - loss: 1.8684 - accuracy: 0.3387 Epoch 5/20 60000/60000 [==============================] - 24s 393us/step - loss: 1.8839 - accuracy: 0.3402 Epoch 6/20 60000/60000 [==============================] - 24s 400us/step - loss: 1.8838 - accuracy: 0.3337 Epoch 7/20 60000/60000 [==============================] - 24s 405us/step - loss: 1.9669 - accuracy: 0.3087 Epoch 8/20 60000/60000 [==============================] - 25s 414us/step - loss: 1.8700 - accuracy: 0.3469 Epoch 9/20 60000/60000 [==============================] - 23s 386us/step - loss: 1.9151 - accuracy: 0.3499 Epoch 10/20 60000/60000 [==============================] - 23s 378us/step - loss: 1.8500 - accuracy: 0.3426 Epoch 11/20 60000/60000 [==============================] - 23s 379us/step - loss: 1.8752 - accuracy: 0.3445 Epoch 12/20 60000/60000 [==============================] - 22s 365us/step - loss: 1.9479 - accuracy: 0.3383 Epoch 13/20 60000/60000 [==============================] - 22s 362us/step - loss: 1.8312 - accuracy: 0.3449 Epoch 14/20 60000/60000 [==============================] - 25s 412us/step - loss: 1.8400 - accuracy: 0.3388 Epoch 15/20 60000/60000 [==============================] - 25s 413us/step - loss: 1.8531 - accuracy: 0.3501 Epoch 16/20 60000/60000 [==============================] - 25s 409us/step - loss: 2.0138 - accuracy: 0.3461 Epoch 17/20 60000/60000 [==============================] - 25s 422us/step - loss: 1.9167 - accuracy: 0.3476 Epoch 18/20 60000/60000 [==============================] - 25s 418us/step - loss: 1.8591 - accuracy: 0.3605 Epoch 19/20 60000/60000 [==============================] - 25s 414us/step - loss: 1.8448 - accuracy: 0.3476 Epoch 20/20 60000/60000 [==============================] - 25s 411us/step - loss: 2.1167 - accuracy: 0.3474
plt.imshow(X_test[0])
y_test[0]
7
I have trained my model without CNN so that we can differentiate accuracy and number of features using normal NN and CNN. Please read next post for CNN.
Comments
Post a Comment
Please share your experience.....