Day5_DL_Homework.ipynb

{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Day5_DL_Homework.ipynb","version":"0.3.2","provenance":[],"collapsed_sections":[]},"kernelspec":{"display_name":"Py3 research env","language":"python","name":"py3_research"}},"cells":[{"metadata":{"id":"5szd4N_SQXZX","colab_type":"text"},"cell_type":"markdown","source":["# Домашнее задание №5: Intro to DL"]},{"metadata":{"id":"Zr0vzp4WQXZZ","colab_type":"text"},"cell_type":"markdown","source":["<span style=\"color: red; font-size: 14pt\">Дедлайн: 24.02.2019 23:59</span>"]},{"metadata":{"id":"Go_b3r3uQXZb","colab_type":"text"},"cell_type":"markdown","source":["**Оформление дз**: \n","\n","- Task short name: ``DL``.\n","- Выполненное дз сохраните в файл ``ML2019_<фамилия>_HW#.ipynb``, к примеру -- ``ML2019_ivanov_HW5.ipynb``\n","- Присылайте выполненное задание на почту `` ml_2019_q1@bigdatateam.org `` с темой письма `` HW# Short name. ФИО ``. \n","\n","    Например: `` HW5 DL. Иванов Иван Иванович. ``\n","\n","**Вопросы**:\n","- Свои вопросы присылайте в Telegram.\n","\n","**Фидбек**:\n","- Пожалуйста, оставьте свой отзыв после выполнения домашнего задания по сссылке:\n","\n","    http://bit.ly/ml2019q1_hw5_feedback"]},{"metadata":{"id":"oSkuo7Q9QXZb","colab_type":"code","colab":{}},"cell_type":"code","source":["import sys\n","print(sys.version_info)\n","\n","import keras\n","print(keras.__version__)\n","import numpy as np\n","import matplotlib.pyplot as plt"],"execution_count":0,"outputs":[]},{"metadata":{"id":"c9RZ3K5OQXZf","colab_type":"code","colab":{}},"cell_type":"code","source":["# Dirty Hack, uncomment if necessary\n","\n","# import os\n","# os.environ[\"KMP_DUPLICATE_LIB_OK\"] = \"TRUE\""],"execution_count":0,"outputs":[]},{"metadata":{"id":"ZhhnhN3mQXZj","colab_type":"code","colab":{}},"cell_type":"code","source":["from keras import models\n","from keras import layers"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Mr1a5ou9QXZl","colab_type":"code","colab":{}},"cell_type":"code","source":["from keras.datasets import fashion_mnist\n","\n","# Загрузка данных (загруженные копии могут быть доступны в папке с семинарским занятием)\n","(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()\n","\n","# Смена размерности и нормировка\n","train_images = train_images.reshape((-1, 28 * 28))\n","train_images = train_images.astype('float32') / 255\n","\n","test_images = test_images.reshape((-1, 28 * 28))\n","test_images = test_images.astype('float32') / 255\n","\n","# Кодирование метки класса с помощью one-hot\n","from keras.utils import to_categorical\n","train_labels = to_categorical(train_labels)\n","test_labels = to_categorical(test_labels)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Z8CeAhimQXZo","colab_type":"text"},"cell_type":"markdown","source":["## Часть 1. Практика с различными слоями (20%)\n","\n","Ниже Вам предлагается правильно задать параметры различных слоев нейронной сети для того, чтобы она могла эффективно обучаться на подаваемых данных. После каждой имплементации будет код для самопроверки, который позволит Вам отловить все ошибки до непосредственного построения нейронной сети. \n","\n","Выше данные уже были предобработаны аналогично семинару."]},{"metadata":{"id":"hz1urJEbQXZp","colab_type":"code","colab":{}},"cell_type":"code","source":["basic_input_shape = train_images[0].shape"],"execution_count":0,"outputs":[]},{"metadata":{"id":"xQq_DBlTQXZr","colab_type":"text"},"cell_type":"markdown","source":["Добавьте первый полносвязный слой из десяти нейронов в нейронную сеть (`layers.Dense`)"]},{"metadata":{"id":"gHB36QfGQXZt","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"2E3hl7QmQXZv","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(train_images[0][None, :]).shape == (1, 10)\n","if network.predict(train_images[0][None, :]).shape == (1, 10):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"FRCqPQxiQXZy","colab_type":"text"},"cell_type":"markdown","source":["Пусть теперь первый слой Вашей нейронной сети будет двумерной сверткой (`layers.Conv2D`). Параметры `filters` и `kernel_size` уже заданы в ячейке ниже, используйте их. В качестве функции активации используйте `relu`, параметр `padding='same'`."]},{"metadata":{"id":"0vdkpblYQXZz","colab_type":"code","colab":{}},"cell_type":"code","source":["input_shape_for_conv2d = [28, 28, 1]"],"execution_count":0,"outputs":[]},{"metadata":{"id":"9RGg1N3rQXZ3","colab_type":"text"},"cell_type":"markdown","source":["Обратите внимание на документацию, для двумерного сверточного слоя требуется трехмерное изображение. Его размерность (H, W, C), где H и W это обычные координаты пикселя, а C - номер канала (в нашем случае изображения черно-белые, поэтому канал один. Для цветных изображений, например, каналов три - красный, зеленый и синий в палитре RGB)."]},{"metadata":{"id":"SNbSu1DsQXZ3","colab_type":"code","colab":{}},"cell_type":"code","source":["reshaped_image_for_conv = train_images[0].reshape(input_shape_for_conv2d)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"IG7NDGwWQXZ6","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"8QU7EBWCQXZ8","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 28, 28, 10)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 28, 28, 10):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Ybvq9DBdQXZ-","colab_type":"text"},"cell_type":"markdown","source":["Теперь добавьте вслед за сверточным слоем (из предыдущего пункта) MaxPooling2D слой (`layers.MaxPooling2D`)."]},{"metadata":{"id":"2yCl7TzPQXZ_","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"w1izfRe6QXaB","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 14, 14, 10)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 14, 14, 10):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"2AROrRgkQXaD","colab_type":"text"},"cell_type":"markdown","source":["Возьмите нейронную сеть с прошлого шага и добавьте Flatten слой (`layers.Flatten`)."]},{"metadata":{"id":"ezZ1oJnNQXaE","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"x0IzGFtJQXaG","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 1960)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 1960):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"acgScjm4QXaJ","colab_type":"text"},"cell_type":"markdown","source":["Возьмите нейронную сеть с прошлого шага и добавьте полносвязный слой из 64 нейронов и функцией активации `relu`."]},{"metadata":{"id":"4LL7yv4CQXaK","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"4dddirOpQXaN","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 64)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 64):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Zh4_ADLSQXaP","colab_type":"text"},"cell_type":"markdown","source":["Теперь добавьте Dropout слой после полносвязного (`layers.Dropout`) c `p=0.3`"]},{"metadata":{"id":"1ZB-U9ObQXaQ","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"WVXJPfeMQXaS","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 64)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 64):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"6XyvqqwtQXaU","colab_type":"text"},"cell_type":"markdown","source":["И, наконец, добавьте в конец полносвязный слой из 10 нейронов с функцией активации `softmax`."]},{"metadata":{"id":"LMy_Ckj7QXaW","colab_type":"code","colab":{}},"cell_type":"code","source":["network = models.Sequential()\n","filters = 10\n","kernel_size = 5\n","# your code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"LRO-iKhxQXaY","colab_type":"code","colab":{}},"cell_type":"code","source":["# Проверим Вас\n","network.compile(optimizer='rmsprop',\n","                loss='categorical_crossentropy',\n","                metrics=['accuracy'])\n","assert network.predict(reshaped_image_for_conv[None, :]).shape == (1, 10)\n","if network.predict(reshaped_image_for_conv[None, :]).shape == (1, 10):\n","    print('Отлично, все верно!')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Z4l_dHNXQXaZ","colab_type":"text"},"cell_type":"markdown","source":["Давайте посмотрим на получившуюся модель:"]},{"metadata":{"id":"oOceG4QeQXaa","colab_type":"code","colab":{}},"cell_type":"code","source":["network.summary()"],"execution_count":0,"outputs":[]},{"metadata":{"id":"PfbCyT3XQXab","colab_type":"text"},"cell_type":"markdown","source":["А теперь обучим нейронную сеть и посмотрим на качество полученного классификатора:"]},{"metadata":{"id":"497UWJg3QXac","colab_type":"text"},"cell_type":"markdown","source":["Но перед этим нам необходимо привести `train_images`, `val_images` и `test_images` к нужным размерностям (теперь мы работаем с ними как с черно-белыми изображениями, а не векторами, помните?)"]},{"metadata":{"id":"r9GO5X9ZQXad","colab_type":"code","colab":{}},"cell_type":"code","source":["# Сделайте reshape для изображений, чтобы они подходили по формату к построенной нейронной сети.\n","# your code here (see above cells for hint)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"oZTbrxGiQXaf","colab_type":"text"},"cell_type":"markdown","source":["Для выделения валидационной выборки достаточно указать параметр `validation_split` в методе `.fit`. See docs for more :)"]},{"metadata":{"id":"-NjDZfATQXag","colab_type":"code","colab":{}},"cell_type":"code","source":["network.fit(train_images, train_labels, epochs=5, batch_size=128, ...)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"hCOpHlefQXah","colab_type":"code","colab":{}},"cell_type":"code","source":["test_loss, test_acc = network.evaluate(test_images, test_labels)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"73NT3EJCQXaj","colab_type":"text"},"cell_type":"markdown","source":["Точность полученного классификатора на отложенной выборке составляет:"]},{"metadata":{"id":"ysVEy-pRQXaj","colab_type":"code","colab":{}},"cell_type":"code","source":["print('test_acc:', test_acc)"],"execution_count":0,"outputs":[]},{"metadata":{"id":"L1mLAHxUQXam","colab_type":"text"},"cell_type":"markdown","source":["Полученный результат превосходит (почти наверное) то, чего добилась простая полносвязная сеть на семинаре. Попробуем построить еще более точный классификатор!"]},{"metadata":{"id":"Xluz-cw8QXam","colab_type":"text"},"cell_type":"markdown","source":["## Часть 2: beat'em up (80%)"]},{"metadata":{"id":"zLfRg_K5QXam","colab_type":"text"},"cell_type":"markdown","source":["Теперь ваша задача построить нейронную сеть, которая обеспечит хотя бы 90.5% метрики `accuracy` на тестовой выборке. Вы можете использовать построенную выше нейронную сеть в качестве baseline."]},{"metadata":{"id":"UDqt477bQXan","colab_type":"code","colab":{}},"cell_type":"code","source":["# your great code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"pa5pL3yIQXap","colab_type":"code","colab":{}},"cell_type":"code","source":["if test_acc >= 0.905:\n","    print('Congratulations! Look at the bottom of the notebook for some more challenges.')\n","else:\n","    print('Sorry, not enough. Look below if you have tried everything yet.')"],"execution_count":0,"outputs":[]},{"metadata":{"id":"Ol7Sc1nPQXaq","colab_type":"text"},"cell_type":"markdown","source":["_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","_______\n","### Spoiler alert!\n","Если Вы перепробовали все, что пришло в голову, но 90.5% `accuracy` так и не покорились, попробуйте следующее:\n","* Добавить еще один сверточный слой.\n","* Увеличить размер Dense слоев (кроме последнего).\n","* Поиграть с `rate` в Dropout слоях.\n","* Увеличить число эпох.\n","* Поменять алгоритм градиентного спуска (например, на adam).\n","* Обратиться к вселенскому разуму в интернете."]},{"metadata":{"id":"ldsny0ahQXaq","colab_type":"text"},"cell_type":"markdown","source":["### For the brave one\n","Если вы преуспели с предыдущим заданием и горите желеанием продолжить, попробуйте побить порог в 91.2% accuracy. \n","Успехов!\n","\n","* Данный пункт задания опциональный, его решение не является обязательным."]},{"metadata":{"id":"9CcZAg53QXas","colab_type":"code","colab":{}},"cell_type":"code","source":["# your brave code here"],"execution_count":0,"outputs":[]},{"metadata":{"id":"dEIoDpBCQXas","colab_type":"text"},"cell_type":"markdown","source":["Пожалуйста, оцените данное домашнее задание: http://bit.ly/ml2019q1_hw5_feedback"]}]}