Lab_9_Generative_Models.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lab 9 Generative Models",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "REaV1GblZovs",
        "colab_type": "text"
      },
      "source": [
        "#Generative models\n",
        "In this Lab we will be experimenting with\n",
        "\n",
        "- Convolutional autoencoders\n",
        "- Latent space visualization adn interpolation\n",
        "- Upsampling techniques\n",
        "- Variational autoencoders\n",
        "- Deep Convolutional GANs (DCGANs)\n",
        "\n",
        "If you want to experiemnt with Denoising Autoencoders, revisit Lab 3 (task 5):\n",
        "https://github.com/aivclab/dlcourse/blob/master/Lab3_FunWithMNIST.ipynb\n",
        "\n",
        "**Before we start - remember to set runtime to GPU**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VSIpfu6wUG1r",
        "colab_type": "text"
      },
      "source": [
        "**NOTE:** In case you have trouble running Keras/TensorFlow in Colab, try one of the following:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fZ01fAojUGCV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Try this\n",
        "#!pip install --upgrade tensorflow==1.8.0\n",
        "\n",
        "# ... or this\n",
        "%tensorflow_version 1.x\n",
        "\n",
        "# Check TensorFlow version\n",
        "#import tensorflow as tf\n",
        "#print(tf.__version__)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qc8RJkIJaxap",
        "colab_type": "text"
      },
      "source": [
        "##1. Download the MNIST dataset\n",
        "As usual:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "r-_9ZwQkiGQg",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import print_function\n",
        "import keras\n",
        "from keras.datasets import mnist\n",
        "from keras import backend as K\n",
        "\n",
        "num_classes = 10\n",
        "\n",
        "# input image dimensions\n",
        "img_rows, img_cols = 28, 28\n",
        "\n",
        "# the data, split between train and test sets\n",
        "(x_train, y_train), (x_test, y_test) = mnist.load_data()\n",
        "\n",
        "if K.image_data_format() == 'channels_first':\n",
        "    x_train = x_train.reshape(x_train.shape[0], 1, img_rows, img_cols)\n",
        "    x_test = x_test.reshape(x_test.shape[0], 1, img_rows, img_cols)\n",
        "    input_shape = (1, img_rows, img_cols)\n",
        "else:\n",
        "    x_train = x_train.reshape(x_train.shape[0], img_rows, img_cols, 1)\n",
        "    x_test = x_test.reshape(x_test.shape[0], img_rows, img_cols, 1)\n",
        "    input_shape = (img_rows, img_cols, 1)\n",
        "\n",
        "# Pre-process inputs\n",
        "x_train = x_train.astype('float32')\n",
        "x_test = x_test.astype('float32')\n",
        "x_train /= 255\n",
        "x_test /= 255\n",
        "\n",
        "# Convert class indices to one-hot vectors\n",
        "y_train = keras.utils.to_categorical(y_train, num_classes)\n",
        "y_test = keras.utils.to_categorical(y_test, num_classes)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Son2qjrbQTq",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Input shape: 28 x 28 x 1 = image with one color channel\n",
        "print('input_shape :',input_shape)\n",
        "\n",
        "# Pre-process inputs\n",
        "print('x_train shape:', x_train.shape)\n",
        "print(x_train.shape[0], 'train samples')\n",
        "print(x_test.shape[0], 'test samples')\n",
        "\n",
        "# to_categorical converts class indices to one-hot vectors\n",
        "print('y_train shape:', y_train.shape)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6YJnv8WI7b4B",
        "colab_type": "text"
      },
      "source": [
        "##2. Task 1: Convolutional Autoencoder\n",
        "Here is an example of a Convolutional Autoencoder (CAE) for MNIST.\n",
        "\n",
        "**Note:** There are many ways to implement CAEs. This one is designed to map the input image down to a 2D latent space, so that you can plot the latent vectors in 2D. Also note how we define the encoder and the decoder separately and combine them afterwards to form the final CAE model.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7aABSvT1cME5",
        "colab_type": "text"
      },
      "source": [
        "###2.1 Your task\n",
        "The shape of the decoder's output should match the input shape (28x28x1), but it doesn't. Try for yourself. Your task is to fix this problem by modifying this line of code:\n",
        "\n",
        "```\n",
        "decoded = Conv2D(1, kernel_size=(3, 3), padding='same', activation='sigmoid')(x)\n",
        "```\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BIBw4LuzVEEh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from keras.layers import Input, Dropout, Flatten, Dense, Conv2D, MaxPooling2D\n",
        "from keras.models import Model\n",
        "from keras.layers import UpSampling2D, ZeroPadding2D, Conv2DTranspose, Reshape\n",
        "\n",
        "# Number of latent dimensions\n",
        "latent_dim = 2\n",
        "\n",
        "# Encoder (convolutional base)\n",
        "inputs = Input(shape=(28, 28, 1))\n",
        "x = ZeroPadding2D(padding=(2, 2))(inputs)\n",
        "x = Conv2D(8, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(16, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(32, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "\n",
        "# shape info needed to build decoder model\n",
        "shape = K.int_shape(x)\n",
        "\n",
        "x = Flatten()(x)\n",
        "encoded = Dense(latent_dim)(x)\n",
        "encoder = Model(inputs, encoded)\n",
        "encoder.summary()\n",
        "print((\"shape of encoded\", K.int_shape(encoded)))\n",
        "\n",
        "# Decoder (upsamling)\n",
        "encoding = Input(shape=(1, 1, latent_dim))\n",
        "x = Dense(shape[1] * shape[2] * shape[3], activation='relu')(encoding)\n",
        "x = Reshape((shape[1], shape[2], shape[3]))(x)\n",
        "x = Conv2DTranspose(32, (3,3), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(16, (3,3), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(8, (3,3), strides=(2,2), padding='same')(x)\n",
        "decoded = Conv2D(1, kernel_size=(3, 3), padding='same', activation='sigmoid')(x) # Fix this line !!!\n",
        "decoder = Model(encoding, decoded)\n",
        "decoder.summary()\n",
        "print((\"shape of decoded\", K.int_shape(decoded)))\n",
        "\n",
        "x = encoder(inputs)\n",
        "predictions = decoder(x)\n",
        "autoencoder = Model(input=inputs, output=predictions)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1gIKS_zSWZng",
        "colab_type": "text"
      },
      "source": [
        "###2.2 Questions:\n",
        "1. What does the `ZeroPadding2D` layer do?\n",
        "2. What is the shape of the data before and after zero padding? (Note: for downsampling and upsampling it is more convenient if the shape of the data is a power of 2).\n",
        "3. What is the purpose of the `Reshape`layer in the decoder?\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bKXQlf8tBnmc",
        "colab_type": "text"
      },
      "source": [
        "###2.3 Training\n",
        "Let's train the autoencoder for 30 epochs (add more epochs to improve results):"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Vw_OqAHD5k1-",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "autoencoder.compile(optimizer='adam', loss='mse')\n",
        "history = autoencoder.fit(x_train, x_train, epochs=30, batch_size=256,\n",
        "               shuffle=True, validation_data=(x_test, x_test), verbose=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q_91-fqbZSPp",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "plt.plot(history.history['loss'])\n",
        "plt.plot(history.history['val_loss'])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gcd5EgZuZI1l",
        "colab_type": "text"
      },
      "source": [
        "###2.4 Plot the latent space representation\n",
        "To get some intuition about what our autoencoder has learned, we can plot the latent representation of the training data:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G7Idi6hcEhjx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import matplotlib as mpl\n",
        "mpl.rc('image', cmap='jet')\n",
        "\n",
        "# Get latent representation\n",
        "z = encoder.predict(x_train,batch_size=32)\n",
        "\n",
        "# Plot\n",
        "plt.figure(figsize=(12, 10))\n",
        "plt.scatter(z[:, 0], z[:, 1], c=np.argmax(y_train,axis=1))\n",
        "plt.colorbar()\n",
        "plt.xlabel(\"z[0]\")\n",
        "plt.ylabel(\"z[1]\")\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dlR9C90cfUVu",
        "colab_type": "text"
      },
      "source": [
        "Rather than plotting the latent representation of the training samples, we could also use the CAE to *generate* new samples. We do this by generating latent vectors that span a 2D grid (defined by `grid_x` and `grid_y` below) and then feed each latent vector on the grid into the decoder to generate an image: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Nqk1t_r8_UNu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "n = 20\n",
        "digit_size = 28\n",
        "figure = np.zeros((digit_size * n, digit_size * n))\n",
        "# linearly spaced coordinates corresponding to the 2D plot\n",
        "# of digit classes in the latent space\n",
        "grid_x = np.linspace(-6, 3, n)        # Task : Set range according to your latent representation\n",
        "grid_y = np.linspace(-3, 6, n)[::-1]  # Task : Set range according to your latent representation\n",
        "\n",
        "for i, yi in enumerate(grid_y):\n",
        "    for j, xi in enumerate(grid_x):\n",
        "        z_sample = np.array([[xi, yi]])\n",
        "        x_decoded = decoder.predict(z_sample.reshape(1,1,1,2))\n",
        "        digit = x_decoded[0].reshape(digit_size, digit_size)\n",
        "        figure[i * digit_size: (i + 1) * digit_size,\n",
        "                j * digit_size: (j + 1) * digit_size] = digit\n",
        "\n",
        "plt.figure(figsize=(10, 10))\n",
        "start_range = digit_size // 2\n",
        "end_range = n * digit_size + start_range + 1\n",
        "pixel_range = np.arange(start_range, end_range, digit_size)\n",
        "sample_range_x = np.round(grid_x, 1)\n",
        "sample_range_y = np.round(grid_y, 1)\n",
        "plt.xticks(pixel_range, sample_range_x)\n",
        "plt.yticks(pixel_range, sample_range_y)\n",
        "plt.xlabel(\"z[0]\")\n",
        "plt.ylabel(\"z[1]\")\n",
        "plt.imshow(figure, cmap='Greys_r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FnnBuylfeHO_",
        "colab_type": "text"
      },
      "source": [
        "**Sub-task:** To get the best result, modify the x and y ranges (`grid_x` and `grid_y') so that they approximately match the ranges observed in the previous plot of the training data.\n",
        "\n",
        "###2.5 Question\n",
        "1. Which digits can the autoencoder generate faithfully, which digits does it have trouble generating? Why?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EwXofIvQhOdN",
        "colab_type": "text"
      },
      "source": [
        "###2.6 Encoding, decoding and latent space interpolation\n",
        "Now that we have trained an autoencoder, we can use it to encode existing images and generate new images (from a latent representation). With the latent representation we can also start doing interpolation between training samples.\n",
        "\n",
        "Your task is to \n",
        "\n",
        "1. Encode an image of a 7 and an image of a 9 (or any other pair if you refer)\n",
        "2. Decode the encodings to generate reconstructed images\n",
        "3. Interpolate between the two digits in latent space"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qoBUEtgdSu_i",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Draw two samples (a 7 and a 9) and display them\n",
        "y_test_category = np.argmax(y_test,axis=1)\n",
        "ix7 = np.where(y_test_category==7)[0][1] # Pick a 7\n",
        "ix9 = np.where(y_test_category==9)[0][1] # Pick a 9\n",
        "plt.subplot(221);plt.imshow(x_test[ix7,:].squeeze(),cmap='gray')\n",
        "plt.subplot(222);plt.imshow(x_test[ix9,:].squeeze(),cmap='gray')\n",
        "\n",
        "# Subtask 1 (encoding):\n",
        "# Calculate the latent representation of each sample using the encoder\n",
        "z7 = # encode x_test[ix7,:]\n",
        "z9 = # encode x_test[ix9,:]\n",
        "\n",
        "# Subtask 2 (decoding):\n",
        "# Reconstruct images from the two latent vectors using the decoder\n",
        "x_hat_7 = # decode z7\n",
        "x_hat_9 = # decode z9\n",
        "\n",
        "# Show reconstruction\n",
        "plt.subplot(223);plt.imshow(x_hat_7.squeeze(),cmap='gray')\n",
        "plt.subplot(224);plt.imshow(x_hat_9.squeeze(),cmap='gray')\n",
        "\n",
        "# Subtask 3 (interpolate):\n",
        "# Just run - no changes required)\n",
        "N = 8\n",
        "interp_features = np.zeros((N,latent_dim))\n",
        "for i in range(latent_dim):\n",
        "  interp_features[:,i] = np.linspace(z7[0,i].squeeze(),z9[0,i].squeeze(),N)\n",
        "\n",
        "plt.figure(figsize=(20,6))\n",
        "for i in range(N):\n",
        "  x = interp_features[i,:].reshape(1,1,1,latent_dim)\n",
        "  out = decoder.predict(x)\n",
        "  plt.subplot(1,N,i+1)\n",
        "  plt.imshow(out.squeeze(),cmap='gray')  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8QQDl2AC22AF",
        "colab_type": "text"
      },
      "source": [
        "**Perspectives:** Given a dataset of facial images, you could use latent space interpolation to generate images like these:\n",
        "\n",
        "![alt text](https://github.com/davidsandberg/facenet/wiki/20170708-150701-add_smile.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gE_--D90TNsz",
        "colab_type": "text"
      },
      "source": [
        "##3. Task 2: Implement a CAE from scratch\n",
        "The purpose of this task is to test if you can implement a CAE from scratch. **I recommend you skip ahead and complete the tasks on variational encoders and GANs first, and then return to this task later**.\n",
        "\n",
        "Your task is to implement this CAE archtecture for MNIST:\n",
        "\n",
        "![alt text](https://github.com/aivclab/dlcourse/raw/master/data/Lab9_CAE_architecture.png)\n",
        "\n",
        "**Explanation**:\n",
        "- \"Conv 1\", \"Conv 2\", \"Conv 3\", \"D Conv 1\", \"D Conv 2\", \"D Conv 3\", and \"D Conv 4\" are *all* regular 2D convolutions: [Conv2D](https://keras.io/layers/convolutional/#conv2d).\n",
        "- \"M.P\" is short for Max Pooling\n",
        "- \"U.S\" is short for upsampling. You must use [UpSampling2D](https://keras.io/layers/convolutional/#upsampling2d) and **not** [Conv2DTranspose](https://keras.io/layers/convolutional/#conv2dtranspose). (What's the difference by the way?)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PKKWQM-xVlfI",
        "colab_type": "text"
      },
      "source": [
        "##4. Task 3: Variational Autoencoder\n",
        "Recall that variational autoencoders (VAE) are designed to learn smooth latent space representation (the problem with traditional autoencoders is that they tend to generate gaps in the latent space, making interpolation impossible). The purpose of this task is to see if this is actually the case in practise.\n",
        "\n",
        "Below is an implementation of a convolutional VAE."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "odSqFFagAaCR",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from keras.layers import UpSampling2D, ZeroPadding2D, Conv2DTranspose, Lambda, Reshape\n",
        "\n",
        "batch_size = 256\n",
        "latent_dim = 2\n",
        "\n",
        "def sampling(args):\n",
        "    z_mean, z_log_sigma = args\n",
        "    epsilon = K.random_normal(shape=(batch_size, latent_dim))\n",
        "    return z_mean + K.exp(z_log_sigma) * epsilon\n",
        "\n",
        "# VAE model = encoder + decoder\n",
        "# build encoder model\n",
        "inputs = Input(shape=(28, 28, 1),name='encoder_input')\n",
        "x = ZeroPadding2D(padding=(2, 2))(inputs)\n",
        "x = Conv2D(8, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(16, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(32, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(64, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "x = Conv2D(64, kernel_size=(3, 3), strides=(2,2), activation='relu', padding='same')(x)\n",
        "\n",
        "# shape info needed to build decoder model\n",
        "shape = K.int_shape(x)\n",
        "\n",
        "# generate latent vector Q(z|X)\n",
        "x = Flatten()(x)\n",
        "z_mean = Dense(latent_dim, name='z_mean')(x)\n",
        "z_log_var = Dense(latent_dim, name='z_log_var')(x)\n",
        "\n",
        "# use reparameterization trick to push the sampling out as input\n",
        "# note that \"output_shape\" isn't necessary with the TensorFlow backend\n",
        "z = Lambda(sampling, output_shape=(latent_dim,), name='z')([z_mean, z_log_var])\n",
        "\n",
        "# instantiate encoder model\n",
        "encoder = Model(inputs, [z_mean, z_log_var, z], name='encoder')\n",
        "encoder.summary()\n",
        "\n",
        "# build decoder model\n",
        "latent_inputs = Input(shape=(latent_dim,), name='z_sampling')\n",
        "x = Dense(shape[1] * shape[2] * shape[3], activation='relu')(latent_inputs)\n",
        "x = Reshape((shape[1], shape[2], shape[3]))(x)\n",
        "\n",
        "x = Conv2DTranspose(64, (1,1), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(32, (3,3), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(16, (3,3), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(8, (3,3), strides=(2,2), padding='same')(x)\n",
        "x = Conv2DTranspose(8, (3,3), strides=(2,2), padding='same')(x)\n",
        "outputs = Conv2D(1, kernel_size=(5, 5), padding='valid', activation='sigmoid')(x)\n",
        "\n",
        "# instantiate decoder model\n",
        "decoder = Model(latent_inputs, outputs, name='decoder')\n",
        "decoder.summary()\n",
        "\n",
        "# instantiate VAE model\n",
        "outputs = decoder(encoder(inputs)[2])\n",
        "vae = Model(inputs, outputs, name='vae')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F1oRlpirW_4g",
        "colab_type": "text"
      },
      "source": [
        "###4.1 Questions\n",
        "1. What does the `sampling` function do?\n",
        "2. The encoder outputs three variables: `[z_mean, z_log_var, z]`. What do they represent?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yIYCApwOTA11",
        "colab_type": "text"
      },
      "source": [
        "###4.2 Loss function\n",
        "The loss consists of two terms:\n",
        "\n",
        "- A reconstruction term (or similarity term)\n",
        "- and a KL divergence term\n",
        "\n",
        "You can read more about it here: https://towardsdatascience.com/intuitively-understanding-variational-autoencoders-1bfe67eb5daf\n",
        "\n",
        "The KL term is:\n",
        "\n",
        "![alt text](https://miro.medium.com/max/520/1*uEAxCmyVKxzZOJG6afkCCg.png)\n",
        "\n",
        "**Sub-task:** Identify the individual terms in the code block below."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NkGEWszvFb_A",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from keras.losses import mse\n",
        "reconstruction_loss = mse(K.flatten(inputs), K.flatten(outputs))\n",
        "reconstruction_loss *= 28 * 28\n",
        "kl_loss = K.exp(z_log_var) + K.square(z_mean) - z_log_var - 1\n",
        "kl_loss = K.sum(kl_loss, axis=-1)\n",
        "kl_loss *= 0.5\n",
        "vae_loss = K.mean(reconstruction_loss + kl_loss)\n",
        "vae.add_loss(vae_loss)\n",
        "vae.compile(optimizer='rmsprop')\n",
        "vae.summary()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aMnJJSHMazvZ",
        "colab_type": "text"
      },
      "source": [
        "###4.3 Training"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8B6HwxBBFs9v",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "num_samples = int(np.floor(x_train.shape[0] / batch_size) * batch_size)\n",
        "vae.fit(x_train[0:num_samples,:], epochs=30, batch_size=batch_size,\n",
        "        shuffle=True, verbose=1)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "__jYiiWaa4tp",
        "colab_type": "text"
      },
      "source": [
        "###4.4 Plot the latent space representation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CSyV2LxfvYj_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "z_mean, _, _ = encoder.predict(x_train[0:num_samples,:],\n",
        "                                batch_size=batch_size)\n",
        "import matplotlib as mpl\n",
        "mpl.rc('image', cmap='jet')\n",
        "plt.figure(figsize=(12, 10))\n",
        "plt.scatter(z_mean[:, 0], z_mean[:, 1], c=np.argmax(y_train[0:num_samples,:],axis=1))\n",
        "plt.colorbar()\n",
        "plt.xlabel(\"z[0]\")\n",
        "plt.ylabel(\"z[1]\")\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rmdhxSxk2LKw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "n = 20\n",
        "digit_size = 28\n",
        "figure = np.zeros((digit_size * n, digit_size * n))\n",
        "# linearly spaced coordinates corresponding to the 2D plot\n",
        "# of digit classes in the latent space\n",
        "grid_x = np.linspace(-2, 2, n)\n",
        "grid_y = np.linspace(-2, 2, n)[::-1]\n",
        "\n",
        "for i, yi in enumerate(grid_y):\n",
        "    for j, xi in enumerate(grid_x):\n",
        "        z_sample = np.array([[xi, yi]])\n",
        "        x_decoded = decoder.predict(z_sample)\n",
        "        digit = x_decoded[0].reshape(digit_size, digit_size)\n",
        "        figure[i * digit_size: (i + 1) * digit_size,\n",
        "                j * digit_size: (j + 1) * digit_size] = digit\n",
        "\n",
        "plt.figure(figsize=(10, 10))\n",
        "start_range = digit_size // 2\n",
        "end_range = n * digit_size + start_range + 1\n",
        "pixel_range = np.arange(start_range, end_range, digit_size)\n",
        "sample_range_x = np.round(grid_x, 1)\n",
        "sample_range_y = np.round(grid_y, 1)\n",
        "plt.xticks(pixel_range, sample_range_x)\n",
        "plt.yticks(pixel_range, sample_range_y)\n",
        "plt.xlabel(\"z[0]\")\n",
        "plt.ylabel(\"z[1]\")\n",
        "plt.imshow(figure, cmap='Greys_r')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lRrVCxqraIS5",
        "colab_type": "text"
      },
      "source": [
        "###4.5 Questions\n",
        "1. What do you think of this latent representation? In terms of quality? In terms of smoothness? Compare to the same plot for the traditional autoencoder.\n",
        "2. Which digits does the model faithfully reconstruct? Which digits does it have trouble reconstructing? Why?\n",
        "3. What happens if you set the weight of the KL term to, say 5 (`kl_loss *= 5`), and re-train the model?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wR5jbEkPbYIa",
        "colab_type": "text"
      },
      "source": [
        "###4.6 Encoding, decoding and latent space interpolation\n",
        "Like we did for the traditional autoencoder (see section 2.6), your task is to \n",
        "\n",
        "1. Encode an image of a 7 and an image of a 9 (**warning - this is trickier than you might think!!!**) \n",
        "2. Decode the encodings (to generate reconstructed images)\n",
        "3. Interpolate between the two digits in latent space"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9dHZ9BKb4Tx_",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Draw two samples (a 7 and a 9) and display them\n",
        "y_test_category = np.argmax(y_test,axis=1)\n",
        "ix7 = np.where(y_test_category==7)[0][1]\n",
        "ix9 = np.where(y_test_category==9)[0][1]\n",
        "plt.subplot(221);plt.imshow(x_test[ix7,:].squeeze(),cmap='gray')\n",
        "plt.subplot(222);plt.imshow(x_test[ix9,:].squeeze(),cmap='gray')\n",
        "\n",
        "# Subtask 1 (encoding): Calculate the latent representation of each sample\n",
        "z7 = # encode x_test[ix7,:]\n",
        "z9 = # encode x_test[ix9,:]\n",
        "\n",
        "# Subtask 2 (decoding): Reconstruct images from the two latent vectors\n",
        "x_hat_7 = # decode z7\n",
        "x_hat_9 = # decode z9\n",
        "\n",
        "# Show reconstruction\n",
        "plt.subplot(223);plt.imshow(x_hat_7.squeeze(),cmap='gray')\n",
        "plt.subplot(224);plt.imshow(x_hat_9.squeeze(),cmap='gray')\n",
        "\n",
        "# Subtask 3 (interpolate): Just run - no changes required)\n",
        "N = 8\n",
        "interp_features = np.zeros((N,latent_dim))\n",
        "for i in range(latent_dim):\n",
        "  interp_features[:,i] = np.linspace(z7[0,i].squeeze(),z9[0,i].squeeze(),N)\n",
        "\n",
        "plt.figure(figsize=(20,6))\n",
        "for i in range(N):\n",
        "  x = interp_features[i,:].reshape(1,1,1,latent_dim)\n",
        "  out = decoder.predict(x)\n",
        "  plt.subplot(1,N,i+1)\n",
        "  plt.imshow(out.squeeze(),cmap='gray')  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pg5OtlGMQLYm",
        "colab_type": "text"
      },
      "source": [
        "**Note:** If you want to make nicer reconstructions and better interpolations, increase the latent dimensionality (latent_dim) and re-train the model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a_5WzyIk9LFB",
        "colab_type": "text"
      },
      "source": [
        "##5. Task 4: Generative Adversarial Networks\n",
        "Below is an implementation of a Deep Convolutional GAN (DCGAN) for MNIST.\n",
        "\n",
        "Code here: https://github.com/eriklindernoren/Keras-GAN/blob/master/dcgan/dcgan.py"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_j2dUmszQlVh",
        "colab_type": "text"
      },
      "source": [
        "###5.1 Recap of GANs\n",
        "A DCGAN is a generative model that learns to map random noise vectors into images. Unlike an autoencoder, which  encodes and decodes an image into itself, DCGAN learns to generate images that look real. This means that you must have a data set of real images to compare with.\n",
        "\n",
        "The network consists of two sub-networks that are trained in tandem:\n",
        "\n",
        "- The **Generator** takes a random noise vector and maps it into an image.\n",
        "- The **Discriminator** takes an input image, which is either **\"real\"** (i.e., picked from the database of real images) or **\"fake\"** (i.e., generated by the Generator). It then learns to distingiush between real and fake images.\n",
        "\n",
        "The two networks are competing against each other, and at some point the Generator becomes so good at generating fakes, which look real, that the Discriminator can no longer distuingish fakes from reals.\n",
        "\n",
        "GANs are really hard to train and the above example is just a toy example. The training loop looks like this:\n",
        "\n",
        "![alt text](https://3qeqpr26caki16dnhd19sv6by6v-wpengine.netdna-ssl.com/wp-content/uploads/2019/05/Summary-of-the-Generative-Adversarial-Network-Training-Algorithm-1024x669.png)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TJ7YCEwLSdnK",
        "colab_type": "text"
      },
      "source": [
        "###5.2 Your tasks\n",
        "1. Run the code block to start training the model. In the meantime go through the code and see if you can identify the major steps of the training loop.\n",
        "\n",
        "2. What is the dimensionality of the latent space in this example? Change it to 2 instead.\n",
        "\n",
        "3. Extend the code such that you can train a DCGAN and subsequently make it generate images based on some 2D latent vector that you specify. Use this to make a plot of the 2D latent space, like we did above. What do you observe?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8eYaGm4oV2oo",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import print_function, division\n",
        "\n",
        "from keras.datasets import mnist\n",
        "from keras.layers import Input, Dense, Reshape, Flatten, Dropout\n",
        "from keras.layers import BatchNormalization, Activation, ZeroPadding2D\n",
        "from keras.layers.advanced_activations import LeakyReLU\n",
        "from keras.layers.convolutional import UpSampling2D, Conv2D\n",
        "from keras.models import Sequential, Model\n",
        "from keras.optimizers import Adam\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import sys\n",
        "\n",
        "class DCGAN():\n",
        "    def __init__(self):\n",
        "        # Input shape\n",
        "        self.img_rows = 28\n",
        "        self.img_cols = 28\n",
        "        self.channels = 1\n",
        "        self.img_shape = (self.img_rows, self.img_cols, self.channels)\n",
        "        self.latent_dim = 10\n",
        "\n",
        "        optimizer = Adam(0.0002, 0.5)\n",
        "\n",
        "        # Build and compile the discriminator\n",
        "        self.discriminator = self.build_discriminator()\n",
        "        self.discriminator.compile(loss='binary_crossentropy',\n",
        "            optimizer=optimizer,\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "        # Build the generator\n",
        "        self.generator = self.build_generator()\n",
        "\n",
        "        # The generator takes noise as input and generates imgs\n",
        "        z = Input(shape=(self.latent_dim,))\n",
        "        img = self.generator(z)\n",
        "\n",
        "        # For the combined model we will only train the generator\n",
        "        self.discriminator.trainable = False\n",
        "\n",
        "        # The discriminator takes generated images as input and determines validity\n",
        "        valid = self.discriminator(img)\n",
        "\n",
        "        # The combined model  (stacked generator and discriminator)\n",
        "        # Trains the generator to fool the discriminator\n",
        "        self.combined = Model(z, valid)\n",
        "        self.combined.compile(loss='binary_crossentropy', optimizer=optimizer)\n",
        "\n",
        "    def build_generator(self):\n",
        "\n",
        "        model = Sequential()\n",
        "\n",
        "        model.add(Dense(128 * 7 * 7, activation=\"relu\", input_dim=self.latent_dim))\n",
        "        model.add(Reshape((7, 7, 128)))\n",
        "        model.add(UpSampling2D())\n",
        "        model.add(Conv2D(128, kernel_size=3, padding=\"same\"))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(Activation(\"relu\"))\n",
        "        model.add(UpSampling2D())\n",
        "        model.add(Conv2D(64, kernel_size=3, padding=\"same\"))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(Activation(\"relu\"))\n",
        "        model.add(Conv2D(self.channels, kernel_size=3, padding=\"same\"))\n",
        "        model.add(Activation(\"tanh\"))\n",
        "\n",
        "        model.summary()\n",
        "\n",
        "        noise = Input(shape=(self.latent_dim,))\n",
        "        img = model(noise)\n",
        "\n",
        "        return Model(noise, img)\n",
        "\n",
        "    def build_discriminator(self):\n",
        "\n",
        "        model = Sequential()\n",
        "\n",
        "        model.add(Conv2D(32, kernel_size=3, strides=2, input_shape=self.img_shape, padding=\"same\"))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.25))\n",
        "        model.add(Conv2D(64, kernel_size=3, strides=2, padding=\"same\"))\n",
        "        model.add(ZeroPadding2D(padding=((0,1),(0,1))))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.25))\n",
        "        model.add(Conv2D(128, kernel_size=3, strides=2, padding=\"same\"))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.25))\n",
        "        model.add(Conv2D(256, kernel_size=3, strides=1, padding=\"same\"))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.25))\n",
        "        model.add(Flatten())\n",
        "        model.add(Dense(1, activation='sigmoid'))\n",
        "\n",
        "        model.summary()\n",
        "\n",
        "        img = Input(shape=self.img_shape)\n",
        "        validity = model(img)\n",
        "\n",
        "        return Model(img, validity)\n",
        "\n",
        "    def train(self, epochs, batch_size=128, save_interval=500):\n",
        "\n",
        "        # Load the dataset\n",
        "        (X_train, _), (_, _) = mnist.load_data()\n",
        "\n",
        "        # Rescale -1 to 1\n",
        "        X_train = X_train / 127.5 - 1.\n",
        "        X_train = np.expand_dims(X_train, axis=3)\n",
        "\n",
        "        # Adversarial ground truths\n",
        "        valid = np.ones((batch_size, 1))\n",
        "        fake = np.zeros((batch_size, 1))\n",
        "\n",
        "        for epoch in range(epochs):\n",
        "\n",
        "            # ---------------------\n",
        "            #  Train Discriminator\n",
        "            # ---------------------\n",
        "\n",
        "            # Select a random half of images\n",
        "            idx = np.random.randint(0, X_train.shape[0], batch_size)\n",
        "            imgs = X_train[idx]\n",
        "\n",
        "            # Sample noise and generate a batch of new images\n",
        "            noise = np.random.normal(0, 1, (batch_size, self.latent_dim))\n",
        "            gen_imgs = self.generator.predict(noise)\n",
        "\n",
        "            # Train the discriminator (real classified as ones and generated as zeros)\n",
        "            d_loss_real = self.discriminator.train_on_batch(imgs, valid)\n",
        "            d_loss_fake = self.discriminator.train_on_batch(gen_imgs, fake)\n",
        "            d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)\n",
        "\n",
        "            # ---------------------\n",
        "            #  Train Generator\n",
        "            # ---------------------\n",
        "\n",
        "            # Train the generator (wants discriminator to mistake images as real)\n",
        "            g_loss = self.combined.train_on_batch(noise, valid)\n",
        "\n",
        "            # Plot the progress\n",
        "            print (\"%d [D loss: %f, acc.: %.2f%%] [G loss: %f]\" % (epoch, d_loss[0], 100*d_loss[1], g_loss))\n",
        "\n",
        "            # If at save interval => save generated image samples\n",
        "            if epoch % save_interval == 0:\n",
        "                print (\"%d [D loss: %f, acc.: %.2f%%] [G loss: %f]\" % (epoch, d_loss[0], 100*d_loss[1], g_loss))\n",
        "                self.save_imgs(epoch)\n",
        "\n",
        "    def save_imgs(self, epoch):\n",
        "        r, c = 5, 5\n",
        "        noise = np.random.normal(0, 1, (r * c, self.latent_dim))\n",
        "        gen_imgs = self.generator.predict(noise)\n",
        "\n",
        "        # Rescale images 0 - 1\n",
        "        gen_imgs = 0.5 * gen_imgs + 0.5\n",
        "\n",
        "        plt.figure()\n",
        "        fig, axs = plt.subplots(r, c)\n",
        "        cnt = 0\n",
        "        for i in range(r):\n",
        "            for j in range(c):\n",
        "                axs[i,j].imshow(gen_imgs[cnt, :,:,0], cmap='gray')\n",
        "                axs[i,j].axis('off')\n",
        "                cnt += 1\n",
        "        #fig.savefig(\"mnist_%d.png\" % epoch)\n",
        "        plt.show()\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    dcgan = DCGAN()\n",
        "    dcgan.train(epochs=4000, batch_size=32, save_interval=500)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVHtVbCfQ0_R",
        "colab_type": "text"
      },
      "source": [
        "##5.3 Conditional GAN (optional)\n",
        "The original GAN has no knowledge, and hence no understanding of the data's class labels. CGAN aims to solve this issue by telling both the generator and the discriminator what the class label is. Specifically, CGAN concatenates a one-hot vector y to the random noise vector z to result in an architecture that looks like this:\n",
        "\n",
        "![alt text](https://paper-attachments.dropbox.com/s_D85DDA7D01FD04AEE96825C4B90F1126BC7D080CA4F2947D4A5DEC07FAD6122C_1559840765144_Screenshot+2019-06-06+at+10.35.29+PM.png)\n",
        "\n",
        "If you have more time, try out the CGAN tutorial:\n",
        "\n",
        "- https://github.com/eriklindernoren/Keras-GAN#cgan\n",
        "- https://github.com/eriklindernoren/Keras-GAN/blob/master/cgan/cgan.py\n",
        "\n",
        "How does it work?\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jwzhPsIL9pYa",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from __future__ import print_function, division\n",
        "\n",
        "from keras.datasets import mnist\n",
        "from keras.layers import Input, Dense, Reshape, Flatten, Dropout, multiply\n",
        "from keras.layers import BatchNormalization, Activation, Embedding, ZeroPadding2D\n",
        "from keras.layers.advanced_activations import LeakyReLU\n",
        "from keras.layers.convolutional import UpSampling2D, Conv2D\n",
        "from keras.models import Sequential, Model\n",
        "from keras.optimizers import Adam\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "class CGAN():\n",
        "    def __init__(self):\n",
        "        # Input shape\n",
        "        self.img_rows = 28\n",
        "        self.img_cols = 28\n",
        "        self.channels = 1\n",
        "        self.img_shape = (self.img_rows, self.img_cols, self.channels)\n",
        "        self.num_classes = 10\n",
        "        self.latent_dim = 100\n",
        "\n",
        "        optimizer = Adam(0.0002, 0.5)\n",
        "\n",
        "        # Build and compile the discriminator\n",
        "        self.discriminator = self.build_discriminator()\n",
        "        self.discriminator.compile(loss=['binary_crossentropy'],\n",
        "            optimizer=optimizer,\n",
        "            metrics=['accuracy'])\n",
        "\n",
        "        # Build the generator\n",
        "        self.generator = self.build_generator()\n",
        "\n",
        "        # The generator takes noise and the target label as input\n",
        "        # and generates the corresponding digit of that label\n",
        "        noise = Input(shape=(self.latent_dim,))\n",
        "        label = Input(shape=(1,))\n",
        "        img = self.generator([noise, label])\n",
        "\n",
        "        # For the combined model we will only train the generator\n",
        "        self.discriminator.trainable = False\n",
        "\n",
        "        # The discriminator takes generated image as input and determines validity\n",
        "        # and the label of that image\n",
        "        valid = self.discriminator([img, label])\n",
        "\n",
        "        # The combined model  (stacked generator and discriminator)\n",
        "        # Trains generator to fool discriminator\n",
        "        self.combined = Model([noise, label], valid)\n",
        "        self.combined.compile(loss=['binary_crossentropy'],\n",
        "            optimizer=optimizer)\n",
        "\n",
        "    def build_generator(self):\n",
        "\n",
        "        model = Sequential()\n",
        "\n",
        "        model.add(Dense(256, input_dim=self.latent_dim))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(Dense(512))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(Dense(1024))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(BatchNormalization(momentum=0.8))\n",
        "        model.add(Dense(np.prod(self.img_shape), activation='tanh'))\n",
        "        model.add(Reshape(self.img_shape))\n",
        "\n",
        "        model.summary()\n",
        "\n",
        "        noise = Input(shape=(self.latent_dim,))\n",
        "        label = Input(shape=(1,), dtype='int32')\n",
        "        label_embedding = Flatten()(Embedding(self.num_classes, self.latent_dim)(label))\n",
        "\n",
        "        model_input = multiply([noise, label_embedding])\n",
        "        img = model(model_input)\n",
        "\n",
        "        return Model([noise, label], img)\n",
        "\n",
        "    def build_discriminator(self):\n",
        "\n",
        "        model = Sequential()\n",
        "\n",
        "        model.add(Dense(512, input_dim=np.prod(self.img_shape)))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dense(512))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.4))\n",
        "        model.add(Dense(512))\n",
        "        model.add(LeakyReLU(alpha=0.2))\n",
        "        model.add(Dropout(0.4))\n",
        "        model.add(Dense(1, activation='sigmoid'))\n",
        "        model.summary()\n",
        "\n",
        "        img = Input(shape=self.img_shape)\n",
        "        label = Input(shape=(1,), dtype='int32')\n",
        "\n",
        "        label_embedding = Flatten()(Embedding(self.num_classes, np.prod(self.img_shape))(label))\n",
        "        flat_img = Flatten()(img)\n",
        "\n",
        "        model_input = multiply([flat_img, label_embedding])\n",
        "\n",
        "        validity = model(model_input)\n",
        "\n",
        "        return Model([img, label], validity)\n",
        "\n",
        "    def train(self, epochs, batch_size=128, sample_interval=50):\n",
        "\n",
        "        # Load the dataset\n",
        "        (X_train, y_train), (_, _) = mnist.load_data()\n",
        "\n",
        "        # Configure input\n",
        "        X_train = (X_train.astype(np.float32) - 127.5) / 127.5\n",
        "        X_train = np.expand_dims(X_train, axis=3)\n",
        "        y_train = y_train.reshape(-1, 1)\n",
        "\n",
        "        # Adversarial ground truths\n",
        "        valid = np.ones((batch_size, 1))\n",
        "        fake = np.zeros((batch_size, 1))\n",
        "\n",
        "        for epoch in range(epochs):\n",
        "\n",
        "            # ---------------------\n",
        "            #  Train Discriminator\n",
        "            # ---------------------\n",
        "\n",
        "            # Select a random half batch of images\n",
        "            idx = np.random.randint(0, X_train.shape[0], batch_size)\n",
        "            imgs, labels = X_train[idx], y_train[idx]\n",
        "\n",
        "            # Sample noise as generator input\n",
        "            noise = np.random.normal(0, 1, (batch_size, 100))\n",
        "\n",
        "            # Generate a half batch of new images\n",
        "            gen_imgs = self.generator.predict([noise, labels])\n",
        "\n",
        "            # Train the discriminator\n",
        "            d_loss_real = self.discriminator.train_on_batch([imgs, labels], valid)\n",
        "            d_loss_fake = self.discriminator.train_on_batch([gen_imgs, labels], fake)\n",
        "            d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)\n",
        "\n",
        "            # ---------------------\n",
        "            #  Train Generator\n",
        "            # ---------------------\n",
        "\n",
        "            # Condition on labels\n",
        "            sampled_labels = np.random.randint(0, 10, batch_size).reshape(-1, 1)\n",
        "\n",
        "            # Train the generator\n",
        "            g_loss = self.combined.train_on_batch([noise, sampled_labels], valid)\n",
        "\n",
        "            # Plot the progress\n",
        "            print (\"%d [D loss: %f, acc.: %.2f%%] [G loss: %f]\" % (epoch, d_loss[0], 100*d_loss[1], g_loss))\n",
        "\n",
        "            # If at save interval => save generated image samples\n",
        "            if epoch % sample_interval == 0:\n",
        "                self.sample_images(epoch)\n",
        "\n",
        "    def sample_images(self, epoch):\n",
        "        r, c = 2, 5\n",
        "        noise = np.random.normal(0, 1, (r * c, 100))\n",
        "        sampled_labels = np.arange(0, 10).reshape(-1, 1)\n",
        "\n",
        "        gen_imgs = self.generator.predict([noise, sampled_labels])\n",
        "\n",
        "        # Rescale images 0 - 1\n",
        "        gen_imgs = 0.5 * gen_imgs + 0.5\n",
        "\n",
        "        fig, axs = plt.subplots(r, c)\n",
        "        cnt = 0\n",
        "        for i in range(r):\n",
        "            for j in range(c):\n",
        "                axs[i,j].imshow(gen_imgs[cnt,:,:,0], cmap='gray')\n",
        "                axs[i,j].set_title(\"Digit: %d\" % sampled_labels[cnt])\n",
        "                axs[i,j].axis('off')\n",
        "                cnt += 1\n",
        "        #fig.savefig(\"images/%d.png\" % epoch)\n",
        "        #plt.close()\n",
        "        plt.show()\n",
        "\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    cgan = CGAN()\n",
        "    cgan.train(epochs=4000, batch_size=32, sample_interval=200)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xtW6eqJ2Ujnk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}