Tensorflow/Tensorflow for Python

Convolutional Neural Network (ver. python)

딥스탯 2017. 6. 25. 20:38
MLP

Multi-layer Neural Networks in TensorFlow (ver. python)¶

Convolutional Neural Network¶

The MNIST data-set¶

In [1]:
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("~/MNIST_data/", one_hot=True)
Extracting ~/MNIST_data/train-images-idx3-ubyte.gz
Extracting ~/MNIST_data/train-labels-idx1-ubyte.gz
Extracting ~/MNIST_data/t10k-images-idx3-ubyte.gz
Extracting ~/MNIST_data/t10k-labels-idx1-ubyte.gz

모형에 대한 자세한 설명은 생략하도록 하겠습니다.¶

CNN(convolutional neural network), convolutions, max-pooling, ReLU, softmax, cross entropy, Adam

input -> conv1 -> pool1 -> conv2 -> pool2 -> [inner product -> relu] -> dropout -> [inner product -> softmax] -> output

Loss : cross entropy , Optimizer : Adam

In [2]:
x = tf.placeholder("float", shape = [None, 784])
y_ = tf.placeholder("float", shape = [None, 10])

x_image = tf.reshape(x, [-1, 28, 28, 1])
In [3]:
print("x_image=", x_image)
x_image= Tensor("Reshape:0", shape=(?, 28, 28, 1), dtype=float32)

함수 정의 : weight_variable - truncated normal distribution에서 난수 발생해서 원하는 모양으로 weight tensor를 만드는 함수.

In [4]:
def weight_variable(shape):
    initial = tf.truncated_normal(shape)
    return tf.Variable(initial)

함수 정의 : bias_variable - 원하는 모양으로 bias tensor를 만드는 함수.

In [5]:
def bias_variable(shape):
    initial = tf.constant(1.0, shape = shape)
    return tf.Variable(initial)

함수 정의 : conv2d - 2-D convolution 계산하는 함수

In [6]:
def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides = [1, 1, 1, 1], padding = 'SAME')

함수 정의 : max_pool_2x2 - max-pooling 하는 함수

In [7]:
def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize = [1, 2, 2, 1], strides = [1, 2, 2, 1], padding = 'SAME')

모형 설정¶

convolution 1

In [8]:
W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)

max-pooling 1

In [9]:
h_pool1 = max_pool_2x2(h_conv1)

convolution 2

In [10]:
W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)

max-pooling 2

In [11]:
h_pool2 = max_pool_2x2(h_conv2)

reshaping and [inner product - ReLU (activate function)] 1

In [12]:
W_fc1 = weight_variable([7*7*64, 1024])
b_fc1 = bias_variable([1024])
h_pool2_flat = tf.reshape(h_pool2, [-1, 7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

dropping out

In [13]:
keep_prob = tf.placeholder("float")
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

[inner product - softmax (activate function)] 2

In [14]:
W_fc2 = weight_variable([1024, 10])
b_fc2 = bias_variable([10])
y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)

Loss 와 Optimizer 설정¶

In [15]:
cross_entropy = -tf.reduce_sum(y_*tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

defining accuracy

In [16]:
correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

Session 반복 실행¶

In [17]:
sess = tf.Session()
sess.run(tf.global_variables_initializer())
In [18]:
for i in range(201):
    batch = mnist.train.next_batch(50)
    if i%10 == 0:
        train_accuracy = sess.run(accuracy, feed_dict = {x: batch[0], y_: batch[1], keep_prob: 1.0})
        print("step ", i, "training accuracy ", train_accuracy)
    sess.run(train_step, feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
    if i%50 == 0:
        print("test accuracy ", sess.run(accuracy, feed_dict = {x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}), "step", i)
step  0 training accuracy  0.16
test accuracy  0.098 step 0
step  10 training accuracy  0.12
step  20 training accuracy  0.06
step  30 training accuracy  0.08
step  40 training accuracy  0.1
step  50 training accuracy  0.16
test accuracy  0.098 step 50
step  60 training accuracy  0.16
step  70 training accuracy  0.12
step  80 training accuracy  0.12
step  90 training accuracy  0.04
step  100 training accuracy  0.1
test accuracy  0.098 step 100
step  110 training accuracy  0.16
step  120 training accuracy  0.1
step  130 training accuracy  0.1
step  140 training accuracy  0.14
step  150 training accuracy  0.16
test accuracy  0.098 step 150
step  160 training accuracy  0.04
step  170 training accuracy  0.1
step  180 training accuracy  0.08
step  190 training accuracy  0.08
step  200 training accuracy  0.2
test accuracy  0.098 step 200

같이보기¶

Convolutional Neural Network (ver. R)