Autoencoder (ver.R)

DATA SET 출처¶

http://archive.ics.uci.edu/ml/datasets/Iris (UCI, Iris Data Set)

참고자료¶

https://www.tensorflow.org (TensorFlow)

https://tensorflow.rstudio.com (TensorFlow for R)

함께보기¶

Autoencoder (ver.Python)

Clustering using Autoencoder (ver.R)¶

Iris Data Set¶

In [1]:

head(iris)

Sepal.Length	Sepal.Width	Petal.Length	Petal.Width	Species
5.1	3.5	1.4	0.2	setosa
4.9	3.0	1.4	0.2	setosa
4.7	3.2	1.3	0.2	setosa
4.6	3.1	1.5	0.2	setosa
5.0	3.6	1.4	0.2	setosa
5.4	3.9	1.7	0.4	setosa

In [2]:

str(iris)

'data.frame':	150 obs. of  5 variables:
 $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
 $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
 $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
 $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
 $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...

In [3]:

summary(iris)

  Sepal.Length    Sepal.Width     Petal.Length    Petal.Width   
 Min.   :4.300   Min.   :2.000   Min.   :1.000   Min.   :0.100  
 1st Qu.:5.100   1st Qu.:2.800   1st Qu.:1.600   1st Qu.:0.300  
 Median :5.800   Median :3.000   Median :4.350   Median :1.300  
 Mean   :5.843   Mean   :3.057   Mean   :3.758   Mean   :1.199  
 3rd Qu.:6.400   3rd Qu.:3.300   3rd Qu.:5.100   3rd Qu.:1.800  
 Max.   :7.900   Max.   :4.400   Max.   :6.900   Max.   :2.500  
       Species  
 setosa    :50  
 versicolor:50  
 virginica :50  
                
                
                

Handling the data set¶

In [4]:

set.seed(1)
test_obs <- sample(nrow(iris),50)

In [5]:

training_set <- iris[-test_obs,]
testing_set  <- iris[test_obs,]

In [6]:

training_y <- training_set$Species
training_X <- training_set[,-5]

In [7]:

table(training_y)

training_y
    setosa versicolor  virginica 
        33         28         39 

In [8]:

testing_y <- testing_set$Species
testing_X <- testing_set[,-5]

In [9]:

table(testing_y)

testing_y
    setosa versicolor  virginica 
        17         22         11 

In [10]:

(sd_vec <- apply(training_X,2,sd))
(mean_vec <- apply(training_X,2,mean))

Sepal.Length

0.844054668151467

Sepal.Width

0.433216767538801

Petal.Length

1.83035129376463

Petal.Width

0.792435830496417

Sepal.Length

5.936

Sepal.Width

3.08

Petal.Length

3.854

Petal.Width

1.255

In [11]:

training_X <- t((t(training_X)-mean_vec)/sd_vec)
testing_X  <- t((t(testing_X)-mean_vec)/sd_vec)

In [12]:

dim(training_X)

1. 100
2. 4

모형에 대한 자세한 설명은 생략하도록 하겠습니다.¶

Autoencoder, elu, softmax, Adam

-> encoding : [inner product -> elu] -> [inner product -> elu] -> [inner product -> elu] -> [inner product -> elu] -> [inner product -> softmax]

-> decoding : [inner product -> elu] -> [inner product -> elu] -> [inner product -> elu] -> [inner product -> elu] -> [inner product]

Loss : squared error loss, Optimizer : Adam

In [13]:

require(tensorflow)

Loading required package: tensorflow

In [14]:

x  <- tf$placeholder("float", shape(NULL, 4L))

함수 정의 : weight_variable - truncated normal distribution에서 난수 발생해서 원하는 모양으로 weight tensor를 만드는 함수.

In [15]:

weight_variable <- function(shape){
    initial <- tf$truncated_normal(as.integer(shape))
    return(tf$Variable(initial))
}

함수 정의 : bias_variable - 원하는 모양으로 bias tensor를 만드는 함수.

In [16]:

bias_variable <- function(shape){
    initial <- tf$truncated_normal(as.integer(shape))
    return(tf$Variable(initial))
}

모형 설정¶

Encoder¶

1 [inner product -> elu]

In [17]:

W_en1     <- weight_variable(c(4,6))
b_en1      <- bias_variable(c(1,6))
en_layer1 <- tf$nn$elu(tf$matmul(x, W_en1) + b_en1)

2 [inner product -> elu]

In [18]:

W_en2     <- weight_variable(c(6,6))
b_en2      <- bias_variable(c(1,6))
en_layer2 <- tf$nn$elu(tf$matmul(en_layer1, W_en2) + b_en2)

3 [inner product -> elu]

In [19]:

W_en3     <- weight_variable(c(6,5))
b_en3      <- bias_variable(c(1,5))
en_layer3 <- tf$nn$elu(tf$matmul(en_layer2, W_en3) + b_en3)

4 [inner product -> elu]

In [20]:

W_en4     <- weight_variable(c(5,4))
b_en4      <- bias_variable(c(1,4))
en_layer4 <- tf$nn$elu(tf$matmul(en_layer3, W_en4) + b_en4)

5 [inner product -> softmax]

In [21]:

W_en5     <- weight_variable(c(4,3))
b_en5      <- bias_variable(c(1,3))
en_layer5 <- tf$nn$softmax(tf$matmul(en_layer4, W_en5) + b_en5)

Decoder¶

1 [inner product -> elu]

In [22]:

W_de1     <- weight_variable(c(3,4))
b_de1      <- bias_variable(c(1,4))
de_layer1 <- tf$nn$elu(tf$matmul(en_layer5, W_de1) + b_de1)

2 [inner product -> elu]

In [23]:

W_de2     <- weight_variable(c(4,5))
b_de2      <- bias_variable(c(1,5))
de_layer2 <- tf$nn$elu(tf$matmul(de_layer1, W_de2) + b_de2)

3 [inner product -> elu]

In [24]:

W_de3     <- weight_variable(c(5,6))
b_de3      <- bias_variable(c(1,6))
de_layer3 <- tf$nn$elu(tf$matmul(de_layer2, W_de3) + b_de3)

4 [inner product -> elu]

In [25]:

W_de4     <- weight_variable(c(6,6))
b_de4      <- bias_variable(c(1,6))
de_layer4 <- tf$nn$elu(tf$matmul(de_layer3, W_de4) + b_de4)

5 [inner product]

In [26]:

W_de5     <- weight_variable(c(6,4))
b_de5      <- bias_variable(c(1,4))
de_layer5 <- tf$matmul(de_layer4, W_de5) + b_de5

Loss 와 Optimizer 설정¶

In [27]:

SSE          <- tf$reduce_sum((de_layer5 - x)^2)
train_step <- tf$train$AdamOptimizer(1e-4)$minimize(SSE)

Session 반복 실행¶

In [28]:

sess <- tf$Session()
sess$run(tf$global_variables_initializer())

In [29]:

for(i in 0:500000){
    batch_obs <- sample(100,50)
    sess$run(train_step,feed_dict =
             dict(x = training_X[batch_obs,]))
    if(i%%10000 == 0){
        train_accuracy <- sess$run(SSE, feed_dict =
                                   dict(x = training_X))
        cat("step ", i, "training accuracy ", train_accuracy , "\n")
    }
}

step  0 training accuracy  17789.19 
step  10000 training accuracy  68.0214 
step  20000 training accuracy  29.8632 
step  30000 training accuracy  17.07213 
step  40000 training accuracy  14.8366 
step  50000 training accuracy  13.35122 
step  60000 training accuracy  12.70672 
step  70000 training accuracy  11.99682 
step  80000 training accuracy  10.51179 
step  90000 training accuracy  9.929522 
step  100000 training accuracy  9.65847 
step  110000 training accuracy  9.401793 
step  120000 training accuracy  9.2278 
step  130000 training accuracy  9.02664 
step  140000 training accuracy  8.871257 
step  150000 training accuracy  8.726437 
step  160000 training accuracy  8.569772 
step  170000 training accuracy  8.456591 
step  180000 training accuracy  8.298332 
step  190000 training accuracy  8.208206 
step  200000 training accuracy  8.148201 
step  210000 training accuracy  8.125722 
step  220000 training accuracy  8.015854 
step  230000 training accuracy  7.945371 
step  240000 training accuracy  7.939014 
step  250000 training accuracy  7.856205 
step  260000 training accuracy  7.852313 
step  270000 training accuracy  7.792947 
step  280000 training accuracy  7.78257 
step  290000 training accuracy  7.732402 
step  300000 training accuracy  7.699124 
step  310000 training accuracy  7.667946 
step  320000 training accuracy  7.661506 
step  330000 training accuracy  7.653422 
step  340000 training accuracy  7.612647 
step  350000 training accuracy  7.596587 
step  360000 training accuracy  7.522964 
step  370000 training accuracy  7.485417 
step  380000 training accuracy  7.477603 
step  390000 training accuracy  7.453008 
step  400000 training accuracy  7.386463 
step  410000 training accuracy  7.336392 
step  420000 training accuracy  7.309639 
step  430000 training accuracy  7.2859 
step  440000 training accuracy  7.236358 
step  450000 training accuracy  7.205054 
step  460000 training accuracy  7.140727 
step  470000 training accuracy  7.089565 
step  480000 training accuracy  7.056604 
step  490000 training accuracy  7.012924 
step  500000 training accuracy  6.955946 

Clustering 결과¶

training¶

In [30]:

result <- sess$run(tf$argmax(en_layer5, 1L), feed_dict = dict(x = training_X))
table(clu = result, true = training_y)

   true
clu setosa versicolor virginica
  0     33          0         0
  2      0         28        39

testing¶

In [31]:

result <- sess$run(tf$argmax(en_layer5, 1L), feed_dict = dict(x = testing_X))
table(clu = result, true = testing_y)

   true
clu setosa versicolor virginica
  0     17          0         0
  2      0         22        11

이런 결과가 나온 것에 대해서 당연하게 의문점이 생긴다.

1. 정보적인 측면에서 setosa는 특이할만한 어떤 특징이 뚜렷하게 있는 것인데 반해 다른 것들은 뚜렷한 특징이 없는것일까?

2. training accuracy가 계속 줄어드는 추세이므로 학습을 더 해야 할까? 학습을 더 시키면 versicolor와 virginica도 뚜렷하게 구분해낼까?

In [ ]: