728x90
반응형
In [65]:
"""
- fashion_mnist 데이터를 이용해서 훈련 및 테스트에 대한 독립변수와 종속변수 읽어들이기
--> 변수는 항상 사용하는 변수명으로
- 정규화
"""
Out[65]:
'\n- fashion_mnist 데이터를 이용해서 훈련 및 테스트에 대한 독립변수와 종속변수 읽어들이기\n --> 변수는 항상 사용하는 변수명으로\n- 정규화\n\n'In [66]:
import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
In [67]:
"""실행 결과를 동일하게 하기 위한 처리(완전 동일하지 않을 수도 있음)"""
tf.keras.utils.set_random_seed(42)
"""연산 고정"""
tf.config.experimental.enable_op_determinism()
데이터 수집하기¶
In [68]:
"""keras에서 제공해주는 이미지 데이터셋 사용
- MNIST 패션 이미지
- 이미지 데이터는 픽셀 데이터로 이루어져 있습니다.
"""
(train_input, train_target), (test_input, test_target) = \
keras.datasets.fashion_mnist.load_data()
print(train_input.shape, train_target.shape)
print(test_input.shape, test_target.shape)
(60000, 28, 28) (60000,)
(10000, 28, 28) (10000,)
In [69]:
"""데이터 스케링링 수행하기"""
train_scaled_255 = train_input / 255.0
train_scaled_255[0]
test_scaled_255 = test_input / 255.0
test_scaled_255[0]
train_scaled_255.shape, test_scaled_255.shape
Out[69]:
((60000, 28, 28), (10000, 28, 28))2차원 데이터로 변환하기¶
In [70]:
"""모델 훈련에 사용하기 위해서는 2차원으로 변환"""
train_scaled_2d = train_scaled_255.reshape(-1, 28*28)
test_scaled_2d = test_scaled_255.reshape(-1, 28*28)
train_scaled_2d.shape,test_scaled_2d.shape
Out[70]:
((60000, 784), (10000, 784))훈련 및 검증 데이터로 분류하기(8:2)¶
In [71]:
train_scaled, val_scaled, train_target, val_target = train_test_split(train_scaled_2d,train_target, test_size=0.2, random_state=42 )
In [77]:
print(train_scaled.shape,train_target.shape),
print(val_scaled.shape,val_target.shape),
print(test_scaled_2d.shape,test_target.shape)
(48000, 784) (48000,)
(12000, 784) (12000,)
(10000, 784) (10000,)
신경망 모델에 계층(layer) 추가하는 방법 (3가지)¶
- 층을 먼저 만들고 , 신경망 모델 생성시 추가하기
In [79]:
"""입력 계층 (Input Layer) 생성하기"""
dense1 =keras.layers.Dense(100, activation="sigmoid", input_shape=(784,))
dense1
Out[79]:
<keras.layers.core.dense.Dense at 0x27ef48fd3d0>In [80]:
"""출력 계층(Output Layer) 생성하기"""
dense2 = keras.layers.Dense(10,activation="softmax")
dense2
Out[80]:
<keras.layers.core.dense.Dense at 0x27eeff29220>In [81]:
"""신경망 모델 생성하기"""
model = keras.Sequential([dense1, dense2])
model
Out[81]:
<keras.engine.sequential.Sequential at 0x27eefffad00>In [82]:
"""모델 계층 확인하기"""
model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 100) 78500
dense_2 (Dense) (None, 10) 1010
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________
2. 신경망 모델 생성시 계층(layer)을 함께 추가¶
In [83]:
""" 모델 생성 및 계층 추가하기"""
model = keras.Sequential([
keras.layers.Dense(100, activation="sigmoid", input_shape=(784,), name="Input-Layer"),
keras.layers.Dense(10, activation="softmax", name="Output-Layer"),
], name="Model-2")
model
Out[83]:
<keras.engine.sequential.Sequential at 0x27eeff1b370>In [84]:
model.summary()
Model: "Model-2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Input-Layer (Dense) (None, 100) 78500
Output-Layer (Dense) (None, 10) 1010
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________
3. 신경망 모델을 먼저 생성 후, 계층추가하기¶
In [94]:
"""
신경망 모델 생성하기
- 일반적으로 사용되는 방식
- 위의 1, 2 방법으로 수행 후, 계층을 추가할 필요성이 있을 경우에도 사용됨
"""
model = keras.Sequential()
model
Out[94]:
<keras.engine.sequential.Sequential at 0x27ef1736c10>In [95]:
"""계층 생성 및 모델에 추가하기"""
model.add(keras.layers.Dense(100, activation="sigmoid", input_shape=(784,), name= "Input-Layer"))
model.add(keras.layers.Dense(10, activation="softmax", name= "Output-Layer"))
model
# 출력계층을 제일 마지막에 넣어야 함
Out[95]:
<keras.engine.sequential.Sequential at 0x27ef1736c10>In [96]:
model.summary()
Model: "sequential_3"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Input-Layer (Dense) (None, 100) 78500
Output-Layer (Dense) (None, 10) 1010
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________
모델 설정하기(compile)¶
In [97]:
"""손실함수는 다중분류 사용"""
model.compile(loss="sparse_categorical_crossentropy",
metrics="accuracy")
모델 훈련하기(fit)¶
In [109]:
"""반복횟수10번"""
model.fit(train_scaled, train_target, epochs=100)
Epoch 1/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.2729 - accuracy: 0.9010
Epoch 2/100
1500/1500 [==============================] - 1s 794us/step - loss: 0.2666 - accuracy: 0.9028
Epoch 3/100
1500/1500 [==============================] - 1s 754us/step - loss: 0.2609 - accuracy: 0.9059
Epoch 4/100
1500/1500 [==============================] - 1s 755us/step - loss: 0.2546 - accuracy: 0.9076
Epoch 5/100
1500/1500 [==============================] - 1s 757us/step - loss: 0.2490 - accuracy: 0.9092
Epoch 6/100
1500/1500 [==============================] - 1s 753us/step - loss: 0.2447 - accuracy: 0.9116
Epoch 7/100
1500/1500 [==============================] - 1s 787us/step - loss: 0.2392 - accuracy: 0.9139
Epoch 8/100
1500/1500 [==============================] - 1s 761us/step - loss: 0.2347 - accuracy: 0.9151
Epoch 9/100
1500/1500 [==============================] - 1s 757us/step - loss: 0.2297 - accuracy: 0.9182
Epoch 10/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.2258 - accuracy: 0.9184
Epoch 11/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.2222 - accuracy: 0.9190
Epoch 12/100
1500/1500 [==============================] - 1s 768us/step - loss: 0.2180 - accuracy: 0.9218
Epoch 13/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.2147 - accuracy: 0.9222
Epoch 14/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.2099 - accuracy: 0.9246
Epoch 15/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.2065 - accuracy: 0.9254
Epoch 16/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.2038 - accuracy: 0.9268
Epoch 17/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1993 - accuracy: 0.9293
Epoch 18/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.1973 - accuracy: 0.9295
Epoch 19/100
1500/1500 [==============================] - 1s 779us/step - loss: 0.1945 - accuracy: 0.9305
Epoch 20/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.1909 - accuracy: 0.9328
Epoch 21/100
1500/1500 [==============================] - 1s 759us/step - loss: 0.1879 - accuracy: 0.9319
Epoch 22/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1838 - accuracy: 0.9342
Epoch 23/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.1821 - accuracy: 0.9344
Epoch 24/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.1776 - accuracy: 0.9370
Epoch 25/100
1500/1500 [==============================] - 1s 768us/step - loss: 0.1770 - accuracy: 0.9374
Epoch 26/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.1740 - accuracy: 0.9378
Epoch 27/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.1708 - accuracy: 0.9390
Epoch 28/100
1500/1500 [==============================] - 1s 774us/step - loss: 0.1670 - accuracy: 0.9410
Epoch 29/100
1500/1500 [==============================] - 1s 762us/step - loss: 0.1660 - accuracy: 0.9414
Epoch 30/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1629 - accuracy: 0.9419
Epoch 31/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1601 - accuracy: 0.9441
Epoch 32/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.1572 - accuracy: 0.9446
Epoch 33/100
1500/1500 [==============================] - 1s 761us/step - loss: 0.1543 - accuracy: 0.9446
Epoch 34/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.1529 - accuracy: 0.9457
Epoch 35/100
1500/1500 [==============================] - 1s 767us/step - loss: 0.1504 - accuracy: 0.9464
Epoch 36/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.1479 - accuracy: 0.9469
Epoch 37/100
1500/1500 [==============================] - 1s 773us/step - loss: 0.1457 - accuracy: 0.9482
Epoch 38/100
1500/1500 [==============================] - 1s 762us/step - loss: 0.1458 - accuracy: 0.9494
Epoch 39/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.1420 - accuracy: 0.9492
Epoch 40/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.1395 - accuracy: 0.9514
Epoch 41/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.1366 - accuracy: 0.9513
Epoch 42/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.1344 - accuracy: 0.9527
Epoch 43/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.1333 - accuracy: 0.9537
Epoch 44/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1315 - accuracy: 0.9544
Epoch 45/100
1500/1500 [==============================] - 1s 777us/step - loss: 0.1299 - accuracy: 0.9540
Epoch 46/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.1287 - accuracy: 0.9541
Epoch 47/100
1500/1500 [==============================] - 1s 762us/step - loss: 0.1260 - accuracy: 0.9561
Epoch 48/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.1238 - accuracy: 0.9574
Epoch 49/100
1500/1500 [==============================] - 1s 788us/step - loss: 0.1243 - accuracy: 0.9570
Epoch 50/100
1500/1500 [==============================] - 1s 788us/step - loss: 0.1208 - accuracy: 0.9578
Epoch 51/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.1196 - accuracy: 0.9575
Epoch 52/100
1500/1500 [==============================] - 1s 781us/step - loss: 0.1181 - accuracy: 0.9590
Epoch 53/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.1164 - accuracy: 0.9608
Epoch 54/100
1500/1500 [==============================] - 1s 778us/step - loss: 0.1134 - accuracy: 0.9606
Epoch 55/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.1118 - accuracy: 0.9624
Epoch 56/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.1115 - accuracy: 0.9611
Epoch 57/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.1113 - accuracy: 0.9615
Epoch 58/100
1500/1500 [==============================] - 1s 780us/step - loss: 0.1082 - accuracy: 0.9632
Epoch 59/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.1077 - accuracy: 0.9626
Epoch 60/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.1053 - accuracy: 0.9639
Epoch 61/100
1500/1500 [==============================] - 1s 762us/step - loss: 0.1029 - accuracy: 0.9653
Epoch 62/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.1031 - accuracy: 0.9646
Epoch 63/100
1500/1500 [==============================] - 1s 773us/step - loss: 0.1023 - accuracy: 0.9646
Epoch 64/100
1500/1500 [==============================] - 1s 760us/step - loss: 0.1007 - accuracy: 0.9658
Epoch 65/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0981 - accuracy: 0.9668
Epoch 66/100
1500/1500 [==============================] - 1s 767us/step - loss: 0.0966 - accuracy: 0.9666
Epoch 67/100
1500/1500 [==============================] - 1s 777us/step - loss: 0.0966 - accuracy: 0.9672
Epoch 68/100
1500/1500 [==============================] - 1s 762us/step - loss: 0.0951 - accuracy: 0.9676
Epoch 69/100
1500/1500 [==============================] - 1s 756us/step - loss: 0.0932 - accuracy: 0.9679
Epoch 70/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0916 - accuracy: 0.9694
Epoch 71/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.0907 - accuracy: 0.9685
Epoch 72/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.0910 - accuracy: 0.9691
Epoch 73/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0896 - accuracy: 0.9693
Epoch 74/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0871 - accuracy: 0.9710
Epoch 75/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.0862 - accuracy: 0.9705
Epoch 76/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0853 - accuracy: 0.9711
Epoch 77/100
1500/1500 [==============================] - 1s 774us/step - loss: 0.0839 - accuracy: 0.9716
Epoch 78/100
1500/1500 [==============================] - 1s 772us/step - loss: 0.0830 - accuracy: 0.9719
Epoch 79/100
1500/1500 [==============================] - 1s 777us/step - loss: 0.0828 - accuracy: 0.9720
Epoch 80/100
1500/1500 [==============================] - 1s 775us/step - loss: 0.0810 - accuracy: 0.9722
Epoch 81/100
1500/1500 [==============================] - 1s 771us/step - loss: 0.0797 - accuracy: 0.9725
Epoch 82/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.0798 - accuracy: 0.9734
Epoch 83/100
1500/1500 [==============================] - 1s 816us/step - loss: 0.0773 - accuracy: 0.9738
Epoch 84/100
1500/1500 [==============================] - 1s 821us/step - loss: 0.0764 - accuracy: 0.9740
Epoch 85/100
1500/1500 [==============================] - 1s 825us/step - loss: 0.0776 - accuracy: 0.9737
Epoch 86/100
1500/1500 [==============================] - 1s 785us/step - loss: 0.0761 - accuracy: 0.9736
Epoch 87/100
1500/1500 [==============================] - 1s 796us/step - loss: 0.0731 - accuracy: 0.9751
Epoch 88/100
1500/1500 [==============================] - 1s 793us/step - loss: 0.0742 - accuracy: 0.9751
Epoch 89/100
1500/1500 [==============================] - 1s 770us/step - loss: 0.0723 - accuracy: 0.9755
Epoch 90/100
1500/1500 [==============================] - 1s 768us/step - loss: 0.0706 - accuracy: 0.9756
Epoch 91/100
1500/1500 [==============================] - 1s 768us/step - loss: 0.0711 - accuracy: 0.9763
Epoch 92/100
1500/1500 [==============================] - 1s 779us/step - loss: 0.0706 - accuracy: 0.9764
Epoch 93/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.0692 - accuracy: 0.9770
Epoch 94/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.0682 - accuracy: 0.9772
Epoch 95/100
1500/1500 [==============================] - 1s 764us/step - loss: 0.0656 - accuracy: 0.9784
Epoch 96/100
1500/1500 [==============================] - 1s 766us/step - loss: 0.0654 - accuracy: 0.9776
Epoch 97/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.0654 - accuracy: 0.9775
Epoch 98/100
1500/1500 [==============================] - 1s 774us/step - loss: 0.0646 - accuracy: 0.9788
Epoch 99/100
1500/1500 [==============================] - 1s 801us/step - loss: 0.0626 - accuracy: 0.9789
Epoch 100/100
1500/1500 [==============================] - 1s 855us/step - loss: 0.0635 - accuracy: 0.9786
Out[109]:
<keras.callbacks.History at 0x27ef21c2130>In [108]:
"""성능 평가하기(검증)"""
score = model.evaluate(val_scaled, val_target)
"""손실율과 정확도 출력하기"""
print(f"손실율: {score[0]} / 정확도: {score[1]}")
375/375 [==============================] - 0s 567us/step - loss: 0.3315 - accuracy: 0.8832
손실율: 0.3314509689807892 / 정확도: 0.8831666707992554
In [111]:
"""성능 평가하기(검증)"""
score = model.evaluate(val_scaled, val_target)
"""손실율과 정확도 출력하기"""
print(f"손실율: {score[0]} / 정확도: {score[1]}")
375/375 [==============================] - 0s 517us/step - loss: 0.6101 - accuracy: 0.8826
손실율: 0.6101072430610657 / 정확도: 0.8825833201408386
In [ ]:
성능 향상 방법¶
In [ ]:
"""
<성능 향상 방법>
1. 데이터 증가 시키기
2. 하이퍼파라메터 튜닝
-> 반복횟수 증가
-> 계층 추가 또는 제거(일반적으로 추가)
-> 이외 하이퍼파라메터들...
"""
In [112]:
##### 성능향상 - 은닉계층(Hidden Layer)추가
In [137]:
"""입력계층 추가하기
- 전처리 계층으로 추가
"""
"""
- Flatten() 전처리계층: 차원축소 전처리 계층(1차원으로 축소)
: 훈련에 영향을 미치지는 않음
: 일반적으로 입력계층 다음에 추가하거나, 입력계층으로 사용되기도 함
: 이미지 데이터 처리시에 주로 사용됨
"""
model = keras.Sequential()
model.add(
keras.layers.Flatten(input_shape=(28,28))
)
In [138]:
""" 중간계층 = 은닉 계층 (hidden layer) 생성하기
-Dense() 계층은 모델 성능에 영향을 미침
"""
model.add(keras.layers.Dense(100,activation="relu"))
In [139]:
"""출력 계층 (output-layer) 생성하기"""
model.add(keras.layers.Dense(10,activation="softmax"))
In [143]:
"""모델에 추가된 계층 모두 확인하기"""
model.summary()
Model: "sequential_5"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten_7 (Flatten) (None, 784) 0
dense_9 (Dense) (None, 100) 78500
dense_10 (Dense) (None, 10) 1010
=================================================================
Total params: 79,510
Trainable params: 79,510
Non-trainable params: 0
_________________________________________________________________
모델 설정하기(compile)¶
In [156]:
model.compile(loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [157]:
"""keras에서 제공해주는 이미지 데이터셋 사용
- MNIST 패션 이미지
- 이미지 데이터는 픽셀 데이터로 이루어져 있습니다.
"""
(train_input, train_target), (test_input, test_target) = \
keras.datasets.fashion_mnist.load_data()
print(train_input.shape, train_target.shape)
print(test_input.shape, test_target.shape)
(60000, 28, 28) (60000,)
(10000, 28, 28) (10000,)
In [158]:
"""데이터 스케링링 수행하기"""
train_scaled_255 = train_input / 255.0
train_scaled_255[0]
test_scaled_255 = test_input / 255.0
test_scaled_255[0]
train_scaled_255.shape, test_scaled_255.shape
Out[158]:
((60000, 28, 28), (10000, 28, 28))In [159]:
train_scaled, val_scaled, train_target, val_target = train_test_split(train_scaled_255,train_target, test_size=0.2, random_state=42 )
In [160]:
print(train_scaled.shape,train_target.shape),
print(val_scaled.shape,val_target.shape),
print(test_scaled_255.shape,test_target.shape)
(48000, 28, 28) (48000,)
(12000, 28, 28) (12000,)
(10000, 28, 28) (10000,)
모델 훈련시키기¶
In [161]:
model.fit(train_scaled, train_target, epochs=100)
Epoch 1/100
1500/1500 [==============================] - 1s 742us/step - loss: 0.1651 - accuracy: 0.9485
Epoch 2/100
1500/1500 [==============================] - 1s 761us/step - loss: 0.1602 - accuracy: 0.9488
Epoch 3/100
1500/1500 [==============================] - 1s 738us/step - loss: 0.1556 - accuracy: 0.9495
Epoch 4/100
1500/1500 [==============================] - 1s 747us/step - loss: 0.1584 - accuracy: 0.9502
Epoch 5/100
1500/1500 [==============================] - 1s 758us/step - loss: 0.1543 - accuracy: 0.9509
Epoch 6/100
1500/1500 [==============================] - 1s 757us/step - loss: 0.1526 - accuracy: 0.9519
Epoch 7/100
1500/1500 [==============================] - 1s 749us/step - loss: 0.1514 - accuracy: 0.9524
Epoch 8/100
1500/1500 [==============================] - 1s 723us/step - loss: 0.1527 - accuracy: 0.9524
Epoch 9/100
1500/1500 [==============================] - 1s 752us/step - loss: 0.1497 - accuracy: 0.9534
Epoch 10/100
1500/1500 [==============================] - 1s 743us/step - loss: 0.1477 - accuracy: 0.9530
Epoch 11/100
1500/1500 [==============================] - 1s 737us/step - loss: 0.1468 - accuracy: 0.9540
Epoch 12/100
1500/1500 [==============================] - 1s 756us/step - loss: 0.1446 - accuracy: 0.9545
Epoch 13/100
1500/1500 [==============================] - 1s 726us/step - loss: 0.1417 - accuracy: 0.9545
Epoch 14/100
1500/1500 [==============================] - 1s 748us/step - loss: 0.1443 - accuracy: 0.9546
Epoch 15/100
1500/1500 [==============================] - 1s 729us/step - loss: 0.1379 - accuracy: 0.9577
Epoch 16/100
1500/1500 [==============================] - 1s 788us/step - loss: 0.1431 - accuracy: 0.9554
Epoch 17/100
1500/1500 [==============================] - 1s 784us/step - loss: 0.1406 - accuracy: 0.9567
Epoch 18/100
1500/1500 [==============================] - 1s 755us/step - loss: 0.1367 - accuracy: 0.9580
Epoch 19/100
1500/1500 [==============================] - 1s 749us/step - loss: 0.1395 - accuracy: 0.9574
Epoch 20/100
1500/1500 [==============================] - 1s 801us/step - loss: 0.1402 - accuracy: 0.9586
Epoch 21/100
1500/1500 [==============================] - 1s 740us/step - loss: 0.1372 - accuracy: 0.9580
Epoch 22/100
1500/1500 [==============================] - 1s 755us/step - loss: 0.1317 - accuracy: 0.9590
Epoch 23/100
1500/1500 [==============================] - 1s 751us/step - loss: 0.1321 - accuracy: 0.9595
Epoch 24/100
1500/1500 [==============================] - 1s 748us/step - loss: 0.1291 - accuracy: 0.9600
Epoch 25/100
1500/1500 [==============================] - 1s 740us/step - loss: 0.1295 - accuracy: 0.9608
Epoch 26/100
1500/1500 [==============================] - 1s 773us/step - loss: 0.1278 - accuracy: 0.9600
Epoch 27/100
1500/1500 [==============================] - 1s 743us/step - loss: 0.1320 - accuracy: 0.9611
Epoch 28/100
1500/1500 [==============================] - 1s 789us/step - loss: 0.1266 - accuracy: 0.9610
Epoch 29/100
1500/1500 [==============================] - 1s 742us/step - loss: 0.1297 - accuracy: 0.9611
Epoch 30/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.1243 - accuracy: 0.9607
Epoch 31/100
1500/1500 [==============================] - 1s 735us/step - loss: 0.1267 - accuracy: 0.9625
Epoch 32/100
1500/1500 [==============================] - 1s 773us/step - loss: 0.1256 - accuracy: 0.9625
Epoch 33/100
1500/1500 [==============================] - 1s 818us/step - loss: 0.1230 - accuracy: 0.9632
Epoch 34/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.1212 - accuracy: 0.9622
Epoch 35/100
1500/1500 [==============================] - 1s 771us/step - loss: 0.1210 - accuracy: 0.9646
Epoch 36/100
1500/1500 [==============================] - 1s 731us/step - loss: 0.1192 - accuracy: 0.9644
Epoch 37/100
1500/1500 [==============================] - 1s 731us/step - loss: 0.1159 - accuracy: 0.9654
Epoch 38/100
1500/1500 [==============================] - 1s 729us/step - loss: 0.1187 - accuracy: 0.9647
Epoch 39/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.1178 - accuracy: 0.9650
Epoch 40/100
1500/1500 [==============================] - 1s 729us/step - loss: 0.1160 - accuracy: 0.9656
Epoch 41/100
1500/1500 [==============================] - 1s 769us/step - loss: 0.1174 - accuracy: 0.9655
Epoch 42/100
1500/1500 [==============================] - 1s 812us/step - loss: 0.1149 - accuracy: 0.9654
Epoch 43/100
1500/1500 [==============================] - 1s 727us/step - loss: 0.1112 - accuracy: 0.9669
Epoch 44/100
1500/1500 [==============================] - 1s 729us/step - loss: 0.1126 - accuracy: 0.9665
Epoch 45/100
1500/1500 [==============================] - 1s 725us/step - loss: 0.1088 - accuracy: 0.9674
Epoch 46/100
1500/1500 [==============================] - 1s 745us/step - loss: 0.1092 - accuracy: 0.9672
Epoch 47/100
1500/1500 [==============================] - 1s 721us/step - loss: 0.1096 - accuracy: 0.9678
Epoch 48/100
1500/1500 [==============================] - 1s 724us/step - loss: 0.1085 - accuracy: 0.9679
Epoch 49/100
1500/1500 [==============================] - 1s 724us/step - loss: 0.1079 - accuracy: 0.9679
Epoch 50/100
1500/1500 [==============================] - 1s 740us/step - loss: 0.1064 - accuracy: 0.9678
Epoch 51/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.1089 - accuracy: 0.9683
Epoch 52/100
1500/1500 [==============================] - 1s 755us/step - loss: 0.1090 - accuracy: 0.9678
Epoch 53/100
1500/1500 [==============================] - 1s 750us/step - loss: 0.1078 - accuracy: 0.9689
Epoch 54/100
1500/1500 [==============================] - 1s 758us/step - loss: 0.1054 - accuracy: 0.9695
Epoch 55/100
1500/1500 [==============================] - 1s 776us/step - loss: 0.1032 - accuracy: 0.9694
Epoch 56/100
1500/1500 [==============================] - 1s 847us/step - loss: 0.1072 - accuracy: 0.9683
Epoch 57/100
1500/1500 [==============================] - 1s 750us/step - loss: 0.1088 - accuracy: 0.9694
Epoch 58/100
1500/1500 [==============================] - 1s 741us/step - loss: 0.1037 - accuracy: 0.9699
Epoch 59/100
1500/1500 [==============================] - 1s 745us/step - loss: 0.1030 - accuracy: 0.9707
Epoch 60/100
1500/1500 [==============================] - 1s 740us/step - loss: 0.1061 - accuracy: 0.9701
Epoch 61/100
1500/1500 [==============================] - 1s 775us/step - loss: 0.0995 - accuracy: 0.9719
Epoch 62/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.1024 - accuracy: 0.9714
Epoch 63/100
1500/1500 [==============================] - 1s 846us/step - loss: 0.0984 - accuracy: 0.9721
Epoch 64/100
1500/1500 [==============================] - 1s 854us/step - loss: 0.0980 - accuracy: 0.9711
Epoch 65/100
1500/1500 [==============================] - 1s 796us/step - loss: 0.1002 - accuracy: 0.9707
Epoch 66/100
1500/1500 [==============================] - 1s 763us/step - loss: 0.0966 - accuracy: 0.9722
Epoch 67/100
1500/1500 [==============================] - 1s 731us/step - loss: 0.1001 - accuracy: 0.9723
Epoch 68/100
1500/1500 [==============================] - 1s 733us/step - loss: 0.0964 - accuracy: 0.9722
Epoch 69/100
1500/1500 [==============================] - 1s 732us/step - loss: 0.0973 - accuracy: 0.9717
Epoch 70/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.0946 - accuracy: 0.9717
Epoch 71/100
1500/1500 [==============================] - 1s 733us/step - loss: 0.0946 - accuracy: 0.9725
Epoch 72/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.0953 - accuracy: 0.9733
Epoch 73/100
1500/1500 [==============================] - 1s 731us/step - loss: 0.0945 - accuracy: 0.9734
Epoch 74/100
1500/1500 [==============================] - 1s 765us/step - loss: 0.0886 - accuracy: 0.9741
Epoch 75/100
1500/1500 [==============================] - 1s 743us/step - loss: 0.0930 - accuracy: 0.9737
Epoch 76/100
1500/1500 [==============================] - 1s 740us/step - loss: 0.0940 - accuracy: 0.9741
Epoch 77/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.0961 - accuracy: 0.9728
Epoch 78/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.0923 - accuracy: 0.9738
Epoch 79/100
1500/1500 [==============================] - 1s 738us/step - loss: 0.0890 - accuracy: 0.9751
Epoch 80/100
1500/1500 [==============================] - 1s 735us/step - loss: 0.0877 - accuracy: 0.9750
Epoch 81/100
1500/1500 [==============================] - 1s 739us/step - loss: 0.0903 - accuracy: 0.9735
Epoch 82/100
1500/1500 [==============================] - 1s 744us/step - loss: 0.0874 - accuracy: 0.9754
Epoch 83/100
1500/1500 [==============================] - 1s 751us/step - loss: 0.0896 - accuracy: 0.9745
Epoch 84/100
1500/1500 [==============================] - 1s 732us/step - loss: 0.0876 - accuracy: 0.9749
Epoch 85/100
1500/1500 [==============================] - 1s 750us/step - loss: 0.0903 - accuracy: 0.9749
Epoch 86/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.0879 - accuracy: 0.9757
Epoch 87/100
1500/1500 [==============================] - 1s 737us/step - loss: 0.0898 - accuracy: 0.9754
Epoch 88/100
1500/1500 [==============================] - 1s 737us/step - loss: 0.0904 - accuracy: 0.9750
Epoch 89/100
1500/1500 [==============================] - 1s 737us/step - loss: 0.0858 - accuracy: 0.9756
Epoch 90/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.0847 - accuracy: 0.9768
Epoch 91/100
1500/1500 [==============================] - 1s 734us/step - loss: 0.0884 - accuracy: 0.9762
Epoch 92/100
1500/1500 [==============================] - 1s 732us/step - loss: 0.0880 - accuracy: 0.9762
Epoch 93/100
1500/1500 [==============================] - 1s 748us/step - loss: 0.0851 - accuracy: 0.9757
Epoch 94/100
1500/1500 [==============================] - 1s 746us/step - loss: 0.0832 - accuracy: 0.9767
Epoch 95/100
1500/1500 [==============================] - 1s 735us/step - loss: 0.0794 - accuracy: 0.9775
Epoch 96/100
1500/1500 [==============================] - 1s 741us/step - loss: 0.0823 - accuracy: 0.9770
Epoch 97/100
1500/1500 [==============================] - 1s 739us/step - loss: 0.0842 - accuracy: 0.9769
Epoch 98/100
1500/1500 [==============================] - 1s 741us/step - loss: 0.0819 - accuracy: 0.9775
Epoch 99/100
1500/1500 [==============================] - 1s 736us/step - loss: 0.0807 - accuracy: 0.9785
Epoch 100/100
1500/1500 [==============================] - 1s 739us/step - loss: 0.0840 - accuracy: 0.9778
Out[161]:
<keras.callbacks.History at 0x27ef25d8d60>성능평가 - 검증하기¶
In [162]:
score = model.evaluate(val_scaled,val_target)
score
375/375 [==============================] - 0s 568us/step - loss: 1.7030 - accuracy: 0.8723
Out[162]:
[1.702976942062378, 0.8723333477973938]성능향상 - 옵티마이저(Optimizer)¶
In [164]:
"""
-옵티마이저 설정 위치: compile() 시에 설정함
-손실을 줄여나가기 위한 방법을 설정함
-손실을 줄여나가는 방법을 보통" 경사하강법"이라고 칭합니다.
-"경사하강법"을 이용한 여러가지 방법들 중 하나를 선택하는 것이 옵티마이저 선택입니다.
-옵티마이저 종류 : SGD(확률적 경사하강법), Adagrad, RESProp, Adam 이 있음
* SGD (확률적 경사하강법)
- 현재 위치에서 기울어진 방향을 찾을 때
-> 지그재그 모양으로 탐색해 나가는 방법
* Adagrad
-학습율을 적절하게 설정하기 위해 학습률 감소라는 기술을 사용
-학습 진행 중에 학습률을 줄여가는 방법을 사용
- 처음에는 학습률을 크게 학습하다가 점점 작게 학습한다는 의미
* RMSProp
- Adagrad의 단점을 보완한 방법
- Adagrad는 학습량을 점점 작게 학습하기 때문에 학습량이 0이 되어 전형 갱신되지 않는 (학습되지 않는) 시점이 발생할 수 있는 단점이 있음
- 이러한 단점을 보완하여 과거의 기울기 값을 반영하는 방식 사용
- 먼 과거의 기울기(경사) 값은 조금 반영하고, 최근 기울기 (경사)를 많이 반영
- Optimizer의 기본값 (default)로 사용됨
* Adam
- 공이 굴러가듯이 모멘텀(momentum - > 관성)과 Adagrad를 융합한 방법
- 자주 사용되는 기법으로, 좋은 결과를 얻을 수 있는 방법으로 유명함
*** 모멘텀: 관성과 가속도를 적용하여 이동하던 방향으로 좀 더 유연하게 작동함
- 메모리 사용이 많은 단점으 있음 ( 과거 데이터를 저장해 놓음)
"""
model.compile(
# 옵티마이저 정의 : 손실을 줄여나가는 방법
optimizer = "sgd",
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [166]:
model.fit(train_scaled, train_target, epochs=10)
Epoch 1/10
1500/1500 [==============================] - 1s 624us/step - loss: 0.0232 - accuracy: 0.9930
Epoch 2/10
1500/1500 [==============================] - 1s 625us/step - loss: 0.0216 - accuracy: 0.9935
Epoch 3/10
1500/1500 [==============================] - 1s 637us/step - loss: 0.0208 - accuracy: 0.9937
Epoch 4/10
1500/1500 [==============================] - 1s 628us/step - loss: 0.0203 - accuracy: 0.9940
Epoch 5/10
1500/1500 [==============================] - 1s 639us/step - loss: 0.0196 - accuracy: 0.9942
Epoch 6/10
1500/1500 [==============================] - 1s 627us/step - loss: 0.0192 - accuracy: 0.9944
Epoch 7/10
1500/1500 [==============================] - 1s 634us/step - loss: 0.0190 - accuracy: 0.9945
Epoch 8/10
1500/1500 [==============================] - 1s 627us/step - loss: 0.0181 - accuracy: 0.9945
Epoch 9/10
1500/1500 [==============================] - 1s 628us/step - loss: 0.0182 - accuracy: 0.9946
Epoch 10/10
1500/1500 [==============================] - 1s 628us/step - loss: 0.0181 - accuracy: 0.9947
Out[166]:
<keras.callbacks.History at 0x27ef24970d0>옵티마이저에 학습률 적용하기¶
In [167]:
"""
* 학습률을 적용하는 방법
-사용되는 4개의 옵티마이저를 객체로 생성하여learning_rate(학습률) 값을 설정할 수 있음
-학습률 : 보폭이라고 생각하시면 됩니다.
-학습률이 작을 수록 보폭이 작다고 보시면 됩니다.
-가장 손실이 작은 위치를 찾아서 움직이게 됩니다.
-이때 가장 손실이 작은 위치는 모델이 스스로 찾아서 움직이게 됩니다. (사람 관여하지 않음)
-학습률의 기본값은 = 0.01을 사용( 사용값의 범위 0.1~ 0.0001 정도)
* 과적합을 해소하기 위한 튜닝 방법으로 사용됨
-과대적합이 일어난 경우: 학습률을 크게
-과소적합이 일어난 경우: 학습률을 작게
-과대/과소를 떠나서, 직접 값의 범위를 적용하여 튜닝을 수행한 후 가장 일반화 시점의 학습률 값을 찾는 것이 중요함
"""
Out[167]:
'\n * 학습률을 적용하는 방법\n -사용되는 4개의 옵티마이저를 객체로 생성하여learning_rate(학습률) 값을 설정할 수 있음\n -학습률 : 보폭이라고 생각하시면 됩니다.\n -학습률이 작을 수록 보폭이 작다고 보시면 됩니다.\n -가장 손실이 작은 위치를 찾아서 움직이게 됩니다.\n -이때 가장 손실이 작은 위치는 모델이 스스로 찾아서 움직이게 됩니다. (사람 관여하지 않음)\n -학습률의 기본값은 = 0.01을 사용( 사용값의 범위 0.1~ 0.0001 정도)\n\n * 과적합을 해소하기 위한 튜닝 방법으로 사용됨\n -과대적합이 일어난 경우: 학습률을 크게\n -과소적합이 일어난 경우: 학습률을 작게\n -과대/과소를 떠나서, 직접 값의 범위를 적용하여 튜닝을 수행한 후 가장 일반화 시점의 학습률 값을 찾는 것이 중요함\n'옵티마이저에 학습률 적용하기¶
In [168]:
# 옵티마이저 객체 생성
sgd = keras.optimizers.SGD(learning_rate=0.1)
""" 모델설정 (compile)"""
model.compile(
# 옵티마이저 정의 : 손실을 줄여나가는 방법
optimizer = sgd,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [169]:
model.fit(train_scaled, train_target, epochs=10)
Epoch 1/10
1500/1500 [==============================] - 1s 630us/step - loss: 0.3956 - accuracy: 0.9232
Epoch 2/10
1500/1500 [==============================] - 1s 618us/step - loss: 0.2920 - accuracy: 0.9337
Epoch 3/10
1500/1500 [==============================] - 1s 631us/step - loss: 0.2427 - accuracy: 0.9402
Epoch 4/10
1500/1500 [==============================] - 1s 627us/step - loss: 0.1948 - accuracy: 0.9492
Epoch 5/10
1500/1500 [==============================] - 1s 629us/step - loss: 0.1724 - accuracy: 0.9536
Epoch 6/10
1500/1500 [==============================] - 1s 630us/step - loss: 0.1532 - accuracy: 0.9575
Epoch 7/10
1500/1500 [==============================] - 1s 627us/step - loss: 0.1359 - accuracy: 0.9605
Epoch 8/10
1500/1500 [==============================] - 1s 634us/step - loss: 0.1258 - accuracy: 0.9621
Epoch 9/10
1500/1500 [==============================] - 1s 626us/step - loss: 0.1187 - accuracy: 0.9639
Epoch 10/10
1500/1500 [==============================] - 1s 624us/step - loss: 0.1092 - accuracy: 0.9663
Out[169]:
<keras.callbacks.History at 0x27ef2165280>In [171]:
score= model.evaluate(val_scaled,val_target)
score
375/375 [==============================] - 0s 530us/step - loss: 1.4089 - accuracy: 0.8762
Out[171]:
[1.4089133739471436, 0.8761666417121887]모멘텀(Momentum) 직접 적용하기¶
In [173]:
"""<모멘텀(Momentum)
-과거의 방향(기울기)를 적용하여 -> 관성을 적용시키는 방법
-기본적으로 0.9이상의 값을 적용시킴
-보통 nesterov=True 속성과 함께 사용됨
-> nesterov=True : 모멘텀 방향보다 조금 더 앞서서 경사를 계산하는 방식(미리 체크)
-momentum 속성을 사용할 수 있는 옵티마이저: SGD, RMSProp
"""
sgd = keras.optimizers.SGD(momentum=0.9, nesterov=True, learning_rate=0.1)
sgd
Out[173]:
<keras.optimizer_v2.gradient_descent.SGD at 0x27ef24c3760>In [175]:
model.compile(optimizer=sgd,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [176]:
model.fit(train_scaled, train_target, epochs=10)
Epoch 1/10
1500/1500 [==============================] - 1s 673us/step - loss: 0.9886 - accuracy: 0.6841
Epoch 2/10
1500/1500 [==============================] - 1s 661us/step - loss: 0.6999 - accuracy: 0.7613
Epoch 3/10
1500/1500 [==============================] - 1s 675us/step - loss: 0.6567 - accuracy: 0.7760
Epoch 4/10
1500/1500 [==============================] - 1s 678us/step - loss: 0.6140 - accuracy: 0.7963
Epoch 5/10
1500/1500 [==============================] - 1s 688us/step - loss: 0.6515 - accuracy: 0.7754
Epoch 6/10
1500/1500 [==============================] - 1s 684us/step - loss: 0.6126 - accuracy: 0.7909
Epoch 7/10
1500/1500 [==============================] - 1s 673us/step - loss: 0.5544 - accuracy: 0.8111
Epoch 8/10
1500/1500 [==============================] - 1s 684us/step - loss: 0.5406 - accuracy: 0.8170
Epoch 9/10
1500/1500 [==============================] - 1s 683us/step - loss: 0.5324 - accuracy: 0.8213
Epoch 10/10
1500/1500 [==============================] - 1s 686us/step - loss: 0.5147 - accuracy: 0.8228
Out[176]:
<keras.callbacks.History at 0x27ef24b3940>In [177]:
score= model.evaluate(val_scaled,val_target)
score
375/375 [==============================] - 0s 531us/step - loss: 0.6814 - accuracy: 0.8171
Out[177]:
[0.6814032196998596, 0.8170833587646484]In [179]:
""" adagrad 아다그래드"""
adagrad= keras.optimizers.Adagrad()
model.compile(optimizer=adagrad,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
"""또는"""
model.compile(optimizer="adagrad",
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [180]:
"""RMSProp"""
rmsprop= keras.optimizers.RMSprop()
model.compile(optimizer=rmsprop,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
"""또는"""
model.compile(optimizer="rmsprop",
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [181]:
"""Adam"""
adam= keras.optimizers.Adam()
model.compile(optimizer=adam,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
"""또는"""
model.compile(optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics="accuracy")
In [182]:
"""
1.신경망 모델 생성
2. 계층 추가하기
- 1차원 전처리 계층 추가
- 은닉계층 추가, 활성화 함수 relu 사용, 출력크기 100개
- 최종 출력계층 추가
3. 모델설정하기
- 옵티마이저는 adam 사용, 학습률 0.1 사용
4. 훈련시키기
5. 성능평가하기
"""
Out[182]:
'\n1.신경망 모델 생성\n2. 계층 추가하기\n - 1차원 전처리 계층 추가\n - 은닉계층 추가, 활성화 함수 relu 사용, 출력크기 100개\n - 최종 출력계층 추가\n3. 모델설정하기\n - 옵티마이저는 adam 사용, 학습률 0.1 사용\n4. 훈련시키기\n5. 성능평가하기\n\n\n\n'In [216]:
model= keras.Sequential()
model.add(keras.layers.Flatten(input_shape=(28,28)))
model.add(keras.layers.Dense(units=100, activation="relu" ))
model.add(keras.layers.Dense(units=10, activation="softmax" ))
In [217]:
model
Out[217]:
<keras.engine.sequential.Sequential at 0x27ef29df3a0>In [218]:
adam= keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=adam, loss="sparse_categorical_crossentropy", metrics="accuracy")
In [ ]:
model.fit(train_scaled, train_target, epochs=50)
In [220]:
score= model.evaluate(val_scaled,val_target)
score
375/375 [==============================] - 0s 524us/step - loss: 0.5367 - accuracy: 0.8466
Out[220]:
[0.536652147769928, 0.8465833067893982]In [221]:
"""
<실습>
-옵티마이저에 적용할 학습기법(sgd, adagrad, rmsprop, adam) 중에
-가장 좋은 성능을 나타내는 옵티마이저 학습기법 확인하기
-훈련횟수 : 50회
-학습률은 기본값 사용
-은닉계층 relu 사용
-성능 평가결과가 가장 높을 때의 학습기법과 손실율, 정확도를 출력해주세요.
"""
Out[221]:
'\n<실습>\n-옵티마이저에 적용할 학습기법(sgd, adagard, rmsprop, adam) 중에\n-가장 좋은 성능을 나타내는 옵티마이저 학습기법 확인하기\n-훈련횟수 : 50회\n-학습률은 기본값 사용\n-은닉계층 relu 사용\n\n-성능 평가결과가 가장 높을 때의 학습기법과 손실율, 정확도를 출력해주세요.\n'In [ ]:
adf= {"2":1}
list(adf.values())[0]
list(adf.keys())[0]
list(adf.items())[0]
In [265]:
optimizer_list= ["sgd","adagrad","rmsprop","adam"]
best_loss={"null":1}
best_accuracy={"null":0}
for optimizer in optimizer_list:
model = keras.Sequential()
model.add(keras.layers.Flatten(input_shape=(28,28)))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(10, activation="softmax"))
model.compile(optimizer=optimizer, loss="sparse_categorical_crossentropy", metrics="accuracy")
model.fit(train_scaled, train_target, epochs=50)
result = model.evaluate(val_scaled,val_target)
if result[1]>list(best_accuracy.values())[0]:
best_accuracy={optimizer:result[1]}
if result[0]<list(best_loss.values())[0]:
best_loss={optimizer:result[0]}
print(f"손실율이 가장 낮은 학습 방법 {list(best_loss.keys())[0]} : {list(best_loss.values())[0]} \n
정확도가 가장 높은 학습 방법 {list(best_accuracy.keys())[0]} : {list(best_accuracy.values())[0]}")
Epoch 1/50
1500/1500 [==============================] - 1s 665us/step - loss: 0.8081 - accuracy: 0.7393
Epoch 2/50
1500/1500 [==============================] - 1s 642us/step - loss: 0.5394 - accuracy: 0.8168
Epoch 3/50
1500/1500 [==============================] - 1s 682us/step - loss: 0.4898 - accuracy: 0.8325
Epoch 4/50
1500/1500 [==============================] - 1s 642us/step - loss: 0.4617 - accuracy: 0.8403
Epoch 5/50
1500/1500 [==============================] - 1s 625us/step - loss: 0.4434 - accuracy: 0.8481
Epoch 6/50
1500/1500 [==============================] - 1s 625us/step - loss: 0.4311 - accuracy: 0.8519
Epoch 7/50
1500/1500 [==============================] - 1s 642us/step - loss: 0.4189 - accuracy: 0.8560
Epoch 8/50
1500/1500 [==============================] - 1s 623us/step - loss: 0.4097 - accuracy: 0.8585
Epoch 9/50
1500/1500 [==============================] - 1s 625us/step - loss: 0.3996 - accuracy: 0.8618
Epoch 10/50
1500/1500 [==============================] - 1s 627us/step - loss: 0.3928 - accuracy: 0.8644
Epoch 11/50
1500/1500 [==============================] - 1s 677us/step - loss: 0.3860 - accuracy: 0.8671
Epoch 12/50
1500/1500 [==============================] - 1s 648us/step - loss: 0.3794 - accuracy: 0.8686
Epoch 13/50
1500/1500 [==============================] - 1s 630us/step - loss: 0.3733 - accuracy: 0.8704
Epoch 14/50
1500/1500 [==============================] - 1s 632us/step - loss: 0.3678 - accuracy: 0.8724
Epoch 15/50
1500/1500 [==============================] - 1s 631us/step - loss: 0.3628 - accuracy: 0.8738
Epoch 16/50
1500/1500 [==============================] - 1s 627us/step - loss: 0.3578 - accuracy: 0.8757
Epoch 17/50
1500/1500 [==============================] - 1s 625us/step - loss: 0.3527 - accuracy: 0.8761
Epoch 18/50
1500/1500 [==============================] - 1s 658us/step - loss: 0.3486 - accuracy: 0.8779
Epoch 19/50
1500/1500 [==============================] - 1s 640us/step - loss: 0.3438 - accuracy: 0.8800
Epoch 20/50
1500/1500 [==============================] - 1s 638us/step - loss: 0.3410 - accuracy: 0.8795
Epoch 21/50
1500/1500 [==============================] - 1s 649us/step - loss: 0.3375 - accuracy: 0.8802
Epoch 22/50
1500/1500 [==============================] - 1s 655us/step - loss: 0.3326 - accuracy: 0.8820
Epoch 23/50
1500/1500 [==============================] - 1s 634us/step - loss: 0.3286 - accuracy: 0.8842
Epoch 24/50
1500/1500 [==============================] - 1s 629us/step - loss: 0.3254 - accuracy: 0.8854
Epoch 25/50
1500/1500 [==============================] - 1s 631us/step - loss: 0.3226 - accuracy: 0.8861
Epoch 26/50
1500/1500 [==============================] - 1s 661us/step - loss: 0.3194 - accuracy: 0.8863
Epoch 27/50
1500/1500 [==============================] - 1s 627us/step - loss: 0.3163 - accuracy: 0.8894
Epoch 28/50
1500/1500 [==============================] - 1s 642us/step - loss: 0.3129 - accuracy: 0.8892
Epoch 29/50
1500/1500 [==============================] - 1s 629us/step - loss: 0.3103 - accuracy: 0.8910
Epoch 30/50
1500/1500 [==============================] - 1s 638us/step - loss: 0.3083 - accuracy: 0.8913
Epoch 31/50
1500/1500 [==============================] - 1s 634us/step - loss: 0.3041 - accuracy: 0.8923
Epoch 32/50
1500/1500 [==============================] - 1s 625us/step - loss: 0.3015 - accuracy: 0.8944
Epoch 33/50
1500/1500 [==============================] - 1s 670us/step - loss: 0.2987 - accuracy: 0.8935
Epoch 34/50
1500/1500 [==============================] - 1s 644us/step - loss: 0.2964 - accuracy: 0.8957
Epoch 35/50
1500/1500 [==============================] - 1s 630us/step - loss: 0.2942 - accuracy: 0.8962
Epoch 36/50
1500/1500 [==============================] - 1s 633us/step - loss: 0.2915 - accuracy: 0.8973
Epoch 37/50
1500/1500 [==============================] - 1s 674us/step - loss: 0.2893 - accuracy: 0.8977
Epoch 38/50
1500/1500 [==============================] - 1s 668us/step - loss: 0.2868 - accuracy: 0.8977
Epoch 39/50
1500/1500 [==============================] - 1s 685us/step - loss: 0.2841 - accuracy: 0.9000
Epoch 40/50
1500/1500 [==============================] - 1s 638us/step - loss: 0.2823 - accuracy: 0.9002
Epoch 41/50
1500/1500 [==============================] - 1s 648us/step - loss: 0.2794 - accuracy: 0.9009
Epoch 42/50
1500/1500 [==============================] - 1s 632us/step - loss: 0.2764 - accuracy: 0.9021
Epoch 43/50
1500/1500 [==============================] - 1s 653us/step - loss: 0.2755 - accuracy: 0.9025
Epoch 44/50
1500/1500 [==============================] - 1s 634us/step - loss: 0.2730 - accuracy: 0.9040
Epoch 45/50
1500/1500 [==============================] - 1s 633us/step - loss: 0.2713 - accuracy: 0.9032
Epoch 46/50
1500/1500 [==============================] - 1s 633us/step - loss: 0.2699 - accuracy: 0.9044
Epoch 47/50
1500/1500 [==============================] - 1s 629us/step - loss: 0.2670 - accuracy: 0.9057
Epoch 48/50
1500/1500 [==============================] - 1s 632us/step - loss: 0.2650 - accuracy: 0.9062
Epoch 49/50
1500/1500 [==============================] - 1s 641us/step - loss: 0.2634 - accuracy: 0.9078
Epoch 50/50
1500/1500 [==============================] - 1s 639us/step - loss: 0.2606 - accuracy: 0.9091
375/375 [==============================] - 0s 557us/step - loss: 0.3375 - accuracy: 0.8832
Epoch 1/50
1500/1500 [==============================] - 1s 674us/step - loss: 1.1212 - accuracy: 0.6606
Epoch 2/50
1500/1500 [==============================] - 1s 689us/step - loss: 0.7459 - accuracy: 0.7661
Epoch 3/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.6670 - accuracy: 0.7904
Epoch 4/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.6251 - accuracy: 0.8009
Epoch 5/50
1500/1500 [==============================] - 1s 682us/step - loss: 0.5977 - accuracy: 0.8084
Epoch 6/50
1500/1500 [==============================] - 1s 710us/step - loss: 0.5782 - accuracy: 0.8137
Epoch 7/50
1500/1500 [==============================] - 1s 697us/step - loss: 0.5630 - accuracy: 0.8173
Epoch 8/50
1500/1500 [==============================] - 1s 699us/step - loss: 0.5507 - accuracy: 0.8206
Epoch 9/50
1500/1500 [==============================] - 1s 682us/step - loss: 0.5407 - accuracy: 0.8231
Epoch 10/50
1500/1500 [==============================] - 1s 674us/step - loss: 0.5321 - accuracy: 0.8255
Epoch 11/50
1500/1500 [==============================] - 1s 675us/step - loss: 0.5248 - accuracy: 0.8276
Epoch 12/50
1500/1500 [==============================] - 1s 684us/step - loss: 0.5182 - accuracy: 0.8299
Epoch 13/50
1500/1500 [==============================] - 1s 697us/step - loss: 0.5123 - accuracy: 0.8305
Epoch 14/50
1500/1500 [==============================] - 1s 714us/step - loss: 0.5071 - accuracy: 0.8328
Epoch 15/50
1500/1500 [==============================] - 1s 690us/step - loss: 0.5024 - accuracy: 0.8344
Epoch 16/50
1500/1500 [==============================] - 1s 685us/step - loss: 0.4982 - accuracy: 0.8348
Epoch 17/50
1500/1500 [==============================] - 1s 695us/step - loss: 0.4940 - accuracy: 0.8362
Epoch 18/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.4905 - accuracy: 0.8367
Epoch 19/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.4872 - accuracy: 0.8374
Epoch 20/50
1500/1500 [==============================] - 1s 683us/step - loss: 0.4839 - accuracy: 0.8384
Epoch 21/50
1500/1500 [==============================] - 1s 677us/step - loss: 0.4810 - accuracy: 0.8391
Epoch 22/50
1500/1500 [==============================] - 1s 683us/step - loss: 0.4782 - accuracy: 0.8399
Epoch 23/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.4755 - accuracy: 0.8412
Epoch 24/50
1500/1500 [==============================] - 1s 703us/step - loss: 0.4731 - accuracy: 0.8412
Epoch 25/50
1500/1500 [==============================] - 1s 668us/step - loss: 0.4707 - accuracy: 0.8424
Epoch 26/50
1500/1500 [==============================] - 1s 674us/step - loss: 0.4685 - accuracy: 0.8432
Epoch 27/50
1500/1500 [==============================] - 1s 683us/step - loss: 0.4663 - accuracy: 0.8432
Epoch 28/50
1500/1500 [==============================] - 1s 687us/step - loss: 0.4643 - accuracy: 0.8443
Epoch 29/50
1500/1500 [==============================] - 1s 663us/step - loss: 0.4624 - accuracy: 0.8449
Epoch 30/50
1500/1500 [==============================] - 1s 687us/step - loss: 0.4606 - accuracy: 0.8454
Epoch 31/50
1500/1500 [==============================] - 1s 746us/step - loss: 0.4587 - accuracy: 0.8460
Epoch 32/50
1500/1500 [==============================] - 1s 701us/step - loss: 0.4571 - accuracy: 0.8471
Epoch 33/50
1500/1500 [==============================] - 1s 676us/step - loss: 0.4555 - accuracy: 0.8474
Epoch 34/50
1500/1500 [==============================] - 1s 698us/step - loss: 0.4539 - accuracy: 0.8474
Epoch 35/50
1500/1500 [==============================] - 1s 702us/step - loss: 0.4524 - accuracy: 0.8480
Epoch 36/50
1500/1500 [==============================] - 1s 694us/step - loss: 0.4508 - accuracy: 0.8487
Epoch 37/50
1500/1500 [==============================] - 1s 687us/step - loss: 0.4495 - accuracy: 0.8490
Epoch 38/50
1500/1500 [==============================] - 1s 700us/step - loss: 0.4481 - accuracy: 0.8497
Epoch 39/50
1500/1500 [==============================] - 1s 677us/step - loss: 0.4468 - accuracy: 0.8496
Epoch 40/50
1500/1500 [==============================] - 1s 678us/step - loss: 0.4456 - accuracy: 0.8503
Epoch 41/50
1500/1500 [==============================] - 1s 695us/step - loss: 0.4443 - accuracy: 0.8505
Epoch 42/50
1500/1500 [==============================] - 1s 687us/step - loss: 0.4430 - accuracy: 0.8510
Epoch 43/50
1500/1500 [==============================] - 1s 696us/step - loss: 0.4419 - accuracy: 0.8515
Epoch 44/50
1500/1500 [==============================] - 1s 684us/step - loss: 0.4409 - accuracy: 0.8520
Epoch 45/50
1500/1500 [==============================] - 1s 684us/step - loss: 0.4397 - accuracy: 0.8524
Epoch 46/50
1500/1500 [==============================] - 1s 690us/step - loss: 0.4386 - accuracy: 0.8526
Epoch 47/50
1500/1500 [==============================] - 1s 691us/step - loss: 0.4375 - accuracy: 0.8530
Epoch 48/50
1500/1500 [==============================] - 1s 684us/step - loss: 0.4365 - accuracy: 0.8533
Epoch 49/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.4356 - accuracy: 0.8529
Epoch 50/50
1500/1500 [==============================] - 1s 698us/step - loss: 0.4345 - accuracy: 0.8539
375/375 [==============================] - 0s 538us/step - loss: 0.4490 - accuracy: 0.8457
Epoch 1/50
1500/1500 [==============================] - 1s 762us/step - loss: 0.5320 - accuracy: 0.8105
Epoch 2/50
1500/1500 [==============================] - 1s 744us/step - loss: 0.3917 - accuracy: 0.8590
Epoch 3/50
1500/1500 [==============================] - 1s 785us/step - loss: 0.3537 - accuracy: 0.8728
Epoch 4/50
1500/1500 [==============================] - 1s 784us/step - loss: 0.3314 - accuracy: 0.8821
Epoch 5/50
1500/1500 [==============================] - 1s 786us/step - loss: 0.3163 - accuracy: 0.8860
Epoch 6/50
1500/1500 [==============================] - 1s 774us/step - loss: 0.3089 - accuracy: 0.8910
Epoch 7/50
1500/1500 [==============================] - 1s 794us/step - loss: 0.2986 - accuracy: 0.8947
Epoch 8/50
1500/1500 [==============================] - 1s 737us/step - loss: 0.2907 - accuracy: 0.8968
Epoch 9/50
1500/1500 [==============================] - 1s 739us/step - loss: 0.2827 - accuracy: 0.9015
Epoch 10/50
1500/1500 [==============================] - 1s 738us/step - loss: 0.2774 - accuracy: 0.9031
Epoch 11/50
1500/1500 [==============================] - 1s 780us/step - loss: 0.2702 - accuracy: 0.9067
Epoch 12/50
1500/1500 [==============================] - 1s 793us/step - loss: 0.2650 - accuracy: 0.9069
Epoch 13/50
1500/1500 [==============================] - 1s 776us/step - loss: 0.2615 - accuracy: 0.9087
Epoch 14/50
1500/1500 [==============================] - 1s 759us/step - loss: 0.2525 - accuracy: 0.9120
Epoch 15/50
1500/1500 [==============================] - 1s 753us/step - loss: 0.2511 - accuracy: 0.9141
Epoch 16/50
1500/1500 [==============================] - 1s 743us/step - loss: 0.2473 - accuracy: 0.9153
Epoch 17/50
1500/1500 [==============================] - 1s 750us/step - loss: 0.2408 - accuracy: 0.9170
Epoch 18/50
1500/1500 [==============================] - 1s 744us/step - loss: 0.2386 - accuracy: 0.9186
Epoch 19/50
1500/1500 [==============================] - 1s 744us/step - loss: 0.2349 - accuracy: 0.9200
Epoch 20/50
1500/1500 [==============================] - 1s 748us/step - loss: 0.2304 - accuracy: 0.9205
Epoch 21/50
1500/1500 [==============================] - 1s 741us/step - loss: 0.2261 - accuracy: 0.9209
Epoch 22/50
1500/1500 [==============================] - 1s 745us/step - loss: 0.2224 - accuracy: 0.9255
Epoch 23/50
1500/1500 [==============================] - 1s 802us/step - loss: 0.2218 - accuracy: 0.9252
Epoch 24/50
1500/1500 [==============================] - 1s 807us/step - loss: 0.2188 - accuracy: 0.9272
Epoch 25/50
1500/1500 [==============================] - 1s 820us/step - loss: 0.2132 - accuracy: 0.9287
Epoch 26/50
1500/1500 [==============================] - 1s 809us/step - loss: 0.2111 - accuracy: 0.9282
Epoch 27/50
1500/1500 [==============================] - 1s 785us/step - loss: 0.2076 - accuracy: 0.9310
Epoch 28/50
1500/1500 [==============================] - 1s 767us/step - loss: 0.2050 - accuracy: 0.9309
Epoch 29/50
1500/1500 [==============================] - 1s 815us/step - loss: 0.2016 - accuracy: 0.9328
Epoch 30/50
1500/1500 [==============================] - 1s 821us/step - loss: 0.1984 - accuracy: 0.9334
Epoch 31/50
1500/1500 [==============================] - 1s 778us/step - loss: 0.1950 - accuracy: 0.9357
Epoch 32/50
1500/1500 [==============================] - 1s 765us/step - loss: 0.1924 - accuracy: 0.9355
Epoch 33/50
1500/1500 [==============================] - 1s 785us/step - loss: 0.1935 - accuracy: 0.9370
Epoch 34/50
1500/1500 [==============================] - 1s 779us/step - loss: 0.1891 - accuracy: 0.9371
Epoch 35/50
1500/1500 [==============================] - 1s 776us/step - loss: 0.1859 - accuracy: 0.9382
Epoch 36/50
1500/1500 [==============================] - 1s 786us/step - loss: 0.1871 - accuracy: 0.9383
Epoch 37/50
1500/1500 [==============================] - 1s 770us/step - loss: 0.1837 - accuracy: 0.9401
Epoch 38/50
1500/1500 [==============================] - 1s 764us/step - loss: 0.1809 - accuracy: 0.9417
Epoch 39/50
1500/1500 [==============================] - 1s 783us/step - loss: 0.1772 - accuracy: 0.9411
Epoch 40/50
1500/1500 [==============================] - 1s 768us/step - loss: 0.1755 - accuracy: 0.9426
Epoch 41/50
1500/1500 [==============================] - 1s 785us/step - loss: 0.1757 - accuracy: 0.9420
Epoch 42/50
1500/1500 [==============================] - 1s 794us/step - loss: 0.1751 - accuracy: 0.9434
Epoch 43/50
1500/1500 [==============================] - 1s 769us/step - loss: 0.1736 - accuracy: 0.9448
Epoch 44/50
1500/1500 [==============================] - 1s 764us/step - loss: 0.1722 - accuracy: 0.9439
Epoch 45/50
1500/1500 [==============================] - 1s 769us/step - loss: 0.1691 - accuracy: 0.9459
Epoch 46/50
1500/1500 [==============================] - 1s 765us/step - loss: 0.1673 - accuracy: 0.9459
Epoch 47/50
1500/1500 [==============================] - 1s 764us/step - loss: 0.1652 - accuracy: 0.9464
Epoch 48/50
1500/1500 [==============================] - 1s 771us/step - loss: 0.1614 - accuracy: 0.9471
Epoch 49/50
1500/1500 [==============================] - 1s 794us/step - loss: 0.1612 - accuracy: 0.9479
Epoch 50/50
1500/1500 [==============================] - 1s 758us/step - loss: 0.1575 - accuracy: 0.9476
375/375 [==============================] - 0s 548us/step - loss: 0.7231 - accuracy: 0.8833
Epoch 1/50
1500/1500 [==============================] - 1s 664us/step - loss: 0.5277 - accuracy: 0.8146
Epoch 2/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.3947 - accuracy: 0.8582
Epoch 3/50
1500/1500 [==============================] - 1s 665us/step - loss: 0.3540 - accuracy: 0.8710
Epoch 4/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.3313 - accuracy: 0.8798
Epoch 5/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.3094 - accuracy: 0.8857
Epoch 6/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.2939 - accuracy: 0.8919
Epoch 7/50
1500/1500 [==============================] - 1s 691us/step - loss: 0.2813 - accuracy: 0.8969
Epoch 8/50
1500/1500 [==============================] - 1s 677us/step - loss: 0.2701 - accuracy: 0.9002
Epoch 9/50
1500/1500 [==============================] - 1s 679us/step - loss: 0.2591 - accuracy: 0.9035
Epoch 10/50
1500/1500 [==============================] - 1s 728us/step - loss: 0.2501 - accuracy: 0.9067
Epoch 11/50
1500/1500 [==============================] - 1s 675us/step - loss: 0.2437 - accuracy: 0.9087
Epoch 12/50
1500/1500 [==============================] - 1s 676us/step - loss: 0.2347 - accuracy: 0.9106
Epoch 13/50
1500/1500 [==============================] - 1s 676us/step - loss: 0.2276 - accuracy: 0.9147
Epoch 14/50
1500/1500 [==============================] - 1s 686us/step - loss: 0.2222 - accuracy: 0.9173
Epoch 15/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.2149 - accuracy: 0.9189
Epoch 16/50
1500/1500 [==============================] - 1s 668us/step - loss: 0.2099 - accuracy: 0.9212
Epoch 17/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.2030 - accuracy: 0.9247
Epoch 18/50
1500/1500 [==============================] - 1s 678us/step - loss: 0.2005 - accuracy: 0.9260
Epoch 19/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.1944 - accuracy: 0.9280
Epoch 20/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.1894 - accuracy: 0.9302
Epoch 21/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.1835 - accuracy: 0.9310
Epoch 22/50
1500/1500 [==============================] - 1s 684us/step - loss: 0.1814 - accuracy: 0.9316
Epoch 23/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.1750 - accuracy: 0.9342
Epoch 24/50
1500/1500 [==============================] - 1s 672us/step - loss: 0.1738 - accuracy: 0.9349
Epoch 25/50
1500/1500 [==============================] - 1s 668us/step - loss: 0.1679 - accuracy: 0.9380
Epoch 26/50
1500/1500 [==============================] - 1s 669us/step - loss: 0.1640 - accuracy: 0.9392
Epoch 27/50
1500/1500 [==============================] - 1s 670us/step - loss: 0.1626 - accuracy: 0.9389
Epoch 28/50
1500/1500 [==============================] - 1s 676us/step - loss: 0.1552 - accuracy: 0.9419
Epoch 29/50
1500/1500 [==============================] - 1s 683us/step - loss: 0.1561 - accuracy: 0.9422
Epoch 30/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.1513 - accuracy: 0.9435
Epoch 31/50
1500/1500 [==============================] - 1s 667us/step - loss: 0.1467 - accuracy: 0.9459
Epoch 32/50
1500/1500 [==============================] - 1s 669us/step - loss: 0.1457 - accuracy: 0.9454
Epoch 33/50
1500/1500 [==============================] - 1s 686us/step - loss: 0.1413 - accuracy: 0.9465
Epoch 34/50
1500/1500 [==============================] - 1s 671us/step - loss: 0.1402 - accuracy: 0.9485
Epoch 35/50
1500/1500 [==============================] - 1s 669us/step - loss: 0.1351 - accuracy: 0.9492
Epoch 36/50
1500/1500 [==============================] - 1s 687us/step - loss: 0.1332 - accuracy: 0.9494
Epoch 37/50
1500/1500 [==============================] - 1s 675us/step - loss: 0.1305 - accuracy: 0.9518
Epoch 38/50
1500/1500 [==============================] - 1s 679us/step - loss: 0.1295 - accuracy: 0.9511
Epoch 39/50
1500/1500 [==============================] - 1s 680us/step - loss: 0.1262 - accuracy: 0.9523
Epoch 40/50
1500/1500 [==============================] - 1s 669us/step - loss: 0.1216 - accuracy: 0.9542
Epoch 41/50
1500/1500 [==============================] - 1s 673us/step - loss: 0.1211 - accuracy: 0.9550
Epoch 42/50
1500/1500 [==============================] - 1s 702us/step - loss: 0.1200 - accuracy: 0.9555
Epoch 43/50
1500/1500 [==============================] - 1s 707us/step - loss: 0.1171 - accuracy: 0.9566
Epoch 44/50
1500/1500 [==============================] - 1s 750us/step - loss: 0.1151 - accuracy: 0.9574
Epoch 45/50
1500/1500 [==============================] - 1s 704us/step - loss: 0.1148 - accuracy: 0.9566
Epoch 46/50
1500/1500 [==============================] - 1s 699us/step - loss: 0.1122 - accuracy: 0.9581
Epoch 47/50
1500/1500 [==============================] - 1s 690us/step - loss: 0.1068 - accuracy: 0.9597
Epoch 48/50
1500/1500 [==============================] - 1s 682us/step - loss: 0.1079 - accuracy: 0.9589
Epoch 49/50
1500/1500 [==============================] - 1s 715us/step - loss: 0.1046 - accuracy: 0.9609
Epoch 50/50
1500/1500 [==============================] - 1s 692us/step - loss: 0.1053 - accuracy: 0.9611
375/375 [==============================] - 0s 592us/step - loss: 0.4577 - accuracy: 0.8878
손실율이 가장 낮은 학습 방법sgd : 0.3374978303909302 정확도가 가장 높은 학습 방법adam : 0.8878333568572998
In [280]:
### 옵티마이저 학습방법 및 반복횟수를 받아서 처리할 함수 정의
def getBestEval( opt, epoch):
"""모델 생성"""
model = keras.Sequential()
"""레이어 계층 생성 및 모델에 추가하기"""
model.add(keras.layers.Flatten(input_shape=(28,28)))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(10, activation="softmax"))
"""모델 설정하기"""
model.compile(optimizer=opt,
loss="sparse_categorical_crossentropy",
metrics="accuracy")
"""모델 훈련시키기"""
model.fit(train_scaled, train_target, epochs=epoch)
"""성능평가"""
score = model.evaluate(val_scaled , val_target)
"""성능 결과 반환하기"""
return score
In [281]:
"""함술 호출하기"""
"""옵티마이저를 리스트로 정의하기"""
optimizers = ["sgd","adagrad","rmsprop","adam"]
"""최고 정확도를 담을 변수 정의"""
best_acc = 0
"""최고 정확도 일때의 학습 방법을 담을 변수 정의"""
best_acc_opt=""
"""최저 손실율을 담을 변수 정의"""
best_loss = 1
"""최저 손실율일 때의 학습방법을 담을 변수 정의"""
bset_loss_opt = ""
"""옵티마이저의 학습방법을 반복하여 성능 확인하기"""
for opt in optimizers:
print (f"---------------------{opt}-------------------------")
"""함수호출"""
epoch= 10
rs_score= getBestEval(opt,epoch)
"""가장 높은 정확도와 이때 학습방법 저장하기"""
if best_acc< rs_score[1]:
best_acc = rs_score[1]
best_acc_opt = opt
"""가장 낮은 손실율과 이때 학습방법 저장하기"""
if best_loss> rs_score[0]:
best_loss= rs_score[0]
best_loss_opt= opt
print()
print("전체 실행 종료>>>>>>>>>>>>>>>>>>>>>>>>>>>")
print(f"best_acc_opt: {best_acc_opt} / {best_acc}")
print(f"best_loss_opt: {best_loss_opt} / {best_loss}")
---------------------sgd-------------------------
Epoch 1/10
1500/1500 [==============================] - 1s 698us/step - loss: 0.7921 - accuracy: 0.7409
Epoch 2/10
1500/1500 [==============================] - 1s 693us/step - loss: 0.5380 - accuracy: 0.8171
Epoch 3/10
1500/1500 [==============================] - 1s 710us/step - loss: 0.4898 - accuracy: 0.8314
Epoch 4/10
1500/1500 [==============================] - 1s 705us/step - loss: 0.4612 - accuracy: 0.8407
Epoch 5/10
1500/1500 [==============================] - 1s 689us/step - loss: 0.4427 - accuracy: 0.8473
Epoch 6/10
1500/1500 [==============================] - 1s 691us/step - loss: 0.4303 - accuracy: 0.8518
Epoch 7/10
1500/1500 [==============================] - 1s 691us/step - loss: 0.4175 - accuracy: 0.8565
Epoch 8/10
1500/1500 [==============================] - 1s 685us/step - loss: 0.4080 - accuracy: 0.8591
Epoch 9/10
1500/1500 [==============================] - 1s 700us/step - loss: 0.3975 - accuracy: 0.8632
Epoch 10/10
1500/1500 [==============================] - 1s 691us/step - loss: 0.3905 - accuracy: 0.8650
375/375 [==============================] - 0s 567us/step - loss: 0.4025 - accuracy: 0.8593
---------------------adagrad-------------------------
Epoch 1/10
1500/1500 [==============================] - 1s 710us/step - loss: 1.2241 - accuracy: 0.6352
Epoch 2/10
1500/1500 [==============================] - 1s 704us/step - loss: 0.7989 - accuracy: 0.7460
Epoch 3/10
1500/1500 [==============================] - 1s 702us/step - loss: 0.7027 - accuracy: 0.7772
Epoch 4/10
1500/1500 [==============================] - 1s 706us/step - loss: 0.6524 - accuracy: 0.7922
Epoch 5/10
1500/1500 [==============================] - 1s 705us/step - loss: 0.6197 - accuracy: 0.8015
Epoch 6/10
1500/1500 [==============================] - 1s 708us/step - loss: 0.5969 - accuracy: 0.8075
Epoch 7/10
1500/1500 [==============================] - 1s 709us/step - loss: 0.5792 - accuracy: 0.8112
Epoch 8/10
1500/1500 [==============================] - 1s 721us/step - loss: 0.5652 - accuracy: 0.8153
Epoch 9/10
1500/1500 [==============================] - 1s 706us/step - loss: 0.5537 - accuracy: 0.8177
Epoch 10/10
1500/1500 [==============================] - 1s 710us/step - loss: 0.5439 - accuracy: 0.8210
375/375 [==============================] - 0s 616us/step - loss: 0.5537 - accuracy: 0.8160
---------------------rmsprop-------------------------
Epoch 1/10
1500/1500 [==============================] - 2s 846us/step - loss: 0.5326 - accuracy: 0.8115
Epoch 2/10
1500/1500 [==============================] - 1s 849us/step - loss: 0.3940 - accuracy: 0.8585
Epoch 3/10
1500/1500 [==============================] - 1s 850us/step - loss: 0.3545 - accuracy: 0.8725
Epoch 4/10
1500/1500 [==============================] - 1s 853us/step - loss: 0.3325 - accuracy: 0.8822
Epoch 5/10
1500/1500 [==============================] - 1s 846us/step - loss: 0.3173 - accuracy: 0.8872
Epoch 6/10
1500/1500 [==============================] - 1s 885us/step - loss: 0.3091 - accuracy: 0.8918
Epoch 7/10
1500/1500 [==============================] - 1s 917us/step - loss: 0.2970 - accuracy: 0.8950
Epoch 8/10
1500/1500 [==============================] - 1s 863us/step - loss: 0.2898 - accuracy: 0.8980
Epoch 9/10
1500/1500 [==============================] - 1s 880us/step - loss: 0.2824 - accuracy: 0.9005
Epoch 10/10
1500/1500 [==============================] - 1s 887us/step - loss: 0.2767 - accuracy: 0.9034
375/375 [==============================] - 0s 587us/step - loss: 0.3814 - accuracy: 0.8841
---------------------adam-------------------------
Epoch 1/10
1500/1500 [==============================] - 1s 711us/step - loss: 0.5220 - accuracy: 0.8176
Epoch 2/10
1500/1500 [==============================] - 1s 689us/step - loss: 0.3881 - accuracy: 0.8620
Epoch 3/10
1500/1500 [==============================] - 1s 702us/step - loss: 0.3483 - accuracy: 0.8740
Epoch 4/10
1500/1500 [==============================] - 1s 716us/step - loss: 0.3257 - accuracy: 0.8813
Epoch 5/10
1500/1500 [==============================] - 1s 694us/step - loss: 0.3051 - accuracy: 0.8876
Epoch 6/10
1500/1500 [==============================] - 1s 736us/step - loss: 0.2893 - accuracy: 0.8946
Epoch 7/10
1500/1500 [==============================] - 1s 726us/step - loss: 0.2771 - accuracy: 0.8978
Epoch 8/10
1500/1500 [==============================] - 1s 688us/step - loss: 0.2648 - accuracy: 0.9010
Epoch 9/10
1500/1500 [==============================] - 1s 691us/step - loss: 0.2548 - accuracy: 0.9052
Epoch 10/10
1500/1500 [==============================] - 1s 696us/step - loss: 0.2452 - accuracy: 0.9086
375/375 [==============================] - 0s 559us/step - loss: 0.3348 - accuracy: 0.8805
전체 실행 종료>>>>>>>>>>>>>>>>>>>>>>>>>>>
best_acc_opt: rmsprop / 0.8840833306312561
best_loss_opt: adam / 0.3348091244697571
In [ ]:
In [ ]:
728x90
반응형
'딥러닝' 카테고리의 다른 글
| 심플순환신경망(Simple_RNN (0) | 2024.01.05 |
|---|---|
| 퍼셉트론_분류데이터사용 (0) | 2024.01.05 |
| DNN_분류데이터사용 (0) | 2024.01.05 |
| [딥러닝]인공신경망_훈련모델_맛보기 (3) | 2024.01.03 |
| [딥러닝]심층신경망_훈련_및_성능향상 (0) | 2024.01.03 |
