深度殘差網路+自適應引數化ReLU啟用函式（調參記錄18）Cifar10~94.28%
本文將殘差模組的數量增加到27個。其實之前也這樣做過，現在的區別在於，自適應引數化ReLU啟用函式中第一個全連線層中的神經元個數設定成了特徵通道數量的1/16。同樣是在Cifar10資料集上進行測試。
自適應引數化ReLU啟用函式的基本原理如下：
Keras程式碼如下：
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Apr 14 04:17:45 2020
Implemented using TensorFlow 1.10.0 and Keras 2.2.1
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis,
IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458
@author: Minghang Zhao
"""
from __future__ import print_function
import keras
import numpy as np
from keras.datasets import cifar10
from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
from keras.regularizers import l2
from keras import backend as K
from keras.models import Model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import LearningRateScheduler
K.set_learning_phase(1)
# The data, split between train and test sets
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
# Noised data
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_test = x_test-np.mean(x_train)
x_train = x_train-np.mean(x_train)
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)
# Schedule the learning rate, multiply 0.1 every 1500 epoches
def scheduler(epoch):
if epoch % 1500 == 0 and epoch != 0:
lr = K.get_value(model.optimizer.lr)
K.set_value(model.optimizer.lr, lr * 0.1)
print("lr changed to {}".format(lr * 0.1))
return K.get_value(model.optimizer.lr)
# An adaptively parametric rectifier linear unit (APReLU)
def aprelu(inputs):
# get the number of channels
channels = inputs.get_shape().as_list()[-1]
# get a zero feature map
zeros_input = keras.layers.subtract([inputs, inputs])
# get a feature map with only positive features
pos_input = Activation('relu')(inputs)
# get a feature map with only negative features
neg_input = Minimum()([inputs,zeros_input])
# define a network to obtain the scaling coefficients
scales_p = GlobalAveragePooling2D()(pos_input)
scales_n = GlobalAveragePooling2D()(neg_input)
scales = Concatenate()([scales_n, scales_p])
scales = Dense(channels//16, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
scales = Activation('relu')(scales)
scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
scales = Activation('sigmoid')(scales)
scales = Reshape((1,1,channels))(scales)
# apply a paramtetric relu
neg_part = keras.layers.multiply([scales, neg_input])
return keras.layers.add([pos_input, neg_part])
# Residual Block
def residual_block(incoming, nb_blocks, out_channels, downsample=False,
downsample_strides=2):
residual = incoming
in_channels = incoming.get_shape().as_list()[-1]
for i in range(nb_blocks):
identity = residual
if not downsample:
downsample_strides = 1
residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
residual = aprelu(residual)
residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides),
padding='same', kernel_initializer='he_normal',
kernel_regularizer=l2(1e-4))(residual)
residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
residual = aprelu(residual)
residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal',
kernel_regularizer=l2(1e-4))(residual)
# Downsampling
if downsample_strides > 1:
identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
# Zero_padding to match channels
if in_channels != out_channels:
zeros_identity = keras.layers.subtract([identity, identity])
identity = keras.layers.concatenate([identity, zeros_identity])
in_channels = out_channels
residual = keras.layers.add([residual, identity])
return residual
# define and train a model
inputs = Input(shape=(32, 32, 3))
net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
net = residual_block(net, 9, 32, downsample=False)
net = residual_block(net, 1, 32, downsample=True)
net = residual_block(net, 8, 32, downsample=False)
net = residual_block(net, 1, 64, downsample=True)
net = residual_block(net, 8, 64, downsample=False)
net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
net = aprelu(net)
net = GlobalAveragePooling2D()(net)
outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
model = Model(inputs=inputs, outputs=outputs)
sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# data augmentation
datagen = ImageDataGenerator(
# randomly rotate images in the range (deg 0 to 180)
rotation_range=30,
# Range for random zoom
zoom_range = 0.2,
# shear angle in counter-clockwise direction in degrees
shear_range = 30,
# randomly flip images
horizontal_flip=True,
# randomly shift images horizontally
width_shift_range=0.125,
# randomly shift images vertically
height_shift_range=0.125)
reduce_lr = LearningRateScheduler(scheduler)
# fit the model on the batches generated by datagen.flow().
model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
validation_data=(x_test, y_test), epochs=5000,
verbose=1, callbacks=[reduce_lr], workers=4)
# get results
K.set_learning_phase(0)
DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
print('Train loss:', DRSN_train_score[0])
print('Train accuracy:', DRSN_train_score[1])
DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
print('Test loss:', DRSN_test_score[0])
print('Test accuracy:', DRSN_test_score[1])
實驗結果如下（為了方便觀看，刪掉了訓練記錄中的等號）：
Using TensorFlow backend. x_train shape: (50000, 32, 32, 3) 50000 train samples 10000 test samples Epoch 1/5000 107s 215ms/step - loss: 2.9739 - acc: 0.3942 - val_loss: 2.5752 - val_acc: 0.5134 Epoch 2/5000 77s 155ms/step - loss: 2.4558 - acc: 0.5143 - val_loss: 2.1374 - val_acc: 0.6048 Epoch 3/5000 78s 155ms/step - loss: 2.1359 - acc: 0.5744 - val_loss: 1.8827 - val_acc: 0.6355 Epoch 4/5000 78s 155ms/step - loss: 1.9044 - acc: 0.6088 - val_loss: 1.6888 - val_acc: 0.6620 Epoch 5/5000 78s 155ms/step - loss: 1.7154 - acc: 0.6381 - val_loss: 1.5106 - val_acc: 0.6946 Epoch 6/5000 77s 155ms/step - loss: 1.5849 - acc: 0.6528 - val_loss: 1.3992 - val_acc: 0.7031 Epoch 7/5000 77s 155ms/step - loss: 1.4700 - acc: 0.6686 - val_loss: 1.2869 - val_acc: 0.7208 Epoch 8/5000 77s 155ms/step - loss: 1.3735 - acc: 0.6851 - val_loss: 1.2068 - val_acc: 0.7400 Epoch 9/5000 77s 155ms/step - loss: 1.2940 - acc: 0.6980 - val_loss: 1.1169 - val_acc: 0.7588 Epoch 10/5000 77s 155ms/step - loss: 1.2318 - acc: 0.7058 - val_loss: 1.0696 - val_acc: 0.7647 Epoch 11/5000 77s 155ms/step - loss: 1.1735 - acc: 0.7202 - val_loss: 1.0117 - val_acc: 0.7757 Epoch 12/5000 77s 155ms/step - loss: 1.1366 - acc: 0.7240 - val_loss: 0.9968 - val_acc: 0.7770 Epoch 13/5000 77s 155ms/step - loss: 1.0974 - acc: 0.7308 - val_loss: 0.9548 - val_acc: 0.7824 Epoch 14/5000 77s 155ms/step - loss: 1.0653 - acc: 0.7390 - val_loss: 0.9196 - val_acc: 0.7885 Epoch 15/5000 77s 155ms/step - loss: 1.0409 - acc: 0.7419 - val_loss: 0.9444 - val_acc: 0.7775 Epoch 16/5000 78s 155ms/step - loss: 1.0122 - acc: 0.7526 - val_loss: 0.9039 - val_acc: 0.7893 Epoch 17/5000 78s 155ms/step - loss: 0.9970 - acc: 0.7561 - val_loss: 0.8913 - val_acc: 0.7952 Epoch 18/5000 77s 155ms/step - loss: 0.9819 - acc: 0.7562 - val_loss: 0.8748 - val_acc: 0.7995 Epoch 19/5000 77s 155ms/step - loss: 0.9638 - acc: 0.7615 - val_loss: 0.8445 - val_acc: 0.8063 Epoch 20/5000 78s 155ms/step - loss: 0.9516 - acc: 0.7662 - val_loss: 0.8522 - val_acc: 0.8011 Epoch 21/5000 78s 155ms/step - loss: 0.9401 - acc: 0.7698 - val_loss: 0.8325 - val_acc: 0.8145 Epoch 22/5000 78s 155ms/step - loss: 0.9302 - acc: 0.7731 - val_loss: 0.8266 - val_acc: 0.8141 Epoch 23/5000 78s 155ms/step - loss: 0.9231 - acc: 0.7747 - val_loss: 0.8390 - val_acc: 0.8043 Epoch 24/5000 77s 155ms/step - loss: 0.9105 - acc: 0.7786 - val_loss: 0.8009 - val_acc: 0.8191 Epoch 25/5000 77s 155ms/step - loss: 0.9027 - acc: 0.7795 - val_loss: 0.7903 - val_acc: 0.8218 Epoch 26/5000 77s 155ms/step - loss: 0.8946 - acc: 0.7863 - val_loss: 0.7812 - val_acc: 0.8271 Epoch 27/5000 78s 155ms/step - loss: 0.8923 - acc: 0.7852 - val_loss: 0.7933 - val_acc: 0.8226 Epoch 28/5000 78s 155ms/step - loss: 0.8808 - acc: 0.7890 - val_loss: 0.7597 - val_acc: 0.8346 Epoch 29/5000 78s 155ms/step - loss: 0.8826 - acc: 0.7881 - val_loss: 0.7639 - val_acc: 0.8332 Epoch 30/5000 77s 155ms/step - loss: 0.8685 - acc: 0.7941 - val_loss: 0.7807 - val_acc: 0.8278 Epoch 31/5000 78s 155ms/step - loss: 0.8647 - acc: 0.7936 - val_loss: 0.7686 - val_acc: 0.8304 Epoch 32/5000 77s 155ms/step - loss: 0.8611 - acc: 0.7971 - val_loss: 0.7619 - val_acc: 0.8324 Epoch 33/5000 77s 155ms/step - loss: 0.8584 - acc: 0.7965 - val_loss: 0.7632 - val_acc: 0.8323 Epoch 34/5000 77s 155ms/step - loss: 0.8513 - acc: 0.8000 - val_loss: 0.7575 - val_acc: 0.8363 Epoch 35/5000 77s 155ms/step - loss: 0.8494 - acc: 0.7990 - val_loss: 0.7669 - val_acc: 0.8293 Epoch 36/5000 77s 155ms/step - loss: 0.8486 - acc: 0.8024 - val_loss: 0.7269 - val_acc: 0.8488 Epoch 37/5000 78s 155ms/step - loss: 0.8432 - acc: 0.8032 - val_loss: 0.7633 - val_acc: 0.8355 Epoch 38/5000 78s 155ms/step - loss: 0.8380 - acc: 0.8061 - val_loss: 0.7554 - val_acc: 0.8356 Epoch 39/5000 77s 155ms/step - loss: 0.8340 - acc: 0.8071 - val_loss: 0.7587 - val_acc: 0.8378 Epoch 40/5000 77s 155ms/step - loss: 0.8310 - acc: 0.8098 - val_loss: 0.7495 - val_acc: 0.8345 Epoch 41/5000 78s 155ms/step - loss: 0.8313 - acc: 0.8092 - val_loss: 0.7370 - val_acc: 0.8446 Epoch 42/5000 77s 155ms/step - loss: 0.8239 - acc: 0.8118 - val_loss: 0.7481 - val_acc: 0.8398 Epoch 43/5000 78s 155ms/step - loss: 0.8244 - acc: 0.8125 - val_loss: 0.7351 - val_acc: 0.8473 Epoch 44/5000 77s 155ms/step - loss: 0.8197 - acc: 0.8122 - val_loss: 0.7396 - val_acc: 0.8444 Epoch 45/5000 78s 155ms/step - loss: 0.8152 - acc: 0.8143 - val_loss: 0.7500 - val_acc: 0.8444 Epoch 46/5000 78s 155ms/step - loss: 0.8158 - acc: 0.8162 - val_loss: 0.7657 - val_acc: 0.8358 Epoch 47/5000 77s 155ms/step - loss: 0.8120 - acc: 0.8173 - val_loss: 0.7409 - val_acc: 0.8431 Epoch 48/5000 77s 155ms/step - loss: 0.8141 - acc: 0.8155 - val_loss: 0.7370 - val_acc: 0.8458 Epoch 49/5000 77s 155ms/step - loss: 0.8054 - acc: 0.8202 - val_loss: 0.7485 - val_acc: 0.8412 Epoch 50/5000 77s 154ms/step - loss: 0.8030 - acc: 0.8189 - val_loss: 0.7240 - val_acc: 0.8494 Epoch 51/5000 77s 155ms/step - loss: 0.8020 - acc: 0.8212 - val_loss: 0.7283 - val_acc: 0.8472 Epoch 52/5000 77s 155ms/step - loss: 0.7991 - acc: 0.8221 - val_loss: 0.7443 - val_acc: 0.8416 Epoch 53/5000 77s 155ms/step - loss: 0.7915 - acc: 0.8252 - val_loss: 0.7322 - val_acc: 0.8468 Epoch 54/5000 77s 155ms/step - loss: 0.7956 - acc: 0.8231 - val_loss: 0.7171 - val_acc: 0.8502 Epoch 55/5000 77s 155ms/step - loss: 0.7890 - acc: 0.8257 - val_loss: 0.7102 - val_acc: 0.8563 Epoch 56/5000 77s 155ms/step - loss: 0.7930 - acc: 0.8233 - val_loss: 0.7081 - val_acc: 0.8569 Epoch 57/5000 77s 155ms/step - loss: 0.7912 - acc: 0.8261 - val_loss: 0.7117 - val_acc: 0.8597 Epoch 58/5000 77s 154ms/step - loss: 0.7878 - acc: 0.8259 - val_loss: 0.7217 - val_acc: 0.8504 Epoch 59/5000 78s 155ms/step - loss: 0.7851 - acc: 0.8294 - val_loss: 0.6983 - val_acc: 0.8614 Epoch 60/5000 77s 155ms/step - loss: 0.7844 - acc: 0.8276 - val_loss: 0.7018 - val_acc: 0.8598 Epoch 61/5000 77s 155ms/step - loss: 0.7840 - acc: 0.8290 - val_loss: 0.7333 - val_acc: 0.8507 Epoch 62/5000 77s 154ms/step - loss: 0.7848 - acc: 0.8288 - val_loss: 0.7279 - val_acc: 0.8503 Epoch 63/5000 77s 155ms/step - loss: 0.7819 - acc: 0.8305 - val_loss: 0.7187 - val_acc: 0.8520 Epoch 64/5000 77s 155ms/step - loss: 0.7822 - acc: 0.8290 - val_loss: 0.7123 - val_acc: 0.8568 Epoch 65/5000 77s 154ms/step - loss: 0.7768 - acc: 0.8323 - val_loss: 0.6983 - val_acc: 0.8621 Epoch 66/5000 77s 155ms/step - loss: 0.7759 - acc: 0.8318 - val_loss: 0.7027 - val_acc: 0.8614 Epoch 67/5000 77s 155ms/step - loss: 0.7783 - acc: 0.8329 - val_loss: 0.7249 - val_acc: 0.8558 Epoch 68/5000 77s 155ms/step - loss: 0.7772 - acc: 0.8324 - val_loss: 0.7089 - val_acc: 0.8595 Epoch 69/5000 77s 155ms/step - loss: 0.7760 - acc: 0.8341 - val_loss: 0.7116 - val_acc: 0.8562 Epoch 70/5000 77s 155ms/step - loss: 0.7720 - acc: 0.8344 - val_loss: 0.7378 - val_acc: 0.8486 Epoch 71/5000 77s 155ms/step - loss: 0.7721 - acc: 0.8357 - val_loss: 0.6936 - val_acc: 0.8653 Epoch 72/5000 77s 155ms/step - loss: 0.7706 - acc: 0.8353 - val_loss: 0.7158 - val_acc: 0.8580 Epoch 73/5000 77s 155ms/step - loss: 0.7727 - acc: 0.8345 - val_loss: 0.7100 - val_acc: 0.8605 Epoch 74/5000 77s 155ms/step - loss: 0.7682 - acc: 0.8385 - val_loss: 0.6988 - val_acc: 0.8621 Epoch 75/5000 77s 155ms/step - loss: 0.7705 - acc: 0.8353 - val_loss: 0.7062 - val_acc: 0.8553 Epoch 76/5000 77s 155ms/step - loss: 0.7682 - acc: 0.8365 - val_loss: 0.7227 - val_acc: 0.8571 Epoch 77/5000 77s 155ms/step - loss: 0.7621 - acc: 0.8388 - val_loss: 0.6973 - val_acc: 0.8635 Epoch 78/5000 77s 155ms/step - loss: 0.7617 - acc: 0.8393 - val_loss: 0.7023 - val_acc: 0.8591 Epoch 79/5000 77s 155ms/step - loss: 0.7618 - acc: 0.8374 - val_loss: 0.6919 - val_acc: 0.8656 Epoch 80/5000 77s 155ms/step - loss: 0.7658 - acc: 0.8372 - val_loss: 0.7192 - val_acc: 0.8559 Epoch 81/5000 77s 155ms/step - loss: 0.7600 - acc: 0.8415 - val_loss: 0.7004 - val_acc: 0.8628 Epoch 82/5000 77s 155ms/step - loss: 0.7623 - acc: 0.8378 - val_loss: 0.6683 - val_acc: 0.8748 Epoch 83/5000 77s 154ms/step - loss: 0.7579 - acc: 0.8407 - val_loss: 0.7009 - val_acc: 0.8604 Epoch 84/5000 77s 155ms/step - loss: 0.7569 - acc: 0.8407 - val_loss: 0.6992 - val_acc: 0.8611 Epoch 85/5000 77s 155ms/step - loss: 0.7515 - acc: 0.8431 - val_loss: 0.7052 - val_acc: 0.8606 Epoch 86/5000 77s 155ms/step - loss: 0.7581 - acc: 0.8420 - val_loss: 0.7094 - val_acc: 0.8549 Epoch 87/5000 77s 155ms/step - loss: 0.7555 - acc: 0.8413 - val_loss: 0.7164 - val_acc: 0.8573 Epoch 88/5000 77s 154ms/step - loss: 0.7555 - acc: 0.8413 - val_loss: 0.7003 - val_acc: 0.8671 Epoch 89/5000 77s 155ms/step - loss: 0.7523 - acc: 0.8468 - val_loss: 0.6850 - val_acc: 0.8698 Epoch 90/5000 77s 155ms/step - loss: 0.7511 - acc: 0.8451 - val_loss: 0.6796 - val_acc: 0.8720 Epoch 91/5000 77s 155ms/step - loss: 0.7539 - acc: 0.8430 - val_loss: 0.6969 - val_acc: 0.8632 Epoch 92/5000 77s 155ms/step - loss: 0.7519 - acc: 0.8438 - val_loss: 0.7286 - val_acc: 0.8538 Epoch 93/5000 77s 155ms/step - loss: 0.7509 - acc: 0.8443 - val_loss: 0.6878 - val_acc: 0.8669 Epoch 94/5000 77s 155ms/step - loss: 0.7487 - acc: 0.8455 - val_loss: 0.6985 - val_acc: 0.8661 Epoch 95/5000 77s 155ms/step - loss: 0.7497 - acc: 0.8444 - val_loss: 0.6983 - val_acc: 0.8646 Epoch 96/5000 77s 155ms/step - loss: 0.7501 - acc: 0.8437 - val_loss: 0.6890 - val_acc: 0.8677 Epoch 97/5000 77s 155ms/step - loss: 0.7471 - acc: 0.8461 - val_loss: 0.6912 - val_acc: 0.8662 Epoch 98/5000 77s 154ms/step - loss: 0.7472 - acc: 0.8466 - val_loss: 0.6830 - val_acc: 0.8695 Epoch 99/5000 77s 155ms/step - loss: 0.7476 - acc: 0.8451 - val_loss: 0.7109 - val_acc: 0.8630 Epoch 100/5000 77s 155ms/step - loss: 0.7452 - acc: 0.8473 - val_loss: 0.6877 - val_acc: 0.8684 Epoch 101/5000 78s 155ms/step - loss: 0.7436 - acc: 0.8483 - val_loss: 0.7104 - val_acc: 0.8597 Epoch 102/5000 77s 155ms/step - loss: 0.7479 - acc: 0.8471 - val_loss: 0.6644 - val_acc: 0.8757 Epoch 103/5000 77s 154ms/step - loss: 0.7395 - acc: 0.8489 - val_loss: 0.6755 - val_acc: 0.8717 Epoch 104/5000 77s 155ms/step - loss: 0.7394 - acc: 0.8501 - val_loss: 0.6735 - val_acc: 0.8696 Epoch 105/5000 77s 155ms/step - loss: 0.7442 - acc: 0.8465 - val_loss: 0.6777 - val_acc: 0.8741 Epoch 106/5000 77s 154ms/step - loss: 0.7429 - acc: 0.8485 - val_loss: 0.6860 - val_acc: 0.8713 Epoch 107/5000 77s 155ms/step - loss: 0.7420 - acc: 0.8494 - val_loss: 0.6858 - val_acc: 0.8711 Epoch 108/5000 77s 155ms/step - loss: 0.7438 - acc: 0.8483 - val_loss: 0.6807 - val_acc: 0.8702 Epoch 109/5000 77s 154ms/step - loss: 0.7385 - acc: 0.8506 - val_loss: 0.6778 - val_acc: 0.8713 Epoch 110/5000 77s 155ms/step - loss: 0.7409 - acc: 0.8482 - val_loss: 0.7039 - val_acc: 0.8640 Epoch 111/5000 77s 155ms/step - loss: 0.7376 - acc: 0.8501 - val_loss: 0.6737 - val_acc: 0.8745 Epoch 112/5000 77s 155ms/step - loss: 0.7388 - acc: 0.8497 - val_loss: 0.6785 - val_acc: 0.8711 Epoch 113/5000 77s 155ms/step - loss: 0.7307 - acc: 0.8534 - val_loss: 0.6741 - val_acc: 0.8719 Epoch 114/5000 77s 155ms/step - loss: 0.7443 - acc: 0.8496 - val_loss: 0.6898 - val_acc: 0.8699 Epoch 115/5000 77s 155ms/step - loss: 0.7340 - acc: 0.8514 - val_loss: 0.6953 - val_acc: 0.8666 Epoch 116/5000 77s 155ms/step - loss: 0.7367 - acc: 0.8516 - val_loss: 0.6951 - val_acc: 0.8656 Epoch 117/5000 77s 155ms/step - loss: 0.7367 - acc: 0.8508 - val_loss: 0.6809 - val_acc: 0.8709 Epoch 118/5000 77s 155ms/step - loss: 0.7324 - acc: 0.8521 - val_loss: 0.6809 - val_acc: 0.8746 Epoch 119/5000 77s 155ms/step - loss: 0.7335 - acc: 0.8528 - val_loss: 0.6919 - val_acc: 0.8663 Epoch 120/5000 77s 155ms/step - loss: 0.7363 - acc: 0.8507 - val_loss: 0.6922 - val_acc: 0.8693 Epoch 121/5000 77s 155ms/step - loss: 0.7341 - acc: 0.8530 - val_loss: 0.6859 - val_acc: 0.8717 Epoch 122/5000 77s 155ms/step - loss: 0.7334 - acc: 0.8533 - val_loss: 0.7199 - val_acc: 0.8596 Epoch 123/5000 77s 155ms/step - loss: 0.7328 - acc: 0.8527 - val_loss: 0.6934 - val_acc: 0.8658 Epoch 124/5000 77s 155ms/step - loss: 0.7278 - acc: 0.8532 - val_loss: 0.6822 - val_acc: 0.8684 Epoch 125/5000 77s 155ms/step - loss: 0.7338 - acc: 0.8532 - val_loss: 0.6865 - val_acc: 0.8700 Epoch 126/5000 77s 155ms/step - loss: 0.7319 - acc: 0.8528 - val_loss: 0.6759 - val_acc: 0.8732 Epoch 127/5000 78s 155ms/step - loss: 0.7335 - acc: 0.8519 - val_loss: 0.6776 - val_acc: 0.8744 Epoch 128/5000 77s 155ms/step - loss: 0.7302 - acc: 0.8521 - val_loss: 0.6883 - val_acc: 0.8678 Epoch 129/5000 77s 155ms/step - loss: 0.7299 - acc: 0.8543 - val_loss: 0.6868 - val_acc: 0.8697 Epoch 130/5000 77s 155ms/step - loss: 0.7318 - acc: 0.8527 - val_loss: 0.6784 - val_acc: 0.8735 Epoch 131/5000 77s 155ms/step - loss: 0.7280 - acc: 0.8559 - val_loss: 0.6750 - val_acc: 0.8745 Epoch 132/5000 77s 155ms/step - loss: 0.7301 - acc: 0.8543 - val_loss: 0.6815 - val_acc: 0.8721 Epoch 133/5000 77s 155ms/step - loss: 0.7319 - acc: 0.8536 - val_loss: 0.6647 - val_acc: 0.8762 Epoch 134/5000 77s 155ms/step - loss: 0.7338 - acc: 0.8532 - val_loss: 0.6733 - val_acc: 0.8767 Epoch 135/5000 77s 155ms/step - loss: 0.7330 - acc: 0.8533 - val_loss: 0.6891 - val_acc: 0.8705 Epoch 136/5000 77s 155ms/step - loss: 0.7251 - acc: 0.8563 - val_loss: 0.6765 - val_acc: 0.8781 Epoch 137/5000 77s 154ms/step - loss: 0.7315 - acc: 0.8543 - val_loss: 0.6875 - val_acc: 0.8701 Epoch 138/5000 77s 155ms/step - loss: 0.7290 - acc: 0.8546 - val_loss: 0.6824 - val_acc: 0.8701 Epoch 139/5000 77s 155ms/step - loss: 0.7256 - acc: 0.8565 - val_loss: 0.6717 - val_acc: 0.8731 Epoch 140/5000 77s 155ms/step - loss: 0.7240 - acc: 0.8548 - val_loss: 0.6846 - val_acc: 0.8727 Epoch 141/5000 77s 155ms/step - loss: 0.7244 - acc: 0.8580 - val_loss: 0.6681 - val_acc: 0.8793 Epoch 142/5000 77s 155ms/step - loss: 0.7270 - acc: 0.8555 - val_loss: 0.6734 - val_acc: 0.8781 Epoch 143/5000 77s 154ms/step - loss: 0.7295 - acc: 0.8567 - val_loss: 0.6731 - val_acc: 0.8783 Epoch 144/5000 77s 154ms/step - loss: 0.7271 - acc: 0.8549 - val_loss: 0.6803 - val_acc: 0.8758 Epoch 145/5000 77s 154ms/step - loss: 0.7257 - acc: 0.8544 - val_loss: 0.6675 - val_acc: 0.8763 Epoch 146/5000 78s 157ms/step - loss: 0.7257 - acc: 0.8547 - val_loss: 0.6824 - val_acc: 0.8709 Epoch 147/5000 78s 156ms/step - loss: 0.7259 - acc: 0.8550 - val_loss: 0.6763 - val_acc: 0.8733 Epoch 148/5000 77s 155ms/step - loss: 0.7291 - acc: 0.8542 - val_loss: 0.6908 - val_acc: 0.8683 Epoch 149/5000 78s 156ms/step - loss: 0.7262 - acc: 0.8571 - val_loss: 0.6593 - val_acc: 0.8818 Epoch 150/5000 78s 157ms/step - loss: 0.7260 - acc: 0.8565 - val_loss: 0.6801 - val_acc: 0.8763 Epoch 151/5000 78s 156ms/step - loss: 0.7256 - acc: 0.8574 - val_loss: 0.6660 - val_acc: 0.8800 Epoch 152/5000 78s 155ms/step - loss: 0.7201 - acc: 0.8602 - val_loss: 0.6659 - val_acc: 0.8799 Epoch 153/5000 78s 156ms/step - loss: 0.7257 - acc: 0.8561 - val_loss: 0.6740 - val_acc: 0.8764 Epoch 154/5000 77s 154ms/step - loss: 0.7273 - acc: 0.8552 - val_loss: 0.6778 - val_acc: 0.8751 Epoch 155/5000 77s 154ms/step - loss: 0.7214 - acc: 0.8579 - val_loss: 0.6649 - val_acc: 0.8803 Epoch 156/5000 77s 154ms/step - loss: 0.7220 - acc: 0.8558 - val_loss: 0.6748 - val_acc: 0.8782 Epoch 157/5000 77s 154ms/step - loss: 0.7227 - acc: 0.8584 - val_loss: 0.6671 - val_acc: 0.8779 Epoch 158/5000 77s 154ms/step - loss: 0.7264 - acc: 0.8558 - val_loss: 0.6607 - val_acc: 0.8780 Epoch 159/5000 77s 154ms/step - loss: 0.7202 - acc: 0.8597 - val_loss: 0.6702 - val_acc: 0.8785 Epoch 160/5000 77s 154ms/step - loss: 0.7211 - acc: 0.8563 - val_loss: 0.6630 - val_acc: 0.8812 Epoch 161/5000 77s 154ms/step - loss: 0.7229 - acc: 0.8579 - val_loss: 0.6743 - val_acc: 0.8769 Epoch 162/5000 77s 154ms/step - loss: 0.7216 - acc: 0.8571 - val_loss: 0.6772 - val_acc: 0.8761 Epoch 163/5000 77s 154ms/step - loss: 0.7168 - acc: 0.8599 - val_loss: 0.6661 - val_acc: 0.8786 Epoch 164/5000 77s 154ms/step - loss: 0.7208 - acc: 0.8573 - val_loss: 0.6801 - val_acc: 0.8763 Epoch 165/5000 77s 154ms/step - loss: 0.7240 - acc: 0.8590 - val_loss: 0.6678 - val_acc: 0.8781 Epoch 166/5000 79s 158ms/step - loss: 0.7154 - acc: 0.8621 - val_loss: 0.6768 - val_acc: 0.8744 Epoch 167/5000 78s 155ms/step - loss: 0.7177 - acc: 0.8600 - val_loss: 0.6536 - val_acc: 0.8816 Epoch 168/5000 78s 155ms/step - loss: 0.7160 - acc: 0.8585 - val_loss: 0.7057 - val_acc: 0.8640 Epoch 169/5000 77s 155ms/step - loss: 0.7196 - acc: 0.8598 - val_loss: 0.6517 - val_acc: 0.8854 Epoch 170/5000 77s 154ms/step - loss: 0.7153 - acc: 0.8581 - val_loss: 0.6592 - val_acc: 0.8844 Epoch 171/5000 77s 154ms/step - loss: 0.7189 - acc: 0.8587 - val_loss: 0.6797 - val_acc: 0.8774 Epoch 172/5000 77s 154ms/step - loss: 0.7182 - acc: 0.8583 - val_loss: 0.6625 - val_acc: 0.8767 Epoch 173/5000 77s 154ms/step - loss: 0.7161 - acc: 0.8597 - val_loss: 0.6863 - val_acc: 0.8726 Epoch 174/5000 77s 154ms/step - loss: 0.7171 - acc: 0.8602 - val_loss: 0.6963 - val_acc: 0.8669 Epoch 175/5000 77s 154ms/step - loss: 0.7211 - acc: 0.8589 - val_loss: 0.6638 - val_acc: 0.8830 Epoch 176/5000 77s 154ms/step - loss: 0.7149 - acc: 0.8607 - val_loss: 0.6717 - val_acc: 0.8723 Epoch 177/5000 77s 153ms/step - loss: 0.7135 - acc: 0.8619 - val_loss: 0.7025 - val_acc: 0.8681 Epoch 178/5000 77s 154ms/step - loss: 0.7175 - acc: 0.8587 - val_loss: 0.6832 - val_acc: 0.8726 Epoch 179/5000 77s 154ms/step - loss: 0.7155 - acc: 0.8598 - val_loss: 0.6633 - val_acc: 0.8790 Epoch 180/5000 77s 154ms/step - loss: 0.7189 - acc: 0.8598 - val_loss: 0.6783 - val_acc: 0.8756 Epoch 181/5000 77s 154ms/step - loss: 0.7187 - acc: 0.8605 - val_loss: 0.6682 - val_acc: 0.8805 Epoch 182/5000 77s 154ms/step - loss: 0.7174 - acc: 0.8607 - val_loss: 0.6748 - val_acc: 0.8792 Epoch 183/5000 77s 154ms/step - loss: 0.7133 - acc: 0.8619 - val_loss: 0.6691 - val_acc: 0.8773 Epoch 184/5000 77s 154ms/step - loss: 0.7180 - acc: 0.8589 - val_loss: 0.6698 - val_acc: 0.8804 Epoch 185/5000 77s 154ms/step - loss: 0.7133 - acc: 0.8629 - val_loss: 0.6720 - val_acc: 0.8753 Epoch 186/5000 77s 154ms/step - loss: 0.7128 - acc: 0.8615 - val_loss: 0.6919 - val_acc: 0.8699 Epoch 187/5000 77s 154ms/step - loss: 0.7196 - acc: 0.8598 - val_loss: 0.6849 - val_acc: 0.8721 Epoch 188/5000 77s 154ms/step - loss: 0.7207 - acc: 0.8581 - val_loss: 0.6754 - val_acc: 0.8780 Epoch 189/5000 77s 154ms/step - loss: 0.7150 - acc: 0.8619 - val_loss: 0.6571 - val_acc: 0.8837 Epoch 190/5000 77s 154ms/step - loss: 0.7159 - acc: 0.8617 - val_loss: 0.6698 - val_acc: 0.8783 Epoch 191/5000 77s 154ms/step - loss: 0.7158 - acc: 0.8606 - val_loss: 0.6782 - val_acc: 0.8772 Epoch 192/5000 77s 153ms/step - loss: 0.7121 - acc: 0.8609 - val_loss: 0.6745 - val_acc: 0.8786 Epoch 193/5000 77s 154ms/step - loss: 0.7137 - acc: 0.8624 - val_loss: 0.6800 - val_acc: 0.8765 Epoch 194/5000 77s 154ms/step - loss: 0.7158 - acc: 0.8607 - val_loss: 0.6836 - val_acc: 0.8734 Epoch 195/5000 77s 153ms/step - loss: 0.7067 - acc: 0.8640 - val_loss: 0.6691 - val_acc: 0.8784 Epoch 196/5000 77s 154ms/step - loss: 0.7111 - acc: 0.8629 - val_loss: 0.6716 - val_acc: 0.8759 Epoch 197/5000 77s 154ms/step - loss: 0.7139 - acc: 0.8611 - val_loss: 0.6686 - val_acc: 0.8777 Epoch 198/5000 77s 154ms/step - loss: 0.7134 - acc: 0.8621 - val_loss: 0.6709 - val_acc: 0.8767 Epoch 199/5000 77s 154ms/step - loss: 0.7101 - acc: 0.8625 - val_loss: 0.6513 - val_acc: 0.8863 Epoch 200/5000 77s 154ms/step - loss: 0.7128 - acc: 0.8617 - val_loss: 0.6713 - val_acc: 0.8768 Epoch 201/5000 77s 154ms/step - loss: 0.7171 - acc: 0.8620 - val_loss: 0.6620 - val_acc: 0.8830 Epoch 202/5000 77s 153ms/step - loss: 0.7122 - acc: 0.8617 - val_loss: 0.6694 - val_acc: 0.8821 Epoch 203/5000 77s 154ms/step - loss: 0.7127 - acc: 0.8604 - val_loss: 0.6717 - val_acc: 0.8797 Epoch 204/5000 77s 154ms/step - loss: 0.7103 - acc: 0.8634 - val_loss: 0.6683 - val_acc: 0.8826 Epoch 205/5000 77s 154ms/step - loss: 0.7123 - acc: 0.8612 - val_loss: 0.6576 - val_acc: 0.8831 Epoch 206/5000 77s 154ms/step - loss: 0.7078 - acc: 0.8632 - val_loss: 0.6453 - val_acc: 0.8842 Epoch 207/5000 77s 154ms/step - loss: 0.7135 - acc: 0.8623 - val_loss: 0.6683 - val_acc: 0.8804 Epoch 208/5000 77s 154ms/step - loss: 0.7038 - acc: 0.8655 - val_loss: 0.6795 - val_acc: 0.8755 Epoch 209/5000 77s 154ms/step - loss: 0.7160 - acc: 0.8607 - val_loss: 0.6709 - val_acc: 0.8773 Epoch 210/5000 77s 154ms/step - loss: 0.7122 - acc: 0.8629 - val_loss: 0.6917 - val_acc: 0.8718 Epoch 211/5000 77s 154ms/step - loss: 0.7064 - acc: 0.8643 - val_loss: 0.6831 - val_acc: 0.8701 Epoch 212/5000 77s 154ms/step - loss: 0.7090 - acc: 0.8629 - val_loss: 0.6475 - val_acc: 0.8875 Epoch 213/5000 77s 154ms/step - loss: 0.7086 - acc: 0.8621 - val_loss: 0.6740 - val_acc: 0.8741 Epoch 214/5000 77s 154ms/step - loss: 0.7105 - acc: 0.8611 - val_loss: 0.6682 - val_acc: 0.8792 Epoch 215/5000 77s 154ms/step - loss: 0.7129 - acc: 0.8616 - val_loss: 0.6930 - val_acc: 0.8682 Epoch 216/5000 77s 154ms/step - loss: 0.7105 - acc: 0.8632 - val_loss: 0.6647 - val_acc: 0.8776 Epoch 217/5000 77s 154ms/step - loss: 0.7086 - acc: 0.8634 - val_loss: 0.6555 - val_acc: 0.8832 Epoch 218/5000 77s 154ms/step - loss: 0.7128 - acc: 0.8634 - val_loss: 0.6665 - val_acc: 0.8780 Epoch 219/5000 77s 153ms/step - loss: 0.7050 - acc: 0.8641 - val_loss: 0.6698 - val_acc: 0.8800 Epoch 220/5000 77s 154ms/step - loss: 0.7042 - acc: 0.8628 - val_loss: 0.6882 - val_acc: 0.8706 Epoch 221/5000 77s 154ms/step - loss: 0.7098 - acc: 0.8615 - val_loss: 0.6649 - val_acc: 0.8802 Epoch 222/5000 77s 154ms/step - loss: 0.7053 - acc: 0.8648 - val_loss: 0.6672 - val_acc: 0.8808 Epoch 223/5000 77s 154ms/step - loss: 0.7064 - acc: 0.8638 - val_loss: 0.6609 - val_acc: 0.8823 Epoch 224/5000 77s 154ms/step - loss: 0.7071 - acc: 0.8627 - val_loss: 0.6677 - val_acc: 0.8812 Epoch 225/5000 77s 154ms/step - loss: 0.7049 - acc: 0.8655 - val_loss: 0.6656 - val_acc: 0.8784 Epoch 226/5000 77s 154ms/step - loss: 0.7065 - acc: 0.8649 - val_loss: 0.6582 - val_acc: 0.8839 Epoch 227/5000 77s 154ms/step - loss: 0.7049 - acc: 0.8644 - val_loss: 0.6812 - val_acc: 0.8763 Epoch 228/5000 77s 154ms/step - loss: 0.7052 - acc: 0.8657 - val_loss: 0.6639 - val_acc: 0.8794 Epoch 229/5000 77s 154ms/step - loss: 0.7054 - acc: 0.8654 - val_loss: 0.6662 - val_acc: 0.8816 Epoch 230/5000 77s 154ms/step - loss: 0.7096 - acc: 0.8639 - val_loss: 0.6789 - val_acc: 0.8752 Epoch 231/5000 77s 154ms/step - loss: 0.7049 - acc: 0.8667 - val_loss: 0.6748 - val_acc: 0.8768 Epoch 232/5000 77s 154ms/step - loss: 0.7047 - acc: 0.8654 - val_loss: 0.6710 - val_acc: 0.8773 Epoch 233/5000 78s 155ms/step - loss: 0.7051 - acc: 0.8648 - val_loss: 0.6516 - val_acc: 0.8839 Epoch 234/5000 77s 155ms/step - loss: 0.7056 - acc: 0.8619 - val_loss: 0.6996 - val_acc: 0.8685 Epoch 235/5000 77s 155ms/step - loss: 0.7043 - acc: 0.8652 - val_loss: 0.6481 - val_acc: 0.8843 Epoch 236/5000 77s 155ms/step - loss: 0.7038 - acc: 0.8638 - val_loss: 0.6760 - val_acc: 0.8808 Epoch 237/5000 77s 155ms/step - loss: 0.7035 - acc: 0.8660 - val_loss: 0.6597 - val_acc: 0.8833 Epoch 238/5000 77s 155ms/step - loss: 0.7042 - acc: 0.8642 - val_loss: 0.6615 - val_acc: 0.8825 Epoch 239/5000 77s 155ms/step - loss: 0.7061 - acc: 0.8645 - val_loss: 0.6715 - val_acc: 0.8753 Epoch 240/5000 77s 155ms/step - loss: 0.7057 - acc: 0.8648 - val_loss: 0.6609 - val_acc: 0.8834 Epoch 241/5000 77s 155ms/step - loss: 0.6990 - acc: 0.8671 - val_loss: 0.6591 - val_acc: 0.8849 Epoch 242/5000 77s 155ms/step - loss: 0.7092 - acc: 0.8638 - val_loss: 0.6439 - val_acc: 0.8888 Epoch 243/5000 77s 155ms/step - loss: 0.7043 - acc: 0.8650 - val_loss: 0.6699 - val_acc: 0.8803 Epoch 244/5000 77s 155ms/step - loss: 0.7055 - acc: 0.8641 - val_loss: 0.6740 - val_acc: 0.8780 Epoch 245/5000 78s 155ms/step - loss: 0.7010 - acc: 0.8668 - val_loss: 0.6673 - val_acc: 0.8807 Epoch 246/5000 78s 155ms/step - loss: 0.7043 - acc: 0.8649 - val_loss: 0.6574 - val_acc: 0.8857 Epoch 247/5000 77s 155ms/step - loss: 0.7071 - acc: 0.8643 - val_loss: 0.6640 - val_acc: 0.8831 Epoch 248/5000 78s 155ms/step - loss: 0.7048 - acc: 0.8664 - val_loss: 0.6647 - val_acc: 0.8825 Epoch 249/5000 77s 155ms/step - loss: 0.7033 - acc: 0.8649 - val_loss: 0.6821 - val_acc: 0.8750 Epoch 250/5000 77s 155ms/step - loss: 0.7035 - acc: 0.8665 - val_loss: 0.6776 - val_acc: 0.8757 Epoch 251/5000 77s 155ms/step - loss: 0.7050 - acc: 0.8643 - val_loss: 0.6857 - val_acc: 0.8721 Epoch 252/5000 78s 155ms/step - loss: 0.7044 - acc: 0.8664 - val_loss: 0.6780 - val_acc: 0.8746 Epoch 253/5000 78s 155ms/step - loss: 0.7033 - acc: 0.8657 - val_loss: 0.6739 - val_acc: 0.8799 Epoch 254/5000 78s 155ms/step - loss: 0.7057 - acc: 0.8670 - val_loss: 0.6654 - val_acc: 0.8812 Epoch 255/5000 77s 155ms/step - loss: 0.7065 - acc: 0.8656 - val_loss: 0.6798 - val_acc: 0.8752 Epoch 256/5000 78s 155ms/step - loss: 0.7033 - acc: 0.8656 - val_loss: 0.6670 - val_acc: 0.8814 Epoch 257/5000 78s 155ms/step - loss: 0.7041 - acc: 0.8648 - val_loss: 0.6699 - val_acc: 0.8823 Epoch 258/5000 77s 155ms/step - loss: 0.6975 - acc: 0.8682 - val_loss: 0.6650 - val_acc: 0.8792 Epoch 259/5000 78s 155ms/step - loss: 0.7007 - acc: 0.8681 - val_loss: 0.6572 - val_acc: 0.8843 Epoch 260/5000 77s 155ms/step - loss: 0.6987 - acc: 0.8668 - val_loss: 0.6633 - val_acc: 0.8820 Epoch 261/5000 77s 155ms/step - loss: 0.7003 - acc: 0.8667 - val_loss: 0.6728 - val_acc: 0.8835 Epoch 262/5000 77s 155ms/step - loss: 0.7007 - acc: 0.8669 - val_loss: 0.6813 - val_acc: 0.8791 Epoch 263/5000 78s 155ms/step - loss: 0.7036 - acc: 0.8659 - val_loss: 0.6670 - val_acc: 0.8805 Epoch 264/5000 78s 155ms/step - loss: 0.7001 - acc: 0.8667 - val_loss: 0.6954 - val_acc: 0.8714 Epoch 265/5000 77s 155ms/step - loss: 0.7016 - acc: 0.8678 - val_loss: 0.6727 - val_acc: 0.8774 Epoch 266/5000 78s 155ms/step - loss: 0.7005 - acc: 0.8664 - val_loss: 0.6804 - val_acc: 0.8751 Epoch 267/5000 77s 155ms/step - loss: 0.7004 - acc: 0.8665 - val_loss: 0.6690 - val_acc: 0.8828 Epoch 268/5000 77s 155ms/step - loss: 0.7030 - acc: 0.8672 - val_loss: 0.6656 - val_acc: 0.8791 Epoch 269/5000 78s 155ms/step - loss: 0.7049 - acc: 0.8659 - val_loss: 0.6549 - val_acc: 0.8850 Epoch 270/5000 78s 155ms/step - loss: 0.7012 - acc: 0.8669 - val_loss: 0.6743 - val_acc: 0.8775 Epoch 271/5000 77s 155ms/step - loss: 0.7024 - acc: 0.8673 - val_loss: 0.6573 - val_acc: 0.8847 Epoch 272/5000 77s 155ms/step - loss: 0.6991 - acc: 0.8676 - val_loss: 0.6792 - val_acc: 0.8737 Epoch 273/5000 77s 155ms/step - loss: 0.7014 - acc: 0.8664 - val_loss: 0.6658 - val_acc: 0.8832 Epoch 274/5000 77s 155ms/step - loss: 0.6974 - acc: 0.8688 - val_loss: 0.6721 - val_acc: 0.8805 Epoch 275/5000 77s 155ms/step - loss: 0.6985 - acc: 0.8683 - val_loss: 0.6641 - val_acc: 0.8829 Epoch 276/5000 77s 155ms/step - loss: 0.6971 - acc: 0.8678 - val_loss: 0.6491 - val_acc: 0.8868 Epoch 277/5000 77s 155ms/step - loss: 0.7014 - acc: 0.8653 - val_loss: 0.6572 - val_acc: 0.8849 Epoch 278/5000 77s 155ms/step - loss: 0.6943 - acc: 0.8698 - val_loss: 0.6613 - val_acc: 0.8827 Epoch 279/5000 77s 155ms/step - loss: 0.6974 - acc: 0.8690 - val_loss: 0.6941 - val_acc: 0.8707 Epoch 280/5000 77s 155ms/step - loss: 0.6994 - acc: 0.8688 - val_loss: 0.6803 - val_acc: 0.8740 Epoch 281/5000 78s 155ms/step - loss: 0.6995 - acc: 0.8693 - val_loss: 0.6676 - val_acc: 0.8803 Epoch 282/5000 77s 155ms/step - loss: 0.7024 - acc: 0.8660 - val_loss: 0.6690 - val_acc: 0.8795 Epoch 283/5000 78s 155ms/step - loss: 0.6965 - acc: 0.8688 - val_loss: 0.6608 - val_acc: 0.8861 Epoch 284/5000 77s 155ms/step - loss: 0.6945 - acc: 0.8700 - val_loss: 0.6706 - val_acc: 0.8777 Epoch 285/5000 77s 155ms/step - loss: 0.7016 - acc: 0.8672 - val_loss: 0.6635 - val_acc: 0.8812 Epoch 286/5000 78s 155ms/step - loss: 0.6989 - acc: 0.8673 - val_loss: 0.6629 - val_acc: 0.8844 Epoch 287/5000 77s 155ms/step - loss: 0.7025 - acc: 0.8665 - val_loss: 0.6578 - val_acc: 0.8829 Epoch 288/5000 77s 155ms/step - loss: 0.7009 - acc: 0.8667 - val_loss: 0.6617 - val_acc: 0.8814 Epoch 289/5000 77s 155ms/step - loss: 0.6981 - acc: 0.8687 - val_loss: 0.6838 - val_acc: 0.8745 Epoch 290/5000 77s 155ms/step - loss: 0.7025 - acc: 0.8667 - val_loss: 0.6643 - val_acc: 0.8828 Epoch 291/5000 77s 155ms/step - loss: 0.6984 - acc: 0.8683 - val_loss: 0.6492 - val_acc: 0.8867 Epoch 292/5000 77s 155ms/step - loss: 0.6973 - acc: 0.8692 - val_loss: 0.6480 - val_acc: 0.8868 Epoch 293/5000 78s 155ms/step - loss: 0.6918 - acc: 0.8712 - val_loss: 0.6666 - val_acc: 0.8834 Epoch 294/5000 77s 155ms/step - loss: 0.6997 - acc: 0.8672 - val_loss: 0.6919 - val_acc: 0.8714 Epoch 295/5000 77s 155ms/step - loss: 0.6990 - acc: 0.8692 - val_loss: 0.6528 - val_acc: 0.8848 Epoch 296/5000 77s 155ms/step - loss: 0.6974 - acc: 0.8694 - val_loss: 0.6756 - val_acc: 0.8767 Epoch 297/5000 77s 155ms/step - loss: 0.6972 - acc: 0.8669 - val_loss: 0.6588 - val_acc: 0.8849 Epoch 298/5000 77s 155ms/step - loss: 0.6992 - acc: 0.8689 - val_loss: 0.6678 - val_acc: 0.8807 Epoch 299/5000 78s 155ms/step - loss: 0.6936 - acc: 0.8700 - val_loss: 0.6497 - val_acc: 0.8875 Epoch 300/5000 77s 155ms/step - loss: 0.6958 - acc: 0.8695 - val_loss: 0.6750 - val_acc: 0.8788 Epoch 301/5000 77s 155ms/step - loss: 0.7019 - acc: 0.8670 - val_loss: 0.6845 - val_acc: 0.8746 Epoch 302/5000 78s 155ms/step - loss: 0.6974 - acc: 0.8676 - val_loss: 0.6784 - val_acc: 0.8783 Epoch 303/5000 78s 155ms/step - loss: 0.6962 - acc: 0.8690 - val_loss: 0.6511 - val_acc: 0.8864 Epoch 304/5000 77s 155ms/step - loss: 0.6995 - acc: 0.8689 - val_loss: 0.6477 - val_acc: 0.8922 Epoch 305/5000 77s 155ms/step - loss: 0.6979 - acc: 0.8693 - val_loss: 0.6682 - val_acc: 0.8800 Epoch 306/5000 77s 155ms/step - loss: 0.6963 - acc: 0.8689 - val_loss: 0.6852 - val_acc: 0.8775 Epoch 307/5000 78s 155ms/step - loss: 0.6956 - acc: 0.8703 - val_loss: 0.6773 - val_acc: 0.8776 Epoch 308/5000 77s 155ms/step - loss: 0.6969 - acc: 0.8692 - val_loss: 0.6762 - val_acc: 0.8760 Epoch 309/5000 78s 155ms/step - loss: 0.6962 - acc: 0.8700 - val_loss: 0.6636 - val_acc: 0.8826 Epoch 310/5000 78s 155ms/step - loss: 0.6941 - acc: 0.8700 - val_loss: 0.6491 - val_acc: 0.8877 Epoch 311/5000 77s 155ms/step - loss: 0.6922 - acc: 0.8696 - val_loss: 0.6534 - val_acc: 0.8848 Epoch 312/5000 77s 155ms/step - loss: 0.6936 - acc: 0.8691 - val_loss: 0.6530 - val_acc: 0.8858 Epoch 313/5000 77s 155ms/step - loss: 0.6985 - acc: 0.8692 - val_loss: 0.6421 - val_acc: 0.8904 Epoch 314/5000 77s 155ms/step - loss: 0.7032 - acc: 0.8679 - val_loss: 0.6421 - val_acc: 0.8906 Epoch 315/5000 77s 155ms/step - loss: 0.6911 - acc: 0.8714 - val_loss: 0.6497 - val_acc: 0.8868 Epoch 316/5000 77s 155ms/step - loss: 0.6993 - acc: 0.8690 - val_loss: 0.6567 - val_acc: 0.8860 Epoch 317/5000 77s 155ms/step - loss: 0.6943 - acc: 0.8704 - val_loss: 0.6646 - val_acc: 0.8847 Epoch 318/5000 77s 155ms/step - loss: 0.6950 - acc: 0.8709 - val_loss: 0.6642 - val_acc: 0.8801 Epoch 319/5000 77s 155ms/step - loss: 0.7008 - acc: 0.8676 - val_loss: 0.6778 - val_acc: 0.8788 Epoch 320/5000 78s 155ms/step - loss: 0.6947 - acc: 0.8717 - val_loss: 0.6420 - val_acc: 0.8915 Epoch 321/5000 77s 155ms/step - loss: 0.6945 - acc: 0.8702 - val_loss: 0.6391 - val_acc: 0.8939 Epoch 322/5000 77s 155ms/step - loss: 0.6917 - acc: 0.8712 - val_loss: 0.6846 - val_acc: 0.8776 Epoch 323/5000 78s 155ms/step - loss: 0.6968 - acc: 0.8687 - val_loss: 0.6690 - val_acc: 0.8842 Epoch 324/5000 77s 155ms/step - loss: 0.6949 - acc: 0.8708 - val_loss: 0.6511 - val_acc: 0.8841 Epoch 325/5000 78s 155ms/step - loss: 0.6947 - acc: 0.8701 - val_loss: 0.6517 - val_acc: 0.8892 Epoch 326/5000 77s 155ms/step - loss: 0.6958 - acc: 0.8670 - val_loss: 0.6714 - val_acc: 0.8804 Epoch 327/5000 77s 155ms/step - loss: 0.6959 - acc: 0.8694 - val_loss: 0.6856 - val_acc: 0.8744 Epoch 328/5000 78s 155ms/step - loss: 0.6903 - acc: 0.8703 - val_loss: 0.6671 - val_acc: 0.8798 Epoch 329/5000 77s 155ms/step - loss: 0.6921 - acc: 0.8705 - val_loss: 0.6651 - val_acc: 0.8860 Epoch 330/5000 77s 155ms/step - loss: 0.6956 - acc: 0.8693 - val_loss: 0.6903 - val_acc: 0.8742 Epoch 331/5000 77s 155ms/step - loss: 0.6951 - acc: 0.8692 - val_loss: 0.6567 - val_acc: 0.8875 Epoch 332/5000 78s 155ms/step - loss: 0.6924 - acc: 0.8714 - val_loss: 0.6648 - val_acc: 0.8825 Epoch 333/5000 77s 155ms/step - loss: 0.6998 - acc: 0.8689 - val_loss: 0.6581 - val_acc: 0.8837 Epoch 334/5000 77s 155ms/step - loss: 0.6931 - acc: 0.8703 - val_loss: 0.6723 - val_acc: 0.8789 Epoch 335/5000 78s 155ms/step - loss: 0.6987 - acc: 0.8698 - val_loss: 0.6769 - val_acc: 0.8788 Epoch 336/5000 77s 155ms/step - loss: 0.6917 - acc: 0.8707 - val_loss: 0.6494 - val_acc: 0.8886 Epoch 337/5000 78s 155ms/step - loss: 0.6906 - acc: 0.8724 - val_loss: 0.6733 - val_acc: 0.8784 Epoch 338/5000 77s 155ms/step - loss: 0.6948 - acc: 0.8704 - val_loss: 0.6510 - val_acc: 0.8882 Epoch 339/5000 78s 155ms/step - loss: 0.6908 - acc: 0.8714 - val_loss: 0.6726 - val_acc: 0.8794 Epoch 340/5000 77s 155ms/step - loss: 0.6919 - acc: 0.8726 - val_loss: 0.6660 - val_acc: 0.8814 Epoch 341/5000 78s 155ms/step - loss: 0.6947 - acc: 0.8704 - val_loss: 0.6462 - val_acc: 0.8891 Epoch 342/5000 78s 155ms/step - loss: 0.6911 - acc: 0.8721 - val_loss: 0.6821 - val_acc: 0.8761 Epoch 343/5000 78s 155ms/step - loss: 0.6962 - acc: 0.8683 - val_loss: 0.6709 - val_acc: 0.8791 Epoch 344/5000 77s 155ms/step - loss: 0.6938 - acc: 0.8701 - val_loss: 0.6776 - val_acc: 0.8769 Epoch 345/5000 77s 155ms/step - loss: 0.6917 - acc: 0.8717 - val_loss: 0.6769 - val_acc: 0.8779 Epoch 346/5000 78s 155ms/step - loss: 0.6915 - acc: 0.8697 - val_loss: 0.6473 - val_acc: 0.8895 Epoch 347/5000 77s 155ms/step - loss: 0.6918 - acc: 0.8702 - val_loss: 0.6543 - val_acc: 0.8865 Epoch 348/5000 77s 155ms/step - loss: 0.6907 - acc: 0.8725 - val_loss: 0.6867 - val_acc: 0.8748 Epoch 349/5000 77s 155ms/step - loss: 0.6962 - acc: 0.8687 - val_loss: 0.6580 - val_acc: 0.8866 Epoch 350/5000 77s 155ms/step - loss: 0.6922 - acc: 0.8704 - val_loss: 0.6766 - val_acc: 0.8766 Epoch 351/5000 78s 155ms/step - loss: 0.6901 - acc: 0.8708 - val_loss: 0.6742 - val_acc: 0.8797 Epoch 352/5000 77s 155ms/step - loss: 0.6910 - acc: 0.8723 - val_loss: 0.6587 - val_acc: 0.8828 Epoch 353/5000 78s 155ms/step - loss: 0.6941 - acc: 0.8717 - val_loss: 0.6960 - val_acc: 0.8708 Epoch 354/5000 77s 155ms/step - loss: 0.6929 - acc: 0.8699 - val_loss: 0.6612 - val_acc: 0.8853 Epoch 355/5000 77s 155ms/step - loss: 0.6947 - acc: 0.8716 - val_loss: 0.6795 - val_acc: 0.8772 Epoch 356/5000 77s 155ms/step - loss: 0.6958 - acc: 0.8717 - val_loss: 0.6750 - val_acc: 0.8788 Epoch 357/5000 77s 155ms/step - loss: 0.6885 - acc: 0.8721 - val_loss: 0.6617 - val_acc: 0.8830 Epoch 358/5000 78s 155ms/step - loss: 0.6851 - acc: 0.8742 - val_loss: 0.6671 - val_acc: 0.8781 Epoch 359/5000 78s 155ms/step - loss: 0.6898 - acc: 0.8696 - val_loss: 0.6511 - val_acc: 0.8861 Epoch 360/5000 77s 155ms/step - loss: 0.6967 - acc: 0.8681 - val_loss: 0.6567 - val_acc: 0.8867 Epoch 361/5000 77s 155ms/step - loss: 0.6931 - acc: 0.8700 - val_loss: 0.7052 - val_acc: 0.8687 Epoch 362/5000 77s 155ms/step - loss: 0.6953 - acc: 0.8702 - val_loss: 0.6465 - val_acc: 0.8923 Epoch 363/5000 77s 154ms/step - loss: 0.6956 - acc: 0.8695 - val_loss: 0.6305 - val_acc: 0.8971 Epoch 364/5000 77s 154ms/step - loss: 0.6892 - acc: 0.8712 - val_loss: 0.6787 - val_acc: 0.8796 Epoch 365/5000 77s 154ms/step - loss: 0.6895 - acc: 0.8729 - val_loss: 0.6676 - val_acc: 0.8837 Epoch 366/5000 77s 154ms/step - loss: 0.6955 - acc: 0.8705 - val_loss: 0.6644 - val_acc: 0.8825 Epoch 367/5000 77s 154ms/step - loss: 0.6912 - acc: 0.8723 - val_loss: 0.6712 - val_acc: 0.8805 Epoch 368/5000 77s 153ms/step - loss: 0.6925 - acc: 0.8720 - val_loss: 0.6506 - val_acc: 0.8871 Epoch 369/5000 77s 154ms/step - loss: 0.6904 - acc: 0.8724 - val_loss: 0.6560 - val_acc: 0.8857 Epoch 370/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8724 - val_loss: 0.6637 - val_acc: 0.8838 Epoch 371/5000 77s 154ms/step - loss: 0.6859 - acc: 0.8745 - val_loss: 0.6669 - val_acc: 0.8853 Epoch 372/5000 77s 154ms/step - loss: 0.6888 - acc: 0.8731 - val_loss: 0.6485 - val_acc: 0.8884 Epoch 373/5000 77s 154ms/step - loss: 0.6877 - acc: 0.8723 - val_loss: 0.6571 - val_acc: 0.8869 Epoch 374/5000 77s 154ms/step - loss: 0.6895 - acc: 0.8727 - val_loss: 0.6762 - val_acc: 0.8811 Epoch 375/5000 77s 153ms/step - loss: 0.6924 - acc: 0.8694 - val_loss: 0.6716 - val_acc: 0.8802 Epoch 376/5000 77s 154ms/step - loss: 0.6912 - acc: 0.8705 - val_loss: 0.6669 - val_acc: 0.8884 Epoch 377/5000 77s 154ms/step - loss: 0.6885 - acc: 0.8734 - val_loss: 0.6551 - val_acc: 0.8858 Epoch 378/5000 77s 154ms/step - loss: 0.6916 - acc: 0.8689 - val_loss: 0.6692 - val_acc: 0.8810 Epoch 379/5000 77s 154ms/step - loss: 0.6892 - acc: 0.8721 - val_loss: 0.6846 - val_acc: 0.8797 Epoch 380/5000 77s 154ms/step - loss: 0.6996 - acc: 0.8686 - val_loss: 0.6645 - val_acc: 0.8851 Epoch 381/5000 77s 154ms/step - loss: 0.6925 - acc: 0.8734 - val_loss: 0.6541 - val_acc: 0.8866 Epoch 382/5000 77s 154ms/step - loss: 0.6929 - acc: 0.8708 - val_loss: 0.6599 - val_acc: 0.8842 Epoch 383/5000 77s 154ms/step - loss: 0.6972 - acc: 0.8711 - val_loss: 0.6536 - val_acc: 0.8859 Epoch 384/5000 77s 154ms/step - loss: 0.6874 - acc: 0.8734 - val_loss: 0.6531 - val_acc: 0.8867 Epoch 385/5000 77s 154ms/step - loss: 0.6940 - acc: 0.8716 - val_loss: 0.6656 - val_acc: 0.8816 Epoch 386/5000 77s 153ms/step - loss: 0.6853 - acc: 0.8758 - val_loss: 0.6791 - val_acc: 0.8779 Epoch 387/5000 77s 154ms/step - loss: 0.6880 - acc: 0.8747 - val_loss: 0.6615 - val_acc: 0.8835 Epoch 388/5000 77s 154ms/step - loss: 0.6850 - acc: 0.8731 - val_loss: 0.6602 - val_acc: 0.8832 Epoch 389/5000 77s 154ms/step - loss: 0.6890 - acc: 0.8720 - val_loss: 0.6566 - val_acc: 0.8842 Epoch 390/5000 77s 154ms/step - loss: 0.6863 - acc: 0.8738 - val_loss: 0.6814 - val_acc: 0.8779 Epoch 391/5000 77s 154ms/step - loss: 0.6940 - acc: 0.8706 - val_loss: 0.6519 - val_acc: 0.8866 Epoch 392/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8730 - val_loss: 0.6509 - val_acc: 0.8885 Epoch 393/5000 77s 154ms/step - loss: 0.6915 - acc: 0.8706 - val_loss: 0.6633 - val_acc: 0.8827 Epoch 394/5000 77s 154ms/step - loss: 0.6909 - acc: 0.8722 - val_loss: 0.6619 - val_acc: 0.8841 Epoch 395/5000 77s 154ms/step - loss: 0.6877 - acc: 0.8733 - val_loss: 0.6826 - val_acc: 0.8774 Epoch 396/5000 77s 154ms/step - loss: 0.6941 - acc: 0.8717 - val_loss: 0.6372 - val_acc: 0.8920 Epoch 397/5000 77s 154ms/step - loss: 0.6870 - acc: 0.8733 - val_loss: 0.6681 - val_acc: 0.8806 Epoch 398/5000 77s 154ms/step - loss: 0.6878 - acc: 0.8740 - val_loss: 0.6686 - val_acc: 0.8797 Epoch 399/5000 77s 154ms/step - loss: 0.6923 - acc: 0.8724 - val_loss: 0.6524 - val_acc: 0.8867 Epoch 400/5000 77s 154ms/step - loss: 0.6900 - acc: 0.8730 - val_loss: 0.6754 - val_acc: 0.8808 Epoch 401/5000 77s 154ms/step - loss: 0.6924 - acc: 0.8719 - val_loss: 0.6570 - val_acc: 0.8819 Epoch 402/5000 77s 153ms/step - loss: 0.6906 - acc: 0.8721 - val_loss: 0.6717 - val_acc: 0.8803 Epoch 403/5000 77s 154ms/step - loss: 0.6871 - acc: 0.8722 - val_loss: 0.6586 - val_acc: 0.8869 Epoch 404/5000 77s 154ms/step - loss: 0.6876 - acc: 0.8739 - val_loss: 0.6576 - val_acc: 0.8846 Epoch 405/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8718 - val_loss: 0.6742 - val_acc: 0.8773 Epoch 406/5000 77s 154ms/step - loss: 0.6844 - acc: 0.8731 - val_loss: 0.6549 - val_acc: 0.8814 Epoch 407/5000 77s 154ms/step - loss: 0.6910 - acc: 0.8719 - val_loss: 0.6778 - val_acc: 0.8798 Epoch 408/5000 77s 154ms/step - loss: 0.6917 - acc: 0.8711 - val_loss: 0.6612 - val_acc: 0.8874 Epoch 409/5000 77s 154ms/step - loss: 0.6872 - acc: 0.8726 - val_loss: 0.6490 - val_acc: 0.8903 Epoch 410/5000 77s 154ms/step - loss: 0.6931 - acc: 0.8713 - val_loss: 0.6500 - val_acc: 0.8890 Epoch 411/5000 77s 154ms/step - loss: 0.6852 - acc: 0.8740 - val_loss: 0.6654 - val_acc: 0.8801 Epoch 412/5000 77s 154ms/step - loss: 0.6905 - acc: 0.8734 - val_loss: 0.6788 - val_acc: 0.8795 Epoch 413/5000 77s 154ms/step - loss: 0.6889 - acc: 0.8732 - val_loss: 0.6511 - val_acc: 0.8855 Epoch 414/5000 77s 154ms/step - loss: 0.6884 - acc: 0.8733 - val_loss: 0.6630 - val_acc: 0.8831 Epoch 415/5000 77s 154ms/step - loss: 0.6904 - acc: 0.8733 - val_loss: 0.6667 - val_acc: 0.8823 Epoch 416/5000 77s 154ms/step - loss: 0.6872 - acc: 0.8732 - val_loss: 0.6826 - val_acc: 0.8790 Epoch 417/5000 77s 154ms/step - loss: 0.6893 - acc: 0.8715 - val_loss: 0.6645 - val_acc: 0.8822 Epoch 418/5000 77s 153ms/step - loss: 0.6915 - acc: 0.8721 - val_loss: 0.6428 - val_acc: 0.8916 Epoch 419/5000 77s 154ms/step - loss: 0.6925 - acc: 0.8728 - val_loss: 0.6605 - val_acc: 0.8870 Epoch 420/5000 77s 154ms/step - loss: 0.6916 - acc: 0.8714 - val_loss: 0.6503 - val_acc: 0.8905 Epoch 421/5000 77s 154ms/step - loss: 0.6912 - acc: 0.8719 - val_loss: 0.6260 - val_acc: 0.8953 Epoch 422/5000 77s 154ms/step - loss: 0.6824 - acc: 0.8749 - val_loss: 0.6716 - val_acc: 0.8800 Epoch 423/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8756 - val_loss: 0.6709 - val_acc: 0.8772 Epoch 424/5000 77s 154ms/step - loss: 0.6910 - acc: 0.8717 - val_loss: 0.6628 - val_acc: 0.8827 Epoch 425/5000 77s 154ms/step - loss: 0.6869 - acc: 0.8739 - val_loss: 0.6865 - val_acc: 0.8751 Epoch 426/5000 77s 154ms/step - loss: 0.6904 - acc: 0.8738 - val_loss: 0.6621 - val_acc: 0.8861 Epoch 427/5000 77s 154ms/step - loss: 0.6915 - acc: 0.8735 - val_loss: 0.6622 - val_acc: 0.8862 Epoch 428/5000 77s 154ms/step - loss: 0.6863 - acc: 0.8722 - val_loss: 0.6572 - val_acc: 0.8851 Epoch 429/5000 77s 154ms/step - loss: 0.6877 - acc: 0.8722 - val_loss: 0.6473 - val_acc: 0.8896 Epoch 430/5000 77s 154ms/step - loss: 0.6891 - acc: 0.8737 - val_loss: 0.6699 - val_acc: 0.8829 Epoch 431/5000 77s 154ms/step - loss: 0.6915 - acc: 0.8710 - val_loss: 0.6561 - val_acc: 0.8834 Epoch 432/5000 77s 154ms/step - loss: 0.6849 - acc: 0.8739 - val_loss: 0.6476 - val_acc: 0.8863 Epoch 433/5000 77s 154ms/step - loss: 0.6890 - acc: 0.8730 - val_loss: 0.6679 - val_acc: 0.8827 Epoch 434/5000 77s 154ms/step - loss: 0.6910 - acc: 0.8717 - val_loss: 0.6482 - val_acc: 0.8864 Epoch 435/5000 77s 154ms/step - loss: 0.6866 - acc: 0.8736 - val_loss: 0.6579 - val_acc: 0.8841 Epoch 436/5000 77s 154ms/step - loss: 0.6888 - acc: 0.8732 - val_loss: 0.6615 - val_acc: 0.8804 Epoch 437/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8734 - val_loss: 0.6854 - val_acc: 0.8791 Epoch 438/5000 77s 154ms/step - loss: 0.6917 - acc: 0.8719 - val_loss: 0.6664 - val_acc: 0.8847 Epoch 439/5000 77s 154ms/step - loss: 0.6935 - acc: 0.8706 - val_loss: 0.6469 - val_acc: 0.8897 Epoch 440/5000 77s 154ms/step - loss: 0.6890 - acc: 0.8750 - val_loss: 0.6735 - val_acc: 0.8830 Epoch 441/5000 77s 154ms/step - loss: 0.6907 - acc: 0.8719 - val_loss: 0.6752 - val_acc: 0.8802 Epoch 442/5000 77s 154ms/step - loss: 0.6882 - acc: 0.8747 - val_loss: 0.6486 - val_acc: 0.8879 Epoch 443/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8736 - val_loss: 0.6621 - val_acc: 0.8826 Epoch 444/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8732 - val_loss: 0.6650 - val_acc: 0.8854 Epoch 445/5000 77s 154ms/step - loss: 0.6938 - acc: 0.8702 - val_loss: 0.6406 - val_acc: 0.8921 Epoch 446/5000 77s 154ms/step - loss: 0.6891 - acc: 0.8726 - val_loss: 0.6464 - val_acc: 0.8937 Epoch 447/5000 77s 154ms/step - loss: 0.6890 - acc: 0.8725 - val_loss: 0.6449 - val_acc: 0.8902 Epoch 448/5000 77s 154ms/step - loss: 0.6920 - acc: 0.8716 - val_loss: 0.6620 - val_acc: 0.8858 Epoch 449/5000 77s 154ms/step - loss: 0.6881 - acc: 0.8744 - val_loss: 0.6399 - val_acc: 0.8926 Epoch 450/5000 77s 154ms/step - loss: 0.6872 - acc: 0.8735 - val_loss: 0.6694 - val_acc: 0.8836 Epoch 451/5000 77s 154ms/step - loss: 0.6938 - acc: 0.8697 - val_loss: 0.6440 - val_acc: 0.8905 Epoch 452/5000 77s 154ms/step - loss: 0.6914 - acc: 0.8727 - val_loss: 0.6423 - val_acc: 0.8856 Epoch 453/5000 77s 154ms/step - loss: 0.6845 - acc: 0.8746 - val_loss: 0.6852 - val_acc: 0.8734 Epoch 454/5000 77s 154ms/step - loss: 0.6846 - acc: 0.8745 - val_loss: 0.6488 - val_acc: 0.8892 Epoch 455/5000 77s 154ms/step - loss: 0.6849 - acc: 0.8761 - val_loss: 0.6750 - val_acc: 0.8782 Epoch 456/5000 77s 154ms/step - loss: 0.6879 - acc: 0.8728 - val_loss: 0.6682 - val_acc: 0.8815 Epoch 457/5000 77s 154ms/step - loss: 0.6884 - acc: 0.8723 - val_loss: 0.6584 - val_acc: 0.8852 Epoch 458/5000 77s 154ms/step - loss: 0.6877 - acc: 0.8742 - val_loss: 0.6657 - val_acc: 0.8841 Epoch 459/5000 77s 154ms/step - loss: 0.6946 - acc: 0.8708 - val_loss: 0.6339 - val_acc: 0.8915 Epoch 460/5000 77s 154ms/step - loss: 0.6861 - acc: 0.8745 - val_loss: 0.6607 - val_acc: 0.8820 Epoch 461/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8750 - val_loss: 0.6594 - val_acc: 0.8849 Epoch 462/5000 77s 154ms/step - loss: 0.6900 - acc: 0.8724 - val_loss: 0.6511 - val_acc: 0.8886 Epoch 463/5000 77s 154ms/step - loss: 0.6863 - acc: 0.8749 - val_loss: 0.6682 - val_acc: 0.8837 Epoch 464/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8722 - val_loss: 0.6534 - val_acc: 0.8865 Epoch 465/5000 77s 154ms/step - loss: 0.6808 - acc: 0.8763 - val_loss: 0.6723 - val_acc: 0.8836 Epoch 466/5000 77s 154ms/step - loss: 0.6869 - acc: 0.8734 - val_loss: 0.6576 - val_acc: 0.8862 Epoch 467/5000 77s 154ms/step - loss: 0.6859 - acc: 0.8747 - val_loss: 0.6567 - val_acc: 0.8881 Epoch 468/5000 77s 154ms/step - loss: 0.6849 - acc: 0.8736 - val_loss: 0.6570 - val_acc: 0.8840 Epoch 469/5000 77s 154ms/step - loss: 0.6876 - acc: 0.8738 - val_loss: 0.6499 - val_acc: 0.8893 Epoch 470/5000 77s 154ms/step - loss: 0.6889 - acc: 0.8725 - val_loss: 0.6572 - val_acc: 0.8873 Epoch 471/5000 77s 154ms/step - loss: 0.6887 - acc: 0.8721 - val_loss: 0.6576 - val_acc: 0.8871 Epoch 472/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8749 - val_loss: 0.6491 - val_acc: 0.8910 Epoch 473/5000 77s 153ms/step - loss: 0.6843 - acc: 0.8754 - val_loss: 0.6712 - val_acc: 0.8797 Epoch 474/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8729 - val_loss: 0.6721 - val_acc: 0.8812 Epoch 475/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8742 - val_loss: 0.6723 - val_acc: 0.8828 Epoch 476/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8731 - val_loss: 0.6330 - val_acc: 0.8934 Epoch 477/5000 77s 153ms/step - loss: 0.6854 - acc: 0.8752 - val_loss: 0.6432 - val_acc: 0.8900 Epoch 478/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8754 - val_loss: 0.6474 - val_acc: 0.8915 Epoch 479/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8739 - val_loss: 0.6780 - val_acc: 0.8816 Epoch 480/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8755 - val_loss: 0.6889 - val_acc: 0.8774 Epoch 481/5000 77s 154ms/step - loss: 0.6913 - acc: 0.8726 - val_loss: 0.6380 - val_acc: 0.8928 Epoch 482/5000 77s 154ms/step - loss: 0.6846 - acc: 0.8752 - val_loss: 0.6673 - val_acc: 0.8811 Epoch 483/5000 77s 154ms/step - loss: 0.6879 - acc: 0.8748 - val_loss: 0.6674 - val_acc: 0.8804 Epoch 484/5000 77s 154ms/step - loss: 0.6906 - acc: 0.8737 - val_loss: 0.6809 - val_acc: 0.8786 Epoch 485/5000 77s 154ms/step - loss: 0.6871 - acc: 0.8743 - val_loss: 0.6740 - val_acc: 0.8789 Epoch 486/5000 77s 154ms/step - loss: 0.6876 - acc: 0.8747 - val_loss: 0.6729 - val_acc: 0.8844 Epoch 487/5000 77s 154ms/step - loss: 0.6914 - acc: 0.8741 - val_loss: 0.6650 - val_acc: 0.8830 Epoch 488/5000 77s 154ms/step - loss: 0.6864 - acc: 0.8763 - val_loss: 0.6629 - val_acc: 0.8827 Epoch 489/5000 77s 154ms/step - loss: 0.6844 - acc: 0.8747 - val_loss: 0.6371 - val_acc: 0.8935 Epoch 490/5000 77s 153ms/step - loss: 0.6881 - acc: 0.8728 - val_loss: 0.6862 - val_acc: 0.8786 Epoch 491/5000 77s 153ms/step - loss: 0.6919 - acc: 0.8711 - val_loss: 0.6382 - val_acc: 0.8922 Epoch 492/5000 77s 154ms/step - loss: 0.6858 - acc: 0.8767 - val_loss: 0.6820 - val_acc: 0.8788 Epoch 493/5000 77s 154ms/step - loss: 0.6878 - acc: 0.8746 - val_loss: 0.6815 - val_acc: 0.8766 Epoch 494/5000 77s 154ms/step - loss: 0.6889 - acc: 0.8735 - val_loss: 0.6695 - val_acc: 0.8835 Epoch 495/5000 77s 154ms/step - loss: 0.6822 - acc: 0.8754 - val_loss: 0.6530 - val_acc: 0.8843 Epoch 496/5000 77s 154ms/step - loss: 0.6892 - acc: 0.8729 - val_loss: 0.6570 - val_acc: 0.8874 Epoch 497/5000 77s 154ms/step - loss: 0.6885 - acc: 0.8744 - val_loss: 0.6395 - val_acc: 0.8922 Epoch 498/5000 77s 154ms/step - loss: 0.6862 - acc: 0.8745 - val_loss: 0.6509 - val_acc: 0.8875 Epoch 499/5000 77s 154ms/step - loss: 0.6863 - acc: 0.8745 - val_loss: 0.6554 - val_acc: 0.8890 Epoch 500/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8745 - val_loss: 0.6650 - val_acc: 0.8839 Epoch 501/5000 77s 154ms/step - loss: 0.6805 - acc: 0.8767 - val_loss: 0.6894 - val_acc: 0.8726 Epoch 502/5000 77s 154ms/step - loss: 0.6804 - acc: 0.8767 - val_loss: 0.6739 - val_acc: 0.8831 Epoch 503/5000 77s 154ms/step - loss: 0.6863 - acc: 0.8744 - val_loss: 0.6585 - val_acc: 0.8856 Epoch 504/5000 77s 154ms/step - loss: 0.6836 - acc: 0.8748 - val_loss: 0.6515 - val_acc: 0.8859 Epoch 505/5000 77s 154ms/step - loss: 0.6856 - acc: 0.8731 - val_loss: 0.6709 - val_acc: 0.8807 Epoch 506/5000 77s 154ms/step - loss: 0.6880 - acc: 0.8757 - val_loss: 0.6623 - val_acc: 0.8843 Epoch 507/5000 77s 154ms/step - loss: 0.6860 - acc: 0.8743 - val_loss: 0.6719 - val_acc: 0.8839 Epoch 508/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8740 - val_loss: 0.6653 - val_acc: 0.8870 Epoch 509/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8761 - val_loss: 0.6534 - val_acc: 0.8910 Epoch 510/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8751 - val_loss: 0.6686 - val_acc: 0.8812 Epoch 511/5000 77s 154ms/step - loss: 0.6869 - acc: 0.8756 - val_loss: 0.6690 - val_acc: 0.8810 Epoch 512/5000 77s 154ms/step - loss: 0.6902 - acc: 0.8745 - val_loss: 0.6492 - val_acc: 0.8916 Epoch 513/5000 77s 154ms/step - loss: 0.6870 - acc: 0.8737 - val_loss: 0.6780 - val_acc: 0.8828 Epoch 514/5000 77s 153ms/step - loss: 0.6879 - acc: 0.8737 - val_loss: 0.6663 - val_acc: 0.8829 Epoch 515/5000 77s 154ms/step - loss: 0.6866 - acc: 0.8751 - val_loss: 0.6440 - val_acc: 0.8907 Epoch 516/5000 77s 153ms/step - loss: 0.6885 - acc: 0.8742 - val_loss: 0.6478 - val_acc: 0.8873 Epoch 517/5000 77s 154ms/step - loss: 0.6850 - acc: 0.8748 - val_loss: 0.6541 - val_acc: 0.8885 Epoch 518/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8740 - val_loss: 0.6685 - val_acc: 0.8788 Epoch 519/5000 77s 154ms/step - loss: 0.6906 - acc: 0.8721 - val_loss: 0.6836 - val_acc: 0.8762 Epoch 520/5000 77s 154ms/step - loss: 0.6894 - acc: 0.8730 - val_loss: 0.6596 - val_acc: 0.8839 Epoch 521/5000 77s 154ms/step - loss: 0.6879 - acc: 0.8742 - val_loss: 0.6729 - val_acc: 0.8801 Epoch 522/5000 77s 154ms/step - loss: 0.6874 - acc: 0.8755 - val_loss: 0.6422 - val_acc: 0.8915 Epoch 523/5000 77s 154ms/step - loss: 0.6831 - acc: 0.8772 - val_loss: 0.6386 - val_acc: 0.8902 Epoch 524/5000 77s 154ms/step - loss: 0.6869 - acc: 0.8741 - val_loss: 0.6578 - val_acc: 0.8890 Epoch 525/5000 77s 154ms/step - loss: 0.6882 - acc: 0.8734 - val_loss: 0.6563 - val_acc: 0.8845 Epoch 526/5000 77s 154ms/step - loss: 0.6823 - acc: 0.8762 - val_loss: 0.6540 - val_acc: 0.8893 Epoch 527/5000 77s 154ms/step - loss: 0.6853 - acc: 0.8751 - val_loss: 0.6709 - val_acc: 0.8814 Epoch 528/5000 77s 154ms/step - loss: 0.6831 - acc: 0.8774 - val_loss: 0.6622 - val_acc: 0.8838 Epoch 529/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8770 - val_loss: 0.6478 - val_acc: 0.8889 Epoch 530/5000 77s 154ms/step - loss: 0.6910 - acc: 0.8741 - val_loss: 0.6444 - val_acc: 0.8884 Epoch 531/5000 77s 154ms/step - loss: 0.6854 - acc: 0.8759 - val_loss: 0.6519 - val_acc: 0.8891 Epoch 532/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8752 - val_loss: 0.6509 - val_acc: 0.8904 Epoch 533/5000 77s 154ms/step - loss: 0.6842 - acc: 0.8738 - val_loss: 0.6692 - val_acc: 0.8837 Epoch 534/5000 77s 154ms/step - loss: 0.6834 - acc: 0.8764 - val_loss: 0.6551 - val_acc: 0.8883 Epoch 535/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8750 - val_loss: 0.6708 - val_acc: 0.8809 Epoch 536/5000 77s 154ms/step - loss: 0.6861 - acc: 0.8746 - val_loss: 0.6537 - val_acc: 0.8897 Epoch 537/5000 77s 154ms/step - loss: 0.6815 - acc: 0.8771 - val_loss: 0.6546 - val_acc: 0.8885 Epoch 538/5000 77s 154ms/step - loss: 0.6878 - acc: 0.8752 - val_loss: 0.6486 - val_acc: 0.8917 Epoch 539/5000 77s 154ms/step - loss: 0.6833 - acc: 0.8778 - val_loss: 0.6455 - val_acc: 0.8915 Epoch 540/5000 77s 154ms/step - loss: 0.6849 - acc: 0.8758 - val_loss: 0.6467 - val_acc: 0.8881 Epoch 541/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8751 - val_loss: 0.6578 - val_acc: 0.8845 Epoch 542/5000 77s 154ms/step - loss: 0.6830 - acc: 0.8759 - val_loss: 0.6427 - val_acc: 0.8890 Epoch 543/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8750 - val_loss: 0.6498 - val_acc: 0.8903 Epoch 544/5000 77s 153ms/step - loss: 0.6875 - acc: 0.8736 - val_loss: 0.6620 - val_acc: 0.8841 Epoch 545/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8738 - val_loss: 0.6460 - val_acc: 0.8906 Epoch 546/5000 77s 153ms/step - loss: 0.6855 - acc: 0.8769 - val_loss: 0.6529 - val_acc: 0.8920 Epoch 547/5000 77s 154ms/step - loss: 0.6874 - acc: 0.8749 - val_loss: 0.6537 - val_acc: 0.8911 Epoch 548/5000 77s 154ms/step - loss: 0.6899 - acc: 0.8732 - val_loss: 0.6800 - val_acc: 0.8805 Epoch 549/5000 77s 154ms/step - loss: 0.6891 - acc: 0.8760 - val_loss: 0.6508 - val_acc: 0.8879 Epoch 550/5000 77s 154ms/step - loss: 0.6827 - acc: 0.8766 - val_loss: 0.6610 - val_acc: 0.8843 Epoch 551/5000 77s 154ms/step - loss: 0.6906 - acc: 0.8732 - val_loss: 0.6425 - val_acc: 0.8926 Epoch 552/5000 77s 153ms/step - loss: 0.6815 - acc: 0.8767 - val_loss: 0.6834 - val_acc: 0.8760 Epoch 553/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8740 - val_loss: 0.6426 - val_acc: 0.8912 Epoch 554/5000 77s 154ms/step - loss: 0.6799 - acc: 0.8779 - val_loss: 0.6460 - val_acc: 0.8883 Epoch 555/5000 77s 154ms/step - loss: 0.6837 - acc: 0.8749 - val_loss: 0.6652 - val_acc: 0.8841 Epoch 556/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8739 - val_loss: 0.6418 - val_acc: 0.8925 Epoch 557/5000 77s 154ms/step - loss: 0.6866 - acc: 0.8745 - val_loss: 0.6479 - val_acc: 0.8929 Epoch 558/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8743 - val_loss: 0.6716 - val_acc: 0.8860 Epoch 559/5000 77s 154ms/step - loss: 0.6812 - acc: 0.8758 - val_loss: 0.6542 - val_acc: 0.8885 Epoch 560/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8747 - val_loss: 0.6537 - val_acc: 0.8907 Epoch 561/5000 77s 154ms/step - loss: 0.6864 - acc: 0.8770 - val_loss: 0.6436 - val_acc: 0.8925 Epoch 562/5000 77s 154ms/step - loss: 0.6883 - acc: 0.8750 - val_loss: 0.6498 - val_acc: 0.8896 Epoch 563/5000 77s 154ms/step - loss: 0.6871 - acc: 0.8737 - val_loss: 0.6590 - val_acc: 0.8880 Epoch 564/5000 77s 154ms/step - loss: 0.6820 - acc: 0.8765 - val_loss: 0.6520 - val_acc: 0.8902 Epoch 565/5000 77s 154ms/step - loss: 0.6882 - acc: 0.8753 - val_loss: 0.6522 - val_acc: 0.8904 Epoch 566/5000 77s 154ms/step - loss: 0.6828 - acc: 0.8767 - val_loss: 0.6712 - val_acc: 0.8854 Epoch 567/5000 77s 153ms/step - loss: 0.6862 - acc: 0.8742 - val_loss: 0.6891 - val_acc: 0.8789 Epoch 568/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8757 - val_loss: 0.6508 - val_acc: 0.8916 Epoch 569/5000 77s 154ms/step - loss: 0.6831 - acc: 0.8768 - val_loss: 0.6687 - val_acc: 0.8826 Epoch 570/5000 77s 154ms/step - loss: 0.6897 - acc: 0.8738 - val_loss: 0.6624 - val_acc: 0.8880 Epoch 571/5000 77s 153ms/step - loss: 0.6916 - acc: 0.8730 - val_loss: 0.6588 - val_acc: 0.8866 Epoch 572/5000 77s 154ms/step - loss: 0.6834 - acc: 0.8762 - val_loss: 0.6897 - val_acc: 0.8768 Epoch 573/5000 77s 154ms/step - loss: 0.6827 - acc: 0.8787 - val_loss: 0.6428 - val_acc: 0.8931 Epoch 574/5000 77s 154ms/step - loss: 0.6891 - acc: 0.8751 - val_loss: 0.6741 - val_acc: 0.8827 Epoch 575/5000 77s 154ms/step - loss: 0.6855 - acc: 0.8752 - val_loss: 0.6770 - val_acc: 0.8799 Epoch 576/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8770 - val_loss: 0.6548 - val_acc: 0.8877 Epoch 577/5000 77s 154ms/step - loss: 0.6868 - acc: 0.8756 - val_loss: 0.6592 - val_acc: 0.8907 Epoch 578/5000 77s 153ms/step - loss: 0.6837 - acc: 0.8762 - val_loss: 0.6952 - val_acc: 0.8734 Epoch 579/5000 77s 154ms/step - loss: 0.6907 - acc: 0.8747 - val_loss: 0.6539 - val_acc: 0.8858 Epoch 580/5000 77s 153ms/step - loss: 0.6859 - acc: 0.8766 - val_loss: 0.6604 - val_acc: 0.8856 Epoch 581/5000 77s 154ms/step - loss: 0.6792 - acc: 0.8772 - val_loss: 0.6459 - val_acc: 0.8907 Epoch 582/5000 77s 154ms/step - loss: 0.6826 - acc: 0.8766 - val_loss: 0.6629 - val_acc: 0.8886 Epoch 583/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8748 - val_loss: 0.6689 - val_acc: 0.8842 Epoch 584/5000 77s 154ms/step - loss: 0.6823 - acc: 0.8767 - val_loss: 0.6514 - val_acc: 0.8908 Epoch 585/5000 77s 154ms/step - loss: 0.6822 - acc: 0.8770 - val_loss: 0.6637 - val_acc: 0.8838 Epoch 586/5000 77s 154ms/step - loss: 0.6823 - acc: 0.8775 - val_loss: 0.6616 - val_acc: 0.8854 Epoch 587/5000 77s 154ms/step - loss: 0.6843 - acc: 0.8760 - val_loss: 0.6617 - val_acc: 0.8833 Epoch 588/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8756 - val_loss: 0.6399 - val_acc: 0.8939 Epoch 589/5000 77s 154ms/step - loss: 0.6828 - acc: 0.8775 - val_loss: 0.6497 - val_acc: 0.8886 Epoch 590/5000 77s 154ms/step - loss: 0.6848 - acc: 0.8754 - val_loss: 0.6451 - val_acc: 0.8922 Epoch 591/5000 77s 154ms/step - loss: 0.6892 - acc: 0.8751 - val_loss: 0.6585 - val_acc: 0.8875 Epoch 592/5000 77s 154ms/step - loss: 0.6856 - acc: 0.8765 - val_loss: 0.6616 - val_acc: 0.8853 Epoch 593/5000 77s 154ms/step - loss: 0.6842 - acc: 0.8755 - val_loss: 0.6380 - val_acc: 0.8936 Epoch 594/5000 77s 154ms/step - loss: 0.6850 - acc: 0.8766 - val_loss: 0.6702 - val_acc: 0.8830 Epoch 595/5000 77s 154ms/step - loss: 0.6842 - acc: 0.8752 - val_loss: 0.6391 - val_acc: 0.8935 Epoch 596/5000 77s 154ms/step - loss: 0.6832 - acc: 0.8759 - val_loss: 0.6665 - val_acc: 0.8857 Epoch 597/5000 77s 154ms/step - loss: 0.6822 - acc: 0.8764 - val_loss: 0.6539 - val_acc: 0.8870 Epoch 598/5000 77s 154ms/step - loss: 0.6842 - acc: 0.8767 - val_loss: 0.6744 - val_acc: 0.8814 Epoch 599/5000 77s 153ms/step - loss: 0.6876 - acc: 0.8749 - val_loss: 0.6829 - val_acc: 0.8813 Epoch 600/5000 77s 154ms/step - loss: 0.6835 - acc: 0.8768 - val_loss: 0.6625 - val_acc: 0.8873 Epoch 601/5000 77s 153ms/step - loss: 0.6840 - acc: 0.8774 - val_loss: 0.6809 - val_acc: 0.8794 Epoch 602/5000 77s 154ms/step - loss: 0.6846 - acc: 0.8764 - val_loss: 0.6467 - val_acc: 0.8893 Epoch 603/5000 77s 154ms/step - loss: 0.6808 - acc: 0.8759 - val_loss: 0.6447 - val_acc: 0.8912 Epoch 604/5000 77s 154ms/step - loss: 0.6831 - acc: 0.8756 - val_loss: 0.6614 - val_acc: 0.8851 Epoch 605/5000 77s 154ms/step - loss: 0.6844 - acc: 0.8761 - val_loss: 0.6354 - val_acc: 0.8979 Epoch 606/5000 77s 154ms/step - loss: 0.6824 - acc: 0.8773 - val_loss: 0.6695 - val_acc: 0.8832 Epoch 607/5000 77s 154ms/step - loss: 0.6899 - acc: 0.8751 - val_loss: 0.6583 - val_acc: 0.8843 Epoch 608/5000 77s 154ms/step - loss: 0.6804 - acc: 0.8779 - val_loss: 0.6470 - val_acc: 0.8909 Epoch 609/5000 77s 154ms/step - loss: 0.6865 - acc: 0.8755 - val_loss: 0.6763 - val_acc: 0.8801 Epoch 610/5000 77s 154ms/step - loss: 0.6839 - acc: 0.8756 - val_loss: 0.6519 - val_acc: 0.8881 Epoch 611/5000 77s 154ms/step - loss: 0.6853 - acc: 0.8756 - val_loss: 0.6613 - val_acc: 0.8857 Epoch 612/5000 77s 154ms/step - loss: 0.6806 - acc: 0.8773 - val_loss: 0.6511 - val_acc: 0.8934 Epoch 613/5000 77s 154ms/step - loss: 0.6856 - acc: 0.8760 - val_loss: 0.6386 - val_acc: 0.8942 Epoch 614/5000 77s 154ms/step - loss: 0.6830 - acc: 0.8759 - val_loss: 0.6596 - val_acc: 0.8844 Epoch 615/5000 77s 154ms/step - loss: 0.6812 - acc: 0.8776 - val_loss: 0.6517 - val_acc: 0.8885 Epoch 616/5000 77s 154ms/step - loss: 0.6776 - acc: 0.8775 - val_loss: 0.6609 - val_acc: 0.8885 Epoch 617/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8750 - val_loss: 0.6522 - val_acc: 0.8857 Epoch 618/5000 77s 154ms/step - loss: 0.6830 - acc: 0.8770 - val_loss: 0.6656 - val_acc: 0.8847 Epoch 619/5000 77s 154ms/step - loss: 0.6813 - acc: 0.8759 - val_loss: 0.6539 - val_acc: 0.8874 Epoch 620/5000 77s 154ms/step - loss: 0.6870 - acc: 0.8750 - val_loss: 0.6632 - val_acc: 0.8852 Epoch 621/5000 77s 154ms/step - loss: 0.6853 - acc: 0.8757 - val_loss: 0.6495 - val_acc: 0.8894 Epoch 622/5000 77s 154ms/step - loss: 0.6811 - acc: 0.8757 - val_loss: 0.6762 - val_acc: 0.8799 Epoch 623/5000 77s 154ms/step - loss: 0.6810 - acc: 0.8759 - val_loss: 0.6653 - val_acc: 0.8860 Epoch 624/5000 77s 154ms/step - loss: 0.6814 - acc: 0.8765 - val_loss: 0.6669 - val_acc: 0.8845 Epoch 625/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8760 - val_loss: 0.6505 - val_acc: 0.8888 Epoch 626/5000 77s 154ms/step - loss: 0.6809 - acc: 0.8780 - val_loss: 0.6794 - val_acc: 0.8811 Epoch 627/5000 77s 154ms/step - loss: 0.6793 - acc: 0.8779 - val_loss: 0.6671 - val_acc: 0.8862 Epoch 628/5000 77s 153ms/step - loss: 0.6836 - acc: 0.8756 - val_loss: 0.6555 - val_acc: 0.8853 Epoch 629/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8748 - val_loss: 0.6482 - val_acc: 0.8880 Epoch 630/5000 77s 154ms/step - loss: 0.6838 - acc: 0.8769 - val_loss: 0.6464 - val_acc: 0.8900 Epoch 631/5000 77s 154ms/step - loss: 0.6812 - acc: 0.8774 - val_loss: 0.6706 - val_acc: 0.8830 Epoch 632/5000 77s 154ms/step - loss: 0.6827 - acc: 0.8767 - val_loss: 0.6543 - val_acc: 0.8902 Epoch 633/5000 77s 154ms/step - loss: 0.6812 - acc: 0.8762 - val_loss: 0.6733 - val_acc: 0.8829 Epoch 634/5000 77s 154ms/step - loss: 0.6862 - acc: 0.8747 - val_loss: 0.6691 - val_acc: 0.8824 Epoch 635/5000 77s 153ms/step - loss: 0.6814 - acc: 0.8781 - val_loss: 0.6578 - val_acc: 0.8855 Epoch 636/5000 77s 154ms/step - loss: 0.6767 - acc: 0.8784 - val_loss: 0.6637 - val_acc: 0.8846 Epoch 637/5000 77s 154ms/step - loss: 0.6806 - acc: 0.8763 - val_loss: 0.6529 - val_acc: 0.8884 Epoch 638/5000 77s 154ms/step - loss: 0.6837 - acc: 0.8766 - val_loss: 0.6346 - val_acc: 0.8939 Epoch 639/5000 77s 154ms/step - loss: 0.6836 - acc: 0.8746 - val_loss: 0.6439 - val_acc: 0.8929 Epoch 640/5000 77s 153ms/step - loss: 0.6777 - acc: 0.8765 - val_loss: 0.6639 - val_acc: 0.8832 Epoch 641/5000 77s 154ms/step - loss: 0.6824 - acc: 0.8764 - val_loss: 0.6721 - val_acc: 0.8819 Epoch 642/5000 77s 154ms/step - loss: 0.6856 - acc: 0.8757 - val_loss: 0.6614 - val_acc: 0.8842 Epoch 643/5000 77s 154ms/step - loss: 0.6832 - acc: 0.8766 - val_loss: 0.6578 - val_acc: 0.8866 Epoch 644/5000 77s 154ms/step - loss: 0.6805 - acc: 0.8765 - val_loss: 0.6534 - val_acc: 0.8888 Epoch 645/5000 77s 154ms/step - loss: 0.6833 - acc: 0.8755 - val_loss: 0.6613 - val_acc: 0.8855 Epoch 646/5000 77s 154ms/step - loss: 0.6778 - acc: 0.8777 - val_loss: 0.6597 - val_acc: 0.8863 Epoch 647/5000 77s 154ms/step - loss: 0.6833 - acc: 0.8756 - val_loss: 0.6577 - val_acc: 0.8888 Epoch 648/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8762 - val_loss: 0.6539 - val_acc: 0.8885 Epoch 649/5000 77s 154ms/step - loss: 0.6829 - acc: 0.8768 - val_loss: 0.6575 - val_acc: 0.8875 Epoch 650/5000 77s 154ms/step - loss: 0.6892 - acc: 0.8735 - val_loss: 0.6865 - val_acc: 0.8777 Epoch 651/5000 77s 154ms/step - loss: 0.6776 - acc: 0.8790 - val_loss: 0.6533 - val_acc: 0.8857 Epoch 652/5000 77s 154ms/step - loss: 0.6860 - acc: 0.8752 - val_loss: 0.6643 - val_acc: 0.8880 Epoch 653/5000 77s 154ms/step - loss: 0.6848 - acc: 0.8749 - val_loss: 0.6565 - val_acc: 0.8882 Epoch 654/5000 77s 154ms/step - loss: 0.6811 - acc: 0.8774 - val_loss: 0.6638 - val_acc: 0.8852 Epoch 655/5000 77s 154ms/step - loss: 0.6781 - acc: 0.8780 - val_loss: 0.6566 - val_acc: 0.8894 Epoch 656/5000 77s 154ms/step - loss: 0.6818 - acc: 0.8764 - val_loss: 0.6804 - val_acc: 0.8822 Epoch 657/5000 77s 154ms/step - loss: 0.6834 - acc: 0.8766 - val_loss: 0.6487 - val_acc: 0.8894 Epoch 658/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8747 - val_loss: 0.6576 - val_acc: 0.8872 Epoch 659/5000 77s 154ms/step - loss: 0.6823 - acc: 0.8772 - val_loss: 0.6532 - val_acc: 0.8875 Epoch 660/5000 77s 154ms/step - loss: 0.6814 - acc: 0.8769 - val_loss: 0.6576 - val_acc: 0.8865 Epoch 661/5000 77s 154ms/step - loss: 0.6831 - acc: 0.8776 - val_loss: 0.6648 - val_acc: 0.8811 Epoch 662/5000 77s 154ms/step - loss: 0.6822 - acc: 0.8758 - val_loss: 0.6648 - val_acc: 0.8863 Epoch 663/5000 77s 154ms/step - loss: 0.6912 - acc: 0.8741 - val_loss: 0.6724 - val_acc: 0.8799 Epoch 664/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8753 - val_loss: 0.6441 - val_acc: 0.8906 Epoch 665/5000 77s 154ms/step - loss: 0.6847 - acc: 0.8755 - val_loss: 0.6666 - val_acc: 0.8847 Epoch 666/5000 77s 154ms/step - loss: 0.6803 - acc: 0.8782 - val_loss: 0.6387 - val_acc: 0.8931 Epoch 667/5000 77s 154ms/step - loss: 0.6838 - acc: 0.8767 - val_loss: 0.6705 - val_acc: 0.8837 Epoch 668/5000 77s 154ms/step - loss: 0.6882 - acc: 0.8747 - val_loss: 0.6715 - val_acc: 0.8845 Epoch 669/5000 77s 154ms/step - loss: 0.6791 - acc: 0.8779 - val_loss: 0.6596 - val_acc: 0.8866 Epoch 670/5000 77s 154ms/step - loss: 0.6838 - acc: 0.8769 - val_loss: 0.6623 - val_acc: 0.8865 Epoch 671/5000 77s 154ms/step - loss: 0.6776 - acc: 0.8773 - val_loss: 0.6627 - val_acc: 0.8866 Epoch 672/5000 77s 154ms/step - loss: 0.6815 - acc: 0.8779 - val_loss: 0.6596 - val_acc: 0.8901 Epoch 673/5000 77s 154ms/step - loss: 0.6819 - acc: 0.8766 - val_loss: 0.6718 - val_acc: 0.8856 Epoch 674/5000 77s 154ms/step - loss: 0.6836 - acc: 0.8764 - val_loss: 0.6477 - val_acc: 0.8911 Epoch 675/5000 77s 154ms/step - loss: 0.6816 - acc: 0.8783 - val_loss: 0.6656 - val_acc: 0.8874 Epoch 676/5000 77s 154ms/step - loss: 0.6835 - acc: 0.8786 - val_loss: 0.6672 - val_acc: 0.8852 Epoch 677/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8768 - val_loss: 0.6772 - val_acc: 0.8818 Epoch 678/5000 77s 154ms/step - loss: 0.6828 - acc: 0.8784 - val_loss: 0.6708 - val_acc: 0.8828 Epoch 679/5000 77s 153ms/step - loss: 0.6812 - acc: 0.8779 - val_loss: 0.6756 - val_acc: 0.8799 Epoch 680/5000 77s 154ms/step - loss: 0.6825 - acc: 0.8766 - val_loss: 0.6512 - val_acc: 0.8882 Epoch 681/5000 77s 153ms/step - loss: 0.6803 - acc: 0.8769 - val_loss: 0.6505 - val_acc: 0.8896 Epoch 682/5000 77s 154ms/step - loss: 0.6829 - acc: 0.8768 - val_loss: 0.6531 - val_acc: 0.8841 Epoch 683/5000 77s 154ms/step - loss: 0.6834 - acc: 0.8759 - val_loss: 0.6509 - val_acc: 0.8884 Epoch 684/5000 77s 154ms/step - loss: 0.6832 - acc: 0.8780 - val_loss: 0.6385 - val_acc: 0.8940 Epoch 685/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8769 - val_loss: 0.6699 - val_acc: 0.8840 Epoch 686/5000 77s 154ms/step - loss: 0.6790 - acc: 0.8785 - val_loss: 0.6789 - val_acc: 0.8772 Epoch 687/5000 77s 154ms/step - loss: 0.6809 - acc: 0.8766 - val_loss: 0.6421 - val_acc: 0.8935 Epoch 688/5000 77s 153ms/step - loss: 0.6797 - acc: 0.8783 - val_loss: 0.6627 - val_acc: 0.8870 Epoch 689/5000 77s 154ms/step - loss: 0.6870 - acc: 0.8772 - val_loss: 0.6521 - val_acc: 0.8903 Epoch 690/5000 77s 154ms/step - loss: 0.6787 - acc: 0.8784 - val_loss: 0.6540 - val_acc: 0.8898 Epoch 691/5000 77s 154ms/step - loss: 0.6826 - acc: 0.8769 - val_loss: 0.6506 - val_acc: 0.8897 Epoch 692/5000 77s 153ms/step - loss: 0.6843 - acc: 0.8767 - val_loss: 0.6663 - val_acc: 0.8815 Epoch 693/5000 77s 154ms/step - loss: 0.6844 - acc: 0.8769 - val_loss: 0.6549 - val_acc: 0.8897 Epoch 694/5000 77s 154ms/step - loss: 0.6817 - acc: 0.8774 - val_loss: 0.6547 - val_acc: 0.8894 Epoch 695/5000 77s 153ms/step - loss: 0.6850 - acc: 0.8753 - val_loss: 0.6532 - val_acc: 0.8897 Epoch 696/5000 77s 154ms/step - loss: 0.6781 - acc: 0.8787 - val_loss: 0.6379 - val_acc: 0.8958 Epoch 697/5000 77s 154ms/step - loss: 0.6797 - acc: 0.8784 - val_loss: 0.6609 - val_acc: 0.8869 Epoch 698/5000 77s 154ms/step - loss: 0.6886 - acc: 0.8741 - val_loss: 0.6434 - val_acc: 0.8932 Epoch 699/5000 77s 154ms/step - loss: 0.6795 - acc: 0.8790 - val_loss: 0.6545 - val_acc: 0.8889 Epoch 700/5000 77s 154ms/step - loss: 0.6818 - acc: 0.8773 - val_loss: 0.6663 - val_acc: 0.8834 Epoch 701/5000 77s 154ms/step - loss: 0.6857 - acc: 0.8775 - val_loss: 0.6370 - val_acc: 0.8934 Epoch 702/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8775 - val_loss: 0.6452 - val_acc: 0.8928 Epoch 703/5000 77s 154ms/step - loss: 0.6810 - acc: 0.8767 - val_loss: 0.6575 - val_acc: 0.8869 Epoch 704/5000 77s 154ms/step - loss: 0.6809 - acc: 0.8789 - val_loss: 0.6608 - val_acc: 0.8849 Epoch 705/5000 77s 153ms/step - loss: 0.6819 - acc: 0.8765 - val_loss: 0.6537 - val_acc: 0.8892 Epoch 706/5000 77s 154ms/step - loss: 0.6882 - acc: 0.8763 - val_loss: 0.6601 - val_acc: 0.8859 Epoch 707/5000 77s 154ms/step - loss: 0.6836 - acc: 0.8762 - val_loss: 0.6612 - val_acc: 0.8854 Epoch 708/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8771 - val_loss: 0.6675 - val_acc: 0.8843 Epoch 709/5000 77s 154ms/step - loss: 0.6875 - acc: 0.8745 - val_loss: 0.6473 - val_acc: 0.8926 Epoch 710/5000 77s 154ms/step - loss: 0.6820 - acc: 0.8781 - val_loss: 0.6590 - val_acc: 0.8875 Epoch 711/5000 77s 154ms/step - loss: 0.6790 - acc: 0.8798 - val_loss: 0.6470 - val_acc: 0.8877 Epoch 712/5000 77s 154ms/step - loss: 0.6789 - acc: 0.8774 - val_loss: 0.6745 - val_acc: 0.8832 Epoch 713/5000 77s 154ms/step - loss: 0.6833 - acc: 0.8761 - val_loss: 0.6637 - val_acc: 0.8839 Epoch 714/5000 77s 154ms/step - loss: 0.6774 - acc: 0.8790 - val_loss: 0.6672 - val_acc: 0.8818 Epoch 715/5000 77s 154ms/step - loss: 0.6808 - acc: 0.8781 - val_loss: 0.6399 - val_acc: 0.8952 Epoch 716/5000 77s 154ms/step - loss: 0.6845 - acc: 0.8766 - val_loss: 0.6635 - val_acc: 0.8836 Epoch 717/5000 77s 154ms/step - loss: 0.6810 - acc: 0.8782 - val_loss: 0.6382 - val_acc: 0.8934 Epoch 718/5000 77s 154ms/step - loss: 0.6801 - acc: 0.8783 - val_loss: 0.6439 - val_acc: 0.8950 Epoch 719/5000 77s 154ms/step - loss: 0.6800 - acc: 0.8797 - val_loss: 0.6493 - val_acc: 0.8909 Epoch 720/5000 77s 154ms/step - loss: 0.6856 - acc: 0.8763 - val_loss: 0.6469 - val_acc: 0.8922 Epoch 721/5000 77s 154ms/step - loss: 0.6836 - acc: 0.8763 - val_loss: 0.6656 - val_acc: 0.8876 Epoch 722/5000 77s 153ms/step - loss: 0.6830 - acc: 0.8782 - val_loss: 0.6732 - val_acc: 0.8826 Epoch 723/5000 77s 154ms/step - loss: 0.6845 - acc: 0.8765 - val_loss: 0.6785 - val_acc: 0.8832 Epoch 724/5000 77s 154ms/step - loss: 0.6820 - acc: 0.8767 - val_loss: 0.6567 - val_acc: 0.8892 Epoch 725/5000 77s 154ms/step - loss: 0.6851 - acc: 0.8769 - val_loss: 0.6530 - val_acc: 0.8895 Epoch 726/5000 77s 154ms/step - loss: 0.6806 - acc: 0.8784 - val_loss: 0.6473 - val_acc: 0.8929 Epoch 727/5000 77s 153ms/step - loss: 0.6786 - acc: 0.8796 - val_loss: 0.6540 - val_acc: 0.8894 Epoch 728/5000 77s 154ms/step - loss: 0.6790 - acc: 0.8780 - val_loss: 0.6413 - val_acc: 0.8925 Epoch 729/5000 77s 154ms/step - loss: 0.6811 - acc: 0.8781 - val_loss: 0.6519 - val_acc: 0.8885 Epoch 730/5000 77s 154ms/step - loss: 0.6837 - acc: 0.8769 - val_loss: 0.6731 - val_acc: 0.8800 Epoch 731/5000 77s 154ms/step - loss: 0.6895 - acc: 0.8746 - val_loss: 0.6662 - val_acc: 0.8863 Epoch 732/5000 77s 154ms/step - loss: 0.6798 - acc: 0.8784 - val_loss: 0.6768 - val_acc: 0.8820 Epoch 733/5000 77s 154ms/step - loss: 0.6816 - acc: 0.8793 - val_loss: 0.6514 - val_acc: 0.8919 Epoch 734/5000 77s 154ms/step - loss: 0.6867 - acc: 0.8767 - val_loss: 0.6460 - val_acc: 0.8928 Epoch 735/5000 77s 154ms/step - loss: 0.6802 - acc: 0.8799 - val_loss: 0.6523 - val_acc: 0.8875 Epoch 736/5000 77s 153ms/step - loss: 0.6807 - acc: 0.8794 - val_loss: 0.6487 - val_acc: 0.8873 Epoch 737/5000 77s 153ms/step - loss: 0.6811 - acc: 0.8774 - val_loss: 0.6670 - val_acc: 0.8856 Epoch 738/5000 77s 154ms/step - loss: 0.6821 - acc: 0.8763 - val_loss: 0.6545 - val_acc: 0.8863 Epoch 739/5000 77s 154ms/step - loss: 0.6877 - acc: 0.8743 - val_loss: 0.6712 - val_acc: 0.8833 Epoch 740/5000 77s 154ms/step - loss: 0.6763 - acc: 0.8809 - val_loss: 0.6697 - val_acc: 0.8837 Epoch 741/5000 77s 154ms/step - loss: 0.6873 - acc: 0.8766 - val_loss: 0.6560 - val_acc: 0.8911 Epoch 742/5000 77s 154ms/step - loss: 0.6803 - acc: 0.8771 - val_loss: 0.6483 - val_acc: 0.8907 Epoch 743/5000 77s 153ms/step - loss: 0.6794 - acc: 0.8782 - val_loss: 0.6755 - val_acc: 0.8816 Epoch 744/5000 77s 154ms/step - loss: 0.6835 - acc: 0.8758 - val_loss: 0.6565 - val_acc: 0.8884 Epoch 745/5000 77s 154ms/step - loss: 0.6842 - acc: 0.8782 - val_loss: 0.6857 - val_acc: 0.8824 Epoch 746/5000 77s 154ms/step - loss: 0.6768 - acc: 0.8800 - val_loss: 0.6802 - val_acc: 0.8786 Epoch 747/5000 77s 154ms/step - loss: 0.6825 - acc: 0.8771 - val_loss: 0.6430 - val_acc: 0.8961 Epoch 748/5000 77s 154ms/step - loss: 0.6787 - acc: 0.8800 - val_loss: 0.6699 - val_acc: 0.8807 Epoch 749/5000 77s 154ms/step - loss: 0.6822 - acc: 0.8767 - val_loss: 0.6622 - val_acc: 0.8885 Epoch 750/5000 77s 153ms/step - loss: 0.6758 - acc: 0.8808 - val_loss: 0.6511 - val_acc: 0.8901 Epoch 751/5000 77s 154ms/step - loss: 0.6827 - acc: 0.8769 - val_loss: 0.6777 - val_acc: 0.8799 Epoch 752/5000 77s 154ms/step - loss: 0.6855 - acc: 0.8775 - val_loss: 0.6472 - val_acc: 0.8924 Epoch 753/5000 77s 153ms/step - loss: 0.6783 - acc: 0.8784 - val_loss: 0.6535 - val_acc: 0.8919 Epoch 754/5000 77s 154ms/step - loss: 0.6796 - acc: 0.8792 - val_loss: 0.6624 - val_acc: 0.8861 Epoch 755/5000 77s 153ms/step - loss: 0.6798 - acc: 0.8788 - val_loss: 0.6661 - val_acc: 0.8880 Epoch 756/5000 77s 154ms/step - loss: 0.6814 - acc: 0.8766 - val_loss: 0.6466 - val_acc: 0.8930 Epoch 757/5000 77s 154ms/step - loss: 0.6808 - acc: 0.8775 - val_loss: 0.6528 - val_acc: 0.8890 Epoch 758/5000 77s 154ms/step - loss: 0.6818 - acc: 0.8782 - val_loss: 0.6547 - val_acc: 0.8883 Epoch 759/5000 77s 154ms/step - loss: 0.6806 - acc: 0.8788 - val_loss: 0.6509 - val_acc: 0.8889 Epoch 760/5000 77s 154ms/step - loss: 0.6782 - acc: 0.8786 - val_loss: 0.6486 - val_acc: 0.8865 Epoch 761/5000 77s 154ms/step - loss: 0.6808 - acc: 0.8785 - val_loss: 0.6484 - val_acc: 0.8888 Epoch 762/5000 77s 154ms/step - loss: 0.6802 - acc: 0.8775 - val_loss: 0.6838 - val_acc: 0.8819 Epoch 763/5000 77s 154ms/step - loss: 0.6797 - acc: 0.8776 - val_loss: 0.6411 - val_acc: 0.8942 Epoch 764/5000 77s 154ms/step - loss: 0.6811 - acc: 0.8762 - val_loss: 0.6767 - val_acc: 0.8821 Epoch 765/5000 77s 154ms/step - loss: 0.6790 - acc: 0.8778 - val_loss: 0.6430 - val_acc: 0.8942 Epoch 766/5000 77s 154ms/step - loss: 0.6819 - acc: 0.8780 - val_loss: 0.6648 - val_acc: 0.8846 Epoch 767/5000 77s 154ms/step - loss: 0.6843 - acc: 0.8786 - val_loss: 0.6458 - val_acc: 0.8936 ... Epoch 1491/5000 77s 154ms/step - loss: 0.6771 - acc: 0.8807 - val_loss: 0.6669 - val_acc: 0.8886 Epoch 1492/5000 77s 154ms/step - loss: 0.6735 - acc: 0.8827 - val_loss: 0.6475 - val_acc: 0.8920 Epoch 1493/5000 77s 154ms/step - loss: 0.6773 - acc: 0.8814 - val_loss: 0.6660 - val_acc: 0.8891 Epoch 1494/5000 77s 154ms/step - loss: 0.6807 - acc: 0.8792 - val_loss: 0.6478 - val_acc: 0.8925 Epoch 1495/5000 77s 154ms/step - loss: 0.6784 - acc: 0.8819 - val_loss: 0.6947 - val_acc: 0.8754 Epoch 1496/5000 77s 154ms/step - loss: 0.6815 - acc: 0.8802 - val_loss: 0.6643 - val_acc: 0.8882 Epoch 1497/5000 77s 154ms/step - loss: 0.6844 - acc: 0.8787 - val_loss: 0.6254 - val_acc: 0.9019 Epoch 1498/5000 77s 154ms/step - loss: 0.6783 - acc: 0.8815 - val_loss: 0.6634 - val_acc: 0.8876 Epoch 1499/5000 77s 154ms/step - loss: 0.6787 - acc: 0.8800 - val_loss: 0.6708 - val_acc: 0.8862 Epoch 1500/5000 77s 154ms/step - loss: 0.6815 - acc: 0.8810 - val_loss: 0.6554 - val_acc: 0.8919 Epoch 1501/5000 lr changed to 0.010000000149011612 77s 154ms/step - loss: 0.5717 - acc: 0.9188 - val_loss: 0.5618 - val_acc: 0.9242 Epoch 1502/5000 77s 154ms/step - loss: 0.5116 - acc: 0.9373 - val_loss: 0.5485 - val_acc: 0.9261 Epoch 1503/5000 77s 154ms/step - loss: 0.4922 - acc: 0.9420 - val_loss: 0.5352 - val_acc: 0.9278 Epoch 1504/5000 77s 154ms/step - loss: 0.4747 - acc: 0.9466 - val_loss: 0.5291 - val_acc: 0.9268 Epoch 1505/5000 77s 154ms/step - loss: 0.4657 - acc: 0.9481 - val_loss: 0.5215 - val_acc: 0.9288 Epoch 1506/5000 77s 154ms/step - loss: 0.4528 - acc: 0.9497 - val_loss: 0.5122 - val_acc: 0.9292 Epoch 1507/5000 77s 154ms/step - loss: 0.4453 - acc: 0.9513 - val_loss: 0.5116 - val_acc: 0.9304 Epoch 1508/5000 77s 154ms/step - loss: 0.4360 - acc: 0.9530 - val_loss: 0.5029 - val_acc: 0.9325 Epoch 1509/5000 77s 154ms/step - loss: 0.4239 - acc: 0.9547 - val_loss: 0.5028 - val_acc: 0.9297 Epoch 1510/5000 77s 154ms/step - loss: 0.4195 - acc: 0.9549 - val_loss: 0.4906 - val_acc: 0.9322 Epoch 1511/5000 77s 154ms/step - loss: 0.4109 - acc: 0.9565 - val_loss: 0.4917 - val_acc: 0.9308 Epoch 1512/5000 77s 154ms/step - loss: 0.4071 - acc: 0.9561 - val_loss: 0.4811 - val_acc: 0.9324 Epoch 1513/5000 77s 154ms/step - loss: 0.3979 - acc: 0.9580 - val_loss: 0.4752 - val_acc: 0.9348 Epoch 1514/5000 77s 154ms/step - loss: 0.3896 - acc: 0.9590 - val_loss: 0.4839 - val_acc: 0.9303 Epoch 1515/5000 77s 154ms/step - loss: 0.3846 - acc: 0.9599 - val_loss: 0.4811 - val_acc: 0.9307 Epoch 1516/5000 77s 153ms/step - loss: 0.3815 - acc: 0.9592 - val_loss: 0.4753 - val_acc: 0.9317 Epoch 1517/5000 77s 154ms/step - loss: 0.3730 - acc: 0.9610 - val_loss: 0.4688 - val_acc: 0.9302 Epoch 1518/5000 77s 154ms/step - loss: 0.3661 - acc: 0.9624 - val_loss: 0.4653 - val_acc: 0.9324 Epoch 1519/5000 77s 154ms/step - loss: 0.3621 - acc: 0.9618 - val_loss: 0.4602 - val_acc: 0.9334 Epoch 1520/5000 77s 154ms/step - loss: 0.3581 - acc: 0.9636 - val_loss: 0.4604 - val_acc: 0.9310 Epoch 1521/5000 77s 154ms/step - loss: 0.3542 - acc: 0.9622 - val_loss: 0.4598 - val_acc: 0.9299 Epoch 1522/5000 77s 154ms/step - loss: 0.3498 - acc: 0.9632 - val_loss: 0.4564 - val_acc: 0.9307 Epoch 1523/5000 77s 153ms/step - loss: 0.3445 - acc: 0.9631 - val_loss: 0.4461 - val_acc: 0.9310 Epoch 1524/5000 77s 154ms/step - loss: 0.3386 - acc: 0.9641 - val_loss: 0.4460 - val_acc: 0.9319 Epoch 1525/5000 77s 154ms/step - loss: 0.3348 - acc: 0.9636 - val_loss: 0.4432 - val_acc: 0.9323 Epoch 1526/5000 77s 154ms/step - loss: 0.3320 - acc: 0.9652 - val_loss: 0.4571 - val_acc: 0.9249 Epoch 1527/5000 77s 154ms/step - loss: 0.3278 - acc: 0.9650 - val_loss: 0.4385 - val_acc: 0.9324 Epoch 1528/5000 77s 154ms/step - loss: 0.3279 - acc: 0.9639 - val_loss: 0.4325 - val_acc: 0.9330 Epoch 1529/5000 77s 154ms/step - loss: 0.3228 - acc: 0.9647 - val_loss: 0.4303 - val_acc: 0.9326 Epoch 1530/5000 77s 154ms/step - loss: 0.3179 - acc: 0.9658 - val_loss: 0.4323 - val_acc: 0.9314 Epoch 1531/5000 77s 154ms/step - loss: 0.3145 - acc: 0.9658 - val_loss: 0.4339 - val_acc: 0.9307 Epoch 1532/5000 77s 154ms/step - loss: 0.3142 - acc: 0.9658 - val_loss: 0.4335 - val_acc: 0.9287 Epoch 1533/5000 77s 154ms/step - loss: 0.3100 - acc: 0.9661 - val_loss: 0.4267 - val_acc: 0.9305 Epoch 1534/5000 77s 154ms/step - loss: 0.3067 - acc: 0.9660 - val_loss: 0.4233 - val_acc: 0.9339 Epoch 1535/5000 77s 154ms/step - loss: 0.3067 - acc: 0.9651 - val_loss: 0.4267 - val_acc: 0.9280 Epoch 1536/5000 77s 154ms/step - loss: 0.3007 - acc: 0.9660 - val_loss: 0.4275 - val_acc: 0.9288 Epoch 1537/5000 77s 154ms/step - loss: 0.3011 - acc: 0.9663 - val_loss: 0.4170 - val_acc: 0.9306 Epoch 1538/5000 77s 154ms/step - loss: 0.2959 - acc: 0.9666 - val_loss: 0.4172 - val_acc: 0.9331 Epoch 1539/5000 77s 154ms/step - loss: 0.2929 - acc: 0.9671 - val_loss: 0.4187 - val_acc: 0.9317 Epoch 1540/5000 77s 154ms/step - loss: 0.2908 - acc: 0.9676 - val_loss: 0.4157 - val_acc: 0.9299 Epoch 1541/5000 77s 154ms/step - loss: 0.2892 - acc: 0.9670 - val_loss: 0.4234 - val_acc: 0.9252 Epoch 1542/5000 77s 154ms/step - loss: 0.2905 - acc: 0.9659 - val_loss: 0.4111 - val_acc: 0.9299 Epoch 1543/5000 77s 154ms/step - loss: 0.2855 - acc: 0.9674 - val_loss: 0.4094 - val_acc: 0.9291 Epoch 1544/5000 77s 154ms/step - loss: 0.2842 - acc: 0.9673 - val_loss: 0.4163 - val_acc: 0.9286 Epoch 1545/5000 77s 154ms/step - loss: 0.2817 - acc: 0.9670 - val_loss: 0.4068 - val_acc: 0.9331 Epoch 1546/5000 77s 154ms/step - loss: 0.2843 - acc: 0.9661 - val_loss: 0.4202 - val_acc: 0.9255 Epoch 1547/5000 77s 154ms/step - loss: 0.2851 - acc: 0.9649 - val_loss: 0.4121 - val_acc: 0.9295 Epoch 1548/5000 77s 154ms/step - loss: 0.2782 - acc: 0.9665 - val_loss: 0.4117 - val_acc: 0.9308 Epoch 1549/5000 77s 154ms/step - loss: 0.2766 - acc: 0.9671 - val_loss: 0.4029 - val_acc: 0.9302 Epoch 1550/5000 77s 154ms/step - loss: 0.2736 - acc: 0.9679 - val_loss: 0.4064 - val_acc: 0.9299 Epoch 1551/5000 77s 154ms/step - loss: 0.2745 - acc: 0.9663 - val_loss: 0.3985 - val_acc: 0.9300 Epoch 1552/5000 77s 154ms/step - loss: 0.2745 - acc: 0.9661 - val_loss: 0.4098 - val_acc: 0.9250 Epoch 1553/5000 77s 154ms/step - loss: 0.2717 - acc: 0.9662 - val_loss: 0.4028 - val_acc: 0.9292 Epoch 1554/5000 77s 154ms/step - loss: 0.2730 - acc: 0.9657 - val_loss: 0.4019 - val_acc: 0.9276 Epoch 1555/5000 77s 154ms/step - loss: 0.2751 - acc: 0.9645 - val_loss: 0.3969 - val_acc: 0.9287 Epoch 1556/5000 77s 154ms/step - loss: 0.2708 - acc: 0.9657 - val_loss: 0.4027 - val_acc: 0.9271 Epoch 1557/5000 77s 154ms/step - loss: 0.2670 - acc: 0.9669 - val_loss: 0.4042 - val_acc: 0.9275 Epoch 1558/5000 77s 154ms/step - loss: 0.2662 - acc: 0.9669 - val_loss: 0.4038 - val_acc: 0.9258 Epoch 1559/5000 77s 154ms/step - loss: 0.2653 - acc: 0.9661 - val_loss: 0.4098 - val_acc: 0.9251 Epoch 1560/5000 77s 154ms/step - loss: 0.2664 - acc: 0.9657 - val_loss: 0.3950 - val_acc: 0.9269 Epoch 1561/5000 77s 154ms/step - loss: 0.2614 - acc: 0.9675 - val_loss: 0.4088 - val_acc: 0.9232 Epoch 1562/5000 77s 153ms/step - loss: 0.2619 - acc: 0.9669 - val_loss: 0.4005 - val_acc: 0.9274 Epoch 1563/5000 77s 154ms/step - loss: 0.2619 - acc: 0.9666 - val_loss: 0.4021 - val_acc: 0.9257 Epoch 1564/5000 77s 154ms/step - loss: 0.2606 - acc: 0.9669 - val_loss: 0.3961 - val_acc: 0.9282 Epoch 1565/5000 77s 154ms/step - loss: 0.2597 - acc: 0.9664 - val_loss: 0.4045 - val_acc: 0.9229 Epoch 1566/5000 77s 154ms/step - loss: 0.2620 - acc: 0.9650 - val_loss: 0.3903 - val_acc: 0.9284 Epoch 1567/5000 77s 154ms/step - loss: 0.2634 - acc: 0.9654 - val_loss: 0.3865 - val_acc: 0.9297 Epoch 1568/5000 77s 154ms/step - loss: 0.2564 - acc: 0.9668 - val_loss: 0.3962 - val_acc: 0.9276 Epoch 1569/5000 77s 154ms/step - loss: 0.2559 - acc: 0.9662 - val_loss: 0.4041 - val_acc: 0.9266 Epoch 1570/5000 77s 153ms/step - loss: 0.2581 - acc: 0.9656 - val_loss: 0.4054 - val_acc: 0.9238 Epoch 1571/5000 77s 154ms/step - loss: 0.2569 - acc: 0.9659 - val_loss: 0.3901 - val_acc: 0.9273 Epoch 1572/5000 77s 154ms/step - loss: 0.2618 - acc: 0.9639 - val_loss: 0.3958 - val_acc: 0.9262 Epoch 1573/5000 77s 154ms/step - loss: 0.2550 - acc: 0.9667 - val_loss: 0.3916 - val_acc: 0.9262 Epoch 1574/5000 77s 154ms/step - loss: 0.2564 - acc: 0.9659 - val_loss: 0.3992 - val_acc: 0.9261 Epoch 1575/5000 77s 154ms/step - loss: 0.2591 - acc: 0.9641 - val_loss: 0.3978 - val_acc: 0.9257 Epoch 1576/5000 77s 154ms/step - loss: 0.2535 - acc: 0.9654 - val_loss: 0.3850 - val_acc: 0.9286 Epoch 1577/5000 77s 154ms/step - loss: 0.2527 - acc: 0.9656 - val_loss: 0.3918 - val_acc: 0.9251 Epoch 1578/5000 77s 154ms/step - loss: 0.2522 - acc: 0.9673 - val_loss: 0.3841 - val_acc: 0.9280 Epoch 1579/5000 77s 154ms/step - loss: 0.2573 - acc: 0.9644 - val_loss: 0.3839 - val_acc: 0.9285 Epoch 1580/5000 77s 153ms/step - loss: 0.2524 - acc: 0.9664 - val_loss: 0.3796 - val_acc: 0.9299 Epoch 1581/5000 77s 154ms/step - loss: 0.2532 - acc: 0.9648 - val_loss: 0.3851 - val_acc: 0.9284 Epoch 1582/5000 77s 154ms/step - loss: 0.2485 - acc: 0.9678 - val_loss: 0.3811 - val_acc: 0.9280 Epoch 1583/5000 77s 154ms/step - loss: 0.2525 - acc: 0.9657 - val_loss: 0.3967 - val_acc: 0.9230 Epoch 1584/5000 77s 154ms/step - loss: 0.2519 - acc: 0.9658 - val_loss: 0.3781 - val_acc: 0.9285 Epoch 1585/5000 77s 154ms/step - loss: 0.2516 - acc: 0.9656 - val_loss: 0.3820 - val_acc: 0.9263 Epoch 1586/5000 77s 154ms/step - loss: 0.2551 - acc: 0.9645 - val_loss: 0.3922 - val_acc: 0.9266 Epoch 1587/5000 77s 154ms/step - loss: 0.2503 - acc: 0.9650 - val_loss: 0.3849 - val_acc: 0.9250 Epoch 1588/5000 77s 153ms/step - loss: 0.2543 - acc: 0.9640 - val_loss: 0.3732 - val_acc: 0.9298 Epoch 1589/5000 77s 154ms/step - loss: 0.2514 - acc: 0.9654 - val_loss: 0.3804 - val_acc: 0.9302 Epoch 1590/5000 77s 154ms/step - loss: 0.2528 - acc: 0.9651 - val_loss: 0.3872 - val_acc: 0.9252 Epoch 1591/5000 77s 154ms/step - loss: 0.2501 - acc: 0.9657 - val_loss: 0.3836 - val_acc: 0.9272 Epoch 1592/5000 77s 154ms/step - loss: 0.2508 - acc: 0.9652 - val_loss: 0.4037 - val_acc: 0.9209 Epoch 1593/5000 77s 154ms/step - loss: 0.2498 - acc: 0.9651 - val_loss: 0.3844 - val_acc: 0.9262 Epoch 1594/5000 77s 154ms/step - loss: 0.2532 - acc: 0.9637 - val_loss: 0.3932 - val_acc: 0.9251 Epoch 1595/5000 77s 154ms/step - loss: 0.2453 - acc: 0.9673 - val_loss: 0.3754 - val_acc: 0.9283 Epoch 1596/5000 77s 154ms/step - loss: 0.2484 - acc: 0.9652 - val_loss: 0.3797 - val_acc: 0.9283 Epoch 1597/5000 77s 153ms/step - loss: 0.2418 - acc: 0.9672 - val_loss: 0.3826 - val_acc: 0.9272 Epoch 1598/5000 77s 154ms/step - loss: 0.2513 - acc: 0.9640 - val_loss: 0.3885 - val_acc: 0.9248 Epoch 1599/5000 77s 154ms/step - loss: 0.2538 - acc: 0.9636 - val_loss: 0.3797 - val_acc: 0.9250 Epoch 1600/5000 77s 154ms/step - loss: 0.2552 - acc: 0.9623 - val_loss: 0.3814 - val_acc: 0.9288 ... Epoch 2930/5000 77s 154ms/step - loss: 0.2255 - acc: 0.9725 - val_loss: 0.4132 - val_acc: 0.9188 Epoch 2931/5000 77s 154ms/step - loss: 0.2247 - acc: 0.9720 - val_loss: 0.4201 - val_acc: 0.9200 Epoch 2932/5000 77s 154ms/step - loss: 0.2271 - acc: 0.9725 - val_loss: 0.4074 - val_acc: 0.9228 Epoch 2933/5000 77s 154ms/step - loss: 0.2280 - acc: 0.9711 - val_loss: 0.4094 - val_acc: 0.9220 Epoch 2934/5000 77s 154ms/step - loss: 0.2290 - acc: 0.9709 - val_loss: 0.4011 - val_acc: 0.9232 Epoch 2935/5000 77s 154ms/step - loss: 0.2266 - acc: 0.9717 - val_loss: 0.4038 - val_acc: 0.9212 Epoch 2936/5000 77s 154ms/step - loss: 0.2253 - acc: 0.9726 - val_loss: 0.4277 - val_acc: 0.9151 Epoch 2937/5000 77s 154ms/step - loss: 0.2256 - acc: 0.9728 - val_loss: 0.4134 - val_acc: 0.9203 Epoch 2938/5000 77s 154ms/step - loss: 0.2254 - acc: 0.9726 - val_loss: 0.4020 - val_acc: 0.9260 Epoch 2939/5000 77s 154ms/step - loss: 0.2264 - acc: 0.9724 - val_loss: 0.4004 - val_acc: 0.9227 Epoch 2940/5000 77s 154ms/step - loss: 0.2249 - acc: 0.9721 - val_loss: 0.4046 - val_acc: 0.9197 Epoch 2941/5000 77s 154ms/step - loss: 0.2242 - acc: 0.9728 - val_loss: 0.4049 - val_acc: 0.9197 Epoch 2942/5000 77s 154ms/step - loss: 0.2252 - acc: 0.9722 - val_loss: 0.4010 - val_acc: 0.9244 Epoch 2943/5000 77s 154ms/step - loss: 0.2289 - acc: 0.9713 - val_loss: 0.4085 - val_acc: 0.9193 Epoch 2944/5000 77s 154ms/step - loss: 0.2258 - acc: 0.9724 - val_loss: 0.4128 - val_acc: 0.9214 Epoch 2945/5000 77s 154ms/step - loss: 0.2286 - acc: 0.9707 - val_loss: 0.4222 - val_acc: 0.9145 Epoch 2946/5000 77s 154ms/step - loss: 0.2256 - acc: 0.9726 - val_loss: 0.4081 - val_acc: 0.9215 Epoch 2947/5000 77s 154ms/step - loss: 0.2228 - acc: 0.9733 - val_loss: 0.4001 - val_acc: 0.9236 Epoch 2948/5000 77s 154ms/step - loss: 0.2260 - acc: 0.9722 - val_loss: 0.3945 - val_acc: 0.9239 Epoch 2949/5000 77s 154ms/step - loss: 0.2226 - acc: 0.9732 - val_loss: 0.4005 - val_acc: 0.9223 Epoch 2950/5000 77s 154ms/step - loss: 0.2280 - acc: 0.9710 - val_loss: 0.3977 - val_acc: 0.9220 Epoch 2951/5000 77s 154ms/step - loss: 0.2277 - acc: 0.9707 - val_loss: 0.3953 - val_acc: 0.9245 Epoch 2952/5000 77s 154ms/step - loss: 0.2270 - acc: 0.9715 - val_loss: 0.3916 - val_acc: 0.9249 Epoch 2953/5000 77s 154ms/step - loss: 0.2251 - acc: 0.9724 - val_loss: 0.4020 - val_acc: 0.9232 Epoch 2954/5000 77s 154ms/step - loss: 0.2277 - acc: 0.9714 - val_loss: 0.3946 - val_acc: 0.9222 Epoch 2955/5000 77s 154ms/step - loss: 0.2282 - acc: 0.9719 - val_loss: 0.3926 - val_acc: 0.9239 Epoch 2956/5000 77s 154ms/step - loss: 0.2209 - acc: 0.9742 - val_loss: 0.3995 - val_acc: 0.9219 Epoch 2957/5000 77s 154ms/step - loss: 0.2250 - acc: 0.9729 - val_loss: 0.4095 - val_acc: 0.9195 Epoch 2958/5000 77s 154ms/step - loss: 0.2200 - acc: 0.9744 - val_loss: 0.4053 - val_acc: 0.9213 Epoch 2959/5000 77s 154ms/step - loss: 0.2259 - acc: 0.9725 - val_loss: 0.3938 - val_acc: 0.9243 Epoch 2960/5000 77s 154ms/step - loss: 0.2225 - acc: 0.9733 - val_loss: 0.4226 - val_acc: 0.9149 Epoch 2961/5000 77s 154ms/step - loss: 0.2228 - acc: 0.9733 - val_loss: 0.4012 - val_acc: 0.9231 Epoch 2962/5000 77s 154ms/step - loss: 0.2266 - acc: 0.9722 - val_loss: 0.3923 - val_acc: 0.9220 Epoch 2963/5000 77s 154ms/step - loss: 0.2257 - acc: 0.9720 - val_loss: 0.4094 - val_acc: 0.9207 Epoch 2964/5000 77s 154ms/step - loss: 0.2232 - acc: 0.9719 - val_loss: 0.4030 - val_acc: 0.9195 Epoch 2965/5000 77s 154ms/step - loss: 0.2257 - acc: 0.9726 - val_loss: 0.4015 - val_acc: 0.9208 Epoch 2966/5000 77s 154ms/step - loss: 0.2260 - acc: 0.9720 - val_loss: 0.3923 - val_acc: 0.9252 Epoch 2967/5000 77s 154ms/step - loss: 0.2298 - acc: 0.9702 - val_loss: 0.4055 - val_acc: 0.9212 Epoch 2968/5000 77s 154ms/step - loss: 0.2341 - acc: 0.9689 - val_loss: 0.3956 - val_acc: 0.9204 Epoch 2969/5000 78s 155ms/step - loss: 0.2216 - acc: 0.9743 - val_loss: 0.4065 - val_acc: 0.9227 Epoch 2970/5000 78s 155ms/step - loss: 0.2271 - acc: 0.9713 - val_loss: 0.4127 - val_acc: 0.9191 Epoch 2971/5000 77s 155ms/step - loss: 0.2275 - acc: 0.9716 - val_loss: 0.3983 - val_acc: 0.9206 Epoch 2972/5000 78s 155ms/step - loss: 0.2237 - acc: 0.9723 - val_loss: 0.3819 - val_acc: 0.9278 Epoch 2973/5000 78s 155ms/step - loss: 0.2237 - acc: 0.9740 - val_loss: 0.3824 - val_acc: 0.9241 Epoch 2974/5000 78s 155ms/step - loss: 0.2282 - acc: 0.9711 - val_loss: 0.4099 - val_acc: 0.9220 Epoch 2975/5000 77s 155ms/step - loss: 0.2226 - acc: 0.9730 - val_loss: 0.3993 - val_acc: 0.9249 Epoch 2976/5000 77s 155ms/step - loss: 0.2274 - acc: 0.9716 - val_loss: 0.3942 - val_acc: 0.9241 Epoch 2977/5000 78s 155ms/step - loss: 0.2303 - acc: 0.9703 - val_loss: 0.3946 - val_acc: 0.9228 Epoch 2978/5000 78s 155ms/step - loss: 0.2236 - acc: 0.9729 - val_loss: 0.4192 - val_acc: 0.9195 Epoch 2979/5000 77s 155ms/step - loss: 0.2303 - acc: 0.9711 - val_loss: 0.4105 - val_acc: 0.9204 Epoch 2980/5000 77s 155ms/step - loss: 0.2281 - acc: 0.9708 - val_loss: 0.4206 - val_acc: 0.9166 Epoch 2981/5000 78s 155ms/step - loss: 0.2293 - acc: 0.9714 - val_loss: 0.3983 - val_acc: 0.9233 Epoch 2982/5000 77s 155ms/step - loss: 0.2212 - acc: 0.9744 - val_loss: 0.4093 - val_acc: 0.9220 Epoch 2983/5000 78s 155ms/step - loss: 0.2282 - acc: 0.9713 - val_loss: 0.3909 - val_acc: 0.9266 Epoch 2984/5000 78s 155ms/step - loss: 0.2220 - acc: 0.9738 - val_loss: 0.4007 - val_acc: 0.9222 Epoch 2985/5000 77s 155ms/step - loss: 0.2246 - acc: 0.9728 - val_loss: 0.4016 - val_acc: 0.9231 Epoch 2986/5000 78s 155ms/step - loss: 0.2263 - acc: 0.9714 - val_loss: 0.3954 - val_acc: 0.9229 Epoch 2987/5000 77s 155ms/step - loss: 0.2253 - acc: 0.9724 - val_loss: 0.3986 - val_acc: 0.9254 Epoch 2988/5000 77s 155ms/step - loss: 0.2278 - acc: 0.9718 - val_loss: 0.3938 - val_acc: 0.9237 Epoch 2989/5000 77s 155ms/step - loss: 0.2291 - acc: 0.9715 - val_loss: 0.4007 - val_acc: 0.9208 Epoch 2990/5000 78s 155ms/step - loss: 0.2288 - acc: 0.9708 - val_loss: 0.3921 - val_acc: 0.9235 Epoch 2991/5000 77s 155ms/step - loss: 0.2186 - acc: 0.9752 - val_loss: 0.4077 - val_acc: 0.9234 Epoch 2992/5000 78s 155ms/step - loss: 0.2271 - acc: 0.9711 - val_loss: 0.3884 - val_acc: 0.9262 Epoch 2993/5000 78s 155ms/step - loss: 0.2236 - acc: 0.9722 - val_loss: 0.3975 - val_acc: 0.9241 Epoch 2994/5000 77s 155ms/step - loss: 0.2231 - acc: 0.9732 - val_loss: 0.3987 - val_acc: 0.9255 Epoch 2995/5000 77s 155ms/step - loss: 0.2234 - acc: 0.9729 - val_loss: 0.4180 - val_acc: 0.9175 Epoch 2996/5000 77s 155ms/step - loss: 0.2252 - acc: 0.9726 - val_loss: 0.4069 - val_acc: 0.9222 Epoch 2997/5000 78s 155ms/step - loss: 0.2273 - acc: 0.9727 - val_loss: 0.3979 - val_acc: 0.9229 Epoch 2998/5000 77s 155ms/step - loss: 0.2260 - acc: 0.9722 - val_loss: 0.4036 - val_acc: 0.9208 Epoch 2999/5000 78s 155ms/step - loss: 0.2254 - acc: 0.9721 - val_loss: 0.3880 - val_acc: 0.9252 Epoch 3000/5000 78s 155ms/step - loss: 0.2262 - acc: 0.9725 - val_loss: 0.4040 - val_acc: 0.9217 Epoch 3001/5000 lr changed to 0.0009999999776482583 78s 155ms/step - loss: 0.2012 - acc: 0.9814 - val_loss: 0.3627 - val_acc: 0.9343 Epoch 3002/5000 78s 155ms/step - loss: 0.1846 - acc: 0.9875 - val_loss: 0.3587 - val_acc: 0.9344 Epoch 3003/5000 78s 155ms/step - loss: 0.1793 - acc: 0.9893 - val_loss: 0.3559 - val_acc: 0.9357 Epoch 3004/5000 78s 155ms/step - loss: 0.1770 - acc: 0.9901 - val_loss: 0.3559 - val_acc: 0.9361 Epoch 3005/5000 78s 155ms/step - loss: 0.1752 - acc: 0.9909 - val_loss: 0.3563 - val_acc: 0.9363 Epoch 3006/5000 78s 155ms/step - loss: 0.1730 - acc: 0.9916 - val_loss: 0.3536 - val_acc: 0.9366 Epoch 3007/5000 78s 155ms/step - loss: 0.1744 - acc: 0.9905 - val_loss: 0.3543 - val_acc: 0.9388 Epoch 3008/5000 78s 155ms/step - loss: 0.1692 - acc: 0.9927 - val_loss: 0.3553 - val_acc: 0.9374 Epoch 3009/5000 78s 155ms/step - loss: 0.1693 - acc: 0.9925 - val_loss: 0.3557 - val_acc: 0.9370 Epoch 3010/5000 78s 155ms/step - loss: 0.1688 - acc: 0.9926 - val_loss: 0.3541 - val_acc: 0.9389 Epoch 3011/5000 78s 155ms/step - loss: 0.1664 - acc: 0.9934 - val_loss: 0.3545 - val_acc: 0.9384 Epoch 3012/5000 78s 155ms/step - loss: 0.1671 - acc: 0.9931 - val_loss: 0.3572 - val_acc: 0.9387 Epoch 3013/5000 78s 155ms/step - loss: 0.1644 - acc: 0.9940 - val_loss: 0.3561 - val_acc: 0.9387 Epoch 3014/5000 77s 155ms/step - loss: 0.1646 - acc: 0.9937 - val_loss: 0.3563 - val_acc: 0.9388 Epoch 3015/5000 78s 155ms/step - loss: 0.1644 - acc: 0.9936 - val_loss: 0.3564 - val_acc: 0.9379 Epoch 3016/5000 78s 155ms/step - loss: 0.1632 - acc: 0.9939 - val_loss: 0.3542 - val_acc: 0.9376 Epoch 3017/5000 78s 155ms/step - loss: 0.1622 - acc: 0.9941 - val_loss: 0.3562 - val_acc: 0.9380 Epoch 3018/5000 78s 155ms/step - loss: 0.1619 - acc: 0.9943 - val_loss: 0.3545 - val_acc: 0.9385 Epoch 3019/5000 77s 155ms/step - loss: 0.1609 - acc: 0.9947 - val_loss: 0.3553 - val_acc: 0.9393 Epoch 3020/5000 78s 155ms/step - loss: 0.1607 - acc: 0.9945 - val_loss: 0.3575 - val_acc: 0.9373 ... Epoch 4492/5000 77s 154ms/step - loss: 0.0646 - acc: 0.9953 - val_loss: 0.3020 - val_acc: 0.9352 Epoch 4493/5000 77s 153ms/step - loss: 0.0649 - acc: 0.9947 - val_loss: 0.3092 - val_acc: 0.9344 Epoch 4494/5000 77s 154ms/step - loss: 0.0642 - acc: 0.9952 - val_loss: 0.2993 - val_acc: 0.9351 Epoch 4495/5000 77s 154ms/step - loss: 0.0645 - acc: 0.9949 - val_loss: 0.2898 - val_acc: 0.9371 Epoch 4496/5000 77s 154ms/step - loss: 0.0652 - acc: 0.9952 - val_loss: 0.3040 - val_acc: 0.9364 Epoch 4497/5000 77s 154ms/step - loss: 0.0646 - acc: 0.9949 - val_loss: 0.3002 - val_acc: 0.9357 Epoch 4498/5000 77s 153ms/step - loss: 0.0655 - acc: 0.9947 - val_loss: 0.2957 - val_acc: 0.9385 Epoch 4499/5000 77s 153ms/step - loss: 0.0648 - acc: 0.9947 - val_loss: 0.2993 - val_acc: 0.9371 Epoch 4500/5000 77s 153ms/step - loss: 0.0657 - acc: 0.9944 - val_loss: 0.3034 - val_acc: 0.9349 Epoch 4501/5000 lr changed to 9.999999310821295e-05 77s 154ms/step - loss: 0.0631 - acc: 0.9958 - val_loss: 0.2941 - val_acc: 0.9384 Epoch 4502/5000 77s 154ms/step - loss: 0.0606 - acc: 0.9963 - val_loss: 0.2887 - val_acc: 0.9393 Epoch 4503/5000 77s 154ms/step - loss: 0.0585 - acc: 0.9971 - val_loss: 0.2866 - val_acc: 0.9398 Epoch 4504/5000 77s 154ms/step - loss: 0.0579 - acc: 0.9974 - val_loss: 0.2861 - val_acc: 0.9408 Epoch 4505/5000 77s 153ms/step - loss: 0.0568 - acc: 0.9976 - val_loss: 0.2838 - val_acc: 0.9415 Epoch 4506/5000 77s 154ms/step - loss: 0.0579 - acc: 0.9975 - val_loss: 0.2825 - val_acc: 0.9419 Epoch 4507/5000 77s 154ms/step - loss: 0.0567 - acc: 0.9977 - val_loss: 0.2818 - val_acc: 0.9417 Epoch 4508/5000 77s 154ms/step - loss: 0.0570 - acc: 0.9977 - val_loss: 0.2814 - val_acc: 0.9416 Epoch 4509/5000 77s 154ms/step - loss: 0.0557 - acc: 0.9982 - val_loss: 0.2794 - val_acc: 0.9421 Epoch 4510/5000 77s 154ms/step - loss: 0.0555 - acc: 0.9982 - val_loss: 0.2793 - val_acc: 0.9428 Epoch 4511/5000 77s 153ms/step - loss: 0.0565 - acc: 0.9979 - val_loss: 0.2789 - val_acc: 0.9418 Epoch 4512/5000 77s 154ms/step - loss: 0.0555 - acc: 0.9981 - val_loss: 0.2806 - val_acc: 0.9408 Epoch 4513/5000 77s 153ms/step - loss: 0.0561 - acc: 0.9977 - val_loss: 0.2801 - val_acc: 0.9417 Epoch 4514/5000 77s 153ms/step - loss: 0.0552 - acc: 0.9983 - val_loss: 0.2791 - val_acc: 0.9415 Epoch 4515/5000 77s 154ms/step - loss: 0.0550 - acc: 0.9985 - val_loss: 0.2796 - val_acc: 0.9423 Epoch 4516/5000 77s 153ms/step - loss: 0.0547 - acc: 0.9982 - val_loss: 0.2817 - val_acc: 0.9416 Epoch 4517/5000 77s 153ms/step - loss: 0.0559 - acc: 0.9980 - val_loss: 0.2821 - val_acc: 0.9423 Epoch 4518/5000 77s 154ms/step - loss: 0.0548 - acc: 0.9985 - val_loss: 0.2807 - val_acc: 0.9430 Epoch 4519/5000 77s 154ms/step - loss: 0.0545 - acc: 0.9986 - val_loss: 0.2801 - val_acc: 0.9433 Epoch 4520/5000 77s 154ms/step - loss: 0.0550 - acc: 0.9980 - val_loss: 0.2814 - val_acc: 0.9422 Epoch 4521/5000 77s 154ms/step - loss: 0.0554 - acc: 0.9981 - val_loss: 0.2811 - val_acc: 0.9422 Epoch 4522/5000 77s 154ms/step - loss: 0.0545 - acc: 0.9984 - val_loss: 0.2792 - val_acc: 0.9432 Epoch 4523/5000 77s 154ms/step - loss: 0.0541 - acc: 0.9986 - val_loss: 0.2798 - val_acc: 0.9421 Epoch 4524/5000 77s 154ms/step - loss: 0.0543 - acc: 0.9985 - val_loss: 0.2789 - val_acc: 0.9435 Epoch 4525/5000 77s 154ms/step - loss: 0.0543 - acc: 0.9986 - val_loss: 0.2788 - val_acc: 0.9428 Epoch 4526/5000 77s 154ms/step - loss: 0.0547 - acc: 0.9984 - val_loss: 0.2786 - val_acc: 0.9429 Epoch 4527/5000 77s 154ms/step - loss: 0.0541 - acc: 0.9985 - val_loss: 0.2791 - val_acc: 0.9425 Epoch 4528/5000 77s 155ms/step - loss: 0.0539 - acc: 0.9986 - val_loss: 0.2797 - val_acc: 0.9427 Epoch 4529/5000 78s 156ms/step - loss: 0.0553 - acc: 0.9980 - val_loss: 0.2777 - val_acc: 0.9432 Epoch 4530/5000 78s 155ms/step - loss: 0.0542 - acc: 0.9985 - val_loss: 0.2787 - val_acc: 0.9422 Epoch 4531/5000 77s 155ms/step - loss: 0.0548 - acc: 0.9982 - val_loss: 0.2792 - val_acc: 0.9427 Epoch 4532/5000 77s 155ms/step - loss: 0.0547 - acc: 0.9983 - val_loss: 0.2791 - val_acc: 0.9425 Epoch 4533/5000 78s 155ms/step - loss: 0.0539 - acc: 0.9986 - val_loss: 0.2811 - val_acc: 0.9423 Epoch 4534/5000 78s 155ms/step - loss: 0.0536 - acc: 0.9985 - val_loss: 0.2808 - val_acc: 0.9422 Epoch 4535/5000 78s 155ms/step - loss: 0.0542 - acc: 0.9984 - val_loss: 0.2802 - val_acc: 0.9426 Epoch 4536/5000 78s 155ms/step - loss: 0.0535 - acc: 0.9988 - val_loss: 0.2804 - val_acc: 0.9422 Epoch 4537/5000 77s 155ms/step - loss: 0.0536 - acc: 0.9987 - val_loss: 0.2790 - val_acc: 0.9425 Epoch 4538/5000 78s 155ms/step - loss: 0.0539 - acc: 0.9986 - val_loss: 0.2798 - val_acc: 0.9433 Epoch 4539/5000 78s 155ms/step - loss: 0.0540 - acc: 0.9985 - val_loss: 0.2786 - val_acc: 0.9436 Epoch 4540/5000 78s 155ms/step - loss: 0.0540 - acc: 0.9985 - val_loss: 0.2788 - val_acc: 0.9433 Epoch 4541/5000 77s 155ms/step - loss: 0.0543 - acc: 0.9985 - val_loss: 0.2774 - val_acc: 0.9424 Epoch 4542/5000 77s 155ms/step - loss: 0.0536 - acc: 0.9987 - val_loss: 0.2771 - val_acc: 0.9431 Epoch 4543/5000 77s 155ms/step - loss: 0.0536 - acc: 0.9984 - val_loss: 0.2785 - val_acc: 0.9427 Epoch 4544/5000 77s 155ms/step - loss: 0.0539 - acc: 0.9984 - val_loss: 0.2806 - val_acc: 0.9428 Epoch 4545/5000 78s 155ms/step - loss: 0.0537 - acc: 0.9985 - val_loss: 0.2803 - val_acc: 0.9428 Epoch 4546/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9991 - val_loss: 0.2806 - val_acc: 0.9434 Epoch 4547/5000 77s 155ms/step - loss: 0.0534 - acc: 0.9987 - val_loss: 0.2808 - val_acc: 0.9427 Epoch 4548/5000 77s 155ms/step - loss: 0.0537 - acc: 0.9985 - val_loss: 0.2799 - val_acc: 0.9427 Epoch 4549/5000 77s 155ms/step - loss: 0.0538 - acc: 0.9984 - val_loss: 0.2814 - val_acc: 0.9424 Epoch 4550/5000 77s 155ms/step - loss: 0.0537 - acc: 0.9987 - val_loss: 0.2806 - val_acc: 0.9434 Epoch 4551/5000 77s 155ms/step - loss: 0.0532 - acc: 0.9988 - val_loss: 0.2804 - val_acc: 0.9435 Epoch 4552/5000 78s 155ms/step - loss: 0.0534 - acc: 0.9989 - val_loss: 0.2813 - val_acc: 0.9424 Epoch 4553/5000 78s 155ms/step - loss: 0.0531 - acc: 0.9988 - val_loss: 0.2812 - val_acc: 0.9411 Epoch 4554/5000 78s 155ms/step - loss: 0.0535 - acc: 0.9986 - val_loss: 0.2807 - val_acc: 0.9419 Epoch 4555/5000 77s 155ms/step - loss: 0.0529 - acc: 0.9989 - val_loss: 0.2800 - val_acc: 0.9428 Epoch 4556/5000 77s 155ms/step - loss: 0.0533 - acc: 0.9985 - val_loss: 0.2802 - val_acc: 0.9425 Epoch 4557/5000 78s 155ms/step - loss: 0.0534 - acc: 0.9987 - val_loss: 0.2819 - val_acc: 0.9422 Epoch 4558/5000 78s 155ms/step - loss: 0.0536 - acc: 0.9985 - val_loss: 0.2806 - val_acc: 0.9434 Epoch 4559/5000 77s 155ms/step - loss: 0.0540 - acc: 0.9985 - val_loss: 0.2812 - val_acc: 0.9432 Epoch 4560/5000 78s 155ms/step - loss: 0.0532 - acc: 0.9988 - val_loss: 0.2813 - val_acc: 0.9441 Epoch 4561/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9988 - val_loss: 0.2817 - val_acc: 0.9442 Epoch 4562/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9990 - val_loss: 0.2805 - val_acc: 0.9444 Epoch 4563/5000 77s 155ms/step - loss: 0.0528 - acc: 0.9989 - val_loss: 0.2810 - val_acc: 0.9445 Epoch 4564/5000 77s 155ms/step - loss: 0.0525 - acc: 0.9990 - val_loss: 0.2790 - val_acc: 0.9443 Epoch 4565/5000 78s 155ms/step - loss: 0.0529 - acc: 0.9986 - val_loss: 0.2793 - val_acc: 0.9430 Epoch 4566/5000 78s 155ms/step - loss: 0.0530 - acc: 0.9989 - val_loss: 0.2784 - val_acc: 0.9439 Epoch 4567/5000 78s 155ms/step - loss: 0.0535 - acc: 0.9986 - val_loss: 0.2802 - val_acc: 0.9435 Epoch 4568/5000 77s 155ms/step - loss: 0.0525 - acc: 0.9989 - val_loss: 0.2805 - val_acc: 0.9437 Epoch 4569/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9988 - val_loss: 0.2807 - val_acc: 0.9438 Epoch 4570/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9989 - val_loss: 0.2813 - val_acc: 0.9431 Epoch 4571/5000 77s 155ms/step - loss: 0.0528 - acc: 0.9990 - val_loss: 0.2822 - val_acc: 0.9427 Epoch 4572/5000 77s 155ms/step - loss: 0.0531 - acc: 0.9989 - val_loss: 0.2822 - val_acc: 0.9424 Epoch 4573/5000 77s 155ms/step - loss: 0.0520 - acc: 0.9992 - val_loss: 0.2835 - val_acc: 0.9431 Epoch 4574/5000 78s 155ms/step - loss: 0.0530 - acc: 0.9988 - val_loss: 0.2815 - val_acc: 0.9439 Epoch 4575/5000 77s 155ms/step - loss: 0.0530 - acc: 0.9988 - val_loss: 0.2827 - val_acc: 0.9435 Epoch 4576/5000 77s 155ms/step - loss: 0.0528 - acc: 0.9987 - val_loss: 0.2822 - val_acc: 0.9445 Epoch 4577/5000 77s 155ms/step - loss: 0.0529 - acc: 0.9988 - val_loss: 0.2817 - val_acc: 0.9433 Epoch 4578/5000 77s 155ms/step - loss: 0.0530 - acc: 0.9988 - val_loss: 0.2821 - val_acc: 0.9426 Epoch 4579/5000 77s 155ms/step - loss: 0.0526 - acc: 0.9989 - val_loss: 0.2822 - val_acc: 0.9432 Epoch 4580/5000 77s 155ms/step - loss: 0.0529 - acc: 0.9988 - val_loss: 0.2840 - val_acc: 0.9429 Epoch 4581/5000 77s 155ms/step - loss: 0.0534 - acc: 0.9984 - val_loss: 0.2832 - val_acc: 0.9430 Epoch 4582/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9989 - val_loss: 0.2832 - val_acc: 0.9437 Epoch 4583/5000 78s 155ms/step - loss: 0.0523 - acc: 0.9991 - val_loss: 0.2831 - val_acc: 0.9428 Epoch 4584/5000 77s 155ms/step - loss: 0.0527 - acc: 0.9987 - val_loss: 0.2835 - val_acc: 0.9426 Epoch 4585/5000 77s 155ms/step - loss: 0.0529 - acc: 0.9988 - val_loss: 0.2833 - val_acc: 0.9432 ... Epoch 4958/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9992 - val_loss: 0.2819 - val_acc: 0.9435 Epoch 4959/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9992 - val_loss: 0.2826 - val_acc: 0.9437 Epoch 4960/5000 77s 154ms/step - loss: 0.0482 - acc: 0.9994 - val_loss: 0.2830 - val_acc: 0.9423 Epoch 4961/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9992 - val_loss: 0.2827 - val_acc: 0.9436 Epoch 4962/5000 77s 154ms/step - loss: 0.0487 - acc: 0.9990 - val_loss: 0.2831 - val_acc: 0.9434 Epoch 4963/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9993 - val_loss: 0.2835 - val_acc: 0.9440 Epoch 4964/5000 77s 154ms/step - loss: 0.0488 - acc: 0.9991 - val_loss: 0.2843 - val_acc: 0.9433 Epoch 4965/5000 77s 154ms/step - loss: 0.0486 - acc: 0.9992 - val_loss: 0.2821 - val_acc: 0.9434 Epoch 4966/5000 77s 153ms/step - loss: 0.0487 - acc: 0.9992 - val_loss: 0.2824 - val_acc: 0.9440 Epoch 4967/5000 77s 154ms/step - loss: 0.0482 - acc: 0.9992 - val_loss: 0.2817 - val_acc: 0.9449 Epoch 4968/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9992 - val_loss: 0.2830 - val_acc: 0.9449 Epoch 4969/5000 77s 154ms/step - loss: 0.0492 - acc: 0.9990 - val_loss: 0.2856 - val_acc: 0.9439 Epoch 4970/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9991 - val_loss: 0.2860 - val_acc: 0.9429 Epoch 4971/5000 77s 153ms/step - loss: 0.0486 - acc: 0.9991 - val_loss: 0.2864 - val_acc: 0.9429 Epoch 4972/5000 77s 154ms/step - loss: 0.0483 - acc: 0.9994 - val_loss: 0.2878 - val_acc: 0.9422 Epoch 4973/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9991 - val_loss: 0.2874 - val_acc: 0.9423 Epoch 4974/5000 77s 154ms/step - loss: 0.0490 - acc: 0.9991 - val_loss: 0.2848 - val_acc: 0.9439 Epoch 4975/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9993 - val_loss: 0.2867 - val_acc: 0.9438 Epoch 4976/5000 77s 154ms/step - loss: 0.0480 - acc: 0.9993 - val_loss: 0.2857 - val_acc: 0.9439 Epoch 4977/5000 77s 154ms/step - loss: 0.0483 - acc: 0.9992 - val_loss: 0.2848 - val_acc: 0.9436 Epoch 4978/5000 77s 154ms/step - loss: 0.0483 - acc: 0.9992 - val_loss: 0.2860 - val_acc: 0.9440 Epoch 4979/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9993 - val_loss: 0.2872 - val_acc: 0.9433 Epoch 4980/5000 77s 154ms/step - loss: 0.0482 - acc: 0.9993 - val_loss: 0.2867 - val_acc: 0.9436 Epoch 4981/5000 77s 154ms/step - loss: 0.0480 - acc: 0.9994 - val_loss: 0.2871 - val_acc: 0.9431 Epoch 4982/5000 77s 154ms/step - loss: 0.0477 - acc: 0.9995 - val_loss: 0.2867 - val_acc: 0.9431 Epoch 4983/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9992 - val_loss: 0.2864 - val_acc: 0.9423 Epoch 4984/5000 77s 153ms/step - loss: 0.0480 - acc: 0.9994 - val_loss: 0.2852 - val_acc: 0.9437 Epoch 4985/5000 77s 154ms/step - loss: 0.0479 - acc: 0.9994 - val_loss: 0.2846 - val_acc: 0.9437 Epoch 4986/5000 77s 154ms/step - loss: 0.0488 - acc: 0.9990 - val_loss: 0.2852 - val_acc: 0.9433 Epoch 4987/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9990 - val_loss: 0.2831 - val_acc: 0.9443 Epoch 4988/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9993 - val_loss: 0.2850 - val_acc: 0.9440 Epoch 4989/5000 77s 154ms/step - loss: 0.0484 - acc: 0.9991 - val_loss: 0.2871 - val_acc: 0.9431 Epoch 4990/5000 77s 154ms/step - loss: 0.0479 - acc: 0.9993 - val_loss: 0.2863 - val_acc: 0.9433 Epoch 4991/5000 77s 153ms/step - loss: 0.0483 - acc: 0.9993 - val_loss: 0.2868 - val_acc: 0.9434 Epoch 4992/5000 77s 154ms/step - loss: 0.0482 - acc: 0.9992 - val_loss: 0.2847 - val_acc: 0.9431 Epoch 4993/5000 77s 154ms/step - loss: 0.0485 - acc: 0.9992 - val_loss: 0.2853 - val_acc: 0.9417 Epoch 4994/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9994 - val_loss: 0.2835 - val_acc: 0.9422 Epoch 4995/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9994 - val_loss: 0.2836 - val_acc: 0.9432 Epoch 4996/5000 77s 153ms/step - loss: 0.0483 - acc: 0.9991 - val_loss: 0.2852 - val_acc: 0.9427 Epoch 4997/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9994 - val_loss: 0.2855 - val_acc: 0.9427 Epoch 4998/5000 77s 154ms/step - loss: 0.0481 - acc: 0.9993 - val_loss: 0.2839 - val_acc: 0.9430 Epoch 4999/5000 77s 154ms/step - loss: 0.0478 - acc: 0.9994 - val_loss: 0.2855 - val_acc: 0.9424 Epoch 5000/5000 77s 154ms/step - loss: 0.0480 - acc: 0.9993 - val_loss: 0.2855 - val_acc: 0.9428 Train loss: 0.04765264599025249 Train accuracy: 0.9993600006103516 Test loss: 0.2855186524987221 Test accuracy: 0.9428000026941299
測試準確率第一次突破了94%。
其實，在訓練的後半階段還是出現了過擬合，說明還要針對性地調整超引數。
同時，似乎沒必要訓練5000個epoch。因為epoch再多，loss也不怎麼下降。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis,
IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458
https://ieeexplore.ieee.org/document/8998530
————————————————
版權宣告：本文為CSDN博主「dangqing1988」的原創文章，遵循CC 4.0 BY-SA版權協議，轉載請附上原文出處連結及本宣告。
原文連結：https://blog.csdn.net/dangqing1988/article/details/105878193
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/69972329/viewspace-2689916/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄26）Cifar10~95.92%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄19）Cifar10~93.96%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄23）Cifar10~95.47%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄24）Cifar10~95.80%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄20）Cifar10~94.17%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄22）Cifar10~95.25%函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄1）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄2）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄3）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄4）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄5）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄6）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄7）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄8）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄9）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄10）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄11）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄12）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄13）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄14）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄15）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄16）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄17）函式
- 深度殘差網路+自適應引數化ReLU啟用函式（調參記錄21）Cifar10~95.12%函式
- 注意力機制下的啟用函式：自適應引數化ReLU函式
- 深度殘差網路(ResNet)
- 深度學習之殘差網路深度學習
- 殘差網路再升級之深度殘差收縮網路（附Keras程式碼）Keras
- 深度殘差收縮網路：（三）網路結構
- 深度學習故障診斷——深度殘差收縮網路深度學習
- 深度殘差收縮網路：（一）背景知識
- 深度殘差收縮網路：（二）整體思路
- 學習筆記16：殘差網路筆記
- 十分鐘弄懂深度殘差收縮網路
- 深度殘差收縮網路：（五）實驗驗證
- 深度殘差收縮網路：（六）程式碼實現
- 從ReLU到GELU，一文概覽神經網路的啟用函式神經網路函式
- PHP函式，引數,可變參函式.PHP函式