keras 上新增 roc auc指標

weixin_33868027發表於2018-03-04
Keras指標

https://stackoverflow.com/questions/41032551/how-to-compute-receiving-operating-characteristic-roc-and-auc-in-keras

https://github.com/keras-team/keras/issues/3230#issuecomment-319208366

http://www.luozhipeng.com/?p=1225

http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html

http://blog.csdn.net/zdy0_2004/article/details/44948511

 

keras 輸出roc指標，不能每個batch輸出一次，需要全部計算完再一次計算一次。使用sklearn中的metrics roc來計算。幾個帖子類似

class roc_callback(keras.callbacks.Callback):
    def __init__(self,training_data,validation_data):
        
        self.x = training_data[0]
        self.y = training_data[1]
        self.x_val = validation_data[0]
        self.y_val = validation_data[1]
        
    
    def on_train_begin(self, logs={}):
        return
 
    def on_train_end(self, logs={}):
        return
 
    def on_epoch_begin(self, epoch, logs={}):
        return
 
    def on_epoch_end(self, epoch, logs={}):        
        y_pred = self.model.predict(self.x)
        roc = roc_auc_score(self.y, y_pred)      
        
        y_pred_val = self.model.predict(self.x_val)
        roc_val = roc_auc_score(self.y_val, y_pred_val)      
        
        print('\rroc-auc: %s - roc-auc_val: %s' % (str(round(roc,4)),str(round(roc_val,4))),end=100*' '+'\n')
        return
 
    def on_batch_begin(self, batch, logs={}):
        return
 
    def on_batch_end(self, batch, logs={}):
        return   


callbacks=[roc_callback(training_data=training_data,validation_data=validation_data)]

 

 

首先建立callbacks指令碼，

my_callbacks.py如下：

import keras
from sklearn.metrics import roc_auc_score
import numpy as np

 

class Histories(keras.callbacks.Callback):
6
    def on_train_begin(self, logs={}):
7
        self.aucs = []
8
        self.losses = []
9
 
10
    def on_train_end(self, logs={}):
11
        return
12
 
13
    def on_epoch_begin(self, epoch, logs={}):
14
        return
15
 
16
    def on_epoch_end(self, epoch, logs={}):
17
        self.losses.append(logs.get('loss'))
18
        y_pred = self.model.predict(self.validation_data[0:2])
19
 
20
        yp = []
21
        for i in xrange(0, len(y_pred)):
22
            yp.append(y_pred[i][0])
23
        yt = []
24
        for x in self.validation_data[2]:
25
            yt.append(x[0])
26
         
27
        auc = roc_auc_score(yt, yp)
28
        self.aucs.append(auc)
29
        print 'val-loss',logs.get('loss'), ' val-auc: ',auc,
30
        print '\n'
31
         
32
        return
33
 
34
    def on_batch_begin(self, batch, logs={}):
35
        return
36
 
37
    def on_batch_end(self, batch, logs={}):
38
        return

 

 

模型的輸入為：

1 model = Model(inputs=[keyword1, keyword2], outputs=y)

 

在每個epoch結束時計算auc並輸出：

1 histories = my_callbacks.Histories()
2  
3 model.fit(train_x, train_y, batch_size=1024, epochs=20,shuffle=True, class_weight={1:1.0, 0:0.25}, validation_split=0.2, callbacks=[histories, model_check, lr])

相關文章