1、背景
1.1、背景
舊服務改造為多租戶服務後,log4j日誌列印在一起不能區分是哪個租戶的,日誌太多,太雜,不好定位排除問題
,排查問題較難。
1.2、前提
不改動以前的日誌程式碼(工作量太大)
1.3、列印日誌示例
package com.cherf.sauth.controller;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.cherf.common.ResultVo;
import io.swagger.annotations.ApiOperation;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* @author cherf
* @description: test
* @date 2023/03/01 17:33
**/
@RestController
@RequestMapping("/v1")
public class TestController {
private static Logger log = LoggerFactory.getLogger(TestController.class);
@PostMapping("/test")
@ApiOperation(value = "test", notes = "test")
public ResultVo<?> authorization() {
log.trace("test:{}", "trace");
log.debug("test:{}", "debug");
log.info("test:{}", "info");
log.warn("test:{}", "warn");
log.error("test:{}", "error");
return ResultVo.ok();
}
}
2、實現
2.1、版本依賴
nacos: 2.1.0
slf4j-api: 1.7.36
slf4j-log4j12: 1.7.36
spring-boot:2.6.14
spring-cloud:2021.0.1
spring-cloud-alibaba:2021.0.1.0
(注:
log4j-api
和 logback-core
用的是 spring-boot-starter-test 2.6.14
中的版本分別是 2.17.2
1.2.11
)
2.2、實現思路
2.2.1、日誌分租戶列印
logback透過載入ogback.xml 配置,透過 Appender 介面來實現列印,原理請看:logback
透過跟蹤原始碼,可以找到AppenderAttachableImpl
這個類,其中透過Appender.doAppend
方法實現根據logabck配置中的類(預設是:RollingFileAppender
)來列印日誌,我們需要自定義重寫doAppend
方法來實現
2.2.2、logback配置動態生效
透過JoranConfiguration
來實現 。Joran
是 logback
使用的一個配置載入庫,動態生效 logback 的配置可以透過joranConfigurator.doConfigure
方法實現,(實現程式碼在下面);
2.2.3、新增租戶時新增logback配置
主要思路是透過nacos
來動態修改和釋出配置從而實現logback.xml動態修改;XML格式比較煩,如果配置較多可以使用DOM4J來實現修改XML;(我們的較為簡單,所以只是透過字串替換來實現~)
2.3、實現
2.3.1、日誌分離列印
2.3.1.1 重寫doAppend
方法
日誌分離列印主要的實現就是重寫doAppend
方法,示例如下:
其中TenantContextHolder
是用來存放租戶id本地變數,實現可參考:多租戶改造(欄位隔離和表隔離混合模式)(也可使用自己的方法)
package com.cherf.common.logback;
import ch.qos.logback.core.rolling.RollingFileAppender;
import com.cherf.common.context.TenantContextHolder;
import com.cherf.common.util.StringUtil;
/**
* @author cherf
* @description:logback複寫
* @date 2023/02/24 11:49
**/
public class TenantRollingFileAppender<E> extends RollingFileAppender<E> {
/**
* 日誌列印會呼叫此方法,進行復寫,判斷租戶,根據租戶列印到不同日誌檔案
*
* @param eventObject
*/
public void doAppend(E eventObject) {
String tenantId = TenantContextHolder.getTenantId();
if (StringUtil.isBlank(tenantId)) {
//沒有租戶id的日誌,列印到public下面
tenantId = "public";
}
// this.getName() 是在logback.xml中配置的<appender name="appenderName" class="com.cherf.common.logback.TenantRollingFileAppender">
// 只列印當前租戶的Append,RollingFileAppender追加器以租戶型別標識開頭的執行追加
if (this.getName().startsWith(tenantId)) {
super.doAppend(eventObject);
}
}
}
2.3.1.2 logback
配置示例
logback.xml
需要將配置中的appender
標籤的class
屬性修改為剛剛重寫的方法全限定類名:com.cherf.common.logback.TenantRollingFileAppender
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="log.path" value="../../logs/system" />
<property name="log.pattern" value="%date [%level] [%thread] %logger{80} [%file : %line] %msg%n" />
<appender class="ch.qos.logback.core.ConsoleAppender" name="STDOUT">
<encoder>
<pattern>${log.pattern}</pattern>
</encoder>
</appender>
<!-- public 公共-->
<appender class="com.cherf.common.logback.TenantRollingFileAppender" name="public">
<file>${log.path}/public/sys-api.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/public/sys-api.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- 日誌最大的歷史 60天 -->
<maxHistory>60</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${log.pattern}</pattern>
</encoder>
<!-- 保留INFO級別及以上的日誌 -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<!-- 預設租戶 -->
<appender class="com.cherf.common.logback.TenantRollingFileAppender" name="tid20220831114008942">
<file>${log.path}/tid20220831114008942/sys-api.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/tid20220831114008942/sys-api.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- 日誌最大的歷史 60天 -->
<maxHistory>60</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${log.pattern}</pattern>
</encoder>
<!-- 保留INFO級別及以上的日誌 -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<logger additivity="false" name="com.cherf.system">
<appender-ref ref="public" />
<appender-ref ref="STDOUT" />
<appender-ref ref="tid20220831114008942" />
</logger>
<root level="INFO">
<appender-ref ref="public" />
<appender-ref ref="STDOUT" />
<appender-ref ref="tid20220831114008942" />
</root>
</configuration>
2.3.2、配置動態生效
2.3.2.1、專案啟動讀取nacos
配置
服務啟動時logback
預設載入 classpath:logback.xml
配置,需要在yml
中指定logback
配置 (logging.config
後配置)
配置如下:
spring:
cloud:
nacos:
discovery:
# 不使用nacos的配置
# enabled: false
server-addr: 127.0.0.1:8848
#日誌列印
logging:
config: http://${spring.cloud.nacos.discovery.server-addr}/nacos/v1/cs/configs?group=${logback.group}&tenant=public&dataId=${logback.systemDataId}
level:
com.cherf: info
com.cherf.mapper: info
org.springframework: info
org.spring.springboot.dao: info
logback:
group: logback
systemDataId: system-logback.xml
2.3.2.2、nacos
配置監聽+logback
動態載入配置
主要採用nacos
ConfigService
的addListener
方法來監聽;
注意:
網上很多直接透過 NacosFactory.createConfigService()
來建立ConfigService
的方法可能會重複建立例項,導致CPU
上升,詳情可參考:記一次CPU佔用持續上升問題排查(Nacos動態路由引起)
1、nacos
動態配置監聽器
package com.cherf.common.nacos;
import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.joran.JoranConfigurator;
import com.alibaba.nacos.api.config.ConfigService;
import com.alibaba.nacos.api.config.ConfigType;
import com.alibaba.nacos.api.config.listener.Listener;
import com.alibaba.nacos.api.exception.NacosException;
import com.cherf.common.nacos.NacosConfigService;
import com.cherf.common.util.StringUtil;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import javax.annotation.Resource;
import java.io.ByteArrayInputStream;
import java.io.InputStream;
import java.util.concurrent.Executor;
/**
* @author cherf
* @description: nacos監聽器,修改logback.xml後動態生效
* @date 2023/03/04 16:35
**/
@Component
public class NacosDynamicLogbackService {
private static final Logger log = LoggerFactory.getLogger(NacosDynamicLogbackService.class);
/**
* 配置 ID
*/
@Value("${logback.systemDataId}")
private String dataId;
/**
* 配置 分組
*/
@Value("${logback.group}")
private String group;
@Resource
private NacosConfigService nacosConfigService;
@PostConstruct
public void dynamicLogbackByNacosListener() {
try {
ConfigService configService = nacosConfigService.getInstance();
if (configService != null) {
configService.getConfig(dataId, group, 5000);
configService.addListener(dataId, group, new Listener() {
@Override
public void receiveConfigInfo(String configInfo) {
if (StringUtil.isNotBlank(configInfo)) {
System.out.println("configInfo=============================>" + configInfo);
try {
LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
JoranConfigurator configurator = new JoranConfigurator();
configurator.setContext(context);
context.reset();
//獲取nacos配置,生成inputStream
InputStream inputStreamRoute = new ByteArrayInputStream(new String(configInfo).getBytes());
//configurator.doConfigure("/logback.xml");
configurator.doConfigure(inputStreamRoute);
context.start();
} catch (Exception e) {
log.error("載入logback.xml配置發生錯誤", e);
}
}
}
@Override
public Executor getExecutor() {
return null;
}
});
}
} catch (NacosException e) {
log.error("獲取logback.xml配置發生錯誤", e);
}
}
/**
* 獲取配置檔案內容
*
* @return
*/
public String getLogBackConfig(String dataId, String group) {
try {
ConfigService configService = nacosConfigService.getInstance();
// 根據dataId、group定位到具體配置檔案,獲取其內容. 方法中的三個引數分別是: dataId, group, 超時時間
String content = configService.getConfig(dataId, group, 5000L);
return content;
} catch (NacosException e) {
log.error(e.getErrMsg());
}
return null;
}
/**
* 釋出配置
*
* @param logbackXml
* @return
*/
public boolean publishLogBackConfig(String dataId, String group,String logbackXml) {
try {
ConfigService configService = nacosConfigService.getInstance();
boolean isPublishOk = configService.publishConfig(dataId, group, logbackXml, ConfigType.XML.getType());
return isPublishOk;
} catch (Exception e) {
log.error(e.getMessage());
}
return false;
}
}
2、 ConfigService
單例
package com.cherf.common.nacos;
import cn.hutool.log.Log;
import cn.hutool.log.LogFactory;
import com.alibaba.nacos.api.NacosFactory;
import com.alibaba.nacos.api.PropertyKeyConst;
import com.alibaba.nacos.api.config.ConfigService;
import com.alibaba.nacos.api.exception.NacosException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
import java.util.Properties;
/**
* @author cherf
* @description: NacosConfigservice單例
* @date 2023/03/4 16:35
**/
@Component
public class NacosConfigService {
private static final Log log = LogFactory.get(NacosConfigService.class);
/**
* nacos地址
*/
@Value("${spring.cloud.nacos.discovery.server-addr}")
private String ipAddress;
//宣告變數, 使用volatile關鍵字確保絕對執行緒安全
private volatile ConfigService configService = null;
@Bean
public ConfigService getInstance() throws NacosException {
if (configService == null) {
//對單例類進行加鎖
synchronized (NacosConfigService.class) {
if (configService == null) {
Properties properties = new Properties();
// nacos伺服器地址,127.0.0.1:8848
properties.put(PropertyKeyConst.SERVER_ADDR, ipAddress);
//建立例項
configService = NacosFactory.createConfigService(properties);
log.info("==========建立configService例項===============");
}
}
}
return configService;
}
}
2.3.3、logback
配置動態修改
新增租戶後,需要在logback.xml
裡新增新增租戶的配置資訊,以我們的為例是在其中新增如下三段配置
中間較大的這一段可以寫在Resource
目錄下,然後讀出來替換{tenantId}
即可使用,配置如下
<!-- {tenantId} 租戶日誌 -->
<!--system日誌-->
<appender class="com.cherf.common.logback.TenantRollingFileAppender" name="{tenantId}">
<file>${log.path}/{tenantId}/sys-api.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/{tenantId}/sys-api.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- 日誌最大的歷史 60天 -->
<maxHistory>60</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${log.pattern}</pattern>
</encoder>
<!-- 保留INFO級別及以上的日誌 -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
研究了半天DOM4J
用法,感覺很麻煩,為了省事就直接使用字串替換來完成了;先壓縮讀到的XML配置,然後替換需要新增或刪除的配置資訊,再格式化XML,最後再去釋出
可以參考下面程式碼,包括了讀取ini
配置,XML壓縮,XML格式化,從nacos
獲取配置到釋出配置到nacos
都有示例;(使用了最笨的方法來實現,有好的思路大家可以發出來探討探討)。
package com.cherf.common.logback;
import cn.hutool.core.io.IoUtil;
import cn.hutool.core.io.resource.ClassPathResource;
import cn.hutool.core.io.resource.Resource;
import cn.hutool.log.Log;
import cn.hutool.log.LogFactory;
import com.cherf.common.constant.StringPool;
import com.cherf.common.context.TenantContextHolder;
import com.cherf.common.nacos.NacosDynamicLogbackService;
import com.cherf.common.util.StringUtil;
import com.isearch.common.logback.LogbackXmlContent;
import com.sun.org.apache.xml.internal.serialize.OutputFormat;
import com.sun.org.apache.xml.internal.serialize.XMLSerializer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.w3c.dom.Document;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import java.io.*;
/**
* @author cherf
* @description:
* @date 2023/03/04 11:29
**/
@Component
public class DynamicModifyLogback {
private static final Log log = LogFactory.get(DynamicModifyLogback.class);
//group
@Value("${logback.group}")
private String group;
//dataId
@Value("${logback.systemDataId}")
private String systemDataId;
@Autowired
private NacosDynamicLogbackService nacosDynamicLogbackService;
/**
* 新增配置
*/
public void addLogbackXml() {
String logBackConfig = this.getLogBackConfig(systemDataId);
String addSystemXml = addSystemXml(logBackConfig);
//釋出配置
publishLogBackConfig(systemDataId, addSystemXml);
}
/**
* 刪除配置
*/
public void removeLogbackXml() {
String logBackConfig = this.getLogBackConfig(systemDataId);
String removeSystemXml = removeSystemXml(logBackConfig);
//釋出配置
publishLogBackConfig(systemDataId, removeSystemXml);
}
public static String addSystemXml(String logBackXml) {
String systemAppender = getIniResourec("logback/system-logback.ini").replace("{tenantId}", TenantContextHolder.getTenantId());
log.info("appender:", systemAppender);
String systemRef = "<appender-ref ref=\"tid20220831114008942\"/>".replace("tid20220831114008942", TenantContextHolder.getTenantId());
log.info("ref", systemRef);
return replaceLogBack(logBackXml, systemAppender);
}
public static String removeSystemXml(String logBackXml) {
String systemAppender = getIniResourec("logback/system-logback.ini").replace("{tenantId}", TenantContextHolder.getTenantId());
log.info("appender:", systemAppender);
//logBackXml = format(logBackXml);
//壓縮xml
return packXml(logBackXml, systemAppender);
}
private static String replaceLogBack(String logBackXml, String appender) {
String appenderRep = LogbackXmlContent.appenderRep;
logBackXml = StringUtil.replaceLast(logBackXml, appenderRep, LogbackXmlContent.NULL + appenderRep + appender);
logBackXml = logBackXml.replace("<appender-ref ref=\"tid20220831114008942\"/>", "<appender-ref ref=\"tid20220831114008942\"/>" + StringPool.NEWLINE + "<appender-ref ref=\"tid20220831114008942\"/>".replace("tid20220831114008942", TenantContextHolder.getTenantId()));
logBackXml = format(logBackXml);
log.info(logBackXml);
return logBackXml;
}
private static String packXml(String logBackXml, String appender) {
logBackXml = convertFromXml(logBackXml).replace(StringPool.ELEVE_SPACE, StringPool.EMPTY).replace(StringPool.NEWLINE, StringPool.EMPTY).trim();
appender = convertFromXml(appender).trim();
String systemRef = "<appender-ref ref=\"tid20220831114008942\"/>".replace("tid20220831114008942", TenantContextHolder.getTenantId());
log.info("ref", systemRef);
logBackXml = StringUtil.replaceLast(logBackXml, appender, StringPool.EMPTY);
logBackXml = logBackXml.replace(systemRef, StringPool.EMPTY);
logBackXml = format(logBackXml);
log.info(logBackXml);
return logBackXml;
}
/**
* 獲取配置
*/
public String getLogBackConfig(String dataId) {
return nacosDynamicLogbackService.getLogBackConfig(dataId, group);
}
/**
* 釋出
*/
public Boolean publishLogBackConfig(String dataId, String logbackXml) {
return nacosDynamicLogbackService.publishLogBackConfig(dataId, group, logbackXml);
}
/**
* 格式化xml
*
* @param unformattedXml
* @return
*/
public static String format(String unformattedXml) {
try {
final Document document = parseXmlFile(unformattedXml);
OutputFormat format = new OutputFormat(document);
format.setLineWidth(256);
format.setIndenting(true);
format.setIndent(2);
Writer out = new StringWriter();
XMLSerializer serializer = new XMLSerializer(out, format);
serializer.serialize(document);
return out.toString();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
private static Document parseXmlFile(String in) {
try {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
InputSource is = new InputSource(new StringReader(in));
return db.parse(is);
} catch (ParserConfigurationException e) {
throw new RuntimeException(e);
} catch (SAXException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
/**
* 獲取配置
*
* @param fileName
* @return
*/
private static String getIniResourec(String fileName) {
String xml = StringPool.EMPTY;
Resource resource = new ClassPathResource(fileName);
InputStream is = resource.getStream();
xml = IoUtil.readUtf8(is);
return xml;
}
/**
* 壓縮xml
*
* @param str
* @return
*/
public static String convertFromXml(String str) {
boolean flag = true;
boolean quotesFlag = true;
StringBuffer ans = new StringBuffer();
String tmp = "";
for (int i = 0; i < str.length(); i++) {
if ('"' == str.charAt(i)) {
ans.append(str.charAt(i));
quotesFlag = !quotesFlag;
} else if ('<' == str.charAt(i)) {
tmp = tmp.trim();
ans.append(tmp);
flag = true;
ans.append(str.charAt(i));
} else if ('>' == str.charAt(i)) {
if (quotesFlag) {
flag = false;
ans.append(str.charAt(i));
tmp = "";
} else {
ans.append(">");
}
} else if (flag) {
ans.append(str.charAt(i));
} else {
tmp += str.charAt(i);
}
}
return ans.toString();
}
}
3、總結
前面都還可以,只是由於時間關係,XML
修改的方法確實有點挫,後期有時間再研究著改吧!
其中TenantContextHolder
實現可以參考另一篇文章多租戶改造(欄位隔離和表隔離混合模式)
日誌分離列印部分參考大佬實現:springboot logback多租戶根據請求列印日誌到不同檔案