Arthas原理系列(三):服務端啟動流程
歷史文章推薦:
前言
本篇文章主要講我們在終端中敲入的命令是如何被 arthas 伺服器識別並且解釋的。要注意這個過程是 arthas 對所有命令執行過程的抽閒個,對於具體命令的執行過程我會在後面的系列文章中再說。
arthas 服務端的啟動
在上一篇文章中,我們跟蹤了整個 arthas 工程的入口方法:com.taobao.arthas.agent334.AgentBootstrap#main
,在這個方法中,最重要的一個步驟就是啟動過了一個繫結執行緒
private static synchronized void main(String args, final Instrumentation inst) {
try {
// 1. 程式執行前的校驗,
// arthas如果已經存在,則直接返回
// 入參中必須要包含arthas core等
// 這些程式碼細節不會影響我們對主流程的理解,因此暫時刪除
final ClassLoader agentLoader = getClassLoader(inst, arthasCoreJarFile);
Thread bindingThread = new Thread() {
@Override
public void run() {
try {
bind(inst, agentLoader, agentArgs);
} catch (Throwable throwable) {
throwable.printStackTrace(ps);
}
}
};
bindingThread.setName("arthas-binding-thread");
bindingThread.start();
bindingThread.join();
} catch (Throwable t) {
t.printStackTrace(ps);
try {
if (ps != System.err) {
ps.close();
}
} catch (Throwable tt) {
// ignore
}
throw new RuntimeException(t);
}
bind
這個執行緒的執行時會呼叫com.taobao.arthas.agent334.AgentBootstrap#bind
,這個方法的詳細程式碼如下:
private static void bind(Instrumentation inst, ClassLoader agentLoader, String args) throws Throwable {
/**
* <pre>
* ArthasBootstrap bootstrap = ArthasBootstrap.getInstance(inst);
* </pre>
*/
Class<?> bootstrapClass = agentLoader.loadClass(ARTHAS_BOOTSTRAP);
Object bootstrap = bootstrapClass.getMethod(GET_INSTANCE, Instrumentation.class, String.class).invoke(null, inst, args);
boolean isBind = (Boolean) bootstrapClass.getMethod(IS_BIND).invoke(bootstrap);
if (!isBind) {
String errorMsg = "Arthas server port binding failed! Please check $HOME/logs/arthas/arthas.log for more details.";
ps.println(errorMsg);
throw new RuntimeException(errorMsg);
}
ps.println("Arthas server already bind.");
}
這段方法用反射的方法呼叫了com.taobao.arthas.core.server.ArthasBootstrap
的靜態方法getInstance
,並且把從main
方法中解析到引數再傳到這個getInstance
中。
getInstance
從這個名字看就是返回一個ArthasBootstrap
的例項,事實上程式碼的邏輯也是這樣的,其中最關鍵的就是ArthasBootstrap
的建構函式函式:
private ArthasBootstrap(Instrumentation instrumentation, Map<String, String> args) throws Throwable {
this.instrumentation = instrumentation;
String outputPath = System.getProperty("arthas.output.dir", "arthas-output");
arthasOutputDir = new File(outputPath);
arthasOutputDir.mkdirs();
// 1. initSpy()
// 載入SpyAPI這個類
initSpy(instrumentation);
// 2. ArthasEnvironment
// 初始化arthas執行的環境變數
initArthasEnvironment(args);
// 3. init logger
loggerContext = LogUtil.initLooger(arthasEnvironment);
// 4. init beans
// 初始化結果渲染和歷史命令管理的相關類
initBeans();
// 5. start agent server
// 啟動server,開始監聽
bind(configure);
// 註冊一些鉤子函式
executorService = Executors.newScheduledThreadPool(1, new ThreadFactory() {
@Override
public Thread newThread(Runnable r) {
final Thread t = new Thread(r, "arthas-command-execute");
t.setDaemon(true);
return t;
}
});
shutdown = new Thread("as-shutdown-hooker") {
@Override
public void run() {
ArthasBootstrap.this.destroy();
}
};
transformerManager = new TransformerManager(instrumentation);
Runtime.getRuntime().addShutdownHook(shutdown);
}
在這個建構函式中,最重要的就是com.taobao.arthas.core.server.ArthasBootstrap#bind
這個方法
private void bind(Configure configure) throws Throwable {
// 無關緊要的一些前置操作,先刪除掉
try {
// 關於arthas tunnel server,請參考:
// https://arthas.aliyun.com/doc/tunnel.html
if (configure.getTunnelServer() != null) {
tunnelClient = new TunnelClient();
tunnelClient.setAppName(configure.getAppName());
tunnelClient.setId(configure.getAgentId());
tunnelClient.setTunnelServerUrl(configure.getTunnelServer());
tunnelClient.setVersion(ArthasBanner.version());
ChannelFuture channelFuture = tunnelClient.start();
channelFuture.await(10, TimeUnit.SECONDS);
}
} catch (Throwable t) {
logger().error("start tunnel client error", t);
}
try {
// 將一些非常關鍵的引數包裝成ShellServerOptions物件
ShellServerOptions options = new ShellServerOptions()
.setInstrumentation(instrumentation)
.setPid(PidUtils.currentLongPid())
.setWelcomeMessage(ArthasBanner.welcome());
if (configure.getSessionTimeout() != null) {
options.setSessionTimeout(configure.getSessionTimeout() * 1000);
}
// new 一個shellServer,用於監聽命令
shellServer = new ShellServerImpl(options);
// BuiltinCommandPack物件首次出現,包含了所有的內建命令
BuiltinCommandPack builtinCommands = new BuiltinCommandPack();
List<CommandResolver> resolvers = new ArrayList<CommandResolver>();
resolvers.add(builtinCommands);
//worker group
workerGroup = new NioEventLoopGroup(new DefaultThreadFactory("arthas-TermServer", true));
// TODO: discover user provided command resolver
if (configure.getTelnetPort() != null && configure.getTelnetPort() > 0) {
shellServer.registerTermServer(new HttpTelnetTermServer(configure.getIp(), configure.getTelnetPort(),
options.getConnectionTimeout(), workerGroup));
} else {
logger().info("telnet port is {}, skip bind telnet server.", configure.getTelnetPort());
}
if (configure.getHttpPort() != null && configure.getHttpPort() > 0) {
shellServer.registerTermServer(new HttpTermServer(configure.getIp(), configure.getHttpPort(),
options.getConnectionTimeout(), workerGroup));
} else {
// listen local address in VM communication
if (configure.getTunnelServer() != null) {
shellServer.registerTermServer(new HttpTermServer(configure.getIp(), configure.getHttpPort(),
options.getConnectionTimeout(), workerGroup));
}
logger().info("http port is {}, skip bind http server.", configure.getHttpPort());
}
for (CommandResolver resolver : resolvers) {
shellServer.registerCommandResolver(resolver);
}
shellServer.listen(new BindHandler(isBindRef));
if (!isBind()) {
throw new IllegalStateException("Arthas failed to bind telnet or http port.");
}
//http api session manager
sessionManager = new SessionManagerImpl(options, shellServer.getCommandManager(), shellServer.getJobController());
//http api handler
httpApiHandler = new HttpApiHandler(historyManager, sessionManager);
logger().info("as-server listening on network={};telnet={};http={};timeout={};", configure.getIp(),
configure.getTelnetPort(), configure.getHttpPort(), options.getConnectionTimeout());
// 非同步回報啟動次數
if (configure.getStatUrl() != null) {
logger().info("arthas stat url: {}", configure.getStatUrl());
}
UserStatUtil.setStatUrl(configure.getStatUrl());
UserStatUtil.arthasStart();
try {
SpyAPI.init();
} catch (Throwable e) {
// ignore
}
logger().info("as-server started in {} ms", System.currentTimeMillis() - start);
} catch (Throwable e) {
logger().error("Error during start as-server", e);
destroy();
throw e;
}
}
這個方法使我們到目前為止見到的最複雜的一個方法,裡面還是有很多的旁枝末節的干擾,總結一下,這個方法全都是圍繞著如何構建一個ShellServer
物件來進行的:
- 第一步會將一些非常重要的入參包裝
ShellServerOptions
傳入ShellServer
- 然後會在
ShellerServer
上註冊命令直譯器BuiltinCommandPack
,點開BuiltinCommandPack
會發現,所有的命令都已經包含在內了 - 根據入參的不同在
ShellerServer
上註冊不同的TermServer
,比如HttpTermServer
或者是HttpTelnetTermServer
- 伺服器開啟監聽指令
BuiltinCommandPack
的實現如下所示:
public class BuiltinCommandPack implements CommandResolver {
private static List<Command> commands = new ArrayList<Command>();
static {
initCommands();
}
@Override
public List<Command> commands() {
return commands;
}
private static void initCommands() {
commands.add(Command.create(HelpCommand.class));
commands.add(Command.create(KeymapCommand.class));
commands.add(Command.create(SearchClassCommand.class));
commands.add(Command.create(SearchMethodCommand.class));
// ...
}
}
服務端對命令列的監聽和處理
接下來我們分析arthas服務端的監聽過程
@Override
public ShellServer listen(final Handler<Future<Void>> listenHandler) {
final List<TermServer> toStart;
synchronized (this) {
if (!closed) {
throw new IllegalStateException("Server listening");
}
toStart = termServers;
}
final AtomicInteger count = new AtomicInteger(toStart.size());
if (count.get() == 0) {
setClosed(false);
listenHandler.handle(Future.<Void>succeededFuture());
return this;
}
Handler<Future<TermServer>> handler = new TermServerListenHandler(this, listenHandler, toStart);
for (TermServer termServer : toStart) {
// termHandler是termServer監聽命令的回撥函式
// 當有新的命令通過網路到達server時會呼叫這個回撥函式
termServer.termHandler(new TermServerTermHandler(this));
termServer.listen(handler);
}
return this;
}
我們以HttpTermServer
為例
@Override
public TermServer listen(Handler<Future<TermServer>> listenHandler) {
// TODO: charset and inputrc from options
bootstrap = new NettyWebsocketTtyBootstrap(workerGroup).setHost(hostIp).setPort(port);
try {
bootstrap.start(new Consumer<TtyConnection>() {
@Override
public void accept(final TtyConnection conn) {
termHandler.handle(new TermImpl(Helper.loadKeymap(), conn));
}
}).get(connectionTimeout, TimeUnit.MILLISECONDS);
listenHandler.handle(Future.<TermServer>succeededFuture());
} catch (Throwable t) {
logger.error("Error listening to port " + port, t);
listenHandler.handle(Future.<TermServer>failedFuture(t));
}
return this;
}
會發現程式會最終去非同步的呼叫termHandler
的handle
方法,而termHandler
正是前面註冊的TermServerTermHandler
這個類的例項:
public class TermServerTermHandler implements Handler<Term> {
private ShellServerImpl shellServer;
public TermServerTermHandler(ShellServerImpl shellServer) {
this.shellServer = shellServer;
}
@Override
public void handle(Term term) {
shellServer.handleTerm(term);
}
}
handle
又回撥了shellServer
的handleTerm
方法,我們的視線隨著呼叫流程再回到ShellServer
這個類
public void handleTerm(Term term) {
synchronized (this) {
// That might happen with multiple ser
if (closed) {
term.close();
return;
}
}
ShellImpl session = createShell(term);
tryUpdateWelcomeMessage();
session.setWelcome(welcomeMessage);
session.closedFuture.setHandler(new SessionClosedHandler(this, session));
session.init();
sessions.put(session.id, session); // Put after init so the close handler on the connection is set
session.readline(); // Now readline
}
這個方法中的最後一行程式碼session.readline();
是我們重點要關注的地方
public void readline() {
// 這裡要注意ShellLineHandler這個類,後面readLine的回撥最終會回到這裡來
term.readline(prompt, new ShellLineHandler(this),
new CommandManagerCompletionHandler(commandManager));
}
我們以TermImpl
的實現為例
public void readline(String prompt, Handler<String> lineHandler, Handler<Completion> completionHandler) {
if (conn.getStdinHandler() != echoHandler) {
throw new IllegalStateException();
}
if (inReadline) {
throw new IllegalStateException();
}
inReadline = true;
readline.readline(conn, prompt, new RequestHandler(this, lineHandler), new CompletionHandler(completionHandler, session));
}
這個方法呼叫了readline.readline
方法,並把之前傳進來的ShellLineHandler
也包進了RequestHandler
傳到了readline.readline
中,我們繼續往進看
public void readline(TtyConnection conn, String prompt, Consumer<String> requestHandler, Consumer<Completion> completionHandler) {
synchronized (this) {
if (interaction != null) {
throw new IllegalStateException("Already reading a line");
}
interaction = new Interaction(conn, prompt, requestHandler, completionHandler);
}
interaction.install();
conn.write(prompt);
schedulePendingEvent();
}
interaction.install();
以及schedulePendingEvent();
這兩漢程式碼最終都會呼叫下面的一段方法
private void deliver() {
while (true) {
Interaction handler;
KeyEvent event;
synchronized (this) {
if (decoder.hasNext() && interaction != null && !interaction.paused) {
event = decoder.next();
handler = interaction;
} else {
return;
}
}
handler.handle(event);
}
}
Interaction
是ReadLine
的一個內部類,他的handle方法比較長,我們擷取這個方法的關鍵部分如下所示:
private void handle(KeyEvent event) {
if (event instanceof FunctionEvent) {
FunctionEvent fname = (FunctionEvent) event;
Function function = functions.get(fname.name());
if (function != null) {
synchronized (this) {
paused = true;
}
function.apply(this);
} else {
Logging.READLINE.warn("Unimplemented function " + fname.name());
}
} else {
LineBuffer buf = buffer.copy();
for (int i = 0;i < event.length();i++) {
int codePoint = event.getCodePointAt(i);
try {
buf.insert(codePoint);
} catch (IllegalArgumentException e) {
conn.stdoutHandler().accept(new int[]{'\007'});
}
}
refresh(buf);
}
}
在這段程式碼中,會首先判斷輸入時間是否在預存的functions
這個變數中已經定義,如果有的話,則執行相應apply
方法,否則做快取相關的操作。
在ReadLine這個類新建的時候,值預定義了一個方法,那就是ACCEPT_LINE
public Readline(Keymap keymap) {
// https://github.com/alibaba/termd/issues/42
// this.device = TermInfo.defaultInfo().getDevice("xterm"); // For now use xterm
this.decoder = new EventQueue(keymap);
this.history = new ArrayList<int[]>();
addFunction(ACCEPT_LINE);
}
ACCEPT_LINE
的定義如下:
private final Function ACCEPT_LINE = new Function() {
@Override
public String name() {
return "accept-line";
}
@Override
public void apply(Interaction interaction) {
interaction.line.insert(interaction.buffer.toArray());
LineStatus pb = new LineStatus();
for (int i = 0;i < interaction.line.getSize();i++) {
pb.accept(interaction.line.getAt(i));
}
interaction.buffer.clear();
if (pb.isEscaping()) {
interaction.line.delete(-1); // Remove \
interaction.currentPrompt = "> ";
interaction.conn.write("\n> ");
interaction.resume();
} else {
if (pb.isQuoted()) {
interaction.line.insert('\n');
interaction.conn.write("\n> ");
interaction.currentPrompt = "> ";
interaction.resume();
} else {
String raw = interaction.line.toString();
if (interaction.line.getSize() > 0) {
addToHistory(interaction.line.toArray());
}
interaction.line.clear();
interaction.conn.write("\n");
interaction.end(raw);
}
}
}
}
在ACCEPT_LINE
的apply
方法中,如果程式判定到達伺服器的是一個合法的命令列,則會呼叫io.termd.core.readline.Readline.Interaction#end
方法,而這個方法,最終會呼叫requestHandler.accept(s);
,這個RequestHandler
其實就是封裝了一層ShellLineHandler
。
private boolean end(String s) {
synchronized (Readline.this) {
if (interaction == null) {
return false;
}
interaction = null;
conn.setStdinHandler(prevReadHandler);
conn.setSizeHandler(prevSizeHandler);
conn.setEventHandler(prevEventHandler);
}
requestHandler.accept(s);
return true;
}
通過上面的分析可以看到,後續我們對命令的處理直接看ShellLineHandler
就可以了
命令的執行
public void handle(String line) {
String name = first.value();
if (name.equals("exit") || name.equals("logout") || name.equals("q") || name.equals("quit")) {
handleExit();
return;
} else if (name.equals("jobs")) {
handleJobs();
return;
} else if (name.equals("fg")) {
handleForeground(tokens);
return;
} else if (name.equals("bg")) {
handleBackground(tokens);
return;
} else if (name.equals("kill")) {
handleKill(tokens);
return;
}
Job job = createJob(tokens);
if (job != null) {
job.run();
}
}
在com.taobao.arthas.core.shell.handlers.shell.ShellLineHandler#handle
的設計中,如果是一些簡單的命令,比如說exit, logout,jobs,fg,bg,kill
等,都是直接執行的,而其他的命令都是直接通過建立一個Job來執行的,這一小節,我們主要看arthas是怎麼抽象命令的執行的:從建立Job
開始
@Override
public synchronized Job createJob(List<CliToken> args) {
Job job = jobController.createJob(commandManager, args, session, new ShellJobHandler(this), term, null);
return job;
}
會轉發到:
@Override
public Job createJob(InternalCommandManager commandManager, List<CliToken> tokens, Session session, JobListener jobHandler, Term term, ResultDistributor resultDistributor) {
int jobId = idGenerator.incrementAndGet();
StringBuilder line = new StringBuilder();
for (CliToken arg : tokens) {
line.append(arg.raw());
}
boolean runInBackground = runInBackground(tokens);
Process process = createProcess(tokens, commandManager, jobId, term, resultDistributor);
process.setJobId(jobId);
JobImpl job = new JobImpl(jobId, this, process, line.toString(), runInBackground, session, jobHandler);
jobs.put(jobId, job);
return job;
}
Job
的run
方法是完全委託給Process
的,所以接下來就直接看createProcess
的過程:
private Process createProcess(List<CliToken> line, InternalCommandManager commandManager, int jobId, Term term, ResultDistributor resultDistributor) {
try {
ListIterator<CliToken> tokens = line.listIterator();
while (tokens.hasNext()) {
CliToken token = tokens.next();
if (token.isText()) {
Command command = commandManager.getCommand(token.value());
if (command != null) {
return createCommandProcess(command, tokens, jobId, term, resultDistributor);
} else {
throw new IllegalArgumentException(token.value() + ": command not found");
}
}
}
throw new IllegalArgumentException();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
這段程式碼的意圖比較明顯,根據輸入的命令去找相應的Command
物件,如果找到則建立Process
物件,根據文字找相應Command
的邏輯如下:
public Command getCommand(String commandName) {
Command command = null;
for (CommandResolver resolver : resolvers) {
// 內建命令在ShellLineHandler裡提前處理了,所以這裡不需要再查詢內建命令
if (resolver instanceof BuiltinCommandPack) {
command = getCommand(resolver, commandName);
if (command != null) {
break;
}
}
}
return command;
}
private static Command getCommand(CommandResolver commandResolver, String name) {
List<Command> commands = commandResolver.commands();
if (commands == null || commands.isEmpty()) {
return null;
}
for (Command command : commands) {
if (name.equals(command.name())) {
return command;
}
}
return null;
}
這塊的邏輯還是比較清晰的,我們再看看在找到對應Command
之後如何建立Process
private Process createCommandProcess(Command command, ListIterator<CliToken> tokens, int jobId, Term term, ResultDistributor resultDistributor) throws IOException {
List<CliToken> remaining = new ArrayList<CliToken>();
List<CliToken> pipelineTokens = new ArrayList<CliToken>();
boolean isPipeline = false;
RedirectHandler redirectHandler = null;
List<Function<String, String>> stdoutHandlerChain = new ArrayList<Function<String, String>>();
String cacheLocation = null;
// 刪除中間處理管道和後臺程式的程式碼
ProcessOutput ProcessOutput = new ProcessOutput(stdoutHandlerChain, cacheLocation, term);
ProcessImpl process = new ProcessImpl(command, remaining, command.processHandler(), ProcessOutput, resultDistributor);
process.setTty(term);
return process;
}
在刪除了中間的處理管道和後臺命令的程式碼之後這段程式碼的邏輯也非常清晰,就是根據解析好的Command
物件建立一個Process
物件,值得注意的是,這裡把command.processHandler()
傳進了Process
的建構函式中。檢視com.taobao.arthas.core.shell.system.impl.ProcessImpl#run()
,可以看到最終會呼叫到com.taobao.arthas.core.shell.system.impl.ProcessImpl.CommandProcessTask#run
private class CommandProcessTask implements Runnable {
private CommandProcess process;
public CommandProcessTask(CommandProcess process) {
this.process = process;
}
@Override
public void run() {
try {
handler.handle(process);
} catch (Throwable t) {
logger.error("Error during processing the command:", t);
process.end(1, "Error during processing the command: " + t.getClass().getName() + ", message:" + t.getMessage()
+ ", please check $HOME/logs/arthas/arthas.log for more details." );
}
}
}
這裡的handler
正是建立Process
物件時呼叫command.processHandler()
傳進去的
// processHandler 初始化
private Handler<CommandProcess> processHandler = new ProcessHandler();
@Override
public Handler<CommandProcess> processHandler() {
return processHandler;
}
private class ProcessHandler implements Handler<CommandProcess> {
@Override
public void handle(CommandProcess process) {
process(process);
}
}
private void process(CommandProcess process) {
AnnotatedCommand instance;
try {
instance = clazz.newInstance();
} catch (Exception e) {
process.end();
return;
}
CLIConfigurator.inject(process.commandLine(), instance);
instance.process(process);
UserStatUtil.arthasUsageSuccess(name(), process.args());
}
通過instance.process(process);
就可以呼叫到具體Command
類的process
方法了,比如說我們以watch
命令為例,如果客戶端輸入的是這條命令,則會觸發程式碼的插裝
@Override
public void process(final CommandProcess process) {
// ctrl-C support
process.interruptHandler(new CommandInterruptHandler(process));
// q exit support
process.stdinHandler(new QExitHandler(process));
// start to enhance
enhance(process);
}
小結一下
整個啟動過程還是比較清晰的,需要注意的是在這個過程中有好多的回撥函式,這些回撥函式中才包含真正處理事件的邏輯,需要多翻幾遍上下文才能完全理解
本文詳細的跟了上面這個類圖中類之間的互動,伺服器抽象這個模組主要負責建立起完整的伺服器環境並監聽到達服務端的命令,到達的命令經過初步解析之後通過建立的任務類去執行,在任務的執行中通過在ShellImpl
中持有的ShellServer
引用,可以解析出具體的Command
類,最後,命令的執行會呼叫對應Command
類中的process
方法,從而完成了整個命令的執行。
掃描二維碼關注公眾號,獲取今年最新面試資料和電子書文件
相關文章
- Netty系列文章之服務端啟動分析Netty服務端
- Flutter系列三:Flutter啟動流程分析Flutter
- 聊聊Dubbo(九):核心原始碼-服務端啟動流程2原始碼服務端
- 聊聊Dubbo(九):核心原始碼-服務端啟動流程1原始碼服務端
- windows 啟動流程三Windows
- 本地啟動服務的三種方法
- Spring Cloud系列(三):Eureka原始碼解析之服務端SpringCloud原始碼服務端
- Netty原始碼分析(二):服務端啟動Netty原始碼服務端
- SpringBoot啟動流程分析原理(一)Spring Boot
- 【SpringBoot】服務 Jar 包的啟動過程原理Spring BootJAR
- Spring容器系列-啟動原理Spring
- springboot自動配置原理和啟動流程Spring Boot
- Netty原始碼解析 -- 服務端啟動過程Netty原始碼服務端
- gRPC 的增刪改查系列之啟動服務RPC
- zabbix4.0服務端 部署全流程服務端
- go微服務系列(三) - 服務呼叫(http)Go微服務HTTP
- netty原始碼分析之服務端啟動全解析Netty原始碼服務端
- [MySQL] “MySQL 服務無法啟動”原理及解決方法MySql
- Flutter Android 端啟動流程淺析FlutterAndroid
- Spring Cloud Eureka原理分析(三):註冊資訊讀取(服務端)SpringCloud服務端
- gRPC 的增刪改查系列之cmd啟動服務RPC
- flowable 啟動流程的三種方式
- React 服務端渲染原理及過程React服務端
- 遠端服務不能啟動問題的解決方法
- Netty服務端啟動過程相關原始碼分析Netty服務端原始碼
- 怎樣開啟mongodb服務端?MongoDB服務端
- gitblit 服務啟動不了Git
- linux 下啟動服務Linux
- python 啟動http服務PythonHTTP
- Go gRPC 系列三:流式客戶端和服務端GoRPC客戶端服務端
- win10 如何開啟遠端服務_win10如何開啟遠端連線服務Win10
- springboot系列文章之啟動原理詳解Spring Boot
- Linux使用Ambari啟動服務啟動失敗Linux
- 命令列重啟遠端桌面服務命令列
- Kali Linux常用服務配置教程啟動DHCP服務Linux
- 服務啟動一個程式
- dubbo服務啟動的方式
- DUBBO服務啟動過程