一、概述
上篇文章介紹了木舟透過基於木舟平臺淺談surging 的熱點KEY的解決方法 ，那麼此篇文章將介紹基於surging的木舟平臺如何分散式接入裝置.
木舟 (Kayak) 是什麼?
木舟(Kayak)是基於.NET6.0軟體環境下的surging微服務引擎進行開發的, 平臺包含了微服務和物聯網平臺。支援非同步和響應式程式設計開發，功能包含了物模型,裝置,產品,網路元件的統一管理和微服務平臺下的註冊中心，服務路由，模組，中間服務等管理。還有多協議適配(TCP,MQTT,UDP,CoAP,HTTP,Grpc,websocket,rtmp,httpflv,webservice,等),透過靈活多樣的配置適配能夠接入不同廠家不同協議等裝置。並且透過裝置告警,訊息通知,資料視覺化等功能。能夠讓你能快速建立起微服務物聯網平臺系統。
那麼下面就為大家介紹如何從建立元件、協議、裝置閘道器，裝置到裝置閘道器接入，再到裝置資料上報，把整個流程透過此篇文章進行闡述。
木舟kayal 平臺開源地址：https://github.com/microsurging/
surging 微服務引擎開源地址：https://github.com/fanliang11/surging（後面surging 會移動到microsurging進行維護）
二、網路元件
1.編輯建立Tcp協議的網路元件，可以選擇獨立配置(獨立配置是叢集模式). 下圖是解析方式選擇了自定義指令碼進行解碼操作。
選擇了獨立配置後(共享配置不會進行註冊路由)，如果架設了新的閘道器例項，就會在註冊中心networkroute/tcp路徑下新增服務節點，以consul註冊中心為例，開啟：http://127.0.0.1:8500
三、自定義協議
- 如何建立自定義協議模組
如果是網路程式設計開發，必然會涉及到協議報文的編碼解碼處理，那麼對於平臺也是做到了靈活處理，首先是協議模組建立，透過以下程式碼看出協議模組可以新增協議說明md文件, 身份鑑權處理，訊息編解碼，後設資料配置。下面一一介紹如何進行編寫
public class Demo3ProtocolSupportProvider : ProtocolSupportProvider
{
public override IObservable<ProtocolSupport> Create(ProtocolContext context)
{
var support = new ComplexProtocolSupport();
support.Id = "demo_3";
support.Name = "演示協議3";
support.Description = "演示協議3";
support.AddAuthenticator(MessageTransport.Tcp, new Demo5Authenticator());
support.AddDocument(MessageTransport.Tcp, "Document/document-tcp.md");
support.Script = "\r\nvar decode=function(buffer)\r\n{\r\n parser.Fixed(5).Handler(\r\n function(buffer){ \r\n var bytes = BytesUtils.GetBytes(buffer,1,4);\r\n var len = BytesUtils.LeStrToInt(bytes,1,4);//2. 獲取訊息長度.\r\n var buf = BytesUtils.Slice(buffer,0,5); \r\n parser.Fixed(len).Result(buf); \r\n }).Handler(function(buffer){ parser.Result(buffer).Complete(); \r\n }\r\n )\r\n}\r\nvar encode=function(buffer)\r\n{\r\n}";
support.AddMessageCodecSupport(MessageTransport.Tcp, () => Observable.Return(new ScriptDeviceMessageCodec(support.Script)));
support.AddConfigMetadata(MessageTransport.Tcp, _tcpConfig);
support.AddAuthenticator(MessageTransport.Udp, new Demo5Authenticator());
support.Script = "\r\nvar decode=function(buffer)\r\n{\r\n parser.Fixed(5).Handler(\r\n function(buffer){ \r\n var bytes = BytesUtils.GetBytes(buffer,1,4);\r\n var len = BytesUtils.LeStrToInt(bytes,1,4);//2. 獲取訊息長度.\r\n var buf = BytesUtils.Slice(buffer,0,5); \r\n parser.Fixed(len).Result(buf); \r\n }).Handler(function(buffer){ parser.Result(buffer).Complete(); \r\n }\r\n )\r\n}\r\nvar encode=function(buffer)\r\n{\r\n}";
support.AddMessageCodecSupport(MessageTransport.Udp, () => Observable.Return(new ScriptDeviceMessageCodec(support.Script)));
support.AddConfigMetadata(MessageTransport.Udp, _udpConfig);
return Observable.Return(support);
}
}
1. 新增協議說明文件如程式碼: support.AddDocument(MessageTransport.Tcp, "Document/document-tcp.md");，文件僅支援 markdown檔案，如下所示
### 認證說明
CONNECT報文:
```text
clientId: 裝置ID
password: md5(timestamp+"|"+secureKey)
```
2. 新增身份鑑權如程式碼: support.AddAuthenticator(MessageTransport.Http, new Demo5Authenticator()) ，自定義身份鑑權Demo5Authenticator 程式碼如下：
public class Demo5Authenticator : IAuthenticator
{
public IObservable<AuthenticationResult> Authenticate(IAuthenticationRequest request, IDeviceOperator deviceOperator)
{
var result = Observable.Return<AuthenticationResult>(default);
if (request is DefaultAuthRequest)
{
var authRequest = request as DefaultAuthRequest;
deviceOperator.GetConfig(authRequest.GetTransport()==MessageTransport.Http?"token": "key").Subscribe( config =>
{
var password = config.Convert<string>();
if (authRequest.Password.Equals(password))
{
result= result.Publish(AuthenticationResult.Success(authRequest.DeviceId));
}
else
{
result= result.Publish(AuthenticationResult.Failure(StatusCode.CUSTOM_ERROR, "驗證失敗,密碼錯誤"));
}
});
}
else
result = Observable.Return<AuthenticationResult>(AuthenticationResult.Failure(StatusCode.CUSTOM_ERROR, "不支援請求引數型別"));
return result;
}
public IObservable<AuthenticationResult> Authenticate(IAuthenticationRequest request, IDeviceRegistry registry)
{
var result = Observable.Return<AuthenticationResult>(default);
var authRequest = request as DefaultAuthRequest;
registry
.GetDevice(authRequest.DeviceId)
.Subscribe(async p => {
var config= await p.GetConfig(authRequest.GetTransport() == MessageTransport.Http ? "token" : "key");
var password= config.Convert<string>();
if(authRequest.Password.Equals(password))
{
result= result.Publish(AuthenticationResult.Success(authRequest.DeviceId));
}
else
{
result= result.Publish(AuthenticationResult.Failure(StatusCode.CUSTOM_ERROR, "驗證失敗,密碼錯誤"));
}
});
return result;
}
}
3.新增訊息編解碼程式碼 support.AddMessageCodecSupport(MessageTransport.Tcp, () => Observable.Return(new ScriptDeviceMessageCodec(support.Script)));， 可以自定義編解碼，ScriptDeviceMessageCodec程式碼如下：
using DotNetty.Buffers;
using Jint;
using Jint.Parser;
using Microsoft.CodeAnalysis.Scripting;
using Microsoft.Extensions.Logging;
using RulesEngine.Models;
using Surging.Core.CPlatform.Codecs.Core;
using Surging.Core.CPlatform.Utilities;
using Surging.Core.DeviceGateway.Runtime.Device.Message;
using Surging.Core.DeviceGateway.Runtime.Device.Message.Event;
using Surging.Core.DeviceGateway.Runtime.Device.Message.Property;
using Surging.Core.DeviceGateway.Runtime.Device.MessageCodec;
using Surging.Core.DeviceGateway.Runtime.RuleParser.Implementation;
using Surging.Core.DeviceGateway.Utilities;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Linq;
using System.Reactive.Subjects;
using System.Runtime;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
namespace Surging.Core.DeviceGateway.Runtime.Device.Implementation
{
public class ScriptDeviceMessageCodec : DeviceMessageCodec
{
public string GlobalVariable { get; private set; }
public string EncoderScript { get; private set; }
public string DecoderScript { get; private set; }
public IObservable<Task<RulePipePayloadParser>> _rulePipePayload;
private readonly ILogger<ScriptDeviceMessageCodec> _logger;
public ScriptDeviceMessageCodec(string script) {
_logger = ServiceLocator.GetService<ILogger<ScriptDeviceMessageCodec>>();
RegexOptions options = RegexOptions.Singleline | RegexOptions.IgnoreCase;
string matchStr = Regex.Match(script, @"var\s*[\w$]*\s*\=.*function.*\(.*\)\s*\{[\s\S]*\}.*?v", options).Value;
if (!string.IsNullOrEmpty(matchStr))
{
DecoderScript = matchStr.TrimEnd('v');
DecoderScript= Regex.Replace(DecoderScript, @"var\s*[\w$]*\s*\=[.\r|\n|\t|\s]*?(function)\s*\([\w$]*\s*\)\s*\{", "", RegexOptions.IgnoreCase);
DecoderScript= DecoderScript.Slice(0, DecoderScript.LastIndexOf('}'));
EncoderScript = script.Replace(DecoderScript, "");
}
var matchStr1 = Regex.Matches(script, @"(?<=var).*?(?==)|(?=;)|(?=v)", options).FirstOrDefault(p=>!string.IsNullOrEmpty(p.Value))?.Value;
if (!string.IsNullOrEmpty(matchStr1))
{
GlobalVariable = matchStr1.TrimEnd(';');
}
var ruleWorkflow = new RuleWorkflow(DecoderScript);
_rulePipePayload= Observable.Return( GetParser( GetRuleEngine(ruleWorkflow), ruleWorkflow));
}
public override IObservable<IDeviceMessage> Decode(MessageDecodeContext context)
{
var result = Observable.Return<IDeviceMessage>(null);
_rulePipePayload.Subscribe(async p =>
{
var parser = await p;
parser.Build(context.GetMessage().Payload);
parser.HandlePayload().Subscribe(async p =>
{
try
{
var headerBuffer=parser.GetResult().FirstOrDefault();
var buffer = parser.GetResult().LastOrDefault();
var str = buffer.GetString(buffer.ReaderIndex, buffer.ReadableBytes, Encoding.UTF8);
var session = await context.GetSession();
if (session?.GetOperator() == null)
{
var onlineMessage = JsonSerializer.Deserialize<DeviceOnlineMessage>(str);
result = result.Publish(onlineMessage);
}
else
{
var messageType = headerBuffer.GetString(0, 1, Encoding.UTF8);
if (Enum.Parse<MessageType>(messageType.ToString()) == MessageType.READ_PROPERTY)
{
var onlineMessage = JsonSerializer.Deserialize<ReadPropertyMessage>(str);
result = result.Publish(onlineMessage);
}
else if (Enum.Parse<MessageType>(messageType.ToString()) == MessageType.EVENT)
{
var onlineMessage = JsonSerializer.Deserialize<EventMessage>(str);
result = result.Publish(onlineMessage);
}
}
}
catch (Exception e)
{
}
finally
{
p.Release();
parser.Close();
}
});
});
return result;
}
public override IObservable<IEncodedMessage> Encode(MessageEncodeContext context)
{
context.Reply(((RespondDeviceMessage<IDeviceMessageReply>)context.Message).NewReply().Success(true));
return Observable.Empty<IEncodedMessage>();
}
private RulesEngine.RulesEngine GetRuleEngine(RuleWorkflow ruleWorkflow)
{
var reSettingsWithCustomTypes = new ReSettings { CustomTypes = new Type[] { typeof(RulePipePayloadParser) } };
var result = new RulesEngine.RulesEngine(new Workflow[] { ruleWorkflow.GetWorkflow() }, null, reSettingsWithCustomTypes);
return result;
}
private async Task<RulePipePayloadParser> GetParser(RulesEngine.RulesEngine engine, RuleWorkflow ruleWorkflow)
{
var payloadParser = new RulePipePayloadParser();
var ruleResult = await engine.ExecuteActionWorkflowAsync(ruleWorkflow.WorkflowName, ruleWorkflow.RuleName, new RuleParameter[] { new RuleParameter("parser", payloadParser) });
if (ruleResult.Exception != null && _logger.IsEnabled(LogLevel.Error))
_logger.LogError(ruleResult.Exception, ruleResult.Exception.Message);
return payloadParser;
}
}
}
4.新增後設資料配置程式碼 support.AddConfigMetadata(MessageTransport.Tcp, _tcpConfig); _tcpConfig程式碼如下:
private readonly DefaultConfigMetadata _tcpConfig = new DefaultConfigMetadata(
"TCP認證配置"
, "key為tcp認證金鑰")
.Add("tcp_auth_key", "key", "TCP認證KEY", StringType.Instance);
- 如何載入協議模組，協議模組包含了協議模組支援自定義指令碼、新增引用、上傳熱部署載入。
自定義指令碼，選擇了自定義指令碼解析，如果本地有設定訊息編解碼，會進行覆蓋
引用載入模組
上傳熱部署協議模組
首先利用以下命令釋出模組：
然後打包上傳協議模組
四、裝置閘道器
建立TCP裝置閘道器
五、產品管理
以下是新增產品。
裝置接入
六、裝置管理
新增裝置
Tcp認證配置
新增告警閾值
事件定義
七、測試
利用測試工具進行Tcp測試，以呼叫tcp://127.0.0.1:993為例，
測試裝置上線
字串： 293\0\0{"MessageType":2,"Headers":{"token":"123456"},"DeviceId":"scro-34","Timestamp":1726540220311}
說明：第一個字元表示型別，第二個表示訊息內容長度
16進位制:32393300007b224d65737361676554797065223a322c2248656164657273223a7b22746f6b656e223a22313233343536227d2c224465766963654964223a227363726f2d3334222c2254696d657374616d70223a313732363534303232303331317d
結果如下：
測試上報屬性
字串：195\0\0{"MessageType":1,"Properties":{"temp":"38.24"},"DeviceId":"scro-34","Timestamp":1726560007339}
16進位制:31393500007b224d65737361676554797065223a312c2250726f70657274696573223a7b2274656d70223a2233382e3234227d2c224465766963654964223a227363726f2d3334222c2254696d657374616d70223a313732363536303030373333397d
結果如下：
測試事件
字串：8307\0{"MessageType":8,"Data":{"deviceId":"scro-34","level":"alarm","alarmTime":"2024-11-07 19:47:00","from":"device","alarmType":"裝置告警","coordinate":"33.345,566.33","createTime":"2024-11-07 19:47:00","desc":"溫度超過閾值"},"DeviceId":"scro-34","EventId":"alarm","Timestamp":1726540220311}
16進位制:38333037007b224d65737361676554797065223a382c2244617461223a7b226465766963654964223a227363726f2d3334222c226c6576656c223a22616c61726d222c22616c61726d54696d65223a22323032342d31312d30372031393a34373a3030222c2266726f6d223a22646576696365222c22616c61726d54797065223a22e8aebee5a487e5918ae8ada6222c22636f6f7264696e617465223a2233332e3334352c3536362e3333222c2263726561746554696d65223a22323032342d31312d30372031393a34373a3030222c2264657363223a22e6b8a9e5baa6e8b685e8bf87e99888e580bc227d2c224465766963654964223a227363726f2d3334222c224576656e744964223a22616c61726d222c2254696d657374616d70223a313732363534303232303331317d
結果如下：
可以在平臺介面看到上報的資料
以下是基於http接入裝置的測試
以下是基於mqtt接入裝置測試
八、總結
以上是基於介紹如何分散式接入裝置, 等完成mqtt和國標28181裝置接入，會搭建官方網站和DEMO，敬請期待，預計明年3月前完成，會搭建官方網站和DEMO，敬請期待。