.Net Core下使用RabbitMQ比較完備的兩種方案(雖然程式碼有點慘淡,不過我會完善)

大魔王先生發表於2019-05-24

一、前言

    上篇說給大家來寫C#和Java的方案,最近工作也比較忙,遲到了一些,我先給大家補上C#的方案,另外如果沒看我上篇部落格的人最好看一下,否則你可能看的雲裡霧裡的,這裡我就不進行具體的方案畫圖了;傳送門

二、使用的外掛

    HangFire

    一個開源的.NET任務排程框架,最大特點在於內建提供整合化的控制檯,方便後臺檢視及監控,支援多種儲存方式;在方案中主要使用定時任務做補償機制,後期可能會封裝一些,能通過頁面的形式直接新增任務;

   NLog

   日誌記錄框架,方案中使用記錄日誌,後期可能回整合多個日誌框架;

   Autofac

   依賴注入的框架,應該不用做過多介紹;

  SqlSugar

  ORM框架,這個從剛開始我就在使用了,在現在公司沒有推行起來,不過在上兩家公司都留下的遺產,據說還用的可以,當然我還是最佩服作者;

  Polly

  容錯服務框架,類似於Java下的Hystrix,主要是為了解決分散式系統中,系統之間相互依賴,可能會因為多種因素導致服務不可用的而產生的一套框架,支援服務的超時重試、限流、熔斷器等等;

  RabbitMQ.Client

  官方提供的C#連線RabbitMQ的SDK;

三、方案

  模擬一個簡單訂單下單的場景,沒有進行具體的實現。同時建議下游服務不要寫在web端,最好以服務的形式奔跑,程式碼中是Web端實現的,大家不要這麼搞。整體上還是實現了之前提到的兩種方案:一是入庫打標,二是延時佇列(這塊沒有進行很好的測試,但是估計也沒有很大的問題);當然也是有一些特點:RabbitMQ當機情況下無需重啟服務,網路異常的情況下也可以進行斷線重連。接下來聊下程式碼和各方外掛在系統中的具體應用:

  專案結構:

  

  RabbitMQExtensions:

  採用Autofac按照單例的形式注入,採用Polly進行斷線重連,也開啟了自身斷線重連和心跳檢測機制,配置方面採用最簡單的URI規範進行配置,有興趣參考下官方,整體上這塊程式碼還相對比較規範,以後可能也不會有太多調整;

    /// <summary>
    /// rabbitmq持久化連線
    /// </summary>
    public interface IRabbitMQPersistentConnection
    {
        bool IsConnected { get; }

        bool TryConnect();

        IModel CreateModel();
    }
     /// <summary>
    /// rabbitmq持久化連線具體實現
    /// </summary>
    public class DefaultRabbitMQPersistentConnection : IRabbitMQPersistentConnection
    {
        private readonly IConnectionFactory connectionFactory;
        private readonly ILogger<DefaultRabbitMQPersistentConnection> logger;

        private IConnection connection;

        private const int RETTRYCOUNT = 6;

        private static readonly object lockObj = new object();
        public DefaultRabbitMQPersistentConnection(IConnectionFactory connectionFactory, ILogger<DefaultRabbitMQPersistentConnection> logger)
        {
            this.connectionFactory = connectionFactory;
            this.logger = logger;
        }

        public bool IsConnected
        {
            get
            {
                return connection != null && connection.IsOpen;
            }
        }

        public void Cleanup()
        {
            try
            {
                connection.Dispose();
                connection.Close();
                connection = null;

            }
            catch (IOException ex)
            {
                logger.LogCritical(ex.ToString());
            }
        }

        public IModel CreateModel()
        {
            if (!IsConnected)
            {
                connection.Close();
                throw new InvalidOperationException("連線不到rabbitmq");
            }
            return connection.CreateModel();
        }

        public bool TryConnect()
        {
            logger.LogInformation("RabbitMQ客戶端嘗試連線");

            lock (lockObj)
            {
                if (connection == null)
                {
                    var policy = RetryPolicy.Handle<SocketException>()
                        .Or<BrokerUnreachableException>()
                        .WaitAndRetry(RETTRYCOUNT, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)), (ex, time) =>    
                        {
                            logger.LogWarning(ex.ToString());
                        });

                    policy.Execute(() =>
                    {
                        connection = connectionFactory.CreateConnection();
                    });
                }



                if (IsConnected)
                {
                    connection.ConnectionShutdown += OnConnectionShutdown;
                    connection.CallbackException += OnCallbackException;
                    connection.ConnectionBlocked += OnConnectionBlocked;

                    logger.LogInformation($"RabbitMQ{connection.Endpoint.HostName}獲取了連線");

                    return true;
                }
                else
                {
                    logger.LogCritical("無法建立和開啟RabbitMQ連線");

                    return false;
                }
            }
        }


        private void OnConnectionBlocked(object sender, ConnectionBlockedEventArgs e)
        {

            logger.LogWarning("RabbitMQ連線異常,嘗試重連...");

            Cleanup();
            TryConnect();
        }

        private void OnCallbackException(object sender, CallbackExceptionEventArgs e)
        {

            logger.LogWarning("RabbitMQ連線異常,嘗試重連...");

            Cleanup();
            TryConnect();
        }

        private void OnConnectionShutdown(object sender, ShutdownEventArgs reason)
        {

            logger.LogWarning("RabbitMQ連線異常,嘗試重連...");

            Cleanup();
            TryConnect();
        }
    }
View Code

  OrderDal

  SqlSugar的一些簡單封裝,有些小特點:大家可以可以通過配置來實現讀寫分離,採用倉儲設計。如果不太喜歡這麼寫,也可以參考傑哥的做法;

    //倉儲設計
    public interface IBaseDal<T> where T:class,new()
    {
        DbSqlSugarClient DbContext { get; }

        IBaseDal<T> UserDb(string dbName);
        IInsertable<T> AsInsertable(T t);
        IInsertable<T> AsInsertable(T[] t);
        IInsertable<T> AsInsertable(List<T> t);
        IUpdateable<T> AsUpdateable(T t);
        IUpdateable<T> AsUpdateable(T[] t);
        IUpdateable<T> AsUpdateable(List<T> t);
        IDeleteable<T> AsDeleteable();

        List<T> GetList();
        Task<List<T>> GetListAnsync();

        List<T> GetList(Expression<Func<T,bool>> whereExpression);
        Task<List<T>> GetListAnsync(Expression<Func<T, bool>> whereExpression);

        List<T> GetList(Expression<Func<T, bool>> whereExpression, Expression<Func<T, object>> orderExpression, OrderByType orderByType = OrderByType.Desc);
        Task<List<T>> GetListAnsync(Expression<Func<T, bool>> whereExpression, Expression<Func<T, object>> orderExpression, OrderByType orderByType = OrderByType.Desc);

        List<T> GetPageList(Expression<Func<T, bool>> whereExpression, PageModel page);
        Task<List<T>> GetPageListAsync(Expression<Func<T, bool>> whereExpression, PageModel page);

        List<T> GetPageList(Expression<Func<T, bool>> whereExpression, PageModel page, Expression<Func<T, object>> orderByExpression = null, OrderByType orderByType = OrderByType.Asc);
        Task<List<T>> GetPageListAsync(Expression<Func<T, bool>> whereExpression, PageModel page, Expression<Func<T, object>> orderByExpression = null, OrderByType orderByType = OrderByType.Asc);

        int Count(Expression<Func<T, bool>> whereExpression);
        Task<int> CountAsync(Expression<Func<T, bool>> whereExpression);
        T GetById(dynamic id);
        T GetSingle(Expression<Func<T, bool>> whereExpression);
        Task<T> GetSingleAsync(Expression<Func<T, bool>> whereExpression);
        T GetFirst(Expression<Func<T, bool>> whereExpression);
        Task<T> GetFirstAsync(Expression<Func<T, bool>> whereExpression);

        bool IsAny(Expression<Func<T, bool>> whereExpression);
        Task<bool> IsAnyAsync(Expression<Func<T, bool>> whereExpression);

        bool Insert(T t);
        Task<bool> InsertAsync(T t);
        bool InsertRange(List<T> t);
        Task<bool> InsertRangeAsync(List<T> t);
        bool InsertRange(T[] t);
        Task<bool> InsertRangeAsync(T[] t);
        int InsertReturnIdentity(T t);
        Task<long> InsertReturnIdentityAsync(T t);


        bool Delete(Expression<Func<T, bool>> whereExpression);
        Task<bool> DeleteAsync(Expression<Func<T, bool>> whereExpression);
        bool Delete(T t);
        Task<bool> DeleteAsync(T t);
        bool DeleteById(dynamic id);
        Task<bool> DeleteByIdAsync(dynamic id);
        bool DeleteByIds(dynamic[] ids);
        Task<bool> DeleteByIdsAsync(dynamic[] ids);


        bool Update(Expression<Func<T, T>> columns, Expression<Func<T, bool>> whereExpression);
        Task<bool> UpdateAsync(Expression<Func<T, T>> columns, Expression<Func<T, bool>> whereExpression);
        bool Update(T t);
        Task<bool> UpdateAsync(T t);
        bool UpdateRange(T[] t);
        Task<bool> UpdateRangeAsync(T[] t);


        void BeginTran();
        void CommitTran();
        void RollbackTran();


    }
View Code

  OrderCommon

  定義全域性異常的中介軟體,還有包含一些用到的實體等等,這部分程式碼還可優化拆分一下;

  OrderService

  生產者和消費者的具體實現,這塊我還想在改造一番,將消費和業務分割開,現在寫的很凌亂,不建議這麼寫,先把程式碼放出來,看看大家贊同不贊同我的這些用法,可以討論,也歡迎爭論,雖然這塊程式碼寫的不好,但是其實裡面涉及一些RabbitMQ回撥函式的用法,也是比較重要的,沒有這些函式也就實現不了我上面說那兩個特點;

//RabbitMQ當機以後回撥
//客戶端這塊大家不要採用遞迴呼叫恢復連結
//具體為什麼大家可以測試下,這裡留點小疑問哈哈
connection.ConnectionShutdown += OnConnectionShutdown;

//消費端異常以後回撥
consumerchannel.CallbackException += OnOnConsumerMessageAndWriteMessageLogException;

  Order

  具體的呼叫者,大家應該根據方法名字就能區分出我上面提到的兩種方案的設計,整體的設計思路都是最終一致,延時佇列傳送訊息這塊最終也是可以通過定時任務來實現最終一致,實現方式有很多種,簡單來說下可以通過入庫時生成的快取機制,通過定時任務來進行補償實現,這塊我沒有進行具體實現,有興趣我們可以探討下這個方案;

    [Route("api/[controller]/[action]")]
    [ApiController]
    public class OrderController : ControllerBase
    {
        private readonly IBaseDal<OrderMessageLogEntity> orderBaseDal;

        private readonly IMessageService<OrderMessageLogEntity> messageService;

        private readonly IConsumerMessageService consumerMessageService;

        private const string EXCHANGENAME = "order";

        private const string QUEUENAME = "order";

        private const string ROUTINGKEY = "order";


        public OrderController(IBaseDal<OrderMessageLogEntity> orderBaseDal, IMessageService<OrderMessageLogEntity> messageService,IConsumerMessageService consumerMessageService)
        {
            this.orderBaseDal = orderBaseDal;
            this.messageService = messageService;
            this.consumerMessageService = consumerMessageService;
        }

        /// <summary>
        /// 建立訂單
        /// </summary>
        /// <returns></returns>
        public ActionResult<bool> CreateOrder(long userId)
        {
            //建立訂單成功
            OrderEntity orderEntity = new OrderEntity();
            Random random= new Random();
            orderEntity.OrderId = random.Next();
            orderEntity.OrderNo = random.Next();
            orderEntity.UserId = userId;
            orderEntity.OrderInfo = random.Next() + "詳情";
            //bool isCreateOrderSuccress = orderService.CreateOrder(orderId);
            //if (!isCreateOrderSuccress)
            //{
            //    throw new Exception("建立訂單失敗");
            //}
            //建立訂單成功以後開始入訊息記錄庫
            //訊息建議設計的冗餘一些方便以後好查詢
            //千萬級以後連表太困難
            //建議冗餘的資訊有使用者資訊、訂單資訊、方便以後按照這個核對資訊
            //訊息表的建議是按照不同的業務進行分表儲存
            Random messageRandom = new Random();
            OrderMessageLogEntity orderMessageLog = new OrderMessageLogEntity();
            orderMessageLog.MessageId = messageRandom.Next();
            orderMessageLog.MessageInfo = orderEntity.OrderId+"訂單資訊";
            orderMessageLog.Status = (int)MessageStatusEnum.SENDING;
            orderMessageLog.OrderId = orderEntity.OrderId;
            orderMessageLog.UserId = orderEntity.UserId;
            orderMessageLog.CreateTime = DateTime.Now;
            orderMessageLog.UpdateTime = DateTime.Now;
            orderMessageLog.TryCount = 0;
            orderMessageLog.NextRetryTime = DateTime.Now.AddMinutes(5);
            //必須保證訊息先落庫
            bool isCreateOrderMessageLosSuccess = orderBaseDal.Insert(orderMessageLog);
            if (!isCreateOrderMessageLosSuccess)
                throw new Exception("訊息入庫異常");

            Message message = new Message();
            message.ExchangeName = EXCHANGENAME;
            message.QueueName = QUEUENAME;
            message.MessageId = orderMessageLog.MessageId;
            message.RoutingKey = ROUTINGKEY;
            message.Body = Encoding.UTF8.GetBytes(orderMessageLog.MessageInfo);


            //落庫成功以後開始傳送訊息到MQ
            //這個地方採用最終一致而不去使用分散式事物最終一致
            messageService.SendMessage(message, orderMessageLog);



            return true;
        }


        /// <summary>
        /// 消費訂單
        /// </summary>
        /// <returns></returns>
        public ActionResult<bool> ConsumerOrder()
        {
            Message message = new Message();
            message.ExchangeName = EXCHANGENAME;
            message.QueueName = QUEUENAME;
            message.RoutingKey = ROUTINGKEY;

            consumerMessageService.ConsumerMessage();

            return true;
        }



        /// <summary>
        /// 通過延時佇列傳送訊息
        /// </summary>
        /// <param name="userId"></param>
        /// <returns></returns>
        public ActionResult<bool> CreateDelayCreateOrder(long userId)
        {
            //建立訂單成功
            OrderEntity orderEntity = new OrderEntity();
            Random random = new Random();
            orderEntity.OrderId = random.Next();
            orderEntity.OrderNo = random.Next();
            orderEntity.UserId = userId;
            orderEntity.OrderInfo = random.Next() + "詳情";
            //bool isCreateOrderSuccress = orderService.CreateOrder(orderId);
            //if (!isCreateOrderSuccress)
            //{
            //    throw new Exception("建立訂單失敗");
            //}
            //建立訂單成功以後開始入訊息記錄庫
            //訊息建議設計的冗餘一些方便以後好查詢
            //千萬級以後連表太困難
            //建議冗餘的資訊有使用者資訊、訂單資訊、方便以後按照這個核對資訊
            //訊息表的建議是按照不同的業務進行分表儲存
            Random messageRandom = new Random();
            OrderMessageLogEntity orderMessageLog = new OrderMessageLogEntity();
            orderMessageLog.MessageId = messageRandom.Next();
            orderMessageLog.MessageInfo = orderEntity.OrderId + "訂單資訊";
            orderMessageLog.Status = (int)MessageStatusEnum.SENDING;
            orderMessageLog.OrderId = orderEntity.OrderId;
            orderMessageLog.UserId = orderEntity.UserId;
            orderMessageLog.CreateTime = DateTime.Now;
            orderMessageLog.UpdateTime = DateTime.Now;
            orderMessageLog.TryCount = 0;
            orderMessageLog.NextRetryTime = DateTime.Now.AddMinutes(5);
            ////必須保證訊息先落庫
            //bool isCreateOrderMessageLosSuccess = orderBaseDal.Insert(orderMessageLog);
            //if (!isCreateOrderMessageLosSuccess)
            //    throw new Exception("訊息入庫異常");

            Message message = new Message();
            message.ExchangeName = EXCHANGENAME;
            message.QueueName = QUEUENAME;
            message.MessageId = orderMessageLog.MessageId;
            message.RoutingKey = ROUTINGKEY;
            message.Body = Encoding.UTF8.GetBytes(orderMessageLog.MessageInfo);

            //這裡的設計是不進行落庫
            //假如兩條訊息都失敗必須藉助定時任務去對比訊息庫和訂單庫的訊息id然後進行再補發
            //剩下的只要有一條傳送成功其實就能保證下游必然會消費調這條訊息,排除下游消費異常的情況 這個地方我不在進行實現自己可腦補一下
            //開始傳送訊息到MQ
            messageService.SendMessage(message, orderMessageLog);

            //傳送延時訊息
            messageService.SendDelayMessage(message, orderMessageLog);

            return true;

        }

        /// <summary>
        /// 消費訊息以後併入庫
        /// </summary>
        /// <returns></returns>
        public ActionResult<bool> ConsumerOrderAndWirteMessageLog()
        {
            consumerMessageService.ConsumerMessageAndWriteMessageLog();

            return true;
        }


        /// <summary>
        /// 消費延時訊息
        /// 進行二次檢查核對
        /// </summary>
        /// <returns></returns>
        public ActionResult<bool> ConsumerDelayOrder()
        {
            consumerMessageService.ConsumerDelayMessage();

            return true;
        }
    }
View Code

  HangfireExtensions

  Hangfire定時框架,採用Mysql作為持久層的儲存,寫的也比較清晰,後期就是針對這些進行擴充套件,實現在介面就能新增定時任務;

四、結束

  生產端和消費端這段程式碼寫的凌亂,希望大家不要介意這一點,是有原因的,這裡我就不說了。希望大家看到閃光點,不要在一點上糾結;下次會加入Elasticsearch和監控部分的時候我會把這塊程式碼改掉,還大家一片整潔的世界;

  Github地址:https://github.com/wangtongzhou520/rabbitmq.git  有什麼問題大家可以問我;

  歡迎大家加群438836709!歡迎大家關注我!

  

相關文章