出售短影片平臺，多節點例項資料同步觸發的方式

云豹科技-苏凌霄發表於2024-11-23

出售短影片平臺，多節點例項資料同步觸發的方式
今天標題的內容，主要講同步如何觸發？內容已經圈定死，因此就不談資料同步涉及的一致性，只談如何觸發這個動作。多節點例項觸發的關鍵是，一旦觸發，各個節點都要通知到位。那如何進行多個節點通知呢？答案就是透過廣播。

本案例核心流程圖

實現步驟

1、定義高層廣播抽象介面

@FunctionalInterface
public interface DataSyncTrigger {

    void broadcast(Object data);
}

2、定義通知事件類
注： 本文會採用spring的事件監聽模式實現

public class DataSyncTriggerEvent extends ApplicationEvent {
    /**
     * Create a new ApplicationEvent.
     *
     * @param source the object on which the event initially occurred (never {@code null})
     */
    public DataSyncTriggerEvent(Object source) {
        super(source);
    }
}

3、定義高層抽象廣播的模板基類

@RequiredArgsConstructor
public abstract class BaseDataSyncTrigger implements DataSyncTrigger, ApplicationContextAware {
    protected ApplicationContext applicationContext;
    
    protected final DataSyncTriggerProperty dataSyncTriggerProperty;


    @Override
    public void broadcast(Object data) {
        DataSyncTriggerEvent dataSyncTriggerEvent = new DataSyncTriggerEvent(data);
        applicationContext.publishEvent(dataSyncTriggerEvent);
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext = applicationContext;
    }

    private Collection<DataSyncTriggerCallBack> listDataSyncTriggerCallBacks(){
        try {
            Map<String, DataSyncTriggerCallBack> dataSyncTriggerCallBackMap = applicationContext.getBeansOfType(DataSyncTriggerCallBack.class);
            return Collections.unmodifiableList(dataSyncTriggerCallBackMap.values().stream().collect(Collectors.toList()));
        } catch (BeansException e) {

        }

        return Collections.emptyList();
    }
    
    public void callBack(Object data){
        Collection<DataSyncTriggerCallBack> dataSyncTriggerCallBacks = listDataSyncTriggerCallBacks();
        if(CollectionUtil.isNotEmpty(dataSyncTriggerCallBacks)){
            if(dataSyncTriggerProperty.isTriggerCallBackAsync()){
                callbackAsync(data, dataSyncTriggerCallBacks);
            }else{
                callbackSync(data, dataSyncTriggerCallBacks);
            }
          
        }
    }

    private  void callbackSync(Object data, Collection<DataSyncTriggerCallBack> dataSyncTriggerCallBacks) {
        for (DataSyncTriggerCallBack dataSyncTriggerCallBack : dataSyncTriggerCallBacks) {
            dataSyncTriggerCallBack.execute(data);
        }
    }

    private  void callbackAsync(Object data, Collection<DataSyncTriggerCallBack> dataSyncTriggerCallBacks) {
        for (DataSyncTriggerCallBack dataSyncTriggerCallBack : dataSyncTriggerCallBacks) {
            ThreadUtil.execAsync(()->{
                dataSyncTriggerCallBack.execute(data);
            });
        }
    }
}

4、定義抽象回撥介面【擴充套件點】
當業務收到通知，可以透過該回撥介面進行具體業務操作

@FunctionalInterface
public interface DataSyncTriggerCallBack {

    void execute(Object data);
}

5、定義具體廣播實現類
注： 這個廣播的具體實現方案就很多了，只要天生具備廣播能力或者基於原來特性擴充套件出廣播的元件都可以，比如rocketmq的廣播機制、redis的pubsub機制、zookeeper的分散式協調能力、基於註冊中心服務發現能力改造出來的廣播能力等。本文就以redis的pubsub機制為例

Slf4j
public class RedisDataSyncTrigger extends BaseDataSyncTrigger implements CommandLineRunner {


    private final RedisTemplate redisTemplate;
    

    public RedisDataSyncTrigger(RedisTemplate redisTemplate, DataSyncTriggerProperty dataSyncTriggerProperty) {
        super(dataSyncTriggerProperty);
        this.redisTemplate = redisTemplate;
    }

    @EventListener
    public void listener(DataSyncTriggerEvent dataSyncTriggerEvent){
        SyncDataDTO syncDataDTO = SyncDataDTO.builder()
                .data(dataSyncTriggerEvent.getSource())
                        .timeStamp(System.currentTimeMillis())
                                .build();
        try {
            redisTemplate.convertAndSend(REDIS_CHANNEL_KEY, syncDataDTO);
        } catch (Exception e) {
           log.error("redis publish channel 【" + REDIS_CHANNEL_KEY + "】 fail,cause:" + e.getMessage(),e);
        }
    }


    @Override
    public void run(String... args) throws Exception {
        doSubscribe();
    }

    @SneakyThrows
    private void doSubscribe() {
        RedisConnection connection = redisTemplate.getConnectionFactory().getConnection();
        RedisMessageListener redisMessageListener = applicationContext.getBean(RedisMessageListener.class);
        connection.subscribe(redisMessageListener,REDIS_CHANNEL_KEY.getBytes("utf-8"));
        log.info(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Register listen channel : 【{}】",REDIS_CHANNEL_KEY);
    }
}

具體redis訂閱監聽實現

@RequiredArgsConstructor
@Slf4j
public class RedisMessageListener implements MessageListener{

    private final BaseDataSyncTrigger baseDataSyncTrigger;

    private final RedisTemplate redisTemplate;


    @Override
    public void onMessage(Message message, byte[] pattern) {
        byte[] body = message.getBody();
        String dataJson = StrUtil.str(body, "utf-8");
        if(JSONUtil.isJson(dataJson)){
            try {
                SyncDataDTO dataDTO = (SyncDataDTO) redisTemplate.getHashValueSerializer().deserialize(body);
                baseDataSyncTrigger.callBack(dataDTO.getData());
            } catch (Exception e) {
                log.error(e.getMessage(),e);
            }
        }else{
            log.warn(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Data 【{}】 is not match json format ！！！",dataJson);
        }

    }
}

6、測試驗證
a、編寫業務邏輯類

@Service
@RequiredArgsConstructor
@Slf4j
public class DataService {

    private List<Object> dataList = new CopyOnWriteArrayList<>();

    private final RedisTemplate redisTemplate;

    private final BaseDataSyncTrigger dataSyncTrigger;

    public boolean add(String data){
        try {
            Long count = redisTemplate.opsForList().leftPush(RedisConstant.REDIS_LIST_KEY, data);
            if(count > 0){
                dataSyncTrigger.broadcast(data);
                return true;
            }
        } catch (Exception e) {
           log.error("add fail:" + e.getMessage(),e);
        }

        return false;

    }

    public List<Object> getDataList(){
        return dataList;
    }
}

b、編寫業務控制器

@RestController
@RequestMapping("data")
@RequiredArgsConstructor
public class DataController {
    


    private final DataService dataService;

    @GetMapping("add/{data}")
    public String syncData(@PathVariable("data") String data){
        boolean isSuccess = dataService.add(data);
        return isSuccess ? "success" : "fail";
    }

    @GetMapping("list")
    public List<Object> listData(){
        return dataService.getDataList();
    }
}

c、編寫業務回撥類

@Component
@RequiredArgsConstructor
@Slf4j
public class LocalListDataSyncTriggerCallBack implements DataSyncTriggerCallBack {

    private final DataService dataService;

    @Override
    public void execute(Object data) {
        dataService.getDataList().add(data);
        log.info(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sync data:-->{}",data);
    }
}

d、小細節

注： 當專案重啟時，本地儲存容器是沒內容的，因此需要在專案重啟時，寫一個鉤子，從其他快取介質將資料刷到本地儲存中

@Component
@RequiredArgsConstructor
@Slf4j
public class DataInitTask implements CommandLineRunner {

    private final RedisTemplate redisTemplate;

    private final DataService dataService;


    @Override
    public void run(String... args) throws Exception {
        List redisDataList = redisTemplate.opsForList().range(RedisConstant.REDIS_LIST_KEY, 0, -1);
        if(CollectionUtil.isNotEmpty(redisDataList)){
            dataService.getDataList().addAll(redisDataList);
            log.info(">>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Loaded data from redis finished!!!");
        }

    }


}

e、測試

從一個節點（示例：54860埠）新增資料，如圖

觀察其他節點（示例：59829埠）本地儲存是否接收到資料

從圖可以發現已經收到資料，同時我們觀察控制檯

可以看出業務回撥已經觸發

以上就是出售短影片平臺，多節點例項資料同步觸發的方式， 更多內容歡迎關注之後的文章

相關文章