druid查詢原始碼追蹤

待整理和完善
org.apache.druid.cli.Main --> CliBroker getModules()--> BrokerQueryResource.getQueryTargets()
 --> 1.List<LocatedSegmentDescriptor> ServerViewUtil.getTargetLocations(BrokerServerView,datasource,interval,...) 獲取segmentLocation,   2.ResourceIOReaderWriter.ok()讀取segment

LocatedSegmentDescriptor資訊包括：
	segmentMetadata:segment元資訊(interval,version,partition,size)
	serverMetadata:實時節點和歷史節點地址等。

讀取segment
-->Response.ok(newOutputWriter(null, null, false).writeValueAsString(object), contentType) -->


Jetty server（Jetty+Jersey+guice組合）
Broker初始化：
1.CliBroker繫結了 BrokerServerView，CachingClusteredClient，ClientQuerySegmentWalker，TimelineServerView，BrokerQueryResource等
查詢入口doPost
QueryResource.doPost()-->
1.根據請求建立對應Query物件，
2.執行查詢 final QueryLifecycle.QueryResponse queryResponse = queryLifecycle.execute();

執行查詢，返回sequence結果
final Sequence res = QueryPlus.wrap(baseQuery).withIdentity(authenticationResult.getIdentity()).run(texasRanger, responseContext); -->  QueryRunner.run()
建立QueryRunner，執行QueryRunner.run();
說明：
   1.ClientQuerySegmentWalker texasRanger：建立QueryRunner
   2.ResponseContext responseContext：
	   The context for storing and passing data between chains of {@link org.apache.druid.query.QueryRunner}s.
	   The context is also transferred between Druid nodes with all the data it contains.
   3. QueryPlus 包含Query,QueryMetries,Identity  
   4. ClientQuerySegmentWalker建立QueryRunner
    一種根據Query和interval建立對應的QueryRunner
	一種根據SegmentDescriptor建立對應的QueryRunner

ClientQuerySegmentWalker：根據查詢中的intervals或segments引數來確定呼叫getQueryRunnerForIntervals還是getQueryRunnerForSegments
public <T> QueryRunner<T> getQueryRunnerForIntervals(Query<T> query, Iterable<Interval> intervals)
  {
    return makeRunner(query, baseClient.getQueryRunnerForIntervals(query, intervals));
  }
  
makeRunner：返回ResultLevelCachingQueryRunner，ResultLevelCachingQueryRunner裝飾FluentQueryRunner，FluentQueryRunner裝飾SetAndVerifyContextQueryRunner，SetAndVerifyContextQueryRunner裝飾了RetryQueryRunner，RetryQueryRunner裝飾baseRunner

getQueryRunnerForIntervals():返回匿名baseRunner,內部run呼叫 CachingClusteredClient.run --》CachingClusteredClient內部類SpecificQueryRunnable.run

SpecificQueryRunnable.run: 
Sequence<T> run(final UnaryOperator<TimelineLookup<String, ServerSelector>> timelineConverter)
    {
	//根據datasource獲取timeline
      @Nullable
      TimelineLookup<String, ServerSelector> timeline = serverView.getTimeline(query.getDataSource());
      if (timeline == null) {
        return Sequences.empty();
      }
      timeline = timelineConverter.apply(timeline);
      if (uncoveredIntervalsLimit > 0) {
        computeUncoveredIntervals(timeline);
      }
//根據timeline獲取ServerSelector與對應SegmentDescriptor
      final Set<ServerToSegment> segments = computeSegmentsToQuery(timeline);
	  //根據query獲取快取key
      @Nullable
      final byte[] queryCacheKey = computeQueryCacheKey();
      if (query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH) != null) {
        @Nullable
        final String prevEtag = (String) query.getContext().get(QueryResource.HEADER_IF_NONE_MATCH);
        @Nullable
        final String currentEtag = computeCurrentEtag(segments, queryCacheKey);
        if (currentEtag != null && currentEtag.equals(prevEtag)) {
          return Sequences.empty();
        }
      }
//根據快取key獲取對應的快取結果
      final List<Pair<Interval, byte[]>> alreadyCachedResults = pruneSegmentsWithCachedResults(queryCacheKey, segments);
	  //獲取Server以及所擁有的 SegmentDescriptor集合
      final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer = groupSegmentsByServer(segments);
      return new LazySequence<>(() -> {
        List<Sequence<T>> sequencesByInterval = new ArrayList<>(alreadyCachedResults.size() + segmentsByServer.size());
		//從快取中拉取segment並反序列化成 sequences
        addSequencesFromCache(sequencesByInterval, alreadyCachedResults);
		// 從Server端拉取segment並反序列化成 sequences
        addSequencesFromServer(sequencesByInterval, segmentsByServer);
		// 合併sequences並返回結果sequence
        return merge(sequencesByInterval);
      });
    }

//broker端非同步發起多查詢請求，並快取返回結果
private void addSequencesFromServer(
        final List<Sequence<T>> listOfSequences,
        final SortedMap<DruidServer, List<SegmentDescriptor>> segmentsByServer
    )
    {
	//遍歷segmentsByServer，使用NettyHttpClient 發起查詢請求，並將每個查詢請求返回的sequence新增到集合中
      segmentsByServer.forEach((server, segmentsOfServer) -> {
	     //從 BrokerServerView 獲取 QueryRunner
        final QueryRunner serverRunner = serverView.getQueryRunner(server);

        if (serverRunner == null) {
          log.error("Server[%s] doesn't have a query runner", server);
          return;
        }

        final MultipleSpecificSegmentSpec segmentsOfServerSpec = new MultipleSpecificSegmentSpec(segmentsOfServer);

        // Divide user-provided maxQueuedBytes by the number of servers, and limit each server to that much.
        final long maxQueuedBytes = QueryContexts.getMaxQueuedBytes(query, httpClientConfig.getMaxQueuedBytes());
        final long maxQueuedBytesPerServer = maxQueuedBytes / segmentsByServer.size();
        final Sequence<T> serverResults;

        //query中的context屬性中獲取bySegment，預設false
        if (isBySegment) {
          serverResults = getBySegmentServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        } else if (!server.segmentReplicatable() || !populateCache) {
		//直接查詢，不快取
          serverResults = getSimpleServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        } else {
		//查詢並快取
          serverResults = getAndCacheServerResults(serverRunner, segmentsOfServerSpec, maxQueuedBytesPerServer);
        }
        listOfSequences.add(serverResults);
      });
    }
	
	//直接查詢，不快取，呼叫從 BrokerServerView 獲取的 QueryRunner.run方法
	private Sequence<T> getSimpleServerResults(
        final QueryRunner serverRunner,
        final MultipleSpecificSegmentSpec segmentsOfServerSpec,
        long maxQueuedBytesPerServer
    )
    {
      return serverRunner.run(
	  //統一發起segments查詢方式，server端均呼叫getQueryRunnerForSegments執行查詢請求
          queryPlus.withQuerySegmentSpec(segmentsOfServerSpec).withMaxQueuedBytes(maxQueuedBytesPerServer),
          responseContext
      );
    }


BrokerServerView 中的QueryRunner 為子類 DirectDruidClient.run,其中通過 NettyHttpClient.go 非同步傳送http請求到對應server執行查詢請求，query中攜帶要查詢的segments描述列表MultipleSpecificSegmentSpec
傳送查詢請求，
DirectDruidClient.run:
     ....
	 //httpClient 為NettyHttpClient
      future = httpClient.go(
          new Request(
              HttpMethod.POST,
              new URL(url)
          ).setContent(objectMapper.writeValueAsBytes(QueryContexts.withTimeout(query, timeLeft)))
           .setHeader(
               HttpHeaders.Names.CONTENT_TYPE,
               isSmile ? SmileMediaTypes.APPLICATION_JACKSON_SMILE : MediaType.APPLICATION_JSON
           ),
          responseHandler,
          Duration.millis(timeLeft)
      );

      queryWatcher.registerQuery(query, future);

      openConnections.getAndIncrement();
      Futures.addCallback(
          future,
          new FutureCallback<InputStream>()
          {
            @Override
            public void onSuccess(InputStream result)
            {
              openConnections.getAndDecrement();
            }

            @Override
            public void onFailure(Throwable t)
            {
              openConnections.getAndDecrement();
              if (future.isCancelled()) {
                cancelQuery(query, cancelUrl);
              }
            }
          },
          // The callback is non-blocking and quick, so it's OK to schedule it using directExecutor()
          Execs.directExecutor()
      );
     ....

CliPeon:
初始時繫結關係：QuerySegmentWalker為SingleTaskBackgroundRunner
執行的是 SingleTaskBackgroundRunner.getQueryRunnerForSegments 獲取QueryRunner --> RealtimeIndexTask.getQueryRunner-->RealtimePlumber.getQueryRunner
--> SinkQuerySegmentWalker.getQueryRunnerForIntervals
   追加模式任務 SinkQuerySegmentWalker.getQueryRunnerForSegments:
   final Sink theSink = chunk.getObject();
          final SegmentId sinkSegmentId = theSink.getSegment().getId();

          Iterable<QueryRunner<T>> perHydrantRunners = new SinkQueryRunners<>(
              Iterables.transform(
                  theSink,
                  hydrant -> {
                    // Hydrant might swap at any point, but if it's swapped at the start
                    // then we know it's *definitely* swapped.
                    final boolean hydrantDefinitelySwapped = hydrant.hasSwapped();

                    if (skipIncrementalSegment && !hydrantDefinitelySwapped) {
                      return new Pair<>(Intervals.ETERNITY, new NoopQueryRunner<>());
                    }

                    // Prevent the underlying segment from swapping when its being iterated
					//其中segment為 IncrementalIndexSegment，包含 IncrementalIndex 物件，該物件為擁有實時攝取的資料
                    final Pair<Segment, Closeable> segmentAndCloseable = hydrant.getAndIncrementSegment();
                    try {
					//根據segment獲取QueryRunner物件，factory:TopNQueryRunnerFactory，返回匿名QueryRunner，內部run呼叫 TopNQueryEngine.run
                      QueryRunner<T> runner = factory.createRunner(segmentAndCloseable.lhs);

                      // 1) Only use caching if data is immutable
                      // 2) Hydrants are not the same between replicas, make sure cache is local
                      if (hydrantDefinitelySwapped && cache.isLocal()) {
                        runner = new CachingQueryRunner<>(
                            makeHydrantCacheIdentifier(hydrant),
                            descriptor,
                            objectMapper,
                            cache,
                            toolChest,
                            runner,
                            // Always populate in foreground regardless of config
                            new ForegroundCachePopulator(
                                objectMapper,
                                cachePopulatorStats,
                                cacheConfig.getMaxEntrySize()
                            ),
                            cacheConfig
                        );
                      }
                      // Make it always use Closeable to decrement()
                      runner = QueryRunnerHelper.makeClosingQueryRunner(
                          runner,
                          segmentAndCloseable.rhs
                      );
                      return new Pair<>(segmentAndCloseable.lhs.getDataInterval(), runner);
                    }
                    catch (RuntimeException e) {
                      CloseQuietly.close(segmentAndCloseable.rhs);
                      throw e;
                    }
                  }
              )
          );
    
 實時Peon執行查詢：	
	QueryRunner.run-->TopNQueryEngine.query:
	public Sequence<Result<TopNResultValue>> query(
      final TopNQuery query,
      final StorageAdapter adapter,
      final @Nullable TopNQueryMetrics queryMetrics
  )
  {
    if (adapter == null) {
      throw new SegmentMissingException(
          "Null storage adapter found. Probably trying to issue a query against a segment being memory unmapped."
      );
    }

    final List<Interval> queryIntervals = query.getQuerySegmentSpec().getIntervals();
    final Filter filter = Filters.convertToCNFFromQueryContext(query, Filters.toFilter(query.getDimensionsFilter()));
    final Granularity granularity = query.getGranularity();
	//核心topN查詢演算法，adapter為 IncrementalIndexStorageAdapter
    final TopNMapFn mapFn = getMapFn(query, adapter, queryMetrics);

    Preconditions.checkArgument(
        queryIntervals.size() == 1, "Can only handle a single interval, got[%s]", queryIntervals
    );

    return Sequences.filter(
        Sequences.map(
            adapter.makeCursors(
                filter,
                queryIntervals.get(0),
                query.getVirtualColumns(),
                granularity,
                query.isDescending(),
                queryMetrics
            ),
            input -> {
              if (queryMetrics != null) {
                queryMetrics.cursor(input);
              }
              return mapFn.apply(input, queryMetrics);
            }
        ),
        Predicates.notNull()
    );
  }
  
  TopNMapFn.apply:
  public Result<TopNResultValue> apply(final Cursor cursor, final @Nullable TopNQueryMetrics queryMetrics)
  {
    final ColumnSelectorPlus<TopNColumnAggregatesProcessor<?>> selectorPlus =
        DimensionHandlerUtils.createColumnSelectorPlus(
            new TopNColumnAggregatesProcessorFactory(query.getDimensionSpec().getOutputType()),
            query.getDimensionSpec(),
            cursor.getColumnSelectorFactory()
        );

    if (selectorPlus.getSelector() == null) {
      return null;
    }

    TopNParams params = null;
    try {
      params = topNAlgorithm.makeInitParams(selectorPlus, cursor);
      if (queryMetrics != null) {
        queryMetrics.columnValueSelector(selectorPlus.getSelector());
        queryMetrics.numValuesPerPass(params);
      }


      TopNResultBuilder resultBuilder = BaseTopNAlgorithm.makeResultBuilder(params, query);

      topNAlgorithm.run(params, resultBuilder, null, queryMetrics);

      return resultBuilder.build();
    }
    finally {
      topNAlgorithm.cleanup(params);
    }
  }
  
  
  BaseTopNAlgorithm.run--》runWithCardinalityUnknown()-->HeapBasedTopNAlgorithm.scanAndAggregate(),updateResults()
  
  NullableNumericTopNColumnAggregatesProcessor.scanAndAggregate() updateResults()或
  StringTopNColumnAggregatesProcessor.scanAndAggregateWithCardinalityUnknown():
  private long scanAndAggregateWithCardinalityUnknown(
      TopNQuery query,
      Cursor cursor,
      DimensionSelector selector
  )
  {
    long processedRows = 0;
	//cursor遊標IncrementalIndexStorageAdapter.IncrementalIndexCursor，控制掃描分組條數，實際呼叫RollupFactsHolder.timeRangeIterable(startTime,endTime)返回的迭代器
    while (!cursor.isDone()) {
	//IndexerDimensionSelector.getRow，維度值選擇器返回維度值下標
      final IndexedInts dimValues = selector.getRow();
      for (int i = 0, size = dimValues.size(); i < size; ++i) {//可能是多值維度
        final int dimIndex = dimValues.get(i);
		//根據維度值下標查詢到原維度值(Object型別)，呼叫apply轉成對應型別String或Long...
        final Comparable<?> key = dimensionValueConverter.apply(selector.lookupName(dimIndex));
        Aggregator[] aggs = aggregatesStore.computeIfAbsent(
            key,
            k -> BaseTopNAlgorithm.makeAggregators(cursor, query.getAggregatorSpecs())
        );
        for (Aggregator aggregator : aggs) {
          aggregator.aggregate();
        }
      }
      cursor.advance();
      processedRows++;
    }
    return processedRows;
  }
  
  
  updateResults：
  //TopNNumericResultBuilder
  public void updateResults(TopNResultBuilder resultBuilder)
  {
    for (Map.Entry<?, Aggregator[]> entry : aggregatesStore.entrySet()) {
      Aggregator[] aggs = entry.getValue();
      if (aggs != null) {
        Object[] vals = new Object[aggs.length];
        for (int i = 0; i < aggs.length; i++) {
          vals[i] = aggs[i].get();
        }

        final Comparable<?> key = dimensionValueConverter.apply(entry.getKey());
		//DimValHolder新增到 PriorityQueue 按排序TopN指標大小排序優先順序佇列 threshold 限制佇列長度，新增至佇列時完成了對指標的排序
        resultBuilder.addEntry(key, key, vals);
      }
    }
  }
  

CliHistorical:
繫結關係：QuerySegmentWalker為ServerManager
--> buildAndDecorateQueryRunner:
private <T> QueryRunner<T> buildAndDecorateQueryRunner(
      final QueryRunnerFactory<T, Query<T>> factory,
      final QueryToolChest<T, Query<T>> toolChest,
      final ReferenceCountingSegment adapter,
      final SegmentDescriptor segmentDescriptor,
      final AtomicLong cpuTimeAccumulator
  )
  {
    SpecificSegmentSpec segmentSpec = new SpecificSegmentSpec(segmentDescriptor);
    SegmentId segmentId = adapter.getId();
    String segmentIdString = segmentId.toString();

    MetricsEmittingQueryRunner<T> metricsEmittingQueryRunnerInner = new MetricsEmittingQueryRunner<>(
        emitter,
        toolChest,
        new ReferenceCountingSegmentQueryRunner<>(factory, adapter, segmentDescriptor),
        QueryMetrics::reportSegmentTime,
        queryMetrics -> queryMetrics.segment(segmentIdString)
    );

    CachingQueryRunner<T> cachingQueryRunner = new CachingQueryRunner<>(
        segmentIdString,
        segmentDescriptor,
        objectMapper,
        cache,
        toolChest,
        metricsEmittingQueryRunnerInner,
        cachePopulator,
        cacheConfig
    );

    BySegmentQueryRunner<T> bySegmentQueryRunner = new BySegmentQueryRunner<>(
        segmentId,
        adapter.getDataInterval().getStart(),
        cachingQueryRunner
    );

    MetricsEmittingQueryRunner<T> metricsEmittingQueryRunnerOuter = new MetricsEmittingQueryRunner<>(
        emitter,
        toolChest,
        bySegmentQueryRunner,
        QueryMetrics::reportSegmentAndCacheTime,
        queryMetrics -> queryMetrics.segment(segmentIdString)
    ).withWaitMeasuredFromNow();

    SpecificSegmentQueryRunner<T> specificSegmentQueryRunner = new SpecificSegmentQueryRunner<>(
        metricsEmittingQueryRunnerOuter,
        segmentSpec
    );

    PerSegmentOptimizingQueryRunner<T> perSegmentOptimizingQueryRunner = new PerSegmentOptimizingQueryRunner<>(
        specificSegmentQueryRunner,
        new PerSegmentQueryOptimizationContext(segmentDescriptor)
    );

    return new SetAndVerifyContextQueryRunner<>(
        serverConfig,
        CPUTimeMetricQueryRunner.safeBuild(
            perSegmentOptimizingQueryRunner,
            toolChest,
            emitter,
            cpuTimeAccumulator,
            false
        )
    );
  }

-->TopNQueryRunnerFactory.createRunner:


-->建立列值選擇器：QueryableIndexColumnSelectorFactory
 public ColumnValueSelector<?> makeColumnValueSelector(String columnName)
  {
    Function<String, ColumnValueSelector<?>> mappingFunction = name -> {
      if (virtualColumns.exists(columnName)) {
        ColumnValueSelector<?> selector = virtualColumns.makeColumnValueSelector(columnName, index, offset);
        if (selector == null) {
          return virtualColumns.makeColumnValueSelector(columnName, this);
        } else {
          return selector;
        }
      }

      BaseColumn column = getCachedColumn(columnName, BaseColumn.class);

      if (column != null) {
        return column.makeColumnValueSelector(offset);
      } else {
        return NilColumnValueSelector.instance();
      }
    };

參考資料：
https://www.jianshu.com/p/f8de4531c30c
https://blog.csdn.net/weixin_38736107/article/details/98482951