PostgreSQL 原始碼解讀（101）- 分割槽表#7（資料查詢路由#4-prune part...
本節是PG在查詢分割槽表的時候如何確定查詢的是哪個分割槽邏輯介紹的第二部分。在規劃階段,函式set_rel_size中，如RTE為分割槽表（rte->inh=T），則呼叫set_append_rel_size函式，在set_append_rel_size中透過prune_append_rel_partitions函式獲取“可保留”的分割槽。
本節的內容是介紹prune_append_rel_partitions->get_matching_partitions函式。
一、資料結構
PartitionScheme
分割槽方案,根據設計，分割槽方案只包含分割槽方法的一般屬性(列表與範圍、分割槽列的數量和每個分割槽列的型別資訊)，而不包含特定的分割槽邊界資訊。
/*
* If multiple relations are partitioned the same way, all such partitions
* will have a pointer to the same PartitionScheme. A list of PartitionScheme
* objects is attached to the PlannerInfo. By design, the partition scheme
* incorporates only the general properties of the partition method (LIST vs.
* RANGE, number of partitioning columns and the type information for each)
* and not the specific bounds.
* 如果多個關係以相同的方式分割槽，那麼所有這些分割槽都將具有指向相同PartitionScheme的指標。
* PartitionScheme物件的連結串列附加到PlannerInfo中。
* 根據設計，分割槽方案只包含分割槽方法的一般屬性(列表與範圍、分割槽列的數量和每個分割槽列的型別資訊)，
* 而不包含特定的界限。
*
* We store the opclass-declared input data types instead of the partition key
* datatypes since the former rather than the latter are used to compare
* partition bounds. Since partition key data types and the opclass declared
* input data types are expected to be binary compatible (per ResolveOpClass),
* both of those should have same byval and length properties.
* 我們儲存opclass-declared的輸入資料型別，而不是分割槽鍵資料型別，
* 因為前者用於比較分割槽邊界，而不是後者。
* 由於分割槽鍵資料型別和opclass-declared的輸入資料型別預期是二進位制相容的(每個ResolveOpClass)，
* 所以它們應該具有相同的byval和length屬性。
*/
typedef struct PartitionSchemeData
{
char strategy; /* 分割槽策略;partition strategy */
int16 partnatts; /* 分割槽屬性個數;number of partition attributes */
Oid *partopfamily; /* 運算子族OIDs;OIDs of operator families */
Oid *partopcintype; /* opclass宣告的輸入資料型別的OIDs;OIDs of opclass declared input data types */
Oid *partcollation; /* 分割槽排序規則OIDs;OIDs of partitioning collations */
/* Cached information about partition key data types. */
//快取有關分割槽鍵資料型別的資訊。
int16 *parttyplen;
bool *parttypbyval;
/* Cached information about partition comparison functions. */
//快取有關分割槽比較函式的資訊。
FmgrInfo *partsupfunc;
} PartitionSchemeData;
typedef struct PartitionSchemeData *PartitionScheme;
PartitionPruneXXX
執行Prune期間需要使用的資料結構,包括PartitionPruneStep/PartitionPruneStepOp/PartitionPruneCombineOp/PartitionPruneStepCombine
/*
* Abstract Node type for partition pruning steps (there are no concrete
* Nodes of this type).
* 用於分割槽修剪步驟pruning的抽象節點型別(沒有這種型別的具體節點)。
*
* step_id is the global identifier of the step within its pruning context.
* step_id是步驟在其修剪pruning上下文中的全域性識別符號。
*/
typedef struct PartitionPruneStep
{
NodeTag type;
int step_id;
} PartitionPruneStep;
/*
* PartitionPruneStepOp - Information to prune using a set of mutually AND'd
* OpExpr clauses
* PartitionPruneStepOp - 使用一組AND操作的OpExpr條件子句進行修剪prune的資訊
*
* This contains information extracted from up to partnatts OpExpr clauses,
* where partnatts is the number of partition key columns. 'opstrategy' is the
* strategy of the operator in the clause matched to the last partition key.
* 'exprs' contains expressions which comprise the lookup key to be passed to
* the partition bound search function. 'cmpfns' contains the OIDs of
* comparison functions used to compare aforementioned expressions with
* partition bounds. Both 'exprs' and 'cmpfns' contain the same number of
* items, up to partnatts items.
* 它包含從partnatts OpExpr子句中提取的資訊，
* 其中partnatts是分割槽鍵列的數量。
* “opstrategy”是子句中與最後一個分割槽鍵匹配的運算子的策略。
* 'exprs'包含一些表示式，這些表示式包含要傳遞給分割槽繫結搜尋函式的查詢鍵。
* “cmpfns”包含用於比較上述表示式與分割槽邊界的比較函式的OIDs。
* “exprs”和“cmpfns”包含相同數量的條目，最多包含partnatts個條目。
*
* Once we find the offset of a partition bound using the lookup key, we
* determine which partitions to include in the result based on the value of
* 'opstrategy'. For example, if it were equality, we'd return just the
* partition that would contain that key or a set of partitions if the key
* didn't consist of all partitioning columns. For non-equality strategies,
* we'd need to include other partitions as appropriate.
* 一旦我們使用查詢鍵找到分割槽繫結的偏移量，
* 我們將根據“opstrategy”的值確定在結果中包含哪些分割槽。
* 例如，如果它是相等的，我們只返回包含該鍵的分割槽，或者如果該鍵不包含所有分割槽列，
* 則返回一組分割槽。
* 對於非等值的情況，需要適當地包括其他分割槽。
*
* 'nullkeys' is the set containing the offset of the partition keys (0 to
* partnatts - 1) that were matched to an IS NULL clause. This is only
* considered for hash partitioning as we need to pass which keys are null
* to the hash partition bound search function. It is never possible to
* have an expression be present in 'exprs' for a given partition key and
* the corresponding bit set in 'nullkeys'.
* 'nullkeys'是包含與is NULL子句匹配的分割槽鍵(0到partnatts - 1)偏移量的集合。
* 這隻適用於雜湊分割槽，因為我們需要將哪些鍵為null傳遞給雜湊分割槽繫結搜尋函式。
* 對於給定的分割槽鍵和“nullkeys”中設定的相應bit，不可能在“exprs”中出現表示式。
*/
typedef struct PartitionPruneStepOp
{
PartitionPruneStep step;
StrategyNumber opstrategy;
List *exprs;
List *cmpfns;
Bitmapset *nullkeys;
} PartitionPruneStepOp;
/*
* PartitionPruneStepCombine - Information to prune using a BoolExpr clause
* PartitionPruneStepCombine - 使用BoolExpr條件prune的資訊
*
* For BoolExpr clauses, we combine the set of partitions determined for each
* of the argument clauses.
* 對於BoolExpr子句，我們為每個引數子句確定的分割槽集進行組合。
*/
typedef enum PartitionPruneCombineOp
{
PARTPRUNE_COMBINE_UNION,
PARTPRUNE_COMBINE_INTERSECT
} PartitionPruneCombineOp;
typedef struct PartitionPruneStepCombine
{
PartitionPruneStep step;
PartitionPruneCombineOp combineOp;
List *source_stepids;
} PartitionPruneStepCombine;
/* The result of performing one PartitionPruneStep */
//執行PartitionPruneStep步驟後的結果
typedef struct PruneStepResult
{
/*
* The offsets of bounds (in a table's boundinfo) whose partition is
* selected by the pruning step.
* 被pruning步驟選中的分割槽邊界(在資料表boundinfo中)偏移
*/
Bitmapset *bound_offsets;
bool scan_default; /* 是否掃描預設分割槽? Scan the default partition? */
bool scan_null; /* 是否為NULL值掃描分割槽? Scan the partition for NULL values? */
} PruneStepResult;
二、原始碼解讀
get_matching_partitions函式確定在分割槽pruning後仍然"存活"的分割槽。
/*
* get_matching_partitions
* Determine partitions that survive partition pruning
* 確定在分割槽修剪pruning後仍然存在的分割槽.
*
* Returns a Bitmapset of the RelOptInfo->part_rels indexes of the surviving
* partitions.
* 返回pruning後仍存在的分割槽的RelOptInfo->part_rels索引點陣圖集。
*/
Bitmapset *
get_matching_partitions(PartitionPruneContext *context, List *pruning_steps)
{
Bitmapset *result;
int num_steps = list_length(pruning_steps),
i;
PruneStepResult **results,
*final_result;
ListCell *lc;
/* If there are no pruning steps then all partitions match. */
//沒有pruning步驟,則視為保留所有分割槽
if (num_steps == 0)
{
Assert(context->nparts > 0);
return bms_add_range(NULL, 0, context->nparts - 1);
}
/*
* Allocate space for individual pruning steps to store its result. Each
* slot will hold a PruneStepResult after performing a given pruning step.
* Later steps may use the result of one or more earlier steps. The
* result of applying all pruning steps is the value contained in the slot
* of the last pruning step.
* 為單個修剪步驟分配空間來儲存結果。
* 每個slot將持有pruning後，執行一個給定的pruning步驟。
* 後面的步驟可以使用前面一個或多個步驟的結果。
* 應用所有步驟的結果是最後一個步驟的slot中包含的值。
*/
results = (PruneStepResult **)
palloc0(num_steps * sizeof(PruneStepResult *));
foreach(lc, pruning_steps)//遍歷步驟
{
PartitionPruneStep *step = lfirst(lc);
switch (nodeTag(step))
{
case T_PartitionPruneStepOp:
results[step->step_id] =
perform_pruning_base_step(context,
(PartitionPruneStepOp *) step);//執行pruning基礎步驟
break;
case T_PartitionPruneStepCombine:
results[step->step_id] =
perform_pruning_combine_step(context,
(PartitionPruneStepCombine *) step,
results);//執行pruning組合步驟
break;
default:
elog(ERROR, "invalid pruning step type: %d",
(int) nodeTag(step));
}
}
/*
* At this point we know the offsets of all the datums whose corresponding
* partitions need to be in the result, including special null-accepting
* and default partitions. Collect the actual partition indexes now.
* 到目前為止,我們已經知道結果中需要的相應分割槽的所有資料的偏移量，
* 包括特殊的接受null的分割槽和預設分割槽。
* 現在收集實際的分割槽索引。
*/
final_result = results[num_steps - 1];//最終結果
Assert(final_result != NULL);
i = -1;
result = NULL;
while ((i = bms_next_member(final_result->bound_offsets, i)) >= 0)
{
int partindex = context->boundinfo->indexes[i];//分割槽編號
/*
* In range and hash partitioning cases, some slots may contain -1,
* indicating that no partition has been defined to accept a given
* range of data or for a given remainder, respectively. The default
* partition, if any, in case of range partitioning, will be added to
* the result, because the specified range still satisfies the query's
* conditions.
* 範圍分割槽和雜湊分割槽，一些slot可能包含-1，
* 這表示沒有定義接受給定範圍的資料或給定餘數的分割槽。
* 在範圍分割槽的情況下，預設分割槽(如果有的話)將被新增到結果中，
* 因為指定的範圍仍然滿足查詢的條件。
*/
if (partindex >= 0)
result = bms_add_member(result, partindex);
}
/* Add the null and/or default partition if needed and if present. */
//如果需要，新增NULL和/或預設分割槽。
if (final_result->scan_null)
{
Assert(context->strategy == PARTITION_STRATEGY_LIST);
Assert(partition_bound_accepts_nulls(context->boundinfo));
result = bms_add_member(result, context->boundinfo->null_index);
}
if (final_result->scan_default)
{
Assert(context->strategy == PARTITION_STRATEGY_LIST ||
context->strategy == PARTITION_STRATEGY_RANGE);
Assert(partition_bound_has_default(context->boundinfo));
result = bms_add_member(result, context->boundinfo->default_index);
}
return result;
}
/*
* perform_pruning_base_step
* Determines the indexes of datums that satisfy conditions specified in
* 'opstep'.
* 確定滿足“opstep”中指定條件的資料索引。
*
* Result also contains whether special null-accepting and/or default
* partition need to be scanned.
* 結果還包含是否需要掃描特殊的可接受null和/或預設分割槽。
*/
static PruneStepResult *
perform_pruning_base_step(PartitionPruneContext *context,
PartitionPruneStepOp *opstep)
{
ListCell *lc1,
*lc2;
int keyno,
nvalues;
Datum values[PARTITION_MAX_KEYS];
FmgrInfo *partsupfunc;
int stateidx;
/*
* There better be the same number of expressions and compare functions.
* 最好有相同數量的表示式和比較函式。
*/
Assert(list_length(opstep->exprs) == list_length(opstep->cmpfns));
nvalues = 0;
lc1 = list_head(opstep->exprs);
lc2 = list_head(opstep->cmpfns);
/*
* Generate the partition lookup key that will be used by one of the
* get_matching_*_bounds functions called below.
* 生成將由下面呼叫的get_matching_*_bounds函式使用的分割槽查詢鍵。
*/
for (keyno = 0; keyno < context->partnatts; keyno++)
{
/*
* For hash partitioning, it is possible that values of some keys are
* not provided in operator clauses, but instead the planner found
* that they appeared in a IS NULL clause.
* 對於雜湊分割槽，運算子子句中可能沒有提供某些鍵的值，
* 但是計劃器發現它們出現在is NULL子句中。
*/
if (bms_is_member(keyno, opstep->nullkeys))
continue;
/*
* For range partitioning, we must only perform pruning with values
* for either all partition keys or a prefix thereof.
* 對於範圍分割槽，我們必須只對所有分割槽鍵或其字首執行值修剪pruning。
*/
if (keyno > nvalues && context->strategy == PARTITION_STRATEGY_RANGE)
break;
if (lc1 != NULL)//步驟的條件表示式不為NULL
{
Expr *expr;
Datum datum;
bool isnull;
expr = lfirst(lc1);
stateidx = PruneCxtStateIdx(context->partnatts,
opstep->step.step_id, keyno);
if (partkey_datum_from_expr(context, expr, stateidx,
&datum, &isnull))
{
Oid cmpfn;
/*
* Since we only allow strict operators in pruning steps, any
* null-valued comparison value must cause the comparison to
* fail, so that no partitions could match.
* 由於我們只允許在修剪pruning步驟中使用嚴格的運算子，
* 任何空值比較值都必須導致比較失敗，這樣就沒有分割槽能夠匹配。
*/
if (isnull)
{
PruneStepResult *result;
result = (PruneStepResult *) palloc(sizeof(PruneStepResult));
result->bound_offsets = NULL;
result->scan_default = false;
result->scan_null = false;
return result;
}
/* Set up the stepcmpfuncs entry, unless we already did */
//配置stepcmpfuncs(步驟比較函式)入口
cmpfn = lfirst_oid(lc2);
Assert(OidIsValid(cmpfn));
if (cmpfn != context->stepcmpfuncs[stateidx].fn_oid)
{
/*
* If the needed support function is the same one cached
* in the relation's partition key, copy the cached
* FmgrInfo. Otherwise (i.e., when we have a cross-type
* comparison), an actual lookup is required.
* 如果所需的支援函式與關係分割槽鍵快取的支援函式相同，
* 則複製快取的FmgrInfo。
* 否則(比如存在一個跨型別比較時)，需要實際的查詢。
*/
if (cmpfn == context->partsupfunc[keyno].fn_oid)
fmgr_info_copy(&context->stepcmpfuncs[stateidx],
&context->partsupfunc[keyno],
context->ppccontext);
else
fmgr_info_cxt(cmpfn, &context->stepcmpfuncs[stateidx],
context->ppccontext);
}
values[keyno] = datum;
nvalues++;
}
lc1 = lnext(lc1);
lc2 = lnext(lc2);
}
}
/*
* Point partsupfunc to the entry for the 0th key of this step; the
* additional support functions, if any, follow consecutively.
* 將partsupfunc指向此步驟第0個鍵的條目;
* 附加的支援功能(如果有的話)是連續的。
*/
stateidx = PruneCxtStateIdx(context->partnatts, opstep->step.step_id, 0);
partsupfunc = &context->stepcmpfuncs[stateidx];
switch (context->strategy)
{
case PARTITION_STRATEGY_HASH:
return get_matching_hash_bounds(context,
opstep->opstrategy,
values, nvalues,
partsupfunc,
opstep->nullkeys);
case PARTITION_STRATEGY_LIST:
return get_matching_list_bounds(context,
opstep->opstrategy,
values[0], nvalues,
&partsupfunc[0],
opstep->nullkeys);
case PARTITION_STRATEGY_RANGE:
return get_matching_range_bounds(context,
opstep->opstrategy,
values, nvalues,
partsupfunc,
opstep->nullkeys);
default:
elog(ERROR, "unexpected partition strategy: %d",
(int) context->strategy);
break;
}
return NULL;
}
/*
* perform_pruning_combine_step
* Determines the indexes of datums obtained by combining those given
* by the steps identified by cstep->source_stepids using the specified
* combination method
* 使用指定的組合方法將cstep->source_stepids標識的步驟組合在一起得到資料索引
*
* Since cstep may refer to the result of earlier steps, we also receive
* step_results here.
* 因為cstep可能引用前面步驟的結果，所以在這裡也會收到step_results。
*/
static PruneStepResult *
perform_pruning_combine_step(PartitionPruneContext *context,
PartitionPruneStepCombine *cstep,
PruneStepResult **step_results)
{
ListCell *lc1;
PruneStepResult *result = NULL;
bool firststep;
/*
* A combine step without any source steps is an indication to not perform
* any partition pruning, we just return all partitions.
* 如無源步驟,則不執行分割槽pruning,返回所有分割槽
*/
result = (PruneStepResult *) palloc0(sizeof(PruneStepResult));
if (list_length(cstep->source_stepids) == 0)
{
PartitionBoundInfo boundinfo = context->boundinfo;
result->bound_offsets = bms_add_range(NULL, 0, boundinfo->ndatums - 1);
result->scan_default = partition_bound_has_default(boundinfo);
result->scan_null = partition_bound_accepts_nulls(boundinfo);
return result;
}
switch (cstep->combineOp)//根據組合操作型別確定相應邏輯
{
case PARTPRUNE_COMBINE_UNION://PARTPRUNE_COMBINE_UNION
foreach(lc1, cstep->source_stepids)
{
int step_id = lfirst_int(lc1);
PruneStepResult *step_result;
/*
* step_results[step_id] must contain a valid result, which is
* confirmed by the fact that cstep's step_id is greater than
* step_id and the fact that results of the individual steps
* are evaluated in sequence of their step_ids.
* step_results[step_id]必須包含一個有效的結果，
* cstep的step_id大於step_id，並且各個步驟的結果按其step_id的順序計算，
* 這一點是可以確認的。
*/
if (step_id >= cstep->step.step_id)
elog(ERROR, "invalid pruning combine step argument");
step_result = step_results[step_id];
Assert(step_result != NULL);
/* Record any additional datum indexes from this step */
//記錄從該步驟產生的偏移索引
result->bound_offsets = bms_add_members(result->bound_offsets,
step_result->bound_offsets);
/* Update whether to scan null and default partitions. */
//更新掃描null/default分割槽的標記
if (!result->scan_null)
result->scan_null = step_result->scan_null;
if (!result->scan_default)
result->scan_default = step_result->scan_default;
}
break;
case PARTPRUNE_COMBINE_INTERSECT://PARTPRUNE_COMBINE_INTERSECT
firststep = true;
foreach(lc1, cstep->source_stepids)
{
int step_id = lfirst_int(lc1);
PruneStepResult *step_result;
if (step_id >= cstep->step.step_id)
elog(ERROR, "invalid pruning combine step argument");
step_result = step_results[step_id];
Assert(step_result != NULL);
if (firststep)//第一個步驟
{
/* Copy step's result the first time. */
//第一次,複製步驟的結果
result->bound_offsets =
bms_copy(step_result->bound_offsets);
result->scan_null = step_result->scan_null;
result->scan_default = step_result->scan_default;
firststep = false;
}
else
{
/* Record datum indexes common to both steps */
//記錄其他步驟產生的索引
result->bound_offsets =
bms_int_members(result->bound_offsets,
step_result->bound_offsets);
/* Update whether to scan null and default partitions. */
//更新掃描null/default分割槽的標記
if (result->scan_null)
result->scan_null = step_result->scan_null;
if (result->scan_default)
result->scan_default = step_result->scan_default;
}
}
break;
}
return result;
}
/*
* get_matching_hash_bounds
* Determine offset of the hash bound matching the specified values,
* considering that all the non-null values come from clauses containing
* a compatible hash equality operator and any keys that are null come
* from an IS NULL clause.
* 考慮到所有非空值都來自包含相容的雜湊相等運算子的子句，
* 而所有空鍵都來自IS NULL子句，因此確定匹配指定值的雜湊邊界的偏移量。
*
* Generally this function will return a single matching bound offset,
* although if a partition has not been setup for a given modulus then we may
* return no matches. If the number of clauses found don't cover the entire
* partition key, then we'll need to return all offsets.
* 通常，這個函式會返回一個匹配的邊界偏移量，
* 但是如果沒有為給定的模數設定分割槽，則可能不返回匹配值。
* 如果找到的子句數量不包含整個分割槽鍵，那麼我們需要返回所有偏移量。
*
* 'opstrategy' if non-zero must be HTEqualStrategyNumber.
* opstrategy - 如非0,則為HTEqualStrategyNumber
*
* 'values' contains Datums indexed by the partition key to use for pruning.
* values 包含用於分割槽鍵執行pruning的資料值
*
* 'nvalues', the number of Datums in the 'values' array.
* nvalues values陣列大小
*
* 'partsupfunc' contains partition hashing functions that can produce correct
* hash for the type of the values contained in 'values'.
* partsupfunc 儲存分割槽hash函式,可以為values產生hash值
*
* 'nullkeys' is the set of partition keys that are null.
* nullkeys 為null的分割槽鍵值集合
*/
static PruneStepResult *
get_matching_hash_bounds(PartitionPruneContext *context,
StrategyNumber opstrategy, Datum *values, int nvalues,
FmgrInfo *partsupfunc, Bitmapset *nullkeys)
{
PruneStepResult *result = (PruneStepResult *) palloc0(sizeof(PruneStepResult));
PartitionBoundInfo boundinfo = context->boundinfo;
int *partindices = boundinfo->indexes;
int partnatts = context->partnatts;
bool isnull[PARTITION_MAX_KEYS];
int i;
uint64 rowHash;
int greatest_modulus;
Assert(context->strategy == PARTITION_STRATEGY_HASH);
/*
* For hash partitioning we can only perform pruning based on equality
* clauses to the partition key or IS NULL clauses. We also can only
* prune if we got values for all keys.
* 對於Hash分割槽,只能基於等值條件語句或者是IS NULL條件進行pruning.
* 當然,如果可以拿到所有鍵的值,也可以執行prune
*/
if (nvalues + bms_num_members(nullkeys) == partnatts)
{
/*
* If there are any values, they must have come from clauses
* containing an equality operator compatible with hash partitioning.
* 如存在values,那麼這些值必須從包含一個與hash分割槽相容的等值運算子的條件語句而來
*/
Assert(opstrategy == HTEqualStrategyNumber || nvalues == 0);
for (i = 0; i < partnatts; i++)
isnull[i] = bms_is_member(i, nullkeys);
greatest_modulus = get_hash_partition_greatest_modulus(boundinfo);
rowHash = compute_partition_hash_value(partnatts, partsupfunc,
values, isnull);
if (partindices[rowHash % greatest_modulus] >= 0)
result->bound_offsets =
bms_make_singleton(rowHash % greatest_modulus);
}
else
{
/* Getting here means at least one hash partition exists. */
//程式執行到這裡,意味著至少存在一個hash分割槽
Assert(boundinfo->ndatums > 0);
result->bound_offsets = bms_add_range(NULL, 0,
boundinfo->ndatums - 1);
}
/*
* There is neither a special hash null partition or the default hash
* partition.
* 要麼存在一個特別的hash null分割槽,要麼是預設的hash分割槽
*/
result->scan_null = result->scan_default = false;
return result;
}
三、跟蹤分析
測試指令碼如下
testdb=# explain verbose select * from t_hash_partition where c1 = 1 OR c1 = 2;
QUERY PLAN
-------------------------------------------------------------------------------------
Append (cost=0.00..30.53 rows=6 width=200)
-> Seq Scan on public.t_hash_partition_1 (cost=0.00..15.25 rows=3 width=200)
Output: t_hash_partition_1.c1, t_hash_partition_1.c2, t_hash_partition_1.c3
Filter: ((t_hash_partition_1.c1 = 1) OR (t_hash_partition_1.c1 = 2))
-> Seq Scan on public.t_hash_partition_3 (cost=0.00..15.25 rows=3 width=200)
Output: t_hash_partition_3.c1, t_hash_partition_3.c2, t_hash_partition_3.c3
Filter: ((t_hash_partition_3.c1 = 1) OR (t_hash_partition_3.c1 = 2))
(7 rows)
啟動gdb,設定斷點
(gdb) b get_matching_partitions
Breakpoint 1 at 0x804d3b: file partprune.c, line 619.
(gdb) c
Continuing.
Breakpoint 1, get_matching_partitions (context=0x7fff4a5e3930, pruning_steps=0x14d5300) at partprune.c:619
619 int num_steps = list_length(pruning_steps),
(gdb) n
626 if (num_steps == 0)
檢視輸入引數,pruning_steps是有3個ITEM的連結串列
(gdb) p *pruning_steps
$1 = {type = T_List, length = 3, head = 0x14d52d8, tail = 0x14d58d0}
pruning_steps的3個ITEM型別分別是PartitionPruneStepOp/PartitionPruneStepOp/PartitionPruneStepCombine
第1和2個ITEM的expr是Const,即常量1和2
(gdb) p *(Node *)pruning_steps->head->data.ptr_value
$2 = {type = T_PartitionPruneStepOp} -->Node型別
(gdb) p *(PartitionPruneStepOp *)pruning_steps->head->data.ptr_value
$3 = {step = {type = T_PartitionPruneStepOp, step_id = 0}, opstrategy = 1, exprs = 0x14d52a0, cmpfns = 0x14d5240,
nullkeys = 0x0}
(gdb) set $ppso=(PartitionPruneStepOp *)pruning_steps->head->data.ptr_value
(gdb) p *$ppso->exprs
$4 = {type = T_List, length = 1, head = 0x14d5278, tail = 0x14d5278}
(gdb) p *(Node *)$ppso->exprs->head->data.ptr_value
$5 = {type = T_Const}
(gdb) p *(Const *)$ppso->exprs->head->data.ptr_value
$6 = {xpr = {type = T_Const}, consttype = 23, consttypmod = -1, constcollid = 0, constlen = 4, constvalue = 1,
constisnull = false, constbyval = true, location = 42} -->第1個步驟的表示式,常量1
(gdb) set $ppso=(PartitionPruneStepOp *)pruning_steps->head->next->data.ptr_value
(gdb) p *(Const *)$ppso->exprs->head->data.ptr_value
$7 = {xpr = {type = T_Const}, consttype = 23, consttypmod = -1, constcollid = 0, constlen = 4, constvalue = 2,
constisnull = false, constbyval = true, location = 52} -->第2個步驟的表示式,常量2
(gdb) p *(Node *)pruning_steps->head->next->next->data.ptr_value
$8 = {type = T_PartitionPruneStepCombine}
(gdb) set $ppsc=(PartitionPruneStepCombine *)pruning_steps->head->next->next->data.ptr_value
(gdb) p *$ppsc
$9 = {step = {type = T_PartitionPruneStepCombine, step_id = 2}, combineOp = PARTPRUNE_COMBINE_UNION,
source_stepids = 0x14d5480} -->第3個步驟,組合操作是PARTPRUNE_COMBINE_UNION
為步驟結果分配記憶體
(gdb) n
640 palloc0(num_steps * sizeof(PruneStepResult *));
(gdb) p num_steps
$10 = 3
(gdb) n
639 results = (PruneStepResult **)
(gdb)
641 foreach(lc, pruning_steps)
開始遍歷pruning步驟,進入perform_pruning_base_step函式
(gdb)
643 PartitionPruneStep *step = lfirst(lc);
(gdb)
645 switch (nodeTag(step))
(gdb)
648 results[step->step_id] =
(gdb) step
649 perform_pruning_base_step(context,
(gdb)
perform_pruning_base_step (context=0x7fff4a5e3930, opstep=0x14d4e98) at partprune.c:2995
2995 Assert(list_length(opstep->exprs) == list_length(opstep->cmpfns));
perform_pruning_base_step->檢視輸入引數,比較函式是OID=425的函式
(gdb) p *opstep->cmpfns
$12 = {type = T_OidList, length = 1, head = 0x14d5218, tail = 0x14d5218}
(gdb) p opstep->cmpfns->head->data.oid_value
$16 = 425
(gdb) n
2998 lc1 = list_head(opstep->exprs);
(gdb)
2999 lc2 = list_head(opstep->cmpfns);
(gdb)
perform_pruning_base_step->遍歷分割槽鍵
(gdb)
3005 for (keyno = 0; keyno < context->partnatts; keyno++)
(gdb)
3012 if (bms_is_member(keyno, opstep->nullkeys)) -->沒有null鍵
(gdb)
3019 if (keyno > nvalues && context->strategy == PARTITION_STRATEGY_RANGE) -->nvalues為0
(gdb) p nvalues
$17 = 0
(gdb) n
3022 if (lc1 != NULL)
(gdb)
3028 expr = lfirst(lc1); -->獲取步驟中的表示式,其實是常量1
(gdb)
3029 stateidx = PruneCxtStateIdx(context->partnatts, -->stateidx為0
(gdb) p *expr
$18 = {type = T_Const}
(gdb) p *(Const *)expr
$19 = {xpr = {type = T_Const}, consttype = 23, consttypmod = -1, constcollid = 0, constlen = 4, constvalue = 1,
constisnull = false, constbyval = true, location = 42}
(gdb) n
3031 if (partkey_datum_from_expr(context, expr, stateidx,
(gdb) p stateidx
$20 = 0
(gdb) n
3041 if (isnull) -->非NULL
(gdb) p isnull
$21 = false
perform_pruning_base_step->獲取比較函式進行處理
(gdb) n
3054 cmpfn = lfirst_oid(lc2); --> OID=425
(gdb)
3055 Assert(OidIsValid(cmpfn));
(gdb)
3056 if (cmpfn != context->stepcmpfuncs[stateidx].fn_oid) -->fn_oid為0
(gdb)
3064 if (cmpfn == context->partsupfunc[keyno].fn_oid) -->fn_oid為425
(gdb) p context->stepcmpfuncs[stateidx].fn_oid
$22 = 0
(gdb) p context->partsupfunc[keyno].fn_oid
$23 = 425
(gdb) n
3065 fmgr_info_copy(&context->stepcmpfuncs[stateidx],
(gdb)
3066 &context->partsupfunc[keyno],
(gdb)
3065 fmgr_info_copy(&context->stepcmpfuncs[stateidx],
(gdb)
3066 &context->partsupfunc[keyno],
(gdb)
3065 fmgr_info_copy(&context->stepcmpfuncs[stateidx], --> 複製函式
(gdb)
3073 values[keyno] = datum;-->設定值
(gdb) p datum
$24 = 1
(gdb) n
3074 nvalues++;
(gdb)
3077 lc1 = lnext(lc1);
(gdb)
3078 lc2 = lnext(lc2);
(gdb)
perform_pruning_base_step->完成分割槽鍵遍歷
(gdb) n
3005 for (keyno = 0; keyno < context->partnatts; keyno++)
(gdb)
3086 stateidx = PruneCxtStateIdx(context->partnatts, opstep->step.step_id, 0);
(gdb)
3087 partsupfunc = &context->stepcmpfuncs[stateidx];
(gdb)
3089 switch (context->strategy)
(gdb)
3092 return get_matching_hash_bounds(context,
(gdb)
perform_pruning_base_step->進入get_matching_hash_bounds函式
(gdb)
3092 return get_matching_hash_bounds(context,
(gdb) step
3093 opstep->opstrategy,
(gdb)
3092 return get_matching_hash_bounds(context,
(gdb)
get_matching_hash_bounds (context=0x7fff4a5e3930, opstrategy=1, values=0x7fff4a5e3750, nvalues=1, partsupfunc=0x14d3068,
nullkeys=0x0) at partprune.c:2156
2156 PruneStepResult *result = (PruneStepResult *) palloc0(sizeof(PruneStepResult));
(gdb)
get_matching_hash_bounds->變數賦值
(gdb) n
2157 PartitionBoundInfo boundinfo = context->boundinfo;
(gdb)
2158 int *partindices = boundinfo->indexes;
(gdb)
2159 int partnatts = context->partnatts;
(gdb)
2165 Assert(context->strategy == PARTITION_STRATEGY_HASH);
(gdb)
2172 if (nvalues + bms_num_members(nullkeys) == partnatts)
(gdb)
get_matching_hash_bounds->分割槽邊界資訊,共有6個分割槽,Index分別是0-5
(gdb) p boundinfo
$25 = (PartitionBoundInfo) 0x14d32a0
(gdb) p *boundinfo
$26 = {strategy = 104 'h', ndatums = 6, datums = 0x14d32f8, kind = 0x0, indexes = 0x14d2e00, null_index = -1,
default_index = -1}
(gdb) p *boundinfo->datums
$27 = (Datum *) 0x14d3350
(gdb) p **boundinfo->datums
$28 = 6
(gdb) p **boundinfo->indexes
Cannot access memory at address 0x0
(gdb) p *boundinfo->indexes
$29 = 0
(gdb) p boundinfo->indexes[0]
$30 = 0
(gdb) p boundinfo->indexes[1]
$31 = 1
(gdb) p boundinfo->indexes[5]
$32 = 5
(gdb)
get_matching_hash_bounds->分割槽索引和分割槽鍵數
(gdb) p *partindices
$33 = 0
(gdb) p partnatts
$34 = 1
(gdb)
(gdb) p nvalues
$35 = 1
(gdb) p bms_num_members(nullkeys)
$36 = 0
get_matching_hash_bounds->遍歷分割槽鍵,判斷值是否落在分割槽中
(gdb) n
2178 Assert(opstrategy == HTEqualStrategyNumber || nvalues == 0);
(gdb)
2180 for (i = 0; i < partnatts; i++)
(gdb)
2181 isnull[i] = bms_is_member(i, nullkeys);
(gdb)
2180 for (i = 0; i < partnatts; i++)
(gdb)
2183 greatest_modulus = get_hash_partition_greatest_modulus(boundinfo);
(gdb)
2184 rowHash = compute_partition_hash_value(partnatts, partsupfunc,
(gdb)
2187 if (partindices[rowHash % greatest_modulus] >= 0)
get_matching_hash_bounds->
(gdb) p values
$43 = (Datum *) 0x7fff4a5e3750
(gdb) p *values
$44 = 1 --> 約束條件
(gdb) p isnull[0]
$38 = false -->不為NULL
(gdb) p greatest_modulus
$39 = 6 -->6個分割槽
(gdb) p rowHash
$40 = 11274504255086170040 -->values算出的Hash值
(gdb) p rowHash % greatest_modulus
$41 = 2 --> 所在的分割槽
(gdb) p partindices[2]
$42 = 2 -->存在該分割槽
(gdb)
get_matching_hash_bounds->返回結果(bound_offsets->words=4,即編號2)
(gdb) n
2189 bms_make_singleton(rowHash % greatest_modulus);
(gdb)
2188 result->bound_offsets =
(gdb)
2203 result->scan_null = result->scan_default = false;
(gdb)
2205 return result;
(gdb)
2206 }
(gdb) p *result
$45 = {bound_offsets = 0x14d59b8, scan_default = false, scan_null = false}
(gdb) p *result->bound_offsets
$46 = {nwords = 1, words = 0x14d59bc}
(gdb) p *result->bound_offsets->words
$47 = 4
(gdb)
回到get_matching_partitions
(gdb) n
perform_pruning_base_step (context=0x7fff4a5e3930, opstep=0x14d4e98) at partprune.c:3119
3119 }
(gdb)
get_matching_partitions (context=0x7fff4a5e3930, pruning_steps=0x14d5300) at partprune.c:648
648 results[step->step_id] =
(gdb)
651 break;
(gdb)
641 foreach(lc, pruning_steps)
(gdb)
643 PartitionPruneStep *step = lfirst(lc);
(gdb)
645 switch (nodeTag(step))
(gdb) p
繼續執行步驟,進入perform_pruning_combine_step
654 results[step->step_id] =
(gdb)
655 perform_pruning_combine_step(context,
(gdb) step
perform_pruning_combine_step (context=0x7fff4a5e3930, cstep=0x14d5898, step_results=0x14d5958) at partprune.c:3136
3136 PruneStepResult *result = NULL;
(gdb)
perform_pruning_combine_step->進入PARTPRUNE_COMBINE_UNION處理邏輯
(gdb) n
3143 result = (PruneStepResult *) palloc0(sizeof(PruneStepResult));
(gdb)
3144 if (list_length(cstep->source_stepids) == 0)
(gdb)
3154 switch (cstep->combineOp)
(gdb)
3157 foreach(lc1, cstep->source_stepids)
(gdb)
(gdb) p *cstep
$49 = {step = {type = T_PartitionPruneStepCombine, step_id = 2}, combineOp = PARTPRUNE_COMBINE_UNION,
source_stepids = 0x14d5480}
perform_pruning_combine_step->遍歷組合步驟的源步驟 cstep->source_stepids,合併這些步驟的結果
(gdb) n
3159 int step_id = lfirst_int(lc1);
...
(gdb)
3174 result->bound_offsets = bms_add_members(result->bound_offsets,
(gdb)
3178 if (!result->scan_null)
(gdb)
3179 result->scan_null = step_result->scan_null;
(gdb)
3180 if (!result->scan_default)
(gdb)
3181 result->scan_default = step_result->scan_default;
(gdb)
3157 foreach(lc1, cstep->source_stepids)
(gdb)
3183 break;
(gdb)
3223 return result;
(gdb)
perform_pruning_combine_step->最終結果
(gdb) p *result
$54 = {bound_offsets = 0x14d5a48, scan_default = false, scan_null = false}
(gdb) p *result->bound_offsets
$55 = {nwords = 1, words = 0x14d5a4c}
(gdb) p *result->bound_offsets->words
$56 = 5
perform_pruning_combine_step->回到get_matching_partitions
(gdb) n
3224 }
(gdb)
get_matching_partitions (context=0x7fff4a5e3930, pruning_steps=0x14d5300) at partprune.c:654
654 results[step->step_id] =
完成所有步驟的處理
(gdb) n
658 break;
(gdb)
641 foreach(lc, pruning_steps)
(gdb) n
671 final_result = results[num_steps - 1];
(gdb)
672 Assert(final_result != NULL);
(gdb)
構造結果點陣圖集
...
675 while ((i = bms_next_member(final_result->bound_offsets, i)) >= 0)
(gdb) n
677 int partindex = context->boundinfo->indexes[i];
(gdb)
687 if (partindex >= 0)
(gdb)
688 result = bms_add_member(result, partindex);
(gdb)
完成呼叫
gdb)
675 while ((i = bms_next_member(final_result->bound_offsets, i)) >= 0)
(gdb)
692 if (final_result->scan_null)
(gdb)
698 if (final_result->scan_default)
(gdb)
706 return result;
(gdb)
707 }
(gdb)
DONE!
四、參考資料
Parallel Append implementation
Partition Elimination in PostgreSQL 11
來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/6906/viewspace-2374789/，如需轉載，請註明出處，否則將追究法律責任。
相關文章
- PostgreSQL 原始碼解讀（100）- 分割槽表#6（資料查詢路由#3-prune part...SQL原始碼路由
- PostgreSQL 原始碼解讀（98）- 分割槽表#4（資料查詢路由#1-“擴充套件”分割槽表）SQL原始碼路由套件
- PostgreSQL 原始碼解讀（99）- 分割槽表#5（資料查詢路由#2-RelOptInfo數...SQL原始碼路由
- PostgreSQL 原始碼解讀（92）- 分割槽表#1（資料插入路由#1）SQL原始碼路由
- PostgreSQL 原始碼解讀（94）- 分割槽表#2（資料插入路由#2）SQL原始碼路由
- PostgreSQL 原始碼解讀（102）- 分割槽表#8（資料查詢路由#5-構建APPEND訪問路徑）SQL原始碼路由APP
- PostgreSQL 原始碼解讀（103）- 分割槽表#9（資料查詢路由#6-APPEND初始化和實現）SQL原始碼路由APP
- PostgreSQL 原始碼解讀（96）- 分割槽表#3（資料插入路由#3-獲取分割槽鍵值）SQL原始碼路由
- PostgreSQL 原始碼解讀（185）- 查詢#101(聚合函式#6-simplehash)SQL原始碼函式
- PostgreSQL 原始碼解讀（205）- 查詢#118(資料結構RangeTblEntry)SQL原始碼資料結構
- Oracle查詢Interval partition分割槽表內資料Oracle
- PostgreSQL 原始碼解讀（199）- 查詢#114(排序#7 - inittapes&dumptuples)SQL原始碼排序
- PostgreSQL 原始碼解讀（206）- 查詢#119(資料結構RangSubselect等)SQL原始碼資料結構
- PostgreSQL 原始碼解讀（8）- 插入資料#7（ExecutePlan）SQL原始碼
- PostgreSQL 原始碼解讀（204）- 查詢#117(資料結構SelectStmt&Value)SQL原始碼資料結構
- PostgreSQL 原始碼解讀（207）- 查詢#120(資料結構FromExpr&JoinExpr)SQL原始碼資料結構
- PostgreSQL 原始碼解讀（230）- 查詢#123(NOT IN實現)SQL原始碼
- PostgreSQL 原始碼解讀（24）- 查詢語句#9（查詢重寫）SQL原始碼
- PostgreSQL 原始碼解讀（186）- 查詢#102(聚合函式#7-advance_aggregates)SQL原始碼函式
- PostgreSQL 原始碼解讀（20）- 查詢語句#5（查詢樹Query詳解）SQL原始碼
- PostgreSQL/LightDB 分割槽表之分割槽裁剪SQL
- PostgreSQL 原始碼解讀（164）- 查詢#84(表示式求值)SQL原始碼
- PostgreSQL 原始碼解讀（215）- 查詢#122(varstrfastcmp_locale)SQL原始碼AST
- PostgreSQL 原始碼解讀（231）- 查詢#124(NOT IN實現#2)SQL原始碼
- PostgreSQL 原始碼解讀（233）- 查詢#126(NOT IN實現#4)SQL原始碼
- PostgreSQL 原始碼解讀（234）- 查詢#127(NOT IN實現#5)SQL原始碼
- PostgreSQL 原始碼解讀（232）- 查詢#125(NOT IN實現#3)SQL原始碼
- PostgreSQL 原始碼解讀（197）- 查詢#112(排序#5 - mergeruns)SQL原始碼排序
- PostgreSQL 原始碼解讀（193）- 查詢#109(排序#2 - ExecSort)SQL原始碼排序
- PostgreSQL 原始碼解讀（192）- 查詢#108(排序#1 - ExecInitSort)SQL原始碼排序
- PostgreSQL 原始碼解讀（198）- 查詢#113(排序#6 - Tuplesortstate)SQL原始碼排序
- PostgreSQL 原始碼解讀（46）- 查詢語句#31（query_planner函式#7）SQL原始碼函式
- PostgreSQL 原始碼解讀（22）- 查詢語句#7（PlannedStmt結構詳解-日誌分析）SQL原始碼
- PostgreSQL 原始碼解讀（19）- 查詢語句#4（ParseTree詳解）SQL原始碼
- PostgreSQL 原始碼解讀（17）- 查詢語句#2（查詢優化基礎）SQL原始碼優化
- PostgreSQL 原始碼解讀（25）- 查詢語句#10（查詢優化概覽）SQL原始碼優化
- PostgreSQL 原始碼解讀（29）- 查詢語句#14（查詢優化-上拉子查詢）SQL原始碼優化
- PostgreSQL 原始碼解讀（18）- 查詢語句#3（SQL Parse）SQL原始碼