輸入一條SQL語句,PostgreSQL如何解析輸入的SQL,識別SQL型別以及基表/欄位等資訊?接下來的幾節將逐一進行解析.
本節介紹了PostgreSQL的詞法定義檔案(Flex輸入檔案),在檔案src/backend/parser/scan.l中.
如前所述,Flex輸入檔案由四部分組成:
```
%{
Declarations
%}
Definitions
%%
Rules
%%
User subroutines
```
本節介紹第一部分Declarations

### 一、Declarations
由%{和%}包含的部分為Declarations部分,這一部分都是C程式碼,會原封不動的copy到lex.yy.c檔案中.
比較重要的定義包括:
YYSTYPE-Bison使用一個union聯合體來儲存所有可能型別的值,全域性變數yyvalue的型別是YYSTYPE.
```
%top{
/*-------------------------------------------------------------------------
 *
 * scan.l
 *    lexical scanner for PostgreSQL
 *    PostgreSQL的詞法掃描器  
 *
 * NOTE NOTE NOTE:
 * 特別特別特別注意:
 * The rules in this file must be kept in sync with src/fe_utils/psqlscan.l!
 * 這個檔案中的規則必須與src/fe_utils/psqlscan.l檔案中的規則保持一致!!! 
 *
 * The rules are designed so that the scanner never has to backtrack,
 * in the sense that there is always a rule that can match the input
 * consumed so far (the rule action may internally throw back some input
 * with yyless(), however).  As explained in the flex manual, this makes
 * for a useful speed increase --- about a third faster than a plain -CF
 * lexer, in simple testing.  The extra complexity is mostly in the rules
 * for handling float numbers and continued string literals.  If you change
 * the lexical rules, verify that you haven't broken the no-backtrack
 * property by running flex with the "-b" option and checking that the
 * resulting "lex.backup" file says that no backing up is needed.  (As of
 * Postgres 9.2, this check is made automatically by the Makefile.)
 * 之所以設計這一的規則是便於掃描器不需要回溯,確保對於輸入一定有一條規則與其匹配
 * (但是,規則動作可能在內部用yyless() throw back一些輸入).
 * 正如Flex手冊中所說明的,這可以提升效能 -- 
 *   在簡單測試的情況下,相對於普通的-CF詞法分析器,大概有1/3的效能提升.
 * 額外的複雜性主要體現在處理浮點數和連續字串文字的規則中.
 * 如果修改了詞法規則,透過以-b選項執行Flex以確保沒有打破無回溯的約定,
 *   並且堅持結果檔案"lex.backup"以確認無需備份.
 * (在PG 9.2,該檢查透過Makefile自動執行)
 *
 * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
 * Portions Copyright (c) 1994, Regents of the University of California
 *
 * IDENTIFICATION
 *    src/backend/parser/scan.l
 *
 *-------------------------------------------------------------------------
 */
#include "postgres.h"

#include <ctype.h>
#include <unistd.h>

#include "common/string.h"
#include "parser/gramparse.h"
#include "parser/parser.h"      /* only needed for GUC variables */
#include "parser/scansup.h"
#include "mb/pg_wchar.h"
}

//------------------ 宣告部分
%{

/* LCOV_EXCL_START */

/* Avoid exit() on fatal scanner errors (a bit ugly -- see yy_fatal_error) */
//在掃描器出現致命錯誤時,避免呼叫exit()直接退出
#undef fprintf
#define fprintf(file, fmt, msg)  fprintf_to_ereport(fmt, msg)

static void
fprintf_to_ereport(const char *fmt, const char *msg)
{
    ereport(ERROR, (errmsg_internal("%s", msg)));
}

/*
 * GUC variables.  This is a DIRECT violation of the warning given at the
 * head of gram.y, ie flex/bison code must not depend on any GUC variables;
 * as such, changing their values can induce very unintuitive behavior.
 * But we shall have to live with it until we can remove these variables.
 * GUC引數變數.這直接違反了gram.y中提出的約定,如flex/bison程式碼不能依賴GUC變數;
 * 因此,改變他們的值會導致未知的後果.
 * 但在去掉這些變數前,不得不"活下去"
 */
int         backslash_quote = BACKSLASH_QUOTE_SAFE_ENCODING;
bool        escape_string_warning = true;
bool        standard_conforming_strings = true;

/*
 * Set the type of YYSTYPE.
 * 設定YYSTYPE.
 * 在Bison中,全域性變數yylval的型別為YYSTYPE,預設為int
 * Internally, bison declares each value as a C union that includes all of the types. 
 * You list all of the types in %union declarations. 
 * Bison turns them into a typedef for a union type called YYSTYPE.
 */
#define YYSTYPE core_YYSTYPE

/*
 * Set the type of yyextra.  All state variables used by the scanner should
 * be in yyextra, *not* statically allocated.
 * 設定yyextra的資料型別.所有掃描器使用的狀態變數應在yyextra中,不是靜態分配的.
 */
#define YY_EXTRA_TYPE core_yy_extra_type *

/*
 * Each call to yylex must set yylloc to the location of the found token
 * (expressed as a byte offset from the start of the input text).
 * When we parse a token that requires multiple lexer rules to process,
 * this should be done in the first such rule, else yylloc will point
 * into the middle of the token.
 * 每一次呼叫yylex必須設定yylloc指向發現的token所在的位置.
 * (從輸入文字開始計算的位元組偏移量)
 * 在分析一個需要多個詞法規則進行處理的token時,
 *   在第一次應用規則時就應該完成這個動作,否則的話yylloc會指向到token的中間位置.
 */
#define SET_YYLLOC()  (*(yylloc) = yytext - yyextra->scanbuf)

/*
 * Advance yylloc by the given number of bytes.
 * 透過給定的位元組數調整yylloc的位置
 */
#define ADVANCE_YYLLOC(delta)  ( *(yylloc) += (delta) )

#define startlit()  ( yyextra->literallen = 0 )
static void addlit(char *ytext, int yleng, core_yyscan_t yyscanner);
static void addlitchar(unsigned char ychar, core_yyscan_t yyscanner);
static char *litbufdup(core_yyscan_t yyscanner);
static char *litbuf_udeescape(unsigned char escape, core_yyscan_t yyscanner);
static unsigned char unescape_single_char(unsigned char c, core_yyscan_t yyscanner);
static int  process_integer_literal(const char *token, YYSTYPE *lval);
static bool is_utf16_surrogate_first(pg_wchar c);
static bool is_utf16_surrogate_second(pg_wchar c);
static pg_wchar surrogate_pair_to_codepoint(pg_wchar first, pg_wchar second);
static void addunicode(pg_wchar c, yyscan_t yyscanner);
static bool check_uescapechar(unsigned char escape);

#define yyerror(msg)  scanner_yyerror(msg, yyscanner)

#define lexer_errposition()  scanner_errposition(*(yylloc), yyscanner)

static void check_string_escape_warning(unsigned char ychar, core_yyscan_t yyscanner);
static void check_escape_warning(core_yyscan_t yyscanner);

/*
 * Work around a bug in flex 2.5.35: it emits a couple of functions that
 * it forgets to emit declarations for.  Since we use -Wmissing-prototypes,
 * this would cause warnings.  Providing our own declarations should be
 * harmless even when the bug gets fixed.
 * Flex 2.5.35存在一個bug:忽略了函式但沒有忽略函式宣告.
 * 因為使用了-Wmissing-prototypes選項,這會導致警告出現.
 * 就算bug修復,提供PG的宣告也可能會存在問題.
 */
extern int  core_yyget_column(yyscan_t yyscanner);
extern void core_yyset_column(int column_no, yyscan_t yyscanner);

%}

```

### 二、參考資料
[Flex&Bison](http://web.iitd.ac.in/~sumeet/flex__bison.pdf)</unistd.h></ctype.h>

PostgreSQL 原始碼解讀（168）- 查詢#88(PG中的詞法定義：scanner.l)#1

相關文章