with as 語句真的會把資料存記憶體嘛?(原始碼剖析)

專注的阿熊發表於2021-09-28

STAGE DEPENDENCIES:

  Stage-1 is a root stage

  Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2, Stage-4

  Stage-3

  Stage-0 depends on stages: Stage-3, Stage-2, Stage-5

  Stage-8 depends on stages: Stage-0

  Stage-2

  Stage-4

  Stage-5 depends on stages: Stage-4

  Stage-7 depends on stages: Stage-8

STAGE PLANS:

  Stage: Stage-1

   Map Reduce

    Map Operator Tree:

      TableScan

       alias: test

       Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

       Select Operator

        expressions: id (type: int), source (type: string), channel (type: string)

        outputColumnNames: _col0, _col1, _col2

        Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

        File Output Operator

         compressed: false

         Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

         table:

           input format: org.apache.hadoop.mapred.TextInputFormat

           output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat

           serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

           name: default.atable

  Stage: Stage-6

   Conditional Operator

  Stage: Stage-3

   Move Operator

    files:

      hdfs directory: true

      destination: hdfs://localhost:9000/tmp/hive/bytedance/bae441cb-0ef6-4e9c-9f7a-8a5f97d0e560/_tmp_space.db/ce44793b-6eed-4299-b737-f05c66b2281b/.hive-staging_hive_2021-03-24_20-17-38_169_5695913330535939856-1/-ext-10002

  Stage: Stage-0

   Move Operator

    files:

      hdfs directory: true

      destination: hdfs://localhost:9000/tmp/hive/bytedance/bae441cb-0ef6-4e9c-9f7a-8a5f97d0e560/_tmp_space.db/ce44793b-6eed-4299-b737-f05c66b2281b

  Stage: Stage-8

   Map Reduce

    Map Operator Tree:

      TableScan

       alias: atable

       Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

       Filter Operator

        predicate: (channel = ' 直播 ') (type: boolean)

        Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

        Select Operator

         expressions: source (type: string)

         outputColumnNames: _col0

         Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

         Union

          Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column stats: NONE

          File Output Operator

           compressed: false

           Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column stats: NONE

           table:

             input format: org.apache.hadoop.mapred.SequenceFileInputFormat

             output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat

             serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

      TableScan

       alias: atable

       Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

       Filter Operator

        predicate: (channel = 外匯跟單gendan5.com' 影片 ') (type: boolean)

        Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

        Select Operator

         expressions: source (type: string)

         outputColumnNames: _col0

         Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE

         Union

          Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column stats: NONE

          File Output Operator

           compressed: false

           Statistics: Num rows: 2 Data size: 0 Basic stats: PARTIAL Column stats: NONE

           table:

             input format: org.apache.hadoop.mapred.SequenceFileInputFormat

             output format: org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat

             serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

  Stage: Stage-2

   Map Reduce

    Map Operator Tree:

      TableScan

       File Output Operator

        compressed: false

        table:

          input format: org.apache.hadoop.mapred.TextInputFormat

          output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat

          serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

          name: default.atable

  Stage: Stage-4

   Map Reduce

    Map Operator Tree:

      TableScan

       File Output Operator

        compressed: false

        table:

          input format: org.apache.hadoop.mapred.TextInputFormat

          output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat

          serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

          name: default.atable

  Stage: Stage-5

   Move Operator

    files:

      hdfs directory: true

      destination: hdfs://localhost:9000/tmp/hive/bytedance/bae441cb-0ef6-4e9c-9f7a-8a5f97d0e560/_tmp_space.db/ce44793b-6eed-4299-b737-f05c66b2281b/.hive-staging_hive_2021-03-24_20-17-38_169_5695913330535939856-1/-ext-10002

  Stage: Stage-7

   Fetch Operator

    limit: -1

    Processor Tree:

     ListSink

來自 “ ITPUB部落格 ” ,連結:http://blog.itpub.net/69946337/viewspace-2794248/,如需轉載,請註明出處,否則將追究法律責任。

相關文章