【odoo】【知識雜談】單一例項多庫模式下定時任務的問題分析

老韓頭的開發日常發表於2021-09-06

歡迎轉載,但需標註出處,謝謝!

背景:

有客戶反應有個別模組下的定時任務沒有正常執行,是否是新裝的模組哪些有問題?排查後發現,客戶是在一臺伺服器上跑著一個odoo容器,對應多個資料庫。個別庫的定時任務是正常的,但是一個對接其他平臺的庫的定時任務沒有正常跑起來。

先說結論,看官沒時間支援按說明處理即可,分析過程在下面。

結論

在odoo的配置檔案db_name欄位配置希望後臺一直跑著的庫名稱字串,以英文“,”分割。

分析

直接原始碼

  1. 看odoo日誌,我們知道odoo的任務正常執行時會列印Starting Job 任務名稱,直接vscode全域性查詢,定位到ir_cron.py檔案的_process_jobs函式。
    @classmethod
    def _process_jobs(cls, db_name):
        """ Try to process all cron jobs.

        This selects in database all the jobs that should be processed. It then
        tries to lock each of them and, if it succeeds, run the cron job (if it
        doesn't succeed, it means the job was already locked to be taken care
        of by another thread) and return.

        :raise BadVersion: if the version is different from the worker's
        :raise BadModuleState: if modules are to install/upgrade/remove
        """
        db = odoo.sql_db.db_connect(db_name)
        threading.current_thread().dbname = db_name
        try:
            with db.cursor() as cr:
                # Make sure the database has the same version as the code of
                # base and that no module must be installed/upgraded/removed
                cr.execute("SELECT latest_version FROM ir_module_module WHERE name=%s", ['base'])
                (version,) = cr.fetchone()
                cr.execute("SELECT COUNT(*) FROM ir_module_module WHERE state LIKE %s", ['to %'])
                (changes,) = cr.fetchone()
                if version is None:
                    raise BadModuleState()
                elif version != BASE_VERSION:
                    raise BadVersion()
                # Careful to compare timestamps with 'UTC' - everything is UTC as of v6.1.
                cr.execute("""SELECT * FROM ir_cron
                              WHERE numbercall != 0
                                  AND active AND nextcall <= (now() at time zone 'UTC')
                              ORDER BY priority""")
                jobs = cr.dictfetchall()

            if changes:
                if not jobs:
                    raise BadModuleState()
                # nextcall is never updated if the cron is not executed,
                # it is used as a sentinel value to check whether cron jobs
                # have been locked for a long time (stuck)
                parse = fields.Datetime.from_string
                oldest = min([parse(job['nextcall']) for job in jobs])
                if datetime.now() - oldest > MAX_FAIL_TIME:
                    odoo.modules.reset_modules_state(db_name)
                else:
                    raise BadModuleState()

            for job in jobs:
                lock_cr = db.cursor()
                try:
                    # Try to grab an exclusive lock on the job row from within the task transaction
                    # Restrict to the same conditions as for the search since the job may have already
                    # been run by an other thread when cron is running in multi thread
                    lock_cr.execute("""SELECT *
                                       FROM ir_cron
                                       WHERE numbercall != 0
                                          AND active
                                          AND nextcall <= (now() at time zone 'UTC')
                                          AND id=%s
                                       FOR UPDATE NOWAIT""",
                                   (job['id'],), log_exceptions=False)

                    locked_job = lock_cr.fetchone()
                    if not locked_job:
                        _logger.debug("Job `%s` already executed by another process/thread. skipping it", job['cron_name'])
                        continue
                    # Got the lock on the job row, run its code
                    _logger.info('Starting job `%s`.', job['cron_name'])
                    job_cr = db.cursor()
                    try:
                        registry = odoo.registry(db_name)
                        registry[cls._name]._process_job(job_cr, job, lock_cr)
                        _logger.info('Job `%s` done.', job['cron_name'])
                    except Exception:
                        _logger.exception('Unexpected exception while processing cron job %r', job)
                    finally:
                        job_cr.close()

                except psycopg2.OperationalError as e:
                    if e.pgcode == '55P03':
                        # Class 55: Object not in prerequisite state; 55P03: lock_not_available
                        _logger.debug('Another process/thread is already busy executing job `%s`, skipping it.', job['cron_name'])
                        continue
                    else:
                        # Unexpected OperationalError
                        raise
                finally:
                    # we're exiting due to an exception while acquiring the lock
                    lock_cr.close()

        finally:
            if hasattr(threading.current_thread(), 'dbname'):
                del threading.current_thread().dbname
  1. 看到上面這個函式已經是執行的具體內容了。我們繼續在該檔案查詢_process_jobs函式 => _acquire_job函式 => server.py檔案中的cron_thread函式 => cron_spawn函式。
    其中cron_spawn定了開啟幾個cron執行緒,由max_cron_threads決定。
    在cron_thread函式中,我們可以看到定時任務的呼叫過程
    def cron_thread(self, number):
        from odoo.addons.base.models.ir_cron import ir_cron
        while True:
            time.sleep(SLEEP_INTERVAL + number)     # Steve Reich timing style
            registries = odoo.modules.registry.Registry.registries
            _logger.debug('cron%d polling for jobs', number)
            for db_name, registry in registries.d.items():
                if registry.ready:
                    thread = threading.currentThread()
                    thread.start_time = time.time()
                    try:
                        ir_cron._acquire_job(db_name)
                    except Exception:
                        _logger.warning('cron%d encountered an Exception:', number, exc_info=True)
                    thread.start_time = None
  1. 核心內容是registries.d.items(),cron執行緒將迴圈呼叫registries中的資料庫資訊,那麼這個變數中到底有哪些內容,如何新增的呢?可以全域性搜尋registries,定位到Registry.py檔案(具體的registry類物件)以及server.py中的preload_registries函式以及呼叫該函式的run函式。看名稱可以瞭解將預載入registry資訊。

    def run(self, preload=None, stop=False):
        """ Start the http server and the cron thread then wait for a signal.

        The first SIGINT or SIGTERM signal will initiate a graceful shutdown while
        a second one if any will force an immediate exit.
        """
        self.start(stop=stop)

        rc = preload_registries(preload)

        if stop:
            if config['test_enable']:
                logger = odoo.tests.runner._logger
                with Registry.registries._lock:
                    for db, registry in Registry.registries.d.items():
                        report = registry._assertion_report
                        log = logger.error if not report.wasSuccessful() \
                         else logger.warning if not report.testsRun \
                         else logger.info
                        log("%s when loading database %r", report, db)
            self.stop()
            return rc
  1. 核心是run函式彙總的preload變數,記錄著將初始化哪些資料庫的物件;

  2. 查詢上級為,server.py中的main函式,至此所有思路都清晰了,看原始碼


def main(args):
    check_root_user()
    odoo.tools.config.parse_config(args)
    check_postgres_user()
    report_configuration()

    config = odoo.tools.config

    # the default limit for CSV fields in the module is 128KiB, which is not
    # quite sufficient to import images to store in attachment. 500MiB is a
    # bit overkill, but better safe than sorry I guess
    csv.field_size_limit(500 * 1024 * 1024)

    preload = []
    if config['db_name']:
        preload = config['db_name'].split(',')
        for db_name in preload:
            try:
                odoo.service.db._create_empty_database(db_name)
                config['init']['base'] = True
            except ProgrammingError as err:
                if err.pgcode == errorcodes.INSUFFICIENT_PRIVILEGE:
                    # We use an INFO loglevel on purpose in order to avoid
                    # reporting unnecessary warnings on build environment
                    # using restricted database access.
                    _logger.info("Could not determine if database %s exists, "
                                 "skipping auto-creation: %s", db_name, err)
                else:
                    raise err
            except odoo.service.db.DatabaseExists:
                pass

    if config["translate_out"]:
        export_translation()
        sys.exit(0)

    if config["translate_in"]:
        import_translation()
        sys.exit(0)

    # This needs to be done now to ensure the use of the multiprocessing
    # signaling mecanism for registries loaded with -d
    if config['workers']:
        odoo.multi_process = True

    stop = config["stop_after_init"]

    setup_pid_file()
    rc = odoo.service.server.start(preload=preload, stop=stop)
    sys.exit(rc)

在上面,我們preload為配置檔案中的db_name的值,那麼正向梳理回去就是
a) odoo在啟動的時候載入odoo.conf配置檔案,並讀取db_name的值
b) 載入完成後將通過db_name的值初始化資料庫物件;
c) 並在完成cron執行緒初始化後迴圈呼叫庫物件,執行相關定時任務。

相關文章