【Python】django-celery非同步任務佇列

小亮520cl發表於2017-11-10
PythonDjango非同步佇列

Django中如何使用django-celery完成非同步任務 

安裝Celery

我們可以使用pip在vietualenv中安裝: 

 pip install django-celery  celery
django settings設定
  1. import djcelery

  3. djcelery.setup_loader()
  4. BROKER_URL = 'redis://127.0.0.1:6379/2'   ---使用redis當訊息佇列


註冊
  1. INSTALLED_APPS = [
  2.     'django.contrib.admin',
  3.     'django.contrib.auth',
  4.     'django.contrib.contenttypes',
  5.     'django.contrib.sessions',
  6.     'django.contrib.messages',
  7.     'django.contrib.staticfiles',
  8.     'devops',
  9.     'apps',
  10.     'common',
  11.     'djcelery',
  12. ]


在apps下建立一個task.py檔案
  1. from celery import task

  3. @task
  4. def add(x, y):
  5.     return x + y
  
  @task 
 def pp(): 
 return 'ffffffffffffffffffffff'

返回settings將其匯入進去
  1. import djcelery
  2. djcelery.setup_loader()
  3. BROKER_URL = 'redis://127.0.0.1:6379/2'
  4. CELERY_IMPORTS = ('apps.task')

隨意選個views中的方法呼叫改task測試下
  1. #專案列表
  2. def project_list(request):
  3.     admin = Admin.objects.get(id=get_current_admin_id(request))
  4.     pt=admin.projects.all().order_by('-id')
  5.    
  6.     from apps import task
  7.     tt=task.add.delay(2,2)
  8.     print 'vvvvvvvvvvvvvvvvvvvv',tt              -----訪問該功能時呼叫task的add方法

  10.     page_objects = pages(pt, request, 5) ##分頁
  11.     return render_to_response('project/project_list.html',locals())


安裝啟動redis：略

啟動runserver 與 celery
  1. python manage.py runserver
  2. python manage.py celery worker --loglevel=info


訪問對應的頁面看日誌
  1. [2017-11-10 14:55:17,016: INFO/MainProcess] Task apps.task.add[9bafe6d2-8411-4f5f-8eed-10444da0ae3a] succeeded in 0.00280212797225s: 4   --可以看到worker日誌，返回結果


手工測試task
  1. 開啟新的terminal, 啟用virtualenv, 並切換到django專案目錄:
  2.     $ python manage.py shell
  3.     >>> from apps.task import add
  4.     >>> add.delay(2, 2)

  5.
  6. 此時, 你可以在worker視窗中看到worker執行該task:
  7.     [2014-10-07 08:47:08,076: INFO/MainProcess] Got task from broker: myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc]
  8.     [2014-10-07 08:47:08,299: INFO/MainProcess] Task myapp.tasks.add[e080e047-b2a2-43a7-af74-d7d9d98b02fc] succeeded in 0.183349132538s: 4

Eager模式
  1. Eager模式

  3. 如果在settings.py設定:
  4.     CELERY_ALWAYS_EAGER = True
  1. 那麼Celery便以eager模式執行, 則task便不需要加delay執行:
  2.     # 若啟用eager模式, 則以下兩行程式碼相同
  3.     add.delay(2, 2)
  4.     add(2, 2)


dj-celery 定時任務
  1. settings配置新增
    1. CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler' # 定時任務

    3. CELERYBEAT_SCHEDULE = {
    4.    'add-every-3-seconds': {
    5.        'task': 'apps.task.pp',
    6.        'schedule': timedelta(seconds=3)   ---每隔3秒 執行task下的pp函式
    7.    },
    8. }

     如果需要傳參可以這樣寫
    1. CELERYBEAT_SCHEDULE = {
    2.    'add-every-3-minutes': {
    3.        'task': 'apps.task.add',
    4.        'schedule': timedelta(seconds=3),
    5.         'args': (16, 16)
    6.    },
    7. }



  2. 啟動celery beat
    1. 啟動beat
    2. 執行定時任務時, Celery會透過celerybeat程式來完成. Celerybeat會保持執行, 一旦到了某一定時任務需要執行時, Celerybeat便將其加入到queue中. 不像worker程式, Celerybeat只有需要一個即可.
    3. 啟動：
    4. python manage.py celery beat --loglevel=info

    1. 其實還有一種簡單的啟動方式worker和beat一起啟動：
    2. python manage.py celery worker --loglevel=info --beat

  3. 檢視worker日誌
    1. [2017-11-10 16:17:25,853: INFO/MainProcess] Received task: apps.task.pp[8a3af6fb-5189-4647-91f2-8aa07489dd1e]
    2. [2017-11-10 16:17:25,858: INFO/MainProcess] Task apps.task.pp[8a3af6fb-5189-4647-91f2-8aa07489dd1e] succeeded in 0.00379144400358s: 'ffffffffffffffffffffff'
    3. [2017-11-10 16:17:28,858: INFO/MainProcess] Received task: apps.task.pp[d87e4ea0-8881-449a-b993-e7657f50ef25]
    4. [2017-11-10 16:17:28,864: INFO/MainProcess] Task apps.task.pp[d87e4ea0-8881-449a-b993-e7657f50ef25] succeeded in 0.0049942266196s: 'ffffffffffffffffffffff'
    5. [2017-11-10 16:17:31,859: INFO/MainProcess] Received task: apps.task.pp[4d05b4f3-92ff-4922-a8f4-7e047749239a]
    6. [2017-11-10 16:17:31,865: INFO/MainProcess] Task apps.task.pp[4d05b4f3-92ff-4922-a8f4-7e047749239a] succeeded in 0.00537821277976s: 'ffffffffffffffffffffff'
    7. [2017-11-10 16:17:34,859: INFO/MainProcess] Received task: apps.task.pp[5b21afc1-ebf1-4858-be68-20b9bf318452]
    8. [2017-11-10 16:17:34,865: INFO/MainProcess] Task apps.task.pp[5b21afc1-ebf1-4858-be68-20b9bf318452] succeeded in 0.00530335493386s: 'ffffffffffffffffffffff'

  4. 參考文件：
  5. http://www.codeweblog.com/djcelery%E5%85%A5%E9%97%A8-%E5%AE%9E%E7%8E%B0%E8%BF%90%E8%A1%8C%E5%AE%9A%E6%97%B6%E4%BB%BB%E5%8A%A1/




django-celery參考文件：
http://blog.csdn.net/acm_zl/article/details/53188064
https://www.cnblogs.com/znicy/p/5626040.html
http://www.weiguda.com/blog/73/
https://www.cnblogs.com/Lin-Yi/p/7590971.html


補充還有另外一種方法那就是celery
  1. 具體配置可見參考
  2. http://www.jianshu.com/p/7a869a73b92f



來自 “ ITPUB部落格 ” ，連結：http://blog.itpub.net/29096438/viewspace-2147120/，如需轉載，請註明出處，否則將追究法律責任。

相關文章