{"id":5439,"date":"2014-01-02T21:13:36","date_gmt":"2014-01-02T13:13:36","guid":{"rendered":"https:\/\/kyle.ai\/blog\/?p=5439"},"modified":"2014-01-02T21:13:36","modified_gmt":"2014-01-02T13:13:36","slug":"celery%e7%94%a8redis%e4%bd%9cbroker%ef%bc%8ceta%e4%bb%bb%e5%8a%a1%e9%87%8d%e5%a4%8d%e6%89%a7%e8%a1%8c%e7%9a%84bug","status":"publish","type":"post","link":"https:\/\/kyle.ai\/blog\/5439.html","title":{"rendered":"Celery\u7528redis\u4f5cbroker\uff0cETA\u4efb\u52a1\u91cd\u590d\u6267\u884c\u7684bug"},"content":{"rendered":"<p>\u6211\u4eec\u7528celery\u6846\u67b6\u505a\u5f02\u6b65\u961f\u5217\u6267\u884c\u4efb\u52a1\uff0c\u7528redis\u505acelery\u7684broker\u5b58\u50a8\uff0c\u6700\u8fd1\u9047\u5230\u4e00\u4e2a\u95ee\u9898\u3002<\/p>\n<p>\u5468\u672b\u6211\u4eec\u7528celery\u53bb\u6267\u884c\u4e00\u4e2a\u5ef6\u8fdf\u4efb\u52a1\uff0c\u4e5f\u5c31\u662f\u6307\u5b9a\u6267\u884c\u65f6\u95f4\uff0c\u6bd4\u5982\u5468\u4e94\u8bbe\u7f6e\u5468\u516d\u8ddf\u5468\u65e5\u5206\u522b\u6267\u884c\u63a8\u9001\u3002<\/p>\n<p>\u7136\u540e\u5230\u6267\u884c\u65f6\u95f4\u7684\u65f6\u5019\u53d1\u73b0\uff0c\u4efb\u52a1\u88ab\u591a\u6b21\u6267\u884c\u4e86\u3002<\/p>\n<p>\u4ece\u8868\u9762\u4e0a\u770b\uff0c\u597d\u50cf\u662fcelery\u7684\u6bcf\u4e00\u4e2aworker\u90fd\u5206\u522b\u6267\u884c\u4e86\u4e00\u6b21\u4efb\u52a1\uff0c\u4f46celery\u6536\u5230\u786e\u5b9e\u53ea\u6709\u4e00\u6761\u4efb\u52a1\u3002<\/p>\n<p>\u540e\u6765\uff0c\u6211\u901a\u8fc7\u547d\u4ee4\u89c2\u5bdf\u5b9a\u65f6\u6267\u884c\u7684\u4efb\u52a1\uff1a<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\ncelery inspect scheduled\r\n<\/pre>\n<p>\u53d1\u73b0\uff0c\u65b0\u5efa\u7684\u5b9a\u65f6\u4efb\u52a1\uff0c\u9694\u4e00\u4f1a\u540e\uff0c\u5c31\u81ea\u52a8\u53d8\u6210\u4e24\u6761\u4e86\uff1a<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\nkyle@dev-desktop1:~\/www\/meila_app\/deps\/meila_queue$ date\r\n2013\u5e74 12\u6708 31\u65e5 \u661f\u671f\u4e8c 09:42:44 CST\r\nkyle@dev-desktop1:~\/www\/meila_app\/deps\/meila_queue$ env\/bin\/python run.py inspect scheduled\r\n-&gt; dev-desktop1: OK\r\n    * {u'priority': 6, u'eta': 1388456519.971404, u'request': {u'args': u'&#x5B;691]', u'time_start': None, u'name': u'meila_queue.tasks.pnmessage.push_message_to_group', u'delivery_info': {u'priority': 0, u'routing_key': u'celery', u'exchange': u'celery'}, u'hostname': u'dev-desktop1', u'acknowledged': False, u'kwargs': u'{}', u'id': u'b754d377-4fe6-4b36-a0ee-f0bffc6f90bc', u'worker_pid': None}}\r\n\r\nkyle@dev-desktop1:~\/www\/meila_app\/deps\/meila_queue$ date\r\n2013\u5e74 12\u6708 31\u65e5 \u661f\u671f\u4e8c 09:54:01 CST\r\nkyle@dev-desktop1:~\/www\/meila_app\/deps\/meila_queue$ env\/bin\/python run.py inspect scheduled\r\n-&gt; dev-desktop1: OK\r\n    * {u'priority': 6, u'eta': 1388456519.971404, u'request': {u'args': u'&#x5B;691]', u'time_start': None, u'name': u'meila_queue.tasks.pnmessage.push_message_to_group', u'delivery_info': {u'priority': 0, u'routing_key': u'celery', u'exchange': u'celery'}, u'hostname': u'dev-desktop1', u'acknowledged': False, u'kwargs': u'{}', u'id': u'b754d377-4fe6-4b36-a0ee-f0bffc6f90bc', u'worker_pid': None}}\r\n    * {u'priority': 6, u'eta': 1388456519.971404, u'request': {u'args': u'&#x5B;691]', u'time_start': None, u'name': u'meila_queue.tasks.pnmessage.push_message_to_group', u'delivery_info': {u'priority': 0, u'routing_key': u'celery', u'exchange': u'celery'}, u'hostname': u'dev-desktop1', u'acknowledged': False, u'kwargs': u'{}', u'id': u'b754d377-4fe6-4b36-a0ee-f0bffc6f90bc', u'worker_pid': None}}\r\n<\/pre>\n<p>\u4e00\u6a21\u4e00\u6837\u7684\u4efb\u52a1\uff0c\u5c45\u7136\u81ea\u5df1\u514b\u9686\u51fa\u4e86\u4e24\u6761\uff08visibility_timeout\u8bbe\u7f6e\u621010\u5206\u949f\u540e\u91cd\u73b0\u51fa\u7684\u7ed3\u679c\uff09\u3002<\/p>\n<p>\u540e\u9762\u6211\u67e5celery\u7684redis broker\u6587\u6863 http:\/\/docs.celeryproject.org\/en\/latest\/getting-started\/brokers\/redis.html\uff0c\u53d1\u73b0\u4e86\u8fd9\u4e48\u4e00\u6bb5\u8bdd\uff1a<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">\r\n\r\nIf a task is not acknowledged within the Visibility Timeout the task will be redelivered to another worker and executed.\r\n\r\nThis causes problems with ETA\/countdown\/retry tasks where the time to execute exceeds the visibility timeout; in fact if that happens it will be executed again, and again in a loop.\r\n\r\nSo you have to increase the visibility timeout to match the time of the longest ETA you are planning to use.\r\n\r\nNote that Celery will redeliver messages at worker shutdown, so having a long visibility timeout will only delay the redelivery of \u2018lost\u2019 tasks in the event of a power failure or forcefully terminated workers.\r\n\r\nPeriodic tasks will not be affected by the visibility timeout, as this is a concept separate from ETA\/countdown.\r\n\r\nYou can increase this timeout by configuring a transport option with the same name:\r\n\r\nBROKER_TRANSPORT_OPTIONS = {\u2018visibility_timeout\u2019: 43200}\r\n\r\nThe value must be an int describing the number of seconds.\r\n<\/pre>\n<p>\u4e5f\u5c31\u662f\u8bf4\uff0c\u5f53\u6211\u4eec\u8bbe\u7f6e\u4e00\u4e2aETA\u65f6\u95f4\u6bd4visibility_timeout\u957f\u7684\u4efb\u52a1\u65f6\uff0c\u6bcf\u8fc7\u4e00\u6b21 visibility_timeout \u65f6\u95f4\uff0ccelery\u5c31\u4f1a\u8ba4\u4e3a\u8fd9\u4e2a\u4efb\u52a1\u6ca1\u88abworker\u6267\u884c\u6210\u529f\uff0c\u91cd\u65b0\u5206\u914d\u7ed9\u5176\u5b83worker\u518d\u6267\u884c\u3002celery\u8fd9\u65b9\u9762\u6ca1\u6709\u5904\u7406\u597d\uff0c\u6ca1\u8003\u8651\u5230\u5b9a\u65f6\u4efb\u52a1\u60c5\u51b5\u3002<\/p>\n<p>\u89e3\u51b3\u529e\u6cd5\uff0c\u6309\u6587\u6863\u6240\u8ff0\uff0c\u5c31\u662f\u628a visibility_timeout\u53c2\u6570\u8c03\u5927\uff0c\u6bd4\u6211\u4eecETA\u7684\u65f6\u95f4\u5dee\u8981\u5927\u3002\u4f46\u8fd9\u6837\u4e5f\u4f1a\u6709\u526f\u4f5c\u7528\u3002<\/p>\n<p>celery \u9879\u76ee\u672c\u8eab\u7684\u5b9a\u4f4d\u5c31\u4e3b\u8981\u662f\u5b9e\u65f6\u7684\u5f02\u6b65\u961f\u5217\uff0c\u5bf9\u4e8e\u8fd9\u79cd\u957f\u65f6\u95f4\u5b9a\u65f6\u6267\u884c\uff0c\u652f\u6301\u4e0d\u592a\u597d\u3002<\/p>\n<p>\u4f46\u662f\u73b0\u5728\u6211\u628a\u8fd9\u4e2a\u53c2\u6570\u6539\u5927\u4e86\uff0c\u6539\u6210\u4e862\u5929\uff0c\u7ed3\u679c\u4e00\u4e2a\u5c0f\u65f6\u540e\uff0c\u4efb\u52a1\u7167\u6837\u88ab\u514b\u9686\u4e86\u3002\u3002\u3002<\/p>\n<p>\u6700\u540e\u6ca1\u529e\u6cd5\uff0c\u53ea\u597d\u628abroker\u4eceredis\u6362\u6210\u4e86 rabbitmq<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6211\u4eec\u7528celery\u6846\u67b6\u505a\u5f02\u6b65\u961f\u5217\u6267\u884c\u4efb\u52a1\uff0c\u7528redis\u505acelery\u7684broker\u5b58\u50a8\uff0c\u6700\u8fd1\u9047\u5230\u4e00\u4e2a\u95ee\u9898\u3002 \u5468 [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[],"class_list":["post-5439","post","type-post","status-publish","format-standard","hentry","category-diary"],"_links":{"self":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/5439","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/comments?post=5439"}],"version-history":[{"count":2,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/5439\/revisions"}],"predecessor-version":[{"id":5441,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/posts\/5439\/revisions\/5441"}],"wp:attachment":[{"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/media?parent=5439"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/categories?post=5439"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kyle.ai\/blog\/wp-json\/wp\/v2\/tags?post=5439"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}