Skip to content

asyncio concurrent asyncio retry does not work. #16

Description

@bt-wil

When calling asyncio.gather with functions decorated retry, the retries would finish earlier than expected.

import asyncio
import logging

import riprova


logging.basicConfig(
    format='[%(asctime)s] %(levelname)s - %(message)s',
    datefmt="%Y-%m-%d %H:%M:%S",
    level=logging.INFO
)
LOG = logging.getLogger()


def do_retry(val):
    LOG.info("do retry %s", val)
    return True


@riprova.retry(
    backoff=riprova.ConstantBackoff(retries=5),
    error_evaluator=do_retry,
)
async def do_stuff(name):
    LOG.info('calling %s', name)
    if name == 'bar':
        await asyncio.sleep(1.0)
    else:
        await asyncio.sleep(0.1)

    raise ValueError('value-error-{}'.format(name))


async def process():
    coros = [do_stuff('foo'), do_stuff('bar')]
    # coros = [do_stuff('foo')]
    await asyncio.gather(*coros, return_exceptions=True)


loop = asyncio.get_event_loop()
loop.run_until_complete(process())
loop.close()

results:

[2018-03-29 19:17:42] INFO - calling bar
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:42] INFO - do retry value-error-foo
[2018-03-29 19:17:42] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo
[2018-03-29 19:17:43] INFO - do retry value-error-bar
[2018-03-29 19:17:43] INFO - calling foo
[2018-03-29 19:17:43] INFO - do retry value-error-foo

The expected result is that foo and bar should be both called 6x.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions