Asyncio.gather vs asyncio.wait


148

asyncio.gather并且asyncio.wait似乎有类似的用法:我有一堆我想执行/等待的异步事情(不一定要等到下一个开始之前完成)。它们使用不同的语法,并且在某些细节上有所不同,但是对我来说,拥有2个功能在功能上有如此大的重叠是非常不切实际的。我想念什么?

Answers:


177

尽管在一般情况下类似(“为许多任务运行并获取结果”),但是对于其他情况,每个功能都有一些特定的功能:

asyncio.gather()

返回一个Future实例,允许高层任务分组:

import asyncio
from pprint import pprint

import random


async def coro(tag):
    print(">", tag)
    await asyncio.sleep(random.uniform(1, 3))
    print("<", tag)
    return tag


loop = asyncio.get_event_loop()

group1 = asyncio.gather(*[coro("group 1.{}".format(i)) for i in range(1, 6)])
group2 = asyncio.gather(*[coro("group 2.{}".format(i)) for i in range(1, 4)])
group3 = asyncio.gather(*[coro("group 3.{}".format(i)) for i in range(1, 10)])

all_groups = asyncio.gather(group1, group2, group3)

results = loop.run_until_complete(all_groups)

loop.close()

pprint(results)

群组中的所有任务都可以通过调用group2.cancel()甚至取消all_groups.cancel()。另见.gather(..., return_exceptions=True)

asyncio.wait()

支持在完成第一个任务后或在指定的超时后等待停止,从而降低了操作的精度:

import asyncio
import random


async def coro(tag):
    print(">", tag)
    await asyncio.sleep(random.uniform(0.5, 5))
    print("<", tag)
    return tag


loop = asyncio.get_event_loop()

tasks = [coro(i) for i in range(1, 11)]

print("Get first result:")
finished, unfinished = loop.run_until_complete(
    asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED))

for task in finished:
    print(task.result())
print("unfinished:", len(unfinished))

print("Get more results in 2 seconds:")
finished2, unfinished2 = loop.run_until_complete(
    asyncio.wait(unfinished, timeout=2))

for task in finished2:
    print(task.result())
print("unfinished2:", len(unfinished2))

print("Get all other results:")
finished3, unfinished3 = loop.run_until_complete(asyncio.wait(unfinished2))

for task in finished3:
    print(task.result())

loop.close()

4
“单星号形式(* args)用于传递非关键字的可变长度参数列表,而双星号形式用于传递关键字的可变长度参数列表”
产地

40

asyncio.waitasyncio.gather

顾名思义,asyncio.gather主要集中在收集结果上。它等待一堆期货,并以给定的顺序返回其结果。

asyncio.wait只是等待期货。而不是直接给您结果,而是完成和待处理的任务。您必须手动收集值。

此外,您可以指定等待所有期货完成,或者仅等待第一个期货wait


您说:it waits on a bunch of futures and return their results in a given order。如果我有10000000000000个任务并且所有任务都返回大数据怎么办?所有的结果都会使内存繁荣吗?
Kingname

@Kingname ..wat
Matt Joiner

12

我还注意到,您可以通过简单地指定列表来在wait()中提供一组协程:

result=loop.run_until_complete(asyncio.wait([
        say('first hello', 2),
        say('second hello', 1),
        say('third hello', 4)
    ]))

而通过仅指定多个协程来完成对collect()的分组:

result=loop.run_until_complete(asyncio.gather(
        say('first hello', 2),
        say('second hello', 1),
        say('third hello', 4)
    ))

20
列表也可以用于gather(),例如:asyncio.gather(*task_list)
tehfink

1
发电机也可以
Jab

如何在不阻塞脚本其余部分的情况下使用此聚集?
thebeancounter
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.