我在互联网上找不到任何有效的示例,可以看到它们之间的区别以及为什么要选择一个而不是另一个。
Answers:
第一个带有0个或多个参数,每个参数都是可迭代的,第二个带有一个参数,该参数有望产生可迭代的:
from itertools import chain
chain(list1, list2, list3)
iterables = [list1, list2, list3]
chain.from_iterable(iterables)
但iterables
可以是产生可迭代对象的任何迭代器:
def gen_iterables():
for i in range(10):
yield range(i)
itertools.chain.from_iterable(gen_iterables())
使用第二种形式通常是方便的情况,但是由于它懒惰地遍历输入可迭代对象,因此这也是链接无限数量的有限迭代器的唯一方法:
def gen_iterables():
while True:
for i in range(5, 10):
yield range(i)
chain.from_iterable(gen_iterables())
上面的示例将为您提供一个可迭代的迭代器,该迭代器将产生一个循环的数字模式,该模式将永远不会停止,但永远不会消耗比单个range()
调用所需的内存更多的内存。
# Return an iterator of iterators
def it_it(): return iter( [iter( [11, 22] ), iter( [33, 44] )] )
print( list(itertools.chain.from_iterable(it_it())) )
print( list(itertools.chain(it_it())) )
print( list(itertools.chain(*it_it())) )
第一个最好。第二个没有到达嵌套的迭代器,它返回迭代器,而不是所需的数字;第三个产生正确的输出,但并不是完全懒惰:“ *”强制创建所有迭代器。对于这个愚蠢的输入,没关系。
itertools.chain(*iterables)
我找不到任何有效的示例……在这里我可以看到它们[
chain
和chain.from_iterable
]之间的区别以及为什么要选择一个而不是另一个
公认的答案是彻底的。对于那些寻求快速申请的人,请考虑展平几个列表:
list(itertools.chain(["a", "b", "c"], ["d", "e"], ["f"]))
# ['a', 'b', 'c', 'd', 'e', 'f']
您可能希望稍后再使用这些列表,因此可以对列表进行迭代:
iterable = (["a", "b", "c"], ["d", "e"], ["f"])
尝试
但是,将iterable传入chain
会产生不平坦的结果:
list(itertools.chain(iterable))
# [['a', 'b', 'c'], ['d', 'e'], ['f']]
为什么?您传入了一项(元组)。 chain
分别需要每个列表。
解决方案
如果可能,您可以解开可迭代的文件:
list(itertools.chain(*iterable))
# ['a', 'b', 'c', 'd', 'e', 'f']
list(itertools.chain(*iter(iterable)))
# ['a', 'b', 'c', 'd', 'e', 'f']
更一般地,使用.from_iterable
(因为它也可以与无限迭代器一起使用):
list(itertools.chain.from_iterable(iterable))
# ['a', 'b', 'c', 'd', 'e', 'f']
g = itertools.chain.from_iterable(itertools.cycle(iterable))
next(g)
# "a"
他们做非常相似的事情。对于少量的可迭代对象itertools.chain(*iterables)
,itertools.chain.from_iterable(iterables)
执行类似的操作。
关键的优势from_iterables
在于能够处理大量(可能无限)的可迭代对象,因为在调用时不需要全部可用。
*
操作员是否iterables
开箱吗?
itertools.chain(*iterables)
是一个函数调用。所有参数必须在调用时出现。
from_iterables
速度越来越快。
尽管在迭代过程中对内部项目的访问保持不变,但在实现方面还是明智的,
itertools_chain_from_iterable
(即chain.from_iterable
在Python中)和chain_new
(即chain
在Python中)在CPython实现中,都是chain_new_internal的鸭子类型
使用chain.from_iterable(x)
,x
有可迭代的地方是否有优化的好处?主要目的是最终消耗平整的物料清单?
我们可以尝试使用以下方法进行基准测试:
import random
from itertools import chain
from functools import wraps
from time import time
from tqdm import tqdm
def timing(f):
@wraps(f)
def wrap(*args, **kw):
ts = time()
result = f(*args, **kw)
te = time()
print('func:%r args:[%r, %r] took: %2.4f sec' % (f.__name__, args, kw, te-ts))
return result
return wrap
def generate_nm(m, n):
# Creates m generators of m integers between range 0 to n.
yield iter(random.sample(range(n), n) for _ in range(m))
def chain_star(x):
# Stores an iterable that will unpack and flatten the list of list.
chain_x = chain(*x)
# Consumes the items in the flatten iterable.
for i in chain_x:
pass
def chain_from_iterable(x):
# Stores an iterable that will unpack and flatten the list of list.
chain_x = chain.from_iterable(x)
# Consumes the items in the flatten iterable.
for i in chain_x:
pass
@timing
def versus(f, n, m):
f(generate_nm(n, m))
P / S:正在运行基准测试...等待结果。
for _ in range(10):
versus(chain_star, 1000, 1000)
[出]:
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6494 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6603 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6367 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6350 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6296 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6399 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6341 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6381 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6343 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 1000, 1000), {}] took: 0.6309 sec
for _ in range(10):
versus(chain_from_iterable, 1000, 1000)
[出]:
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6416 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6315 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6535 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6334 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6327 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6471 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6426 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6287 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6353 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 1000, 1000), {}] took: 0.6297 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2659 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2966 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2953 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3141 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2802 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2799 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2848 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3299 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.2730 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 10000, 1000), {}] took: 6.3052 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3129 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3064 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.3071 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2660 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2837 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2877 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2756 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2939 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2715 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 10000, 1000), {}] took: 6.2877 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.7874 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.3744 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.5584 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.3745 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.7982 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.4054 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.6769 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.6476 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 63.7397 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 100000, 1000), {}] took: 62.8980 sec
for _ in range(10):
versus(chain_from_iterable, 100000, 1000)
[出]:
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7227 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7717 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7159 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7569 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7906 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.6211 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.7294 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.8260 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.8356 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 100000, 1000), {}] took: 62.9738 sec
for _ in range(3):
versus(chain_from_iterable, 500000, 1000)
[出]:
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 314.5671 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 313.9270 sec
func:'versus' args:[(<function chain_star at 0x7f5c7188ef28>, 500000, 1000), {}] took: 313.8992 sec
for _ in range(3):
versus(chain_from_iterable, 500000, 1000)
[出]:
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.8301 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.8104 sec
func:'versus' args:[(<function chain_from_iterable at 0x7f5c7188eb70>, 500000, 1000), {}] took: 313.9440 sec
另一种查看方式是使用chain.from_iterable
当您拥有一个可迭代的可迭代对象(例如嵌套可迭代对象(或复合可迭代对象))并且将链用于简单可迭代对象时