如何展望Python生成器中的一个元素（预览）？

我不知道如何展望Python生成器中的一个元素。我一看就不见了。

这是我的意思：

gen = iter([1,2,3])
next_value = gen.next()  # okay, I looked forward and see that next_value = 1
# but now:
list(gen)  # is [2, 3]  -- the first value is gone!

这是一个更真实的示例：

gen = element_generator()
if gen.next_value() == 'STOP':
  quit_application()
else:
  process(gen.next())

谁能帮我写一个生成器，让您向前看一个元素？

Python生成器API是一种方式：您不能回退已阅读的元素。但是您可以使用itertools模块创建一个新的迭代器，并在元素前添加：

import itertools

gen = iter([1,2,3])
peek = gen.next()
print list(itertools.chain([peek], gen))

为了完整起见，该more-itertools软件包（可能应该是任何Python程序员的工具箱的一部分）都包含一个peekable实现此行为的包装器。如文档中的代码示例所示：

>>> p = peekable(['a', 'b'])
>>> p.peek()
'a'
>>> next(p)
'a'

但是，通常可以重写将使用此功能的代码，以便实际上不需要它。例如，问题中的实际代码示例可以这样编写：

gen = element_generator()
command = gen.next_value()
if command == 'STOP':
  quit_application()
else:
  process(command)

（读者注意：在编写本示例时，我保留了该示例中的语法，即使该语法指的是Python的过时版本）

好的-太迟了两年-但是我遇到了这个问题，却没有找到令我满意的答案。想到了这个元生成器：

class Peekorator(object):

    def __init__(self, generator):
        self.empty = False
        self.peek = None
        self.generator = generator
        try:
            self.peek = self.generator.next()
        except StopIteration:
            self.empty = True

    def __iter__(self):
        return self

    def next(self):
        """
        Return the self.peek element, or raise StopIteration
        if empty
        """
        if self.empty:
            raise StopIteration()
        to_return = self.peek
        try:
            self.peek = self.generator.next()
        except StopIteration:
            self.peek = None
            self.empty = True
        return to_return

def simple_iterator():
    for x in range(10):
        yield x*3

pkr = Peekorator(simple_iterator())
for i in pkr:
    print i, pkr.peek, pkr.empty

结果是：

0 3 False
3 6 False
6 9 False
9 12 False    
...
24 27 False
27 None False

也就是说，您在迭代过程中随时可以访问列表中的下一项。

您可以使用itertools.tee生成生成器的轻量级副本。然后先偷看一个副本不会影响第二个副本：

import itertools

def process(seq):
    peeker, items = itertools.tee(seq)

    # initial peek ahead
    # so that peeker is one ahead of items
    if next(peeker) == 'STOP':
        return

    for item in items:

        # peek ahead
        if next(peeker) == "STOP":
            return

        # process items
        print(item)

骚扰“偷窥者”不会影响“项目”生成器。请注意，在调用“ tee”之后，您不应使用原始的“ seq”，否则会导致问题。

FWIW，这是解决此问题的错误方法。任何需要您在生成器中向前看一项的算法都可以替换为使用当前生成器项和上一个项。这样一来，您就不必改变对生成器的使用，您的代码将变得更加简单。请参阅我对这个问题的其他答案。

>>> gen = iter(range(10))
>>> peek = next(gen)
>>> peek
0
>>> gen = (value for g in ([peek], gen) for value in g)
>>> list(gen)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

只是为了好玩，我根据亚伦的建议创建了一个超前类的实现：

import itertools

class lookahead_chain(object):
    def __init__(self, it):
        self._it = iter(it)

    def __iter__(self):
        return self

    def next(self):
        return next(self._it)

    def peek(self, default=None, _chain=itertools.chain):
        it = self._it
        try:
            v = self._it.next()
            self._it = _chain((v,), it)
            return v
        except StopIteration:
            return default

lookahead = lookahead_chain

这样，以下将起作用：

>>> t = lookahead(xrange(8))
>>> list(itertools.islice(t, 3))
[0, 1, 2]
>>> t.peek()
3
>>> list(itertools.islice(t, 3))
[3, 4, 5]

通过这种实现，连续多次调用偷看是一个坏主意。

在查看CPython源代码时，我发现了一个更短，更有效的更好方法：

class lookahead_tee(object):
    def __init__(self, it):
        self._it, = itertools.tee(it, 1)

    def __iter__(self):
        return self._it

    def peek(self, default=None):
        try:
            return self._it.__copy__().next()
        except StopIteration:
            return default

lookahead = lookahead_tee

用法与上面相同，但您无需为此付出任何代价就可以连续多次使用peek。再增加几行，您还可以在迭代器中查找多个项（最多可用RAM）。

您应该使用（i-1，i），其中“ i-1”，而不是使用项目（i，i + 1），其中“ i”是当前项目，而i + 1是“向前看”版本是生成器的先前版本。

以这种方式调整算法将产生与您当前拥有的相同的东西，除了尝试“偷看”的额外不必要的复杂性。

偷看是一个错误，您不应该这样做。

这将起作用-它缓冲一个项目并为每个项目和序列中的下一个项目调用一个函数。

您对序列末尾发生的情况不满意。当您处于最后一个位置时，“向前看”是什么意思？

def process_with_lookahead( iterable, aFunction ):
    prev= iterable.next()
    for item in iterable:
        aFunction( prev, item )
        prev= item
    aFunction( item, None )

def someLookaheadFunction( item, next_item ):
    print item, next_item

一个简单的解决方案是使用如下函数：

def peek(it):
    first = next(it)
    return first, itertools.chain([first], it)

然后，您可以执行以下操作：

>>> it = iter(range(10))
>>> x, it = peek(it)
>>> x
0
>>> next(it)
0
>>> next(it)
1

如果有人感兴趣，如果我错了，请纠正我，但是我相信向任何迭代器添加一些推回功能非常容易。

class Back_pushable_iterator:
    """Class whose constructor takes an iterator as its only parameter, and
    returns an iterator that behaves in the same way, with added push back
    functionality.

    The idea is to be able to push back elements that need to be retrieved once
    more with the iterator semantics. This is particularly useful to implement
    LL(k) parsers that need k tokens of lookahead. Lookahead or push back is
    really a matter of perspective. The pushing back strategy allows a clean
    parser implementation based on recursive parser functions.

    The invoker of this class takes care of storing the elements that should be
    pushed back. A consequence of this is that any elements can be "pushed
    back", even elements that have never been retrieved from the iterator.
    The elements that are pushed back are then retrieved through the iterator
    interface in a LIFO-manner (as should logically be expected).

    This class works for any iterator but is especially meaningful for a
    generator iterator, which offers no obvious push back ability.

    In the LL(k) case mentioned above, the tokenizer can be implemented by a
    standard generator function (clean and simple), that is completed by this
    class for the needs of the actual parser.
    """
    def __init__(self, iterator):
        self.iterator = iterator
        self.pushed_back = []

    def __iter__(self):
        return self

    def __next__(self):
        if self.pushed_back:
            return self.pushed_back.pop()
        else:
            return next(self.iterator)

    def push_back(self, element):
        self.pushed_back.append(element)

it = Back_pushable_iterator(x for x in range(10))

x = next(it) # 0
print(x)
it.push_back(x)
x = next(it) # 0
print(x)
x = next(it) # 1
print(x)
x = next(it) # 2
y = next(it) # 3
print(x)
print(y)
it.push_back(y)
it.push_back(x)
x = next(it) # 2
y = next(it) # 3
print(x)
print(y)

for x in it:
    print(x) # 4-9

@ jonathan-hartley的Python3代码段答案：

def peek(iterator, eoi=None):
    iterator = iter(iterator)

    try:
        prev = next(iterator)
    except StopIteration:
        return iterator

    for elm in iterator:
        yield prev, elm
        prev = elm

    yield prev, eoi


for curr, nxt in peek(range(10)):
    print((curr, nxt))

# (0, 1)
# (1, 2)
# (2, 3)
# (3, 4)
# (4, 5)
# (5, 6)
# (6, 7)
# (7, 8)
# (8, 9)
# (9, None)

创建一个可以在其上执行此操作__iter__并仅生成prev项目并将其放入elm某个属性的类将很简单。

@David Z的帖子，较新的seekable工具可以将包装的迭代器重置为先前的位置。

>>> s = mit.seekable(range(3))
>>> s.next()
# 0

>>> s.seek(0)                                              # reset iterator
>>> s.next()
# 0

>>> s.next()
# 1

>>> s.seek(1)
>>> s.next()
# 1

>>> next(s)
# 2

cytoolz具有偷看功能。

>> from cytoolz import peek
>> gen = iter([1,2,3])
>> first, continuation = peek(gen)
>> first
1
>> list(continuation)
[1, 2, 3]

一个迭代器，它允许窥视下一个元素并且也可以向前看。它会根据需要预先读取，并记住中的值deque。

from collections import deque

class PeekIterator:

    def __init__(self, iterable):
        self.iterator = iter(iterable)
        self.peeked = deque()

    def __iter__(self):
        return self

    def __next__(self):
        if self.peeked:
            return self.peeked.popleft()
        return next(self.iterator)

    def peek(self, ahead=0):
        while len(self.peeked) <= ahead:
            self.peeked.append(next(self.iterator))
        return self.peeked[ahead]

演示：

>>> it = PeekIterator(range(10))
>>> it.peek()
0
>>> it.peek(5)
5
>>> it.peek(13)
Traceback (most recent call last):
  File "<pyshell#68>", line 1, in <module>
    it.peek(13)
  File "[...]", line 15, in peek
    self.peeked.append(next(self.iterator))
StopIteration
>>> it.peek(2)
2
>>> next(it)
0
>>> it.peek(2)
3
>>> list(it)
[1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>

尽管这itertools.chain()是完成任务的自然工具，但请注意以下循环：

for elem in gen:
    ...
    peek = next(gen)
    gen = itertools.chain([peek], gen)

...因为这将消耗线性增长的内存量，并最终停止运行。（此代码本质上似乎创建了一个链表，每个chain（）调用一个节点。）我知道这不是因为我检查了库，而是因为这只会导致我的程序严重减速-摆脱该gen = itertools.chain([peek], gen)行会加速它再次。（Python 3.3）