为什么尝试使用此代码在TLS上运行TLS时握手失败?


67

我尝试实现一种协议,该协议可以twisted.protocols.tls使用内存在BIO上与OpenSSL的接口,从而可以使用TLS在TLS上运行TLS 。

我将其实现为协议包装,它看起来像是常规的TCP传输,但是它具有startTLSstopTLS方法分别用于添加和删除TLS层。这对于TLS的第一层工作正常。如果我在“本机” Twisted TLS传输上运行它,它也可以正常工作。但是,如果我尝试使用startTLS此包装器提供的方法添加第二个TLS层,则会立即出现握手错误,并且连接最终将处于未知的不可用状态。

包装程序和使它工作的两个助手如下所示:

from twisted.python.components import proxyForInterface
from twisted.internet.error import ConnectionDone
from twisted.internet.interfaces import ITCPTransport, IProtocol
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory

class TransportWithoutDisconnection(proxyForInterface(ITCPTransport)):
    """
    A proxy for a normal transport that disables actually closing the connection.
    This is necessary so that when TLSMemoryBIOProtocol notices the SSL EOF it
    doesn't actually close the underlying connection.

    All methods except loseConnection are proxied directly to the real transport.
    """
    def loseConnection(self):
        pass


class ProtocolWithoutConnectionLost(proxyForInterface(IProtocol)):
    """
    A proxy for a normal protocol which captures clean connection shutdown
    notification and sends it to the TLS stacking code instead of the protocol.
    When TLS is shutdown cleanly, this notification will arrive.  Instead of telling
    the protocol that the entire connection is gone, the notification is used to
    unstack the TLS code in OnionProtocol and hidden from the wrapped protocol.  Any
    other kind of connection shutdown (SSL handshake error, network hiccups, etc) are
    treated as real problems and propagated to the wrapped protocol.
    """
    def connectionLost(self, reason):
        if reason.check(ConnectionDone):
            self.onion._stopped()
        else:
            super(ProtocolWithoutConnectionLost, self).connectionLost(reason)


class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol, it can run over
    any other ITransport.  As a transport, it implements stackable TLS.  That is,
    whatever application traffic is generated by the protocol running on top of
    OnionProtocol can be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS conversation can
    be encapsulated in yet *another* TLS conversation.

    Each layer of TLS can use different connection parameters, such as keys, ciphers,
    certificate requirements, etc.  At the remote end of this connection, each has to
    be decrypted separately, starting at the outermost and working in.  OnionProtocol
    can do this itself, of course, just as it can encrypt each layer starting with the
    innermost.
    """
    def makeConnection(self, transport):
        self._tlsStack = []
        ProtocolWrapper.makeConnection(self, transport)


    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given contextFactory.

        If *client* is True, this side of the connection will be an SSL client.
        Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of the SSL handshake
        were received by the protocol running on top of OnionProtocol, they must be
        passed here as the **bytes** parameter.
        """
        # First, create a wrapper around the application-level protocol
        # (wrappedProtocol) which can catch connectionLost and tell this OnionProtocol 
        # about it.  This is necessary to pop from _tlsStack when the outermost TLS
        # layer stops.
        connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
        connLost.onion = self
        # Construct a new TLS layer, delivering events and application data to the
        # wrapper just created.
        tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        # Push the previous transport and protocol onto the stack so they can be
        # retrieved when this new TLS layer stops.
        self._tlsStack.append((self.transport, self.wrappedProtocol))

        # Create a transport for the new TLS layer to talk to.  This is a passthrough
        # to the OnionProtocol's current transport, except for capturing loseConnection
        # to avoid really closing the underlying connection.
        transport = TransportWithoutDisconnection(self.transport)

        # Make the new TLS layer the current protocol and transport.
        self.wrappedProtocol = self.transport = tlsProtocol

        # And connect the new TLS layer to the previous outermost transport.
        self.transport.makeConnection(transport)

        # If the application accidentally got some bytes from the TLS handshake, deliver
        # them to the new TLS layer.
        if bytes is not None:
            self.wrappedProtocol.dataReceived(bytes)


    def stopTLS(self):
        """
        Remove a layer of TLS.
        """
        # Just tell the current TLS layer to shut down.  When it has done so, we'll get
        # notification in *_stopped*.
        self.transport.loseConnection()


    def _stopped(self):
        # A TLS layer has completely shut down.  Throw it away and move back to the
        # TLS layer it was wrapping (or possibly back to the original non-TLS
        # transport).
        self.transport, self.wrappedProtocol = self._tlsStack.pop()

我有用于执行此操作的简单客户端和服务器程序,可从启动板(bzr branch lp:~exarkun/+junk/onion)获得。当我使用它startTLS两次调用上述方法,而没有中间调用时stopTLS,出现此OpenSSL错误:

OpenSSL.SSL.Error: [('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]

为什么出问题了?


2
为什么要在TLS中使用TLS?
Jumbogram 2011年

5
为什么不?;)目前,我实际上没有任何实际用途。另一个stackoverflow问题(很遗憾,此问题已被其作者删除)提示我探索它。但是,实现类似Tor之类的洋葱路由网络可能是有用的事情。
Jean-Paul Calderone

10
我通常的网络调试方法是“使用wireshark”,当您要查看的数据被加密时,该方法不能很好地工作。您是否可以让外部层使用TLS_RSA_WITH_NULL_MD5,以便可以进行有意义的数据包捕获?
Jumbogram 2011年

1
好主意。我会尽快尝试。
Jean-Paul Calderone

2
您可能必须对其进行特殊配置。'openssl ciphers'不会列出它,但是'openssl ciphers NULL'会列出它。
Jumbogram 2011年

Answers:


19

至少有两个问题OnionProtocol

  1. 最内 TLSMemoryBIOProtocol成为wrappedProtocol,当它应该是最外面的;
  2. ProtocolWithoutConnectionLost不会弹出TLSMemoryBIOProtocolsOnionProtocol的堆栈,因为connectionLost仅在FileDescriptorsdoReaddoWrite方法返回断开连接的原因后才调用。

在不改变OnionProtocol管理堆栈的方式的情况下,我们不能解决第一个问题,在弄清楚新的堆栈实现之前,我们无法解决第二个问题。毫不奇怪,正确的设计是Twisted中数据流方式的直接结果,因此我们将从一些数据流分析开始。

Twisted表示与twisted.internet.tcp.Server或的实例建立的连接twisted.internet.tcp.Client。由于程序中唯一的交互发生在中stoptls_client,因此我们将仅考虑往返Client实例的数据流。

让我们用一个最小的LineReceiver客户端进行预热,该客户端回显从端口9999上的本地服务器接收到的行:

from twisted.protocols import basic
from twisted.internet import defer, endpoints, protocol, task

class LineReceiver(basic.LineReceiver):
    def lineReceived(self, line):
        self.sendLine(line)

def main(reactor):
    clientEndpoint = endpoints.clientFromString(
        reactor, "tcp:localhost:9999")
    connected = clientEndpoint.connect(
        protocol.ClientFactory.forProtocol(LineReceiver))
    def waitForever(_):
        return defer.Deferred()
    return connected.addCallback(waitForever)

task.react(main)

一旦建立的连接建立,a便Client成为我们LineReceiver协议的传输并调解输入和输出:

客户端和线路接收器

来自服务器的新数据使反应堆调用ClientdoRead方法,该方法又将接收到LineReceiver的内容传递给的dataReceived方法。最后,至少有一条线路可用时LineReceiver.dataReceived呼叫LineReceiver.lineReceived

我们的应用程序通过调用将一行数据发送回服务器LineReceiver.sendLine。这要求write绑定到协议实例的传输,Client实例与处理传入数据的实例相同。 Client.write安排数据由反应堆Client.doWrite发送,而实际上是通过套接字发送数据。

我们准备看一下OnionClient永不调用的行为startTLS

没有startTLS的OnionClient

OnionClients包装在OnionProtocols中,这是我们尝试嵌套TLS的关键所在。作为的子类twisted.internet.policies.ProtocolWrapper,的实例OnionProtocol是一种协议传输三明治。它以低层传输的协议形式呈现,并以连接时由a所建立的假面包装为协议形式的传输形式WrappingFactory

现在,Client.doRead调用OnionProtocol.dataReceived,将数据代理到OnionClient。作为OnionClient的运输,OnionProtocol.write接受线路从其自身发送OnionClient.sendLine并代理到Client自己的运输。这是ProtocolWrapper,包装协议和自己的传输之间的正常交互,因此自然地往来于彼此的数据流动没有任何麻烦。

OnionProtocol.startTLS做一些不同的事情。它试图在已建立的协议传输对之间插入一个新的ProtocolWrapper(碰巧是一个)。这似乎很容易:a将上层协议作为其属性存储,并将代理和其他属性存储到其自己的传输中。 应该能够通过在其自身和上对该实例进行修补来注入一个新的包裹连接的对象:TLSMemoryBIOProtocolProtocolWrapperwrappedProtocolwritestartTLSTLSMemoryBIOProtocolOnionClientwrappedProtocoltransport

def startTLS(self):
    ...
    connLost = ProtocolWithoutConnectionLost(self.wrappedProtocol)
    connLost.onion = self
    # Construct a new TLS layer, delivering events and application data to the
    # wrapper just created.
    tlsProtocol = TLSMemoryBIOProtocol(None, connLost, False)

    # Push the previous transport and protocol onto the stack so they can be
    # retrieved when this new TLS layer stops.
    self._tlsStack.append((self.transport, self.wrappedProtocol))
    ...
    # Make the new TLS layer the current protocol and transport.
    self.wrappedProtocol = self.transport = tlsProtocol

这是第一次调用后的数据流startTLS

startTLS一个TLSMemoryBIOProtocol,正在工作

如预期的那样,传递到的新数据将OnionProtocol.dataReceived路由到上TLSMemoryBIOProtocol存储的_tlsStack,后者将解密的明文传递到OnionClient.dataReceivedOnionClient.sendLine还将数据传递到TLSMemoryBIOProtocol.write加密并将所产生的密文发送到OnionProtocol.write然后Client.write

遗憾的是,此方案在再次调用时失败startTLS。根本原因是此行:

    self.wrappedProtocol = self.transport = tlsProtocol

每次调用都会用最里面的startTLS替换,即使接收到的数据已被最外面的加密:wrappedProtocol TLSMemoryBIOProtocolClient.doRead

startTLS两个TLSMemoryBIOProtocols,已损坏

transportS,然而,正确的嵌套。 OnionClient.sendLine只能调用其传输write-即OnionProtocol.write-因此OnionProtocol应将其替换transport为最内层,TLSMemoryBIOProtocol以确保写入顺序嵌套在其他加密层中。

该溶液中,然后,是确保数据流经第一 TLSMemoryBIOProtocol_tlsStack依次一个,从而使加密各层按相反的顺序剥离其应用于:

使用两个TLSMemoryBIOProtocols启动TLS

代表_tlsStack作为一个列表中给出这个新的要求似乎不太自然。幸运的是,以线性方式表示传入的数据流表明了一种新的数据结构:

传入数据作为链表遍历

错误的和正确的传入数据流都类似于一个单链表,wrappedProtocol充当ProtocolWrappers的下一个链接,protocol充当Client的。该列表应从向下增加,OnionProtocol并始终以结束OnionClient。发生该错误是因为违反了排序不变式。

单链列表可以将协议推入堆栈,但弹出协议比较麻烦,因为它需要从其头部向下遍历到节点才能删除。当然,这种遍历在每次接收到数据时都会发生,因此需要关注的是附加遍历所隐含的复杂性,而不是最坏情况下的时间复杂性。幸运的是,该列表实际上是双重链接的:

带有协议和传输的双链表

transport属性将每个嵌套协议与其先前的协议链接在一起,以便transport.write可以在最终通过网络发送数据之前先对较低的加密级别进行分层。我们有两个哨兵可以帮助管理列表:Client必须始终位于顶部,并且OnionClient必须始终位于底部。

将两者放在一起,我们得出以下结论:

from twisted.python.components import proxyForInterface
from twisted.internet.interfaces import ITCPTransport
from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
from twisted.protocols.policies import ProtocolWrapper, WrappingFactory


class PopOnDisconnectTransport(proxyForInterface(ITCPTransport)):
    """
    L{TLSMemoryBIOProtocol.loseConnection} shuts down the TLS session
    and calls its own transport's C{loseConnection}.  A zero-length
    read also calls the transport's C{loseConnection}.  This proxy
    uses that behavior to invoke a C{pop} callback when a session has
    ended.  The callback is invoked exactly once because
    C{loseConnection} must be idempotent.
    """
    def __init__(self, pop, **kwargs):
        super(PopOnDisconnectTransport, self).__init__(**kwargs)
        self._pop = pop

    def loseConnection(self):
        self._pop()
        self._pop = lambda: None


class OnionProtocol(ProtocolWrapper):
    """
    OnionProtocol is both a transport and a protocol.  As a protocol,
    it can run over any other ITransport.  As a transport, it
    implements stackable TLS.  That is, whatever application traffic
    is generated by the protocol running on top of OnionProtocol can
    be encapsulated in a TLS conversation.  Or, that TLS conversation
    can be encapsulated in another TLS conversation.  Or **that** TLS
    conversation can be encapsulated in yet *another* TLS
    conversation.

    Each layer of TLS can use different connection parameters, such as
    keys, ciphers, certificate requirements, etc.  At the remote end
    of this connection, each has to be decrypted separately, starting
    at the outermost and working in.  OnionProtocol can do this
    itself, of course, just as it can encrypt each layer starting with
    the innermost.
    """

    def __init__(self, *args, **kwargs):
        ProtocolWrapper.__init__(self, *args, **kwargs)
        # The application level protocol is the sentinel at the tail
        # of the linked list stack of protocol wrappers.  The stack
        # begins at this sentinel.
        self._tailProtocol = self._currentProtocol = self.wrappedProtocol


    def startTLS(self, contextFactory, client, bytes=None):
        """
        Add a layer of TLS, with SSL parameters defined by the given
        contextFactory.

        If *client* is True, this side of the connection will be an
        SSL client.  Otherwise it will be an SSL server.

        If extra bytes which may be (or almost certainly are) part of
        the SSL handshake were received by the protocol running on top
        of OnionProtocol, they must be passed here as the **bytes**
        parameter.
        """
        # The newest TLS session is spliced in between the previous
        # and the application protocol at the tail end of the list.
        tlsProtocol = TLSMemoryBIOProtocol(None, self._tailProtocol, False)
        tlsProtocol.factory = TLSMemoryBIOFactory(contextFactory, client, None)

        if self._currentProtocol is self._tailProtocol:
            # This is the first and thus outermost TLS session.  The
            # transport is the immutable sentinel that no startTLS or
            # stopTLS call will move within the linked list stack.
            # The wrappedProtocol will remain this outermost session
            # until it's terminated.
            self.wrappedProtocol = tlsProtocol
            nextTransport = PopOnDisconnectTransport(
                original=self.transport,
                pop=self._pop
            )
            # Store the proxied transport as the list's head sentinel
            # to enable an easy identity check in _pop.
            self._headTransport = nextTransport
        else:
            # This a later TLS session within the stack.  The previous
            # TLS session becomes its transport.
            nextTransport = PopOnDisconnectTransport(
                original=self._currentProtocol,
                pop=self._pop
            )

        # Splice the new TLS session into the linked list stack.
        # wrappedProtocol serves as the link, so the protocol at the
        # current position takes our new TLS session as its
        # wrappedProtocol.
        self._currentProtocol.wrappedProtocol = tlsProtocol
        # Move down one position in the linked list.
        self._currentProtocol = tlsProtocol
        # Expose the new, innermost TLS session as the transport to
        # the application protocol.
        self.transport = self._currentProtocol
        # Connect the new TLS session to the previous transport.  The
        # transport attribute also serves as the previous link.
        tlsProtocol.makeConnection(nextTransport)

        # Left over bytes are part of the latest handshake.  Pass them
        # on to the innermost TLS session.
        if bytes is not None:
            tlsProtocol.dataReceived(bytes)


    def stopTLS(self):
        self.transport.loseConnection()


    def _pop(self):
        pop = self._currentProtocol
        previous = pop.transport
        # If the previous link is the head sentinel, we've run out of
        # linked list.  Ensure that the application protocol, stored
        # as the tail sentinel, becomes the wrappedProtocol, and the
        # head sentinel, which is the underlying transport, becomes
        # the transport.
        if previous is self._headTransport:
            self._currentProtocol = self.wrappedProtocol = self._tailProtocol
            self.transport = previous
        else:
            # Splice out a protocol from the linked list stack.  The
            # previous transport is a PopOnDisconnectTransport proxy,
            # so first retrieve proxied object off its original
            # attribute.
            previousProtocol = previous.original
            # The previous protocol's next link becomes the popped
            # protocol's next link
            previousProtocol.wrappedProtocol = pop.wrappedProtocol
            # Move up one position in the linked list.
            self._currentProtocol = previousProtocol
            # Expose the new, innermost TLS session as the transport
            # to the application protocol.
            self.transport = self._currentProtocol



class OnionFactory(WrappingFactory):
    """
    A L{WrappingFactory} that overrides
    L{WrappingFactory.registerProtocol} and
    L{WrappingFactory.unregisterProtocol}.  These methods store in and
    remove from a dictionary L{ProtocolWrapper} instances.  The
    C{transport} patching done as part of the linked-list management
    above causes the instances' hash to change, because the
    C{__hash__} is proxied through to the wrapped transport.  They're
    not essential to this program, so the easiest solution is to make
    them do nothing.
    """
    protocol = OnionProtocol

    def registerProtocol(self, protocol):
        pass


    def unregisterProtocol(self, protocol):
        pass

(这也可以在GitHub上获得。)

第二个问题的解决方案在于PopOnDisconnectTransport。原始代码试图通过来从堆栈中弹出TLS会话connectionLost,但是由于仅导致一个关闭的文件描述符connectionLost被调用,因此它未能删除未关闭基础套接字的已停止TLS会话。

在撰写本文时,恰好在两个地方TLSMemoryBIOProtocol调用其传输loseConnection_shutdownTLS_tlsShutdownFinished_shutdownTLS被称为主动关闭(loseConnectionabortConnectionunregisterProducer之后loseConnection所有的未决写已经刷新),而_tlsShutdownFinished被称为被动关闭(握手失败空读取读取错误,并写错误)。这一切都意味着,封闭连接的两端都可以在期间将停止的TLS会话弹出堆栈loseConnectionPopOnDisconnectTransport这样做loseConnection是幂等的,因为通常是幂等的,并且TLSMemoryBIOProtocol肯定会如此。

放入堆栈管理逻辑的不利之处loseConnection在于它取决于实现的细节TLSMemoryBIOProtocol。通用解决方案需要跨多个Twisted级别的新API。

在此之前,我们还停留在Hyrum定律的另一个示例中。


1

如果第二层具备此功能,则可能需要在启动第二层之前通知远程设备您希望启动环境并为其分配资源。


1
谢谢。失败的示例均通过localhost运行。“设备”是bzr分支中的代码lp:~exarkun/+junk/onion
Jean-Paul Calderone

0

如果两层都使用相同的TLS参数,并且要连接到同一主机,则两层加密都可能使用相同的密钥对。尝试对嵌套层使用其他密钥对,例如隧道传输到第三个主机/端口。即:(localhost:30000客户端)-> localhost:8080(使用密钥对A的TLS层1)-> localhost:8081(使用密钥对B的TLS层2)。


1
为什么使用相同的密钥对会有问题?
Jean-Paul Calderone

1
我同意@Jumbogram的观点,即TLS-in-TLS在技术上应该可以工作,因此我想到数学中可能存在某些问题,在使用双重加密的情况下,该问题会崩溃。但是,我一直无法通过直接RSA再现失败。
Leo Accend 2011年

@LeoAccend-如果您无法重现故障,那么在故障发生的地方使用什么代码?
Glyph
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.