为什么通过SSH进行rsync可以给我十倍的SCP吞吐量?


12
  1. scp user@aws-ec2:~/file file
  2. rsync --partial --progress -Pav -e ssh user@aws-ec2:~/file file

scp仅给我200K / s,但rsync给我1.9M / s

我测试了几次,所有结果都一样。

rsync 使用多个线程??

Answers:


7

两种协议均基于SSH。SSH 本身也有一些开销维基

SCP是真正的天真协议,具有真正的天真算法,可以传输一些小文件。它具有很多同步(RTT-往返时间)和小的缓冲区(基本上是2048 B- )。

Rsync是为了提高性能而设计的,因此它可以提供更好的结果并具有更多功能。

10倍加速是特定于您的情况的。如果您要通过高延迟通道在整个世界范围内传输文件,则在这种scp情况下,性能会差很多,但是在本地网络上,性能可能几乎相同。

不,压缩(-Cfor scp)将无济于事。最大的问题是延迟和缓冲区大小。


7

RSYNC与SCP

SCP基本上使用SSH从本地到整个本地或通过网络从源到目的地进行简单的旧复制,但是您可以使用该-C开关启用SSH压缩,从而有可能加快跨网络的数据复制。

RSYNC使用高效的校验和搜索算法在数据传输过程中自动优化网络连接,从而仅通过网络连接传输两组文件之间的差异。

同步

描述

   rsync is a program that behaves in much the same way that rcp does, but
   has many more options and uses  the  rsync  remote-update  protocol  to
   greatly  speed  up  file  transfers  when the destination file is being
   updated.

   The rsync remote-update protocol allows rsync to transfer just the dif-
   ferences between two sets of files across the network connection, using
   an efficient  checksum-search  algorithm  described  in  the  technical
   report that accompanies this package.

资源


SCP

描述

 scp copies files between hosts on a network.  It uses ssh(1) for data
 transfer, and uses the same authentication and provides the same secu‐
 rity as ssh(1).  scp will ask for passwords or passphrases if they are
 needed for authentication.




 File names may contain a user and host specification to indicate that
 the file is to be copied to/from that host.  Local file names can be
 made explicit using absolute or relative pathnames to avoid scp treat‐
 ing file names containing ‘:’ as host specifiers.  Copies between two
 remote hosts are also permitted.

资源


3
但在这种情况下,情况就有些不同:他只复制一个文件。(大概在远端还不存在。)
Daniel B

1
@DanielB您不会认为默认情况下RSYNC甚至可以优化一个文件的数据连接,甚至在传输过程中也可以压缩数据,这样实际上很少有数据块通过管道发送,而没有开关的SCP-C没有。在管道传输过程中压缩数据?
Pimp Juice IT

2
rsync命令行中未指定压缩或校验和。当然,文件内增量算法始终处于活动状态。可能就是这样scp
Daniel B
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.