如何安全地“组合”由多个程序打印的行？

假设我要并行执行多个程序，并将它们的输出组合到一个管道中：

sh -c '
    (echo qqq; echo qqq2; echo qqq3)&
    (echo www; echo www2; echo www3)& 
    (echo eee; echo eee2; echo eee3)& 
  wait; wait; wait'

这种shell方法在这种简单情况下效果很好，但我希望如果程序以缓冲方式输出越来越长的行，则它会失败，例如：

qqq
qqwww
q2
qqq3www2

wwweee3

eee2
eee3

我被提示使用的解决方案之一是tail -f：

tail -n +0 -q -f <(echo qqq; echo qqq2; echo qqq3) <(echo www; echo www2; echo www3) <(echo eee; echo eee2; echo eee3)

，但这是次优的选择：输出数据缓慢，不会终止；在这种情况下，我看到的输出不是按“睡眠”顺序，而是按参数顺序：

tail -n +0 -q -f <(sleep 1; echo qqq; sleep 1; echo qqq2; echo qqq3) <(echo www; echo www2; sleep 10; echo www3) <(echo eee; sleep 4; echo eee2; echo eee3) | cat

我为此实现了一个特殊的小程序，但是相信应该有一些标准的好方法。

如何使用标准工具（没有tail -f缺点）进行操作？

GNU并行。

从日期为2013年8月的发行说明中：

--line-buffer将基于行缓冲输出。--group将整个工作的输出保持在一起。--ungroup允许输出混合在一起，其中一半来自一个作业，一半来自另一份作业。--line-buffer介于两者之间；它会打印一条整行，但允许混合不同作业的行。

例如：

parallel --line-buffer <jobs

其中jobs包含：

./long.sh
./short.sh one
./short.sh two

short.sh：

#!/bin/bash

while true; do
        echo "short line $1"
        sleep .1
done

long.sh：

#!/bin/bash

count=0
while true; do
        echo -n "long line with multiple write()s "
        sleep .1
        count=$((count+1))
        if [ $count -gt 30 ]; then
                count=0
                echo
        fi
done

输出：

short line one
short line two
short line one
short line two
short line one
**-snip-**
short line one
short line one
short line two
short line two
short line one
short line one
short line one
long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s long line with multiple write()s 
short line two
short line two
short line two
short line one

一个实现锁的解决方案：

function putlines () {
   read line || return $?
   while ! ln -s $$ lock >/dev/null 2>&1
   do
      sleep 0.05
   done
   echo "$line" 
}

function getlines () {
     while read lline
     do 
          echo "$lline"
          rm lock
     done
}

# your paralelized jobs  
(  
   job1 | putlines & 
   job2 | putlines & 
   job3 | putlines & 
   wait
) | getlines| final_processing

创建锁的方法应该比使用文件系统更快。

我想不到有什么简单的方法可以对您有所帮助，如果您的行太长，那么一个程序将在它能够完成之前将其发送到睡眠状态，以完成向stdout的写行。

但是，如果您的行足够短以至于无法在过程切换之前完全写入，那么您的问题是，生成一行会花费很长时间，则可以使用read缓冲输出。

例如：

((./script1 | while read line1; do echo $line1; done) & \
(./script2 | while read line2; do echo $line2; done)) | doSomethingWithOutput

您可以使用创建一个命名管道mkfifo，将所有输出转储到命名管道中，并分别从命名管道中读取收集的数据：

mkfifo /tmp/mypipe
job1 > /tmp/mypipe &
job2 > /tmp/mypipe &
job3 > /tmp/mypipe &

cat /tmp/mypipe > /path/to/final_output &

wait; wait; wait; wait