如何为每个请求项创建多个线程


9

我正在尝试在订单级别使用多线程处理以下代码。

List<String> orders = Arrays.asList("order1", "order2", 
                   "order3", "order4", "order1");

当前顺序执行:

orders.stream().forEach(order -> {
    rules.forEach(rule -> {
        finalList.add(beanMapper.getBean(rule)
                .applyRule(createTemplate.apply(getMetaData.apply(rule), command),
                           order));
    });
});

我尝试使用:

orders.parallelStream().forEach(order -> {}} // code snippet.

但是它正在更改rules.forEach(rule-> {}}的顺序。

例如:
输入:

 List<String> orders = Arrays.asList("order1", "order2", 
                         "order3", "order4", "order1");
 List<String> rules = Arrays.asList("rule1", "rule2", "rule3");

预期产量:

order1 with rule1, rule2, rule3
order2 with rule1, rule2, rule3

实际输出parallelStream()

order1 with rule3, rule1, rule2
order1 with rule2, rule1, rule3

我不为命令的顺序而烦恼,但我为规则的顺序而烦恼。订单可以按任何顺序处理,但是规则应针对每个订单以相同的顺序执行。

请帮忙。

Answers:


4

您可以使用 :

orders.stream().parallel().forEachOrdered(// Your rules logic goes here. )

ForEachOrdered保证维持流的顺序。

所以供您参考:

orders.stream().parallel().forEachOrdered( order -> {

            rules.stream().parallel().forEachOrdered ( rule -> {

                 System.out.println( " Order : " + order + " rule :" + rule);
            });

        });

注意:尽管我们可以做到这一点,但应该密切注意性能,因为并行处理和顺序不能很好地结合在一起!

输出量

 Order : order1 rule :rule1
 Order : order1 rule :rule2
 Order : order1 rule :rule3
 Order : order2 rule :rule1
 Order : order2 rule :rule2
 Order : order2 rule :rule3
 Order : order3 rule :rule1
 Order : order3 rule :rule2
 Order : order3 rule :rule3
 Order : order4 rule :rule1
 Order : order4 rule :rule2
 Order : order4 rule :rule3
 Order : order1 rule :rule1
 Order : order1 rule :rule2
 Order : order1 rule :rule3

感谢您的回答。forEachOrdered保证了流的顺序,但同时也降低了性能。我试过了,应用程序花费的时间类似于顺序处理。stream()。parallel&forEachOrdered不互相称赞。
Mayank bisht

是的,我同意我们需要先进行完整的延迟分析。
Pramod S. Nikam

是的,使用此功能我得到了相同的性能,没有任何改善。
mayank bisht

1
密切关注此线程以获得更好的解决方案。
Pramod S. Nikam

我可以使用ExecutorService实现并行处理吗?
mayank bisht

1

您可以同时finalList从不同的线程向中添加元素。这导致将规则应用于不同订单的结果混合在一起(规则未按其订单分组)。

您可以通过为每个临时列表创建一个临时列表order,然后将所有临时列表同步合并到中来修复此问题finalList

这是使用Stream-API(Java 9+)的方法:

List<AppliedRule> finalList = orders.parallelStream().map(order ->
        rules.stream().map(rule -> applyRule(order, rule)).collect(Collectors.toList())
).collect(Collectors.flatMapping(Collection::stream, Collectors.toList()));

注意:Collectors.flatMapping()此处使用它,而不是简单flatMap地在流收集期间同步运行平面映射。


Java 8模拟:

List<AppliedRule> finalList = orders.parallelStream().map(order ->
        rules.stream().map(rule -> applyRule(order, rule)).collect(Collectors.toList())
).collect(Collectors.toList())
        .stream()
        .flatMap(Collection::stream)
        .collect(Collectors.toList());

感谢您的回答。我尝试了您的方法,并且得到了java.util.ConcurrentModificationException:null
Mayank bisht

finalList = orders.parallelStream()。map(顺序-> rules.stream()。map(规则-> beanMapper.getBean(rule).applyRule(createTemplate.apply(getMetaData.apply(rule),command),order)) .collect(Collectors.toList())。collect(Collectors.toList())。stream()。flatMap(Collection :: stream).collect(Collectors.toList());
mayank bisht

@mayankbisht,这意味着它beanMapper.getBean(rule) .applyRule(createTemplate.apply(getMetaData.apply(rule), command), order)不是一个纯函数,因此不能并行使用。尝试消除所有副作用;ConcurrentModificationException堆栈跟踪可以帮助找到它们。
巴南嫩

0

这样行吗?

final int rulesSize = rules.size();
AtomicInteger atomicInteger = new AtomicInteger(0);
orders.stream().parallel().forEach(order -> {
    IntStream.range(0, rulesSize).parallel().forEach( i -> {
        synchronized (atomicInteger) {
            System.out.println(" Order : " + order + " rule :" + rules.get(atomicInteger.getAndIncrement() % rulesSize));
        }
    });
});

输出量

 Order : order1 rule :rule1
 Order : order4 rule :rule2
 Order : order1 rule :rule3
 Order : order3 rule :rule1
 Order : order3 rule :rule2
 Order : order3 rule :rule3
 Order : order2 rule :rule1
 Order : order2 rule :rule2
 Order : order2 rule :rule3
 Order : order1 rule :rule1
 Order : order1 rule :rule2
 Order : order4 rule :rule3
 Order : order1 rule :rule1
 Order : order4 rule :rule2
 Order : order1 rule :rule3

0

顺序可以是任意顺序,但是规则的顺序不应更改。同样,对于特定的顺序规则,应该成组出现。

如果是这样,则没有实际并行性的空间。

什么时候

order1-rule1
order1-rule2
order2-rule1
order2-rule2

order2-rule1
order2-rule2
order1-rule1
order1-rule2

是2个订单和2条规则的唯一有效运行,
并且

order1-rule1
order2-rule1
order1-rule2
order2-rule2

被认为是无效的,不是并行性,只是orders的随机化,大概没有增益。如果您一直无所适从order1,那么可以随机排列列表,仅此而已:

public static void main (String[] args) throws java.lang.Exception
{
    List<String> orders = Arrays.asList("order1", "order2", "order3", "order4");
    List<String> rules = Arrays.asList("rule1", "rule2", "rule3");
    Collections.shuffle(orders);
    orders.forEach(order->{
        rules.forEach(rule->{
            System.out.println(order+"-"+rule);
        });
    });
}

甚至不需要流,只需两个嵌套循环。测试:https//ideone.com/qI3dqd

order2-rule1
order2-rule2
order2-rule3
order4-rule1
order4-rule2
order4-rule3
order1-rule1
order1-rule2
order1-rule3
order3-rule1
order3-rule2
order3-rule3


原始答案

但是它正在更改rules.forEach(rule-> {}}的顺序。

不,不是的。的orderS可以重叠,但顺序ruleS代表每个顺序被保留。为什么非平行人员还会forEach做其他事情?

示例代码:

public static void main (String[] args) throws java.lang.Exception
{
    List<String> orders = Arrays.asList("order1", "order2", "order3", "order4");
    List<String> rules = Arrays.asList("rule1", "rule2", "rule3");
    orders.stream().parallel().forEach(order->{
        rules.forEach(rule->{
            System.out.println(order+"-"+rule);
        });
    });
}

测试:https : //ideone.com/95Cybg
示例输出:

order2-rule1
order2-rule2
order2-rule3
order1-rule1
order1-rule2
order1-rule3
order4-rule1
order4-rule2
order4-rule3
order3-rule1
order3-rule2
order3-rule3

orders 的顺序是混合的,但rules始终为1-2-3。我认为您的输出只是隐藏了配对(实际上您没有显示它是如何生成的)。

当然,可以将其扩展一些延迟,因此orders的处理实际上将重叠:

public static void delay(){
    try{
        Thread.sleep(ThreadLocalRandom.current().nextInt(100,300));
    }catch(Exception ex){}
}

public static void main (String[] args) throws java.lang.Exception
{
    List<String> orders = Arrays.asList("order1", "order2", "order3", "order4");
    List<String> rules = Arrays.asList("rule1", "rule2", "rule3");
    orders.stream().parallel().forEach(order->{
        rules.forEach(rule->{
            delay();
            System.out.println(order+"-"+rule);
        });
    });
}

测试:https : //ideone.com/cSFaqS
示例输出:

order3-rule1
order2-rule1
order2-rule2
order3-rule2
order3-rule3
order2-rule3
order1-rule1
order4-rule1
order1-rule2
order4-rule2
order4-rule3
order1-rule3

这可能是您所看到的,只是没有orderx一部分。随着order有形之可以追踪到rule,使其保持未来1-2-3,order。同样,您的示例列表包含order1两次,这肯定不会帮助您查看正在发生的情况。


感谢您的回答。对于较少的订单,以上输出可能是正确的。但是,如果您增加订单,那么您将获得不同的输出。例如(order4-rule1 order4-rule2 order4-rule1)(order1-rule1 order1-rule2)(order3-rule1 order3-rule2)(order4-rule1 order4-rule2 order4-rule1 order4-rule2)。
Mayank bisht

顺序可以是任意顺序,但规则的顺序不应更改。另外,对于特定的订单规则,应该成组出现。对于前。(order1-规则1 order1-rule2 order1-rule3)而不是(order1-rule1 order2-rule1 order1-
rule2 order1

@mayankbisht我认为这些限制根本不允许并行处理。请参阅更新的答案(我一开始就写了新的部分)。
tevemadar '19

是的,我的确理解这一点,这就是为什么我在这里发布此问题。我以为也许会有另一种方法,或者我们可以改变算法
Mayank bisht

@mayankbisht,您可以描述orders 为什么不能重叠(那些rules 是否是有状态的,并且存在数量有限的副本,也许只有一个副本?)。但是通常情况下,没有并行运行的事物就没有并行性,这毕竟是并行性的全部要点。
tevemadar '19

0

如果您不介意尝试第三方库。这是我的库的示例:abacus-util

StreamEx.of(orders).parallelStream().forEach(order -> {}}

您甚至可以指定线程号:

StreamEx.of(orders).parallelStream(maxThreadNum).forEach(order -> {}}

的顺序rule将保留。

顺便说一句,因为它是并行流,所以这段代码...finalList.add(...很可能无法工作。我认为最好将结果收集到列表中:

StreamEx.of(orders).parallelStream().map/flatMap(order -> {...}}.toList()

即使您order以后出于某些原因想要保留订单的顺序,它也是可行的:

StreamEx.of(orders).indexed().parallelStream()
      .map/flatMap(order -> {...}}.sortedBy(...index).toList()
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.