Products.CMFActivity: Distribute all activities pending distribution
Activity distribution (actually: validation) is fundamentally a bottleneck of the current CMFActivity design: there can be only one distribution/ validation node, and it must check every single activity which express a dependency over any activity, until these dependencies are satisfied. As a result, distribution/validation is in the critical path between an activity being spawned and it being executed, and this work cannot be parallelised. So care should be taken to waste as little time as possible, in order to reduce the activity execution latency. Before this change, CMFActivity would distribute at most 1000 activities (MAX_VALIDATED_LIMIT) per queue per timerserver wake-up. In a typical ERP5 setup, timerserver ticks once per second, which means CMFActivity was unable to validate more than 1000 activities per second per queue. Maybe there are more activities which are possible to validate but still the code was forcing the node to sleep until the next wake-up, which is a tremendous waste of time. This change fixes that issue by having ActivityTool.distribute keep looping until there has been an iteration over all activity queues which simultaneously did not find any activity they could validate. This lets Zope yield CPU control when it would be better spent processing activities (those which are preventing the validation of any activities remaining to validate) without imposing a maximum effective validation-per-second hard limit.
Status | Job ID | Name | Coverage | ||||||
---|---|---|---|---|---|---|---|---|---|
External | |||||||||
passed |
#698500
external
|
ERP5.CodingStyleTest-Master |
01:08:38
|
||||||
failed |
#698529
external
|
ERP5.PerformanceTest-Master |
00:40:17
|
||||||
failed |
#698501
external
|
ERP5.UnitTest-Master |
01:46:36
|
||||||
passed |
#698508
external
|
SlapOS.Eggs.UnitTest-Master.Python2 |
00:14:10
|
||||||
passed |
#698609
external
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:14:07
|
||||||
passed |
#698514
external
|
Wendelin.UnitTest-Master |
00:38:37
|
||||||
passed |
#698523
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:14:37
|
||||||
passed |
#698594
external
retried
|
SlapOS.Eggs.UnitTest-Master.Python3 |
00:14:01
|
||||||