Incorrect total "Execution Duration" for the execution set
How does "Execution Duration" is calculated for an execution set? Apparently, right now it's the cumulative of "Execution Duration" of all individual tests in the set, however since the tests are running in parallel, we need this time to represent the time from start to end of an execution.
Below execution shows "15286" seconds of "Execution Duration" which is 4+ hours, however to total time execution completed was around 40 mins (from 15:45 to 16:25):
Note: "Execution Duration" for individual tests are correct.
Sorry for the delay, this is what we are planning to do in 8.11.0 which is due in less than two weeks:
1. For an execution, we will be adding how long the entire execution took from start to finish. For example, suppose you have an execution set with 10 test cases where every test case takes 5 minutes. With 1 VM, this should take 50 minutes, with 10 VMs, this should take 5 minutes. Deepening on how many VMs a pool will have, we will be reporting the effective execution time.
2. For a test case execution, we will be adding endTime-startTime
3. For an execution, we will be adding endTime-startTime
4. For legacy reasons, we will be keeping the sum of total the steps.
Test case execution already has actual duration that is calculated as (end date - start date).
Unfortunately, same actual duration for execution set execution is not planned for 8.11/8.12. Probably we will add it in 8.13.0. We collect ideas for better calculation in special cases, for example, like re-runs.
Please sign in to leave a comment.