Incorrect total "Execution Duration" for the execution set

How does "Execution Duration" is calculated for an execution set? Apparently, right now it's the cumulative of "Execution Duration" of all individual tests in the set, however since the tests are running in parallel, we need this time to represent the time from start to end of an execution. 

Below execution shows "15286" seconds of "Execution Duration" which is 4+ hours, however to total time execution completed was around 40 mins (from 15:45 to 16:25):

Note: "Execution Duration" for individual tests are correct.

 

Thanks,
Hasib

0
5 comments
Avatar
Joseph Hamdan

Hasib,

Thanks for your post. We are currently evaluating it and will provide a response as soon as possible.

Regards,

Subject7 Team

0
Comment actions Permalink
Avatar
Hasib Rostaiee

Hi Joseph,

Is there any update on this issue. An accurate execution duration time is helpful for us to used it for some metrics we need.

0
Comment actions Permalink
Avatar
Joseph Hamdan

Hi Hasib,

Sorry for the delay, this is what we are planning to do in 8.11.0 which is due in less than two weeks: 

1. For an execution, we will be adding how long the entire execution took from start to finish. For example, suppose you have an execution set with 10 test cases where every test case takes 5 minutes. With 1 VM, this should take 50 minutes, with 10 VMs, this should take 5 minutes. Deepening on how many VMs a pool will have, we will be reporting the effective execution time.
2. For a test case execution, we will be adding endTime-startTime

3. For an execution, we will be adding endTime-startTime

4. For legacy reasons, we will be keeping the sum of total the steps. 

Regards,

Subject7 Team

0
Comment actions Permalink
Avatar
Hasib Rostaiee

Hi Joseph,

Are these changes going to be available on 8.11 release?

0
Comment actions Permalink
Avatar
Jonathan Hotz

Hi Hasib.

Test case execution already has actual duration that is calculated as (end date - start date).

Unfortunately, same actual duration for execution set execution is not planned for 8.11/8.12. Probably we will add it in 8.13.0. We collect ideas for better calculation in special cases, for example, like re-runs.

0
Comment actions Permalink

Please sign in to leave a comment.