Any best practices for using Auto-Test? Compare old with New Auto-test

Working in Studio
1

Hello,

I was wondering if there are any best practices for using the Auto-test functionality within Teneo Studio. I think it is best to use an example:

  • I run an auto-test
  • I make changes
  • I run another auto-test

At that moment I have two reports. I also have the overview of number of tests and percentages Passed / Passed with Warning / Failed
However, is there an easy way to actually see the difference when comparing the before and afterwards scenario except from scrolling manually through both of the full test results (Because it is a huge list of flows and tests and you could lose track what is actually different from before your change)

I would like to just compare the two and get a result of what changed since my changes.

Thanks!

1 Like
2

Hi, wesley.levels, Welcome to the forum!

The explained steps are actually the ones we recommend as well.

For the question, as we do not have a built-in ‘compare’ function, there are two ways to approach this,

  1. In Auto-Test there is an option to filter out results you do not want to see, which can be helpful if you are only interested in ‘Warnings’ or ‘Failed’ results.
    image

  2. If you want to compare the exact results I’d recommend you to select the ‘Get Report’ button to export the results to an Excel file and run an internal ‘diff’ on the two results.This will return the exact difference between the two tests.
    image

Please let me know if you have any other questions,

I’m more than happy to help!
Ramazan

1 Like
3

Hi Wesley (@wesley.levels), great to see you around in the Forum! :grinning:
Ramazan already answered your question, just wanted to add that if you are working on one specific flow and want to see what effects your work has only on the flow’s triggers (as a first step) then you can run the Autotest also at Flow level. In this way, you get a very clear overview. Once you are happy with it, then you should run it obviously also at Solution level to see the overall performance (and possible issues). Maybe you were doing this already, just wanted to mention it.
We have a very nice overview of the Autotest and its different testing scopes here: Run Automated Tests | Teneo Developers
/Benjamin

1 Like