I have a related idea: use something like pitest to test your test suite. and mark all assertions that never fail as suspicious. you would run a longer test task that mutates your code to check if it can get all your assertions to fail. and also display a report of the error message, then you can check that the error messages make sense.
basically taking “only trust an assertion that you have seen fail” to the next level.