4) If you guys tell me that options one and two is not feasible, @Nikky suggested the possibility of creating a plugin instead to read files, which is starting to sound more and more attractive. I could come back to the original idea of creating something similar to ScenicView and use IntelliJ's psi system to read within the files itself and perhaps use sockets to inject a java agent at runtime. Then I'd use scripting to create the test files as I am now.