Used `JupyterReplTestCase` since a long time to au...

# datascience

[Peter]

Used JupyterReplTestCase since a long time to automate some regression testing of notebooks and it always worked fine.

But now I’m trying slightly different setup with a lib.json file containing some imports (before this was handled by JupyterIntegration) and I cannot get it to work.

In my notebook:

%use roboquant@url[<https://roboquant.org/roboquant.json>]

In my unit test:

class NotebookTester() : JupyterReplTestCase(ReplProvider.forLibrariesTesting(listOf("roboquant"))) { ... }

But if I try this, the imports that are defined in the roboquant.json are ignored. Anyone know what I’m doing wrong or how I can get the imports in the roboquant.json files to be used for my unit test (other then specifying them explicitly in my notebooks) ?

[Ilya Muradyan]

It doesn't work only in the unit test or in the notebook too?

[Ilya Muradyan]

It seems you need to write a specific library resolver for this case: it should ignore dependencies and repositories in all (or only in specific) JSON descriptors. I can help you with this, but there is a couple of questions to be sure I understand your case correctly:
1. How do you use JSON descriptor in test? I mean, what do you write in eval("%use ...")?
2. How does your JSON descriptor for the test look like?

[Peter]

It works in the notebook standalone, just not in the unit test.

[Peter]

It is fully automated, so it runs each cell and compares it against the pre-recorded output.

[Ilya Muradyan]

Aha, so it's the same

%use roboquant@url[<https://roboquant.org/roboquant.json>]

in your test

[Peter]

The unit test code

private class NotebookTester(lib: String) : JupyterReplTestCase(ReplProvider.forLibrariesTesting(listOf(lib))) {

 
    fun validateNotebook(notebookPath: String) {
        val notebookFile = File(notebookPath)
        val notebook = JupyterParser.parse(notebookFile)

        for (cell in notebook.cells.filterIsInstance<CodeCell>()) {
            val cellResult = exec(cell.source)
            val result = if (cellResult is MimeTypedResult) cellResult.entries.first().value else cellResult.toString()

            if (cell.outputs.isNotEmpty()) {
                val firstOutput = cell.outputs.first()
                if (firstOutput is ExecuteResult && firstOutput.data.isNotEmpty()) {
                    val value = firstOutput.data.entries.first()
                    assertEquals(value.value, result)
                }
            }
        }

    }

}

[Peter]

Indeed, that is the code. Used to be

%use @https://roboquant.org/roboquant.json

But then the Jupyter Test case doesn’t know the library name (roboquant) and complains about “@default” being unknown.

[Ilya Muradyan]

If you only use descriptors from URLs, following REPL provider should suit you. You should pass it to JupyterReplTestCase constructor. %use @https://roboquant.org/roboquant.json will also work. Dependencies will be taken from classpath, imports and other things will be loaded from descriptor

import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.buildJsonObject
import org.jetbrains.kotlinx.jupyter.ReplForJupyter
import org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl
import org.jetbrains.kotlinx.jupyter.api.libraries.LibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.common.getHttp
import org.jetbrains.kotlinx.jupyter.common.jsonObject
import org.jetbrains.kotlinx.jupyter.defaultRepositories
import org.jetbrains.kotlinx.jupyter.dependencies.ResolverConfig
import org.jetbrains.kotlinx.jupyter.libraries.AbstractLibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.libraries.ByNothingLibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.libraries.ResolutionInfoProvider
import org.jetbrains.kotlinx.jupyter.libraries.SpecificLibraryResolver
import java.io.File
import java.net.URL

object RoboquantReplProvider : ReplProvider {
    override fun invoke(classpath: List<File>): ReplForJupyter {
        return ReplForJupyterImpl(
            resolutionInfoProvider,
            classpath,
            resolverConfig = ResolverConfig(defaultRepositories, urlEditingResolver),
            isEmbedded = true
        ).apply {
            eval { librariesScanner.addLibrariesFromClassLoader(currentClassLoader, this) }
        }
    }

    private val urlEditingResolver = SpecificLibraryResolver(AbstractLibraryResolutionInfo.ByURL::class) { info, _ ->
        val response = getHttp(info.url.toString())
        val json = response.jsonObject
        val newJson = buildJsonObject {
            for ((key, value) in json.entries) {
                if (key == "repositories" || key == "dependencies") continue
                put(key, value)
            }
        }
        Json.encodeToString(newJson)
    }

    private val resolutionInfoProvider = object : ResolutionInfoProvider {
        override var fallback: LibraryResolutionInfo
            get() = ByNothingLibraryResolutionInfo
            set(value) {}

        override fun get(string: String): LibraryResolutionInfo {
            return AbstractLibraryResolutionInfo.ByURL(URL(string))
        }
    }
}

[Peter]

Thanks for the help, very much appreciated. Will give it a try.

[Peter]

Just cut&paste your code snippet and use that new RoboquantReplProvider and works like a charm.

Thank you so much for spending your time in the weekend to help me out 🙏 🙏🙏

P.S really enjoy the combination of Kotlin + Jupyter. In case you are curious what is currently build, click this link and run the notebook: Run Notebook on MyBinder