Peter
04/16/2022, 4:00 PMJupyterReplTestCase
since a long time to automate some regression testing of notebooks and it always worked fine.
But now I’m trying slightly different setup with a lib.json file containing some imports (before this was handled by JupyterIntegration
) and I cannot get it to work.
In my notebook:
%use roboquant@url[<https://roboquant.org/roboquant.json>]
In my unit test:
class NotebookTester() : JupyterReplTestCase(ReplProvider.forLibrariesTesting(listOf("roboquant"))) { ... }
But if I try this, the imports that are defined in the roboquant.json are ignored. Anyone know what I’m doing wrong or how I can get the imports in the roboquant.json files to be used for my unit test (other then specifying them explicitly in my notebooks) ?Ilya Muradyan
04/16/2022, 7:06 PMdependencies
and repositories
in all (or only in specific) JSON descriptors. I can help you with this, but there is a couple of questions to be sure I understand your case correctly:
1. How do you use JSON descriptor in test? I mean, what do you write in eval("%use ...")
?
2. How does your JSON descriptor for the test look like?Peter
04/16/2022, 9:01 PMIlya Muradyan
04/16/2022, 9:04 PM%use roboquant@url[<https://roboquant.org/roboquant.json>]
in your testPeter
04/16/2022, 9:04 PMprivate class NotebookTester(lib: String) : JupyterReplTestCase(ReplProvider.forLibrariesTesting(listOf(lib))) {
fun validateNotebook(notebookPath: String) {
val notebookFile = File(notebookPath)
val notebook = JupyterParser.parse(notebookFile)
for (cell in notebook.cells.filterIsInstance<CodeCell>()) {
val cellResult = exec(cell.source)
val result = if (cellResult is MimeTypedResult) cellResult.entries.first().value else cellResult.toString()
if (cell.outputs.isNotEmpty()) {
val firstOutput = cell.outputs.first()
if (firstOutput is ExecuteResult && firstOutput.data.isNotEmpty()) {
val value = firstOutput.data.entries.first()
assertEquals(value.value, result)
}
}
}
}
}
%use @https://roboquant.org/roboquant.json
But then the Jupyter Test case doesn’t know the library name (roboquant) and complains about “@default” being unknown.Ilya Muradyan
04/16/2022, 10:14 PMJupyterReplTestCase
constructor. %use @https://roboquant.org/roboquant.json
will also work. Dependencies will be taken from classpath, imports and other things will be loaded from descriptor
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.buildJsonObject
import org.jetbrains.kotlinx.jupyter.ReplForJupyter
import org.jetbrains.kotlinx.jupyter.ReplForJupyterImpl
import org.jetbrains.kotlinx.jupyter.api.libraries.LibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.common.getHttp
import org.jetbrains.kotlinx.jupyter.common.jsonObject
import org.jetbrains.kotlinx.jupyter.defaultRepositories
import org.jetbrains.kotlinx.jupyter.dependencies.ResolverConfig
import org.jetbrains.kotlinx.jupyter.libraries.AbstractLibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.libraries.ByNothingLibraryResolutionInfo
import org.jetbrains.kotlinx.jupyter.libraries.ResolutionInfoProvider
import org.jetbrains.kotlinx.jupyter.libraries.SpecificLibraryResolver
import java.io.File
import java.net.URL
object RoboquantReplProvider : ReplProvider {
override fun invoke(classpath: List<File>): ReplForJupyter {
return ReplForJupyterImpl(
resolutionInfoProvider,
classpath,
resolverConfig = ResolverConfig(defaultRepositories, urlEditingResolver),
isEmbedded = true
).apply {
eval { librariesScanner.addLibrariesFromClassLoader(currentClassLoader, this) }
}
}
private val urlEditingResolver = SpecificLibraryResolver(AbstractLibraryResolutionInfo.ByURL::class) { info, _ ->
val response = getHttp(info.url.toString())
val json = response.jsonObject
val newJson = buildJsonObject {
for ((key, value) in json.entries) {
if (key == "repositories" || key == "dependencies") continue
put(key, value)
}
}
Json.encodeToString(newJson)
}
private val resolutionInfoProvider = object : ResolutionInfoProvider {
override var fallback: LibraryResolutionInfo
get() = ByNothingLibraryResolutionInfo
set(value) {}
override fun get(string: String): LibraryResolutionInfo {
return AbstractLibraryResolutionInfo.ByURL(URL(string))
}
}
}
Peter
04/17/2022, 5:37 AM