I have a file I'm reading (250MB), and want to upd...

# react

[Zyle Moore]

I have a file I'm reading (250MB), and want to update a `progress`periodically while deserializing elements from it. I use a delay(1) after each chunked(10000) right now. Without it, there is no delay between chunks, and the page hangs (no render/update) until it's gone through every chunk. There's about 4 million elements that it reads, and takes about 20 seconds (16 seconds without delay). I'm performing this iteration in a useEffect. Is there something else I should be doing?

useEffect(byteString, setElementMarkers, setLastElementMarkerRead) {
    val elementMarkerSequence: Sequence<PluginElementMarker> =
        PluginFormat.decodeMarkerSequenceFromByteString(byteString)

    val tokenList: MutableList<PluginElementMarker> = mutableListOf()
    val toListTime = measureTime {
        elementMarkerSequence
            .chunked(10000)
            .forEach { chunk ->
                tokenList.addAll(chunk)
                setLastElementMarkerRead(chunk.last())
                delay(1)
            }
    }

    println("toList time $toListTime ${tokenList.size}")

    setElementMarkers(tokenList)
}
// ...
progress {
    max = byteString.size.toDouble()
    value = lastElementMarkerRead?.skip ?: 0
}

[turansky]

Is it some binary deserialization?

[Zyle Moore]

Yes, it is a custom Kotlin Serialization BinaryFormat for reading Skyrim mod files. A Type-Length-Value format. I noticed that there's actually useEffectOnce and useEffectOnceWithCleanup, but when making a job and cleanup function, it behaves like it does without a delay (hangs until complete)

[turansky]

Probably it would be better to move calculations in web worker to avoid UI freezes

[turansky]

As draft - you can increase delay

[turansky]

20ms should be enough for 1 paint