https://kotlinlang.org logo
Join Slack
Powered by
Json is `StringFormat` so it has methods like `enc...
# serialization
i
Json is 
StringFormat
so it has methods like 
encodeToString/decodeFromString
but is somehow available direct serialization/deserialization to ByteArray? Comparing to jackson that has 
writeValueAsBytes
(create string with 
encodeToString
and than convert to com.google.protobuf.ByteString, my case for PubSub):
Copy code
Benchmark          Mode  Cnt       Score       Error  Units
JsonTest.jackson  thrpt   25  459596.078 ± 31782.902  ops/s
JsonTest.kotlin   thrpt   25  301485.307 ± 19990.942  ops/s
Oh, even 
encodeToString
(vs writeValueAsString) is somehow slower:
Copy code
And string mode (just Any -> String):

Benchmark          Mode  Cnt       Score        Error  Units
JsonTest.jackson  thrpt    5  709838.180 ± 221485.556  ops/s
JsonTest.kotlin   thrpt    5  316919.843 ±  46396.238  ops/s
m
What’s your actual question? Btw, how do you benchmark that? I wonder how fast mine is 🙂 https://github.com/fluidsonic/fluid-json
i
1. Any -> ByteArray (JSON) in kotlinx.serialization?
2. JMH
I can try, but I'll rather will use kotlinx.serialization anyway, and remove jackson from project.
Currently I use kotlinx.serialization for protobuf and jackson for json
m
There seems to be no way to write JSON into a 
ByteArray
or into a stream 😞 So you’ll always have an intermediate 
String
.
p
Json
is defined as a 
StringFormat
, which is an impl of 
SerialFormat
, and there's an (experimental) 
BinaryFormat
in the same file, so it looks like there's plans for binary support in the future
i
Fluid: 84957.650 ops/s
m
Interesting, thank you @irus
i
I think constructing map very inefficient way of serializing. Fluid should directly write to some byte[buffer/array] structure
m
It’s not creating maps
It even has streaming reader and writer
What’s your test input anyway?
i
Simple k-v object, without maps/arrays/objects
m
In my lib 
@Json
creates a codec that streams the JSON output just like kotlinx.
i
Ok. Time to baseline test 🙂
Just StringBuilder with couple of appends
😁 1
message has been deleted
Copy code
JsonTest.baseline  thrpt   10  2041713.611 ± 259978.102  ops/s
JsonTest.fluid     thrpt   10    98638.922 ±   9891.324  ops/s
JsonTest.jackson   thrpt   10   673197.943 ± 147790.012  ops/s
JsonTest.kotlin    thrpt   10   309573.559 ±  22590.947  ops/s
m
Thanks for sharing. What did you change that makes it do down that far 😮 Looking at the graph there’s some unoptimized string escaping in 
StandardWriter.writeString
.
What tool do you use for testing?
i
JMH: https://github.com/melix/jmh-gradle-plugin Just consume result of mapping into blackhole, serializer is created in setup section (for jackson), others just use static
Looking at the graph there’s some unoptimized string escaping in 
StandardWriter.writeString
.
Also number and depth of calls, jackson visually do less calls, and eventually just write in ByteArray
m
Jackson uses an intermediate buffer and doesn’t write directly to the stream. Allows for some fancy optimizations.
i
Better escaping kotlinx-serialization-json-jvm-1.0.1-sources.jar!/commonMain/kotlinx/serialization/json/internal/StringOps.kt:20
m
Yeah. I prefer Jackson’s approach though. However it’s also more work 😅
v
We probably will add it along with IO support. In the meantime, could you please share the source of your benchmarks?
Copy code
And string mode (just Any -> String):
Benchmark          Mode  Cnt       Score        Error  Units
JsonTest.jackson  thrpt    5  709838.180 ± 221485.556  ops/s
JsonTest.kotlin   thrpt    5  316919.843 ±  46396.238  ops/s
This difference looks too suspicious. Also, please ensure that you do not measuring typeOf-based serializer lookup. E.g. 
Json.encodeToString(value)
is much slower than 
Json.encodeToString(MyValue.serializer() value)
i
Copy code
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import kotlinx.serialization.json.Json
import kotlinx.serialization.serializer
import org.openjdk.jmh.annotations.Benchmark
import org.openjdk.jmh.annotations.Scope
import org.openjdk.jmh.annotations.Setup
import org.openjdk.jmh.annotations.State
import org.openjdk.jmh.infra.Blackhole

@State(Scope.Benchmark)
open class JsonTest {
    private lateinit var objectMapper: ObjectMapper

    @Setup
    open fun setup() {
        objectMapper = jacksonObjectMapper()

    }

    @Benchmark
    open fun jackson(blackhole: Blackhole) {
        blackhole.consume(
            objectMapper.writeValueAsString(
                DefaultPixelEvent(
                    version = 1,
                    dateTime2 = System.currentTimeMillis().toString(),
                    serverName = "some-endpoint-qwer",
                    domain = "<http://some.domain.com|some.domain.com>",
                    method = "POST",
                    clientIp = "127.0.0.1",
                    queryString = "anxa=CASCative&anxv=13.901.16.34566&anxe=FoolbarActive&anxt=E7AFBF15-1761-4343-92C1-78167ED19B1C&anxtv=13.901.16.34566&anxp=%5ECQ6%5Expt292%5ES33656%5Eus&anxsi&anxd=2019-10-08T17%3A03%3A57.246Z&f=00400000&anxr=1571945992297&coid=66abafd0d49f42e58dc7536109395306&userSegment&cwsid=opgkcnbminncdgghighmimmphiooeohh",
                    userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:70.0) Gecko/20100101 Firefox/70.0",
                    contentType = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                    browserLanguage = "en-US,en;q=0.5",
                    postData = "-",
                    cookies = "_ga=GA1.2.971852807.1546968515"
                )
            )
        )
    }

    @Benchmark
    open fun kotlin(blackhole: Blackhole) {
        blackhole.consume(
            Json.encodeToString(
                DefaultPixelEvent.serializer(),
                DefaultPixelEvent(
                    version = 1,
                    dateTime2 = System.currentTimeMillis().toString(),
                    serverName = "some-endpoint-qwer",
                    domain = "<http://some.domain.com|some.domain.com>",
                    method = "POST",
                    clientIp = "127.0.0.1",
                    queryString = "anxa=CASCative&anxv=13.901.16.34566&anxe=FoolbarActive&anxt=E7AFBF15-1761-4343-92C1-78167ED19B1C&anxtv=13.901.16.34566&anxp=%5ECQ6%5Expt292%5ES33656%5Eus&anxsi&anxd=2019-10-08T17%3A03%3A57.246Z&f=00400000&anxr=1571945992297&coid=66abafd0d49f42e58dc7536109395306&userSegment&cwsid=opgkcnbminncdgghighmimmphiooeohh",
                    userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:70.0) Gecko/20100101 Firefox/70.0",
                    contentType = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
                    browserLanguage = "en-US,en;q=0.5",
                    postData = "-",
                    cookies = "_ga=GA1.2.971852807.1546968515"
                )
            )
        )
    }
}
🙏 1
Copy code
@Serializable
data class DefaultPixelEvent(
    val version: Int,
    val dateTime2: String,
    val serverName: String,
    val domain: String,
    val method: String,
    val clientIp: String,
    val queryString: String,
    val userAgent: String,
    val contentType: String,
    val browserLanguage: String,
    val postData: String,
    val cookies: String
)
2 Views
Open in Slack
PreviousNext
Powered by