https://kotlinlang.org logo
Title
c

Colton Idle

05/01/2023, 1:52 PM
Working with an old backend api that for some reason is advertised as taking json as input, but it doesn't use
"
, it uses
'
Is there an easy way to write an interceptor or something that just replace all double quotes with single quotes? I'm using okhttp + retrofit.
c

Chris Lee

05/01/2023, 1:56 PM
that’s unfortunate. don’t know about the interceptor; worth understanding how embedded single-quotes (within strings) are represented so those don’t get inadvertently replaced.
c

Colton Idle

05/01/2023, 2:00 PM
Asked my lead, and he said that embedded single quotes just aren't supported. 😭
c

Chris Lee

05/01/2023, 2:00 PM
😱
advertised as taking json as input
You already know this: that isn’t JSON. Specs: https://www.rfc-editor.org/rfc/rfc8259, https://www.json.org/json-en.html
A string begins and ends with
   quotation marks.
…where quotation mark is 0x22
"
if there’s no chance of a single-quote showing up inside a string, is it possible to do a global replace (
"
->
'
) when generating the json, or is that generation buried in the framework?
c

Colton Idle

05/01/2023, 2:08 PM
Yeah. Best part is if I hand it valid json, I get a raw response of
"Input is not valid json"
which is funny, because even the response isn't proper json because it doesn't have {} lol
e

ephemient

05/01/2023, 2:31 PM
I believe the https://github.com/stleary/JSON-java is lenient enough to parse single-quoted strings, it's a pretty ludicrous solution but you could parse and re-stringify with it
oh wait you need to produce invalid JSON? wtf...
c

Chris Lee

05/01/2023, 2:34 PM
Looks like Jackson may have some support for this.
e

eygraber

05/01/2023, 2:35 PM
Is your backend using python?
y

yschimke

05/01/2023, 2:56 PM
I think that's still valid
Early versions of JSON (such as specified by RFC 4627) required that a valid JSON text must consist of only an object or an array type, which could contain other types within them. This restriction was dropped in RFC 7158, where a JSON text was redefined as any serialized value
c

Chris Lee

05/01/2023, 2:57 PM
even RFC 4627 specifies quotation marks (
"
)
y

yschimke

05/01/2023, 3:00 PM
Sorry, was trying to reply to the error message not being valid JSON.
c

Chris Lee

05/01/2023, 3:01 PM
ah yea, that.
it’s likely the back-end here is taking a shortcut - calling it ‘json’ but not directly parsing it, instead dropping it to language-specific object representation (Python or JS).
which means it’s potentially vulnerable to all kinds of injection attacks 🙄
e

eygraber

05/01/2023, 3:17 PM
Yeah I used to get back True and False from an API. The backend engineers were just dumping python objects ¯\_(ツ)_/¯
c

Chris Lee

05/01/2023, 3:18 PM
“engineers” 🙈
c

Colton Idle

05/01/2023, 3:20 PM
So it seems like moving to jackson is the current front-runner as an option to produce "invalid" json to send to the server?
e

ephemient

05/01/2023, 3:21 PM
well it wouldn't be too hard to write a transformer from
"
-based JSON to
'
-based
c

Chris Lee

05/01/2023, 3:21 PM
what are you using now / does it have a similar option to use single-quotes? If not then you have options for Jackson or a search/replace.
c

Colton Idle

05/01/2023, 3:21 PM
transformer... do you mean interceptor? Or is transformer something I just don't know about. 😄
I'm using moshi now. Let me look at it's docs/issue page.
e

ephemient

05/01/2023, 3:22 PM
well I mean like a simple state machine can handle it, and sure you could put that in an interceptor
private enum class State {
    Outside,
    InsideString,
    BeginEscape,
}

fun jsonHack(json: String): String = buildString(json.length) {
    var state = State.Outside
    for (c in json) {
        when (state) {
            State.Outside -> when (c) {
                '"' -> {
                    append('\'')
                    state = State.InsideString
                }
                else -> append(c)
            }
            State.InsideString -> when (c) {
                '"' -> {
                    append('\'')
                    state = State.Outside
                }
                '\\' -> {
                    append(c)
                    state = State.BeginEscape
                }
                '\'' -> append("\\'")
                else -> append(c)
            }
            State.BeginEscape -> {
                append(c)
                state = State.InsideString
            }
        }
    }
    check(state == State.Outside)
}
c

Colton Idle

05/01/2023, 3:34 PM
Gotcha. Didn't know if an interceptor would essentially just let me do a replace(", ') I'll give this a shot when I'm back at my desk
c

Chris Lee

05/01/2023, 3:35 PM
depends on what altitude the ‘interceptor’ is at - the above will work if you can intercept the generated json. it’s perhaps possible to intercept during generation (over-riding encoding of a string, for example).
e

ephemient

05/01/2023, 3:37 PM
well you get a Sink in ResponseBody so you can either stringify the whole thing or change the above to wrap it in a FilteringSink or something (as long as it's UTF-8 or similar, it'll work fine on bytes)
oh I guess there's a FilterOutputStream but there isn't a FilterSink. well, a custom ForwardingSink or something like that then
ForwardingResponseBody is a test only class, but ForwardingSource is public API of okio.
e

ephemient

05/01/2023, 4:18 PM
well it's a little different on the request side, but if you don't mind the extra conversions, it's easy to
object JsonHack : Interceptor {
    override fun intercept(chain: Interceptor.Chain): Response {
        val request = chain.request()
        return chain.proceed(
            request.newBuilder()
                .method(
                    request.method,
                    request.body?.let { body ->
                        val buffer = Buffer()
                        body.writeTo(buffer)
                        buffer.readUtf8().jsonHack().toRequestBody(body.contentType())
                    }
                )
                .build()
        )
    }
}
using the previous function
j

jessewilson

05/01/2023, 5:05 PM
val lenientJsonAdapter = strictJsonAdapter.lenient()
c

Chris Lee

05/01/2023, 5:06 PM
it’s the outgoing json - the consumer of that “json” - that expects malformed json, and is purportedly not able to change.
j

jessewilson

05/01/2023, 5:06 PM
In that case, I recommend getting the author of that endpoint fired and replaced with somebody else