Ktor works great so far, for HTTP requests at least. But I'm having an issue with the websockets in ...

# ktor

[serebit]

Ktor works great so far, for HTTP requests at least. But I'm having an issue with the websockets in creating a connection to the Discord API servers. Using the same headers and parameters for both Ktor's websocket library and Http4k's nv-websockets, only Http4k manages to create the connection, while Ktor sends an invalid request (403).

[Deactivated User]

If you manage to gather more information (the valid request / and the invalid one) that would help a lot. You can write an issue here: https://github.com/ktorio/ktor/issues

[serebit]

@Deactivated User Will do. I have a few hours to kill anyway

[serebit]

Still looking, haven't found anything obvious that would cause a 403.

[Deactivated User]

Can’t you dump a request that works, and a request that doesn’t?

[serebit]

The request is as default as it gets. The only parameter I give to Http4k is the URI <ws://gateway.discord.gg/>, and the same to Ktor. I can't manually create a websocket request to that endpoint using Ktor, since the Upgrade header is locked.

[serebit]

I'm using the debugger to try to find differences in what's sent to the endpoint.

[Deactivated User]

Yeah, I was thinking for example in wireshark to dump one working and other not working and see differences. Especially it if it is ws and not wss shouldn’t require anything special.

[serebit]

Didn't even know about wireshark, I'll install it and give it a shot. You've just made this 10x easier 😄

[serebit]

It's wss. I had the program request a valid endpoint from Discord before attempting to connect, and upon further inspection, attempting to connect with ws throws a 404, even with Http4k.

[Deactivated User]

I see. I guess it would be possible to intercept Java calls to the stream to get the data without encryption, but I have not investigated that case so I don’t know the details

[serebit]

Ktor connects to the endpoint when using wss, but I need to be able to keep a reference to the WebSocketSession, and there's no webSocketSession complement for the wss function. Any ideas?

[Deactivated User]

in the worst case you can compile the library used from source modifying it to dump the write stuff to a file too

[Deactivated User]

uhm

[Deactivated User]

if that’s the case, maybe you can create a PR or a issue about that missing method. You can also look at the sourcecode and create the extension method in your code in the meantime:

https://github.com/ktorio/ktor/blob/master/ktor-client/ktor-client-features/ktor-client-websocket/src/io/ktor/client/features/websocket/builders.kt

[Deactivated User]

another possible option is to create a WSS reverse proxy and dump stuff there

[Deactivated User]

you can even use ktor for that

[serebit]

The option I've gone for is to store the outgoing SendChannel and use that, while using the wss function to create the socket.

[serebit]

I can still create an issue for the lack of a secureWebSocketSession though, if you'd like

[Deactivated User]

yeah if you think it is missing, makes sense to add it

[serebit]

Run into another issue. It doesn't seem like anything I put into the outgoing channel is actually sent

[Deactivated User]

how are you using websockets?

[Deactivated User]

as far as I understand you are using the ktor websocket client to connect to a wss api from discord, right?

[Deactivated User]

so you are not using channels directly, right?

[serebit]

Yes

[serebit]

Well

[serebit]

Discord's connection process is... let's say "involved".

[Deactivated User]

do you need to send other things other than normal frames?

[serebit]

Once the secure websocket is opened, a payload is sent to the client that contains information about the session. The client is supposed to then send information about itself, including the operating system and bot token.

[Deactivated User]

in the same http request?

[Deactivated User]

once the Ws upgrade is done, it just should let you to send frames

[Deactivated User]

https://en.wikipedia.org/wiki/WebSocket#Protocol_handshake

the request + the server responds with an upgrade
then everything should be frames

[serebit]

client.wss(host = "<http://gateway.discord.gg|gateway.discord.gg>") {
        outgoingChannel = outgoing
}

then, later on, after the session info is received from the socket:

outgoingChannel?.send(Frame.Text(text))

[serebit]

outgoingChannel isn't null at this point either, and no exception is thrown. It just does nothing, as far as I can tell.

[Deactivated User]

but I don’t know why are you trying to change the outgoingChannel

[Deactivated User]

for interception/debugging?

[serebit]

I'm not. outgoingChannel is a class-level variable that I'm using in leiu of storing the WebSocketSession itself.

[Deactivated User]

uhm

[serebit]

So instead of storing the session, I store the outgoing channel, and use that to send frames.

[Deactivated User]

aha

[Deactivated User]

and the frames you are trying to send with those methods

[Deactivated User]

are not written to the socket?

[serebit]

Yep. No exceptions are thrown, but nothing is written to the socket as far as I can tell.

[Deactivated User]

that might be a bug, I can’t tell without all the context. Which version of ktor are you using?

[serebit]

1.0.0-alpha-1. I can commit this and push it to Gitlab if you'd like to take a look.

[Deactivated User]

if it is not too large I think I can have a look

[Deactivated User]

by chance have you tried with an older ktor version just in case? Or created this code after 1.0.0-alpha-1?

[Deactivated User]

(Just to discard regressions)

[serebit]

I can try it with an older version, sure.

[Deactivated User]

Yeah, that would help if it is possibl

[serebit]

Happens with 1.0.0-alpha-2, 0.9.6-alpha-1-rc13, and 0.9.5-rc13.

[Deactivated User]

aha

[Deactivated User]

well try to upload that sample and I will have a look as soon as I have some time. Also I can try to create a snippet for a WS reverse proxy using ktor so you can intercept the communication and debug both directions

[serebit]

Sure. I'll actually try and rewrite it as just a single file, to make it easier to read.

[serebit]

I'll need to give you a token to use for authentication, just promise you'll get rid of it once you're done 🙂

[Deactivated User]

Im not at the computer right now. I will look at it in a few hours. First i will try to read it without executing it to see if i find something obvious. If not we can try the token. I willalso try to create a ws reverse proxy snippet to debug it

[serebit]

https://gitlab.com/serebit/ktor-error-demo

The working Http4k version is on the master branch, the not-working ktor version is on the ktor branch.

[Deactivated User]

Cant find here where are you setting the outgoingChannel: https://gitlab.com/serebit/ktor-error-demo/blob/ktor/src/main/kotlin/internal/network/Socket.kt

[Deactivated User]

Being private I guess you missed the assignment when simplifying the code

[serebit]

Yep, that's what happened. Fixed it.

[Deactivated User]

Sorry for the delay. Except of that can’t find anything else strange.

Also I have created a sample showing how to create a websocket reverse proxy so you can intercept messages and dump stuff and see if data is being sent:
https://github.com/ktorio/ktor-samples/blob/master/other/reverse-proxy-ws/src/ReverseProxyWsApplication.kt#L61

[serebit]

Thanks, I’ll give that a try once I get home.

[serebit]

So as someone who hasn't touched the server side of ktor, how does this reverse proxy work?