https://kotlinlang.org logo
#http4k
Title
# http4k
g

Gopal S Akshintala

05/17/2022, 1:02 PM
Basic question:
urlStr.urlEncoded()
is turning my URL into this:
https%3A%2F%<http://2Fpokeapi.co|2Fpokeapi.co>%2Fapi%2Fv2%2Fpokemon%3Flimit%3D10
which is invalid. What’s the right and http4k idiomatic way to do it?
s

s4nchez

05/17/2022, 1:04 PM
you normally only need to url encode "values" (e.g. query parameters) rather than the full url. If you have a full example of what you're trying to do, maybe it'll be easier to spot where the problem is.
g

Gopal S Akshintala

05/17/2022, 1:06 PM
Sure… one of my urls has spaces something like
{{baseUrl}}/services/data/v{{version}}/query/?q=SELECT Id FROM Profile where Name = 'Standard User'
I don’t wanna string replace spaces with
%20
as there can be all types of urls with spl characters etc, so I wanna leverage encoder
Copy code
val uri = Uri.of(itemRequest.url.raw).let { it.query(it.query.urlEncoded()) }
I tried this, but it is replacing
=
sign also which is causing a problem like
limit%3D10
s

s4nchez

05/17/2022, 1:11 PM
what problems is it causing? The way I'd write is:
Copy code
Uri.of("{{baseUrl}}/services/data/v{{version}}/query/")
    .query("q", "SELECT Id FROM Profile where Name = 'Standard User'")
which translates to
{{baseUrl}}/services/data/v{{version}}/query/?q=SELECT+Id+FROM+Profile+where+Name+%3D+%27Standard+User%27
g

Gopal S Akshintala

05/17/2022, 1:12 PM
But I get these urls as params at runtime, I don’t know them prior.
s

s4nchez

05/17/2022, 1:16 PM
I still can't follow. You get a full URL and what do you need to do with it? The whole setup looks strange (let alone the fact that sending SQL as parameter looks very suspicious security-wise)
g

Gopal S Akshintala

05/17/2022, 1:18 PM
Ah that’s SOQL query (Salesforce). It’s normal to pass… so basically I am trying to parse postman collection json file (something like the one attached). Each item in this json has a url to fire. So that’s how I get the url at run time.
Tried this, seems to be working for my existing usecases:
Copy code
fun encodeUri(url: String): Uri =
  Uri.of(url).let { uri ->
    uri.queries().map { query -> query.first to query.second?.urlEncoded() }
      .foldRight(uri) { encodedQuery, _ -> uri.query(encodedQuery.first, encodedQuery.second) }
}
s

s4nchez

05/17/2022, 3:50 PM
Glad you find a workaround. We could consider adding a way to parse Uris (including query etc), but another alternative would be using
java.net.URI
to do that job and then converting to its various parts to
org.http4k.Uri
g

Gopal S Akshintala

05/17/2022, 3:57 PM
Thanks @s4nchez but if you see this (UrlEncode only query values) as a valid use case, I can find some time to contribute this within http4k 🙂
s

s4nchez

05/18/2022, 7:01 AM
@Gopal S Akshintala that's a good spot! Do you want to submit a PR for a fix or are you ok if I just do it myself?
g

Gopal S Akshintala

05/18/2022, 7:03 AM
I wish to submit a PR, but can only do it on the weekend 🙂 does that work?
s

s4nchez

05/18/2022, 7:05 AM
Or course. That code is almost five years old, so I think it can survive being like that for a few more days 😉 Thank you @Gopal S Akshintala!
🙏 1
j

James Richardson

05/18/2022, 10:49 AM
🤔 i'm not 100% sure about this. I think your query parameter should be properly encoded as:
query/?q=SELECT%20Id%20FROM%20Profile%20where%20Name%20%3D%20'Standard%20User'
note that there is no '=' in the parameter value because it was encoded.
g

Gopal S Akshintala

05/18/2022, 10:53 AM
@James Richardson but to encode, I need to split
q
part and
SELECT …
part first and encode only the value. The problem is with the
split("=")
which returns the value as
SELECT Id FROM Profile where Name
trimming the
'Standard User'
as there is an
=
before that
j

James Richardson

05/18/2022, 10:56 AM
well, where did that come from - i think you're talking about parsing the URL from your json file, right?
not parsing the parameters in the request handler... if i understand right
g

Gopal S Akshintala

05/18/2022, 10:57 AM
Yes right, I have these urls coming from a JSON file
j

James Richardson

05/18/2022, 11:01 AM
the JSON file was produced and is managed by postman. usually it uses placeholder variables for parameters (which it will encode properly before sending) - perhaps the request that is captured in the postman file is itself incorrectly encoded. You could try this by using a variable for the request parameter, then substitute the value at runtime (in postman) and see what is actually sent over the wire. i suspect (i'm not 100%) that it will match what i posted.
Also for extra fun - the stuff that goes into the JSON file will also be json encoded, so needs to be read with care 🙂
g

Gopal S Akshintala

05/18/2022, 11:03 AM
This is the corresponding json in the collection file:
Copy code
{
			"name": "Standard User Profile Query",
			"event": [
				{
					"listen": "test",
					"script": {
						"exec": [
							"var statusCode = pm.response.code;",
							"",
							"var jsonData = JSON.parse(responseBody);",
							"if(jsonData.records[0]!=null){",
							"    pm.environment.set(\"standardUserProfileId\", jsonData.records[0].Id);",
							"}"
						],
						"type": "text/javascript"
					}
				}
			],
			"protocolProfileBehavior": {
				"disableBodyPruning": true
			},
			"request": {
				"method": "GET",
				"header": [
					{
						"key": "Content-Type",
						"name": "Content-Type",
						"type": "text",
						"value": "application/json"
					}
				],
				"body": {
					"mode": "raw",
					"raw": ""
				},
				"url": {
					"raw": "{{baseUrl}}/services/data/v{{version}}/query/?q=SELECT Id FROM Profile where Name = 'Standard User'",
					"host": [
						"{{baseUrl}}"
					],
					"path": [
						"services",
						"data",
						"v{{version}}",
						"query",
						""
					],
					"query": [
						{
							"key": "q",
							"value": "SELECT Id FROM Profile where Name = 'Standard User'"
						}
					]
				}
			},
			"response": []
		},
I hv a logic to replace those placeholder variables, using Moshi factory
j

James Richardson

05/18/2022, 11:05 AM
Yes, I believe your url in the postman file is incorrectly encoded.
g

Gopal S Akshintala

05/18/2022, 11:06 AM
No the json file has whatever is in the screenshot, Postman doesn’t do anything in between
j

James Richardson

05/18/2022, 11:06 AM
Yes, the query that has been typed in, in the screenshot, is incorrectly encoded
See my comment above about using a parameter variable, and see if you get a different request. I believe you will. I think you'll find that the select query is encoded as I posted above. I'm not 100% sure on this, but I think so
well - i tried it and postman doesn't do what i expected at all! the encoding it uses for parameter values doesn't encode
=
🤷 - apologies - might have just thrown confusion into the mix.
encodeURIComponent
on javascript, and also some other encoding thing do, but postman doesn't. hmmm
so maybe the thing to do is what you said right at the start. sorry for that.
🙏 1
s

s4nchez

05/22/2022, 11:55 AM
Build is failing. Did it pass locally?
g

Gopal S Akshintala

05/22/2022, 12:04 PM
I noticed some failing unit tests and I fixed them an hour ago, but when I do
gradle clean build
I get this:
Dust.kt: (3, 12): Unresolved reference: nashorn
which might not be related to the change I did
s

s4nchez

05/22/2022, 12:05 PM
Which jdk are you using? We expect 11
g

Gopal S Akshintala

05/22/2022, 12:13 PM
I see, yes
./gradlew clean build
succeed in my machine
s

s4nchez

05/22/2022, 12:14 PM
Ok I'll check the pr later again (not close to a computer now). Thank you for submitting it!
👍🏼 1
347 Views