Today: • Wanted to try handling _sharedBox_create and stuff around that (looks like it's about capturing vars in lambdas) but bumped into incomplete support for lambdas (we support only lambdas with a single return but in Kotlin many expressions in a lambda are possible). • Spent the most time of the day trying to create a lowering that creates functions for complex lambdas. Took inspiration in CallableReferenceLowering and in InteropCallableReferenceLowering, tried hard, but managed nothing: there are many problems and I still don't know IR well enough. • Tried to continue a temporary solution for lambdas support in the IR2PyAST that we now have, it's ugly but it works. I have +43 box tests passing without regressions now. However, there can be some more improvements so I don't open a PR for now. The next week I will be on vacation so probably I won't do anything. Next time: • Finish with that ugly support for multiline lambdas. • Probably start with _sharedBox_create.

Today: • PR: Finished with support of complex lambdas (multi-lined; with receiver) – +60 box tests pass (+43 before and +17 additionally now). • PR: Supported sharedBox (lambda captures) – +95 box tests pass. • Continued with switching to the new testing infra a bit. Still no luck as a huge amount of code there is bound to JVM

:(

Even classes that look like common ones judging by name. So looks like a deep understanding of the new infra is required to adapt our code to it, almost from scratch. Or maybe when JS IR is migrated to it, we could reuse it again. For now, I just ported some pieces of our code to the new infra, but it's only about 5-10% of what's needed to be ported. Although, there are some good news: I see how we can simplify our current testing even more, there is still some unused stuff left from JS. • Continued with IrTry but now the blocker is that exceptions can't be constructed (there is some JS-specific logic). Next time: • Prettify our testing a bit more: remove more unneeded pieces; take a look at ignored tests for Python Backend and maybe remove those declarations because we now don't depend on JS. • Research constructing exceptions.

This Friday we have a public holiday, so it seems I'm not working again...

thread for discussing/tracking support for stdlib

@ dmitriy.novozhilov pinged me today that JS is now having the new testing infra in master: https://github.com/JetBrains/kotlin/tree/8a1362de03bc930072ca8a5a3ed997e23050423c/js/js.tests/test/org/jetbrains/kotlin/js/testNew

Wow, just found an interesting test (IrPythonCodegenBoxTestGenerated$BinaryOp.testEqNullableDoubles):

fun eq_double_any(a: Double, b: Any) = a == b

eq_double_any(0.0, -0.0)  // should be false but we in Python return true

At the same time:

fun eq_doubleN_double(a: Double?, b: Double) = a == b

eq_doubleN_double(0.0, -0.0)  // should be true (and we return true)

mind blown

Today: • Prettified our custom e2e tests a bit (PR). • Removed few unneeded symbols from our current testing (PR). • Researched exceptions: it seems we really need to get rid of KJS stdlib and introduce our own. • Researched some stdlib calls: same. • Searched what other failure reasons we have. Supported undefined call and dynamic expressions (PR) – only +2 box tests pass because there are other problems, but at least they are explicitly listed now. Next time: • I see there are also some failures like "X object has no attribute 'constructor'", I want to take a look, maybe it's easy to fix. • Inspect other failure reasons. • Probably start with researching of how we can copy and introduce our own stdlib implementation.

FYI, I started introducing our stdlib, copying the JS IR one. Will send some first PR within a week

i think this is a fascinating project, though very much under represented in terms of the gravity. While i was writing some C porting syntax analogs for which I am informed

"<*>" will be illegal in 1.7

I came up with the notion that intellij is dominantly written in kotlin and kotlin has a PSI editor with syntax embedding which is therefore going to have significant export compatibility with kotlin as well, whether currently done or otherwise. in my code I would if i could simply wrap C code into an embedding and accept a guided tour of compiler errors to flesh out the declaration gaps needed. Instead I have been using sed and operator overloads to examine the gap and fill in the machinations that apparently already exist in a kotlin-native exports version of CPP. This also got me to thinking that this python project was making some mention of the steep difficulties in porting while there sits volumes of presumably kotlin-friendly pycharm code. The dichotomy appears unfortunate to me, that we pay and priotize editors which can perform language specific mappings while my experience proposing flexible language constructs in every language development conversation I have undertaken is almost always met with a summary that is not far from "we are very happy with how we solve programming problems and we feel that stability and usability excludes such matters". but if that's the case then why is there in fact so much effort undertaken from scratch to build front-end harnasses for the language that our editor front-ends are already doing a spectacular job of?

"X object has no attribute 'constructor'"

Those and similar are related to suspend functions support.

Inspect other failure reasons.

From what I picked, mostly it's about some stuff in stdlib. And one was about variable names that are keywords in Python ("invalid syntax"). I will take a look at stdlib then. WASM one, as suggested. By the way, as K 1.6.0 is out, maybe it's time to sync with upstream...

With 1.6.0 merged, OOM happens in

python/experiments/generate-box-tests-reports.main.kts

for me pretty stably 😞 I think I disable the plot generation for now

This sync (1.5.21 -> 1.6.0) is much more difficult than the previous one (~1.4.20 -> 1.5.21), btw 😅

Today: • PR: sync with Kotlin 1.6. • PR: a couple of minor infra changes. Next time: let's discuss.

It's quite uncomfortable for me that we have too much code that doesn't actually work. I mean there are pieces including: 1. Stuff that is needed for JS but won't be needed for Python. Example: different logic for ES6 and pre-ES6. I don't think we will support different compilation modes in the foreseeable future. 2. Stuff that can be useful in Python but some adjustments are needed; this stuff doesn't work now because it's bound to JS but can be unbound and adapted to Python. Example: modules support. For now and in the short and mid term, I believe modules won't be supported by us (we are fixing compilation of a single file now). Our Py backend isn't that complex itself but it has a lot of legacy code that isn't clear. So today I got a very strong wish of spending some time on thoughtful inspection of the whole codebase that we've copied and getting rid of unneeded pieces. In the future, when we need some of them and be ready to support them properly, we can restore them from git history or from kjs dirs. I think this can make our life much easier. We will understand the remaining parts better. Also it will be easier to refactor (like switching to the new testing infra) and syncing with the upstream. The only problem is that there is much work, it can easily take a month or even more, but it can be done iteratively. So Piotr, wdyt? I can start with it. And you can continue with kpy stdlib.

(moving to a separate thread) https://kotlinlang.slack.com/archives/C0289CS37AS/p1637517057061400?thread_ts=1637340107.058700&cid=C0289CS37AS

Today: • Some minor checks and issues management. • PR: Removed some Python magic methods to support MicroPython. • PR: As discussed a week ago: removed more JS-related and unused stuff, mostly in testing code. I'm starting to understand the testing code as a whole, I hope it will help to migrate to the new infra in the future. Also, saw that there are two variants of stdlib: "full+kotlin.test" and "minimal for testing". It complicates affairs, but we can't use a single one always because it adds regressions. I think we should solve it in scope of introducing our own stdlib. Next time: • Continue with testing enhancements. I hope I'll finish with it and switch to compiler code.

after this change, we'll be able to better track MicroPython support: [#75] Run MicroPython tests in CI . It doesn't yet fix the differences and issues like MemoryErrors, but makes them explicit

This message was deleted.

regarding unstable master, asked here: https://kotlinlang.slack.com/archives/C0BUHC9HD/p1638448734017700

Today: some work regarding automation of reports generation: https://github.com/krzema12/kotlin-python/pull/90. That's because it's a bit inconvenient to generate them locally when we have more than one target (and we have MicroPython now). I think I'll continue to generate reports for plain Python locally, but for MicroPython, CI will do it. Next time: • If the CI fails, I'll try to resolve the problem. Without this automation, I can't continue well... • Continue with testing enhancements. I hope I'll finish with it and switch to compiler code.

I guess I haven't done such build chains on GH Actions yet, looks cool 🚀 We'll see if I manage to make it right from the first time 👀

Today: turns out there was some job left regarding CI, fixed some problems but not all. The next week I think I will finish it to be almost perfect 😉

Piotr, there is only one question unclear, please answer it until the next Friday: https://github.com/krzema12/kotlin-python/pull/90#discussion_r766858977 Other things are clear, I just need to find some will to finish them

Let me mention it in advance: the next week will be the last one before I take a break. I will have a vacation, then New Year Russian public holidays, then some personal affairs. So after the next week, I guess I will continue coding K/Py only in the end of January 2022

Oh, so endless loops are hard to create in GH Actions: https://github.community/t/github-actions-workflow-not-triggering-with-tag-push/17053/6

Today: I think finished with CI but if some minor problems left, I can fix them. Also improved our e2e tests a bit. Next time: it will be in a month or so, so I don't want to plan too far ahead.

Are there any plans regarding mixed kotlin->python and kotlin->python C API projects?

I once mentioned the problem of duplication in GitHub Actions workflows (YAML doesn't let us do much about it, maybe apart from extracting simple stuff to env vars). Well, I decided to tackle it with a Kotlin DSL which is now published to Maven Central, and I propose to adopt it in kotlin-python: https://github.com/krzema12/kotlin-python/pull/106

Trying to set up kotlin-python on a new computer but having the following error:

> Task :compiler:resolution:compileKotlin
e: error while writing /home/servb/IdeaProjects/kotlin-python/compiler/resolution/build/classes/kotlin/main/org/jetbrains/kotlin/resolve/calls/components/CollectionTypeVariableUsagesInfo$isContainedInInvariantOrContravariantPositionsWithDependencies$isContainedAnyDependentTypeInReturnType$1$1.class (Permission denied)

Any ideas what the problem is? Probably too long path but then I'm wondering why it's worked before...

@Piotr Krzemiński, could you please clarify the status of https://github.com/krzema12/kotlin-python/pull/97? I'm going to continue researching the codebase and removing unneeded stuff. Should I make my branch out of yours?