Is it possible to pass two dir files and assert all files and directories have the same structure an...

# kotest

[Javier]

Is it possible to pass two dir files and assert all files and directories have the same structure and content?

[Jim]

Sounds like a fun assertion to write, I'm not sure that it exists though

[sam]

yeah sounds like a useful addition

[sam]

dir1 shouldHaveSomeStrutureAs dir2

[sam]

feel free to call it what you want

[sam]

sameContents sameTree etc

[LeoColman]

we should differentiate between `deep`/`recursive` in a way (maybe an optional parameter) as well

[Javier]

Should I file an issue?

[Javier]

I am going to create a PR

[Javier]

https://github.com/kotest/kotest/pull/2575

[sam]

val p = Runtime.getRuntime().exec("""reg query "HKU\S-1-5-19"""")

[sam]

what does that do

[Javier]

I called shouldHaveSameTreeAndContents to be more explicit, feels free to suggest if I should change it

[Javier]

I don't know, I didn't write that

[sam]

oh lol

[sam]

you just changed formatting ?

[Javier]

maybe that line is appearing because the format was different in one function

[sam]

the name is a touch verbose, what about shouldHaveSameStructure ?

[Javier]

I rename to that 🙂

[Javier]

Done ✅

[sam]

✅

[sam]

will merge once green

[Javier]

@sam I checking one documentation website is same than another, I got issues related to the content of the file can be different \ instead of / and so on

[Javier]

so I am doing something like this

[Javier]

expectFiles.zip(actualFiles).forEach { (expect, actual) ->
    expect.replaceSlash()
    actual.replaceSlash()
    when {
        expect.extension == "gz" -> println("`.gz` files are ignored")
        expect.extension == "json" -> println("`.gz` files are ignored")
        expect.isDirectory -> actual.shouldBeADirectory()
        expect.isFile -> actual.shouldHaveSameContentAs(actual)
    }
}

[sam]

the filename or the contents of the file ?

[Javier]

what do you thing if we add a transform lambda and filter predicate?

[Javier]

the content inside the file

[Javier]

the gradle plugin I use to generate the content

[Javier]

generates the files differently in windows vs macos

[sam]

the lambdas would work

[sam]

I didn't realize we were checking file contents

[sam]

I thought were just checking file tree

[Javier]

nono, tree + content

[sam]

I mus have missed that

[Javier]

that was the reason I named it shouldHaveSameTreeAndContents

[sam]

yeah I see now

[Jim]

Is this jvm only?

[sam]

yeah

[Javier]

yeah

[Javier]

files api is not ready in kmp

[Jim]

Should we try to use system.fileseperaror?

[sam]

wanna rename it back 🙂

[Javier]

okio is almost there tho

[Javier]

nono

[Javier]

the function is working fine

[Javier]

the merged one

[Javier]

technically the files have different content

[sam]

I think the name same strucutre only makes sense for file names / tree

[sam]

not actual contents

[Javier]

so what should I do?

[Javier]

in my case I was thinking if you see feasible I add another function to bypass those issues

[sam]

I think if we start comparing file contents we're gonna run into all kinds of issues

[Javier]

no no, there is no issues with that, only if files are generated or something so

[sam]

ok

[sam]

I still think it's useful to have an assertion that only checks structure too

[Javier]

for example this gradle plugin generates html files with \ in its content in windows

[sam]

right

[Javier]

what do you think about three functions

[Javier]

structure

[Javier]

structure + content

[Javier]

structure + content + transform + filtering

[sam]

yes that works

[sam]

I like that

[Javier]

ie:

fun File.shouldHaveSameStructure(file: File, transform: transform: (T) -> Unit, filter: (T) -> Boolean)

[Javier]

I am going to add them

[sam]

ok

[Jim]

Can you use "mutate" instead of transform perhaps?

[sam]

I think transform is a more common name than mutate

[Jim]

Huh I deduced what it did from the type, it wasn't clear to me

[Jim]

Also other: File

[sam]

Actually yeah, what is T ?

[Javier]

yeah you have right

[Javier]

should be

[sam]

Do like a ByteArray or something

[Javier]

filter (File, File) -> boolean

[sam]

or file

[Javier]

in my case, I am filtering all gz files

[sam]

why double file on the filter ?

[Javier]

technically you are comparing pairs of files

[sam]

It's a filter, shouldn't it be File -> Boolean

[Javier]

probably having only one file is enough, "because they should be the same", but not sure if there is use case where you can filter using the file B instead of file A or viceversa

[Javier]

or even both

[Javier]

i.e. filter { a, b -> a.readLines().size == 1 && b.readLines().size == 2 }

[sam]

I thought the filter function would "normalize" it

[sam]

if it's actuallying comparing the files then call it compare

[Javier]

fun File.shouldHaveSameStructure(
  file: File,
  filter: (expect: File, actual: File) -> Boolean
) {
  val expectFiles = this.walkTopDown().toList()
  val actualFiles = file.walkTopDown().toList()

  expectFiles shouldBeSameSizeAs actualFiles

  expectFiles.zip(actualFiles) { expect, actual ->
    when {
      filter(expect, actual) -> {}
      expect.isDirectory -> actual.shouldBeADirectory()
      expect.isFile -> expect shouldHavePath actual.path
      else -> error("There is an unexpected error analyzing file trees")
    }
  }
}

[Javier]

what do you think?

[sam]

I wouldn't call it filter. Filter implies you are filtering things out. Here you are comparing. So I would call it compare, or isEqual or something

[sam]

But other than that, I like

[Javier]

techncally, you are filtering that comparison

[sam]

I dunno, it doesn't seem right to me

[sam]

I know you are filtering the file contents, but I'd never expect a function called filter to compare two things

[Javier]

but the function is not comparing, is filtering

[Javier]

one sample

[Javier]

fileA.shouldHaveSameStructure(fileB, filter = { a, b -> a.extension == "html" })

[sam]

so what's b doing here ?

[sam]

you don't use the b arg

[Javier]

when it is going to be compared the file a with b, if a is a json, it is filtered

[Javier]

yeah

[Javier]

but you can do different comparison

[Javier]

ie

[Javier]

fileA.shouldHaveSameStructure(fileB, filter = { a, b -> a.readlines().size == 2 || b.readlines().size == 5 })

[sam]

you're not filtering anything there. You're applying a predicate to the files.

[Jim]

Are we trying to test that two file trees have the same structure ignoring nodes based on some predicate?

[Javier]

I am removing from the comparison all files with have those predicates

[sam]

so you are filtering files FROM the comparison ?

[sam]

If so why can't it just be File -> Boolean

[Javier]

it can, but a and b can be different, so maybe there can be use cases where you need both

[Javier]

@Jim both trees are identical, but you can remove from the comparison some files

[Javier]

for example, a tree generated in windows and a tree generated in macos, I want to filter if the file extension is .gz

[Javier]

what name should I put @sam? compare is weird for me, because I am not comparing, I am comparing and removing from the comparison

[Jim]

Yeah so File -> Boolean makes sense here

[Javier]

and about the filter name?

[Jim]

Yeah I think so, it should apply the same filter to both files in the comparison but hide that from the tester I think, it instead of a, b

[sam]

filter is fine if it's File -> Boolean

[Javier]

fun File.shouldHaveSameStructure(
  file: File,
  filter: (File) -> Boolean
) {
  val expectFiles = this.walkTopDown().toList()
  val actualFiles = file.walkTopDown().toList()

  expectFiles shouldBeSameSizeAs actualFiles

  expectFiles.zip(actualFiles) { expect, actual ->
    when {
      filter(expect) -> {}
      expect.isDirectory -> actual.shouldBeADirectory()
      expect.isFile -> expect shouldHavePath actual.path
      else -> error("There is an unexpected error analyzing file trees")
    }
  }
}

[sam]

if it's File,File -> Boolean then filter is odd

[sam]

Yep, either

fun File.shouldHaveSameStructure(
  file: File,
  filter: (File) -> Boolean
)

or

fun File.shouldHaveSameStructure(
  file: File,
  filterLhs: (File) -> Boolean,
  filterRhs: (File-> Boolean,
)

If you really really want to filter the two sides separately

[Javier]

I like that

[Javier]

about transform, I should do that too?

[sam]

personally, I'd keep it simple for now

[sam]

can always add more stuff later

[Javier]

and about filterLhs and Rhs, should be && or ||

filterLhs(expect) && filterRhs(actual) -> {}

[sam]

|| makes sense to me

[Javier]

yep

[Javier]

I would like to add the transform one because I need it too, I think it can be useful for codegen files comparison where the file content can vary only in separators or something so

[sam]

Ok then :)

[Javier]

should I separate the transform there in two transform functions, or in this case it is better to have only one transform?

[sam]

I don't really have an opinion on that so we can just go with whatver you think is best

[Javier]

my use case a and b have different content because the codegen, so I want to transform the files "to be the same" (if really they are the same and only separators are the problem)

[Javier]

I think it should be one

[sam]

Ok that's cool

[Javier]

well, technically with the filter functions I can achieve that

[Javier]

well, no, because there is separated

[Javier]

fun File.shouldHaveSameStructure(
  file: File,
  filterLhs: (File) -> Boolean,
  filterRhs: (File) -> Boolean,
) {

[Javier]

if I remerge them into one, there is no need to add another with transform

[Javier]

I know understand your compare, my head did "tick"