When I was working with large text files, I was just building necessary indeces by hand and storing ...

# datascience

[altavir]

When I was working with large text files, I was just building necessary indeces by hand and storing them in a separate temporary file. But it all depends on task.