This is rather architectural question. I'm using p...

# spring

[rajesh]

This is rather architectural question. I'm using postgresql as primary database and elastic search as search engine in spring boot. I'm only storing portion of data columns of postgres into elastic search. When user search for any data, I search in elastic search, get their id and retrieve data from postgres to send response. Am I doing anything wrong? Should I store all data in elastic search? If so, there will be overhead of updating data everytime when there's update in postgres.

[Philip Puffinburger]

I usually store all the search result data in ES, so if I was searching Books I would store the DB id, Title, Author, whatever else I’d show in the result list in ES. If someone performed some action on the result item, I’d go hit PG with the id and get the needed data at that time. This has worked pretty well with our datasets (largest around 30M) and updating in ES isn’t that measurable of an overhead for us.

[Joel]

I don't think there's a wrong answer here. The configuration is really dependent on your data and how it's consumed. If it's working well, then you're doing it right!

[ashmelev]

@rajesh I would say it depends on your search/response volumes. If a search returns only one ES document and that results in only on Postgres call. And if you only have a few thousand users and the system will never serve more than 50,000 queries a day - then your architecture should be fine. That said, if it's an application where the ES result set may be 10,000 documents and for each of those you must issue a SQL SELECT (or even something more clever) then I'd advise against your current strategy and store all the data needed for response to ES (yes, even with the additional update overhead to ES)