Kang Andrew

GitHub — Data transfer between Excel and DB

1
Memory-efficient millions of rows' transfer between Excel and a database using Apache POI, Spring Events, and Async Threads

Add a comment

Replies
Best
Kang Andrew
Maker
📌
Memory-efficient millions of rows' transfer between Excel and a database using Apache POI, Spring Events, and Async Threads - Fetch data using pagination in the case of data transfer from the database to Excel Calculate the total data size and then creates a job queue using Spring Events - Set the Max ID value to ignore any data generated after this point to avoid disrupting the pagination process. - Utilize idle threads to perform asynchronous chunked data transfer between Excel and the database.