Memory-efficient millions of rows' transfer between Excel and a database using Apache POI, Spring Events, and Async Threads
- Fetch data using pagination in the case of data transfer from the database to Excel
Calculate the total data size and then creates a job queue using Spring Events
- Set the Max ID value to ignore any data generated after this point to avoid disrupting the pagination process.
- Utilize idle threads to perform asynchronous chunked data transfer between Excel and the database.