Feature Request: Optimize Excel-to-SQL Bulk Insert for 100k+ Row Scalability for initial release

[wenxuangithub]

The current MVP(initialbulkcopy) reliably imports ~16k rows from Excel into SQL Server using SqlBulkCopy within a WinForms UI. However, as the row volume increases (target: 100k+), potential performance and stability issues need to be addressed.

Problem Areas:

• Entire Excel file is loaded into a DataTable, which risks memory exhaustion on large files.
• SqlBulkCopy.WriteToServer() is executed in a single operation — no batching.
• Import is performed on the UI thread, risking application freezing during long operations.
• No progress feedback or cancellation mechanism for long-running tasks.
• No error visibility if a single row causes SqlBulkCopy to fail.

---

Current Enhancements:

• Replace ClosedXML with a streaming Excel reader (e.g., ExcelDataReader) to reduce memory footprint.
• Refactor BulkInsertToSql() to split inserts into batches (e.g., 10,000 rows per call).
• Move import logic to a background task (Task.Run) to avoid blocking the UI thread.
• Add progress indicators (label, progress bar, or log updates) for better UX.
• Add basic validation or preview step to detect issues (e.g., invalid date, string length overflows).
• Add exception logging with contextual information when bulk insert fails.

---

Acceptance Criteria:

• Large Excel files (≥100k rows) can be imported without crashing or freezing the UI.
• Insert duration remains stable or predictable (target: <30s for 100k rows).