Exporting tables from a SQL Server backup (.bak) to Parquet without restoring it

Posted by rferraton 1 day ago

Counter25Comment5OpenOriginal

Comments

Comment by hbarka 1 day ago

I love me my Microsoft SQL Server. Where was this thirty years ago. Finding a table from a bak file in a matter of minutes without restore is miraculous.

Comment by rferraton 7 hours ago

[dead]

Comment by rferraton 4 hours ago

TDE and Encrypted backup are now included in the FastBCP 1.6.6

Comment by mrtimo 23 hours ago

Parquet files are a game changer. I don't know how people are living without them, especially pairing them with DuckDB for basic analysis that was previously happening with pandas etc.

Comment by rferraton 7 hours ago

Agree!

Comment by rferraton 1 day ago

Author : Romain Ferraton

To get one table out of a SQL Server full backup, you normally restore the whole database first: disk space for all of it, an instance, then a SELECT INTO.

FastBCP (our bulk export tool) now treats the .bak as a plain binary file. It reads it once to index it, locates the table’s pages, decodes them into Arrow columns and writes Parquet. No SQL Server instance, no credentials, nothing touches production. Threads split the table’s pages and share one open backup.

On a 320 GB striped, compressed production backup (383 tables), the cold open took 2m29s (~2.2 GB/s from disk). After that, a 175M-row table exported in 29.6 s, and later opens took 0.3 s.

Supported: SQL Server 2017–2025, full and copy-only backups, stripes, MS_XPRESS/ZSTD compression, rowstore and columnstore.

Not supported yet: differential/log backups, TDE or encrypted backups. The only output is Parquet.

Happy to answer questions about the backup format internals.