Tutorial: How to Read Parquet Files in SSIS Using ZappySys ODBC

[daniel]

Introduction

This article demonstrates how to connect to Parquet files using SSIS and ODBC. By leveraging the Parquet (DuckDB) Connector, which is part of the ZappySys ODBC PowerPack, you can seamlessly query Parquet data and integrate it into your SSIS packages.

Prerequisites

• Download and install the ZappySys ODBC PowerPack.

Steps

Step 1: Create a New ODBC Driver

1. Open ODBC Data Sources by typing “ODBC” in the Windows search box and launching ODBC Data Sources (64-bit).
   

   

2. To allow access for SSIS, go to System DSN (recommended for SQL Server Integration Services) and click Add.
   

   

3. From the driver list, select ZappySys JDBC Bridge Driver, then click Continue.
   

   

Step 2: Configure the DuckDB (Parquet) Connector

1. Enter the connection settings and configure the required parameters.

2. Use the following values:

   • Connection string:
     jdbc:duckdb:memory:
   • JDBC driver file(s):
     D:\Drivers\JDBC\duckdb_jdbc.jar

3. Click Test Connection to validate the configuration.
   

   

4. Go to the Preview tab and run a sample query to confirm access to the Parquet file. For more SQL examples, see the JDBC Bridge documentation.

SELECT *
FROM read_parquet('https://hrbrmstr.github.io/large-parquet-test/sample.parquet');

5. Verify that data is returned successfully, then click OK to save the configuration.
   
   

Step 3: Read Parquet (DuckDB) Data in SSIS

1. Open Visual Studio and create a new Integration Services Project, or open an existing one.

2. Set the project name and location, then click OK.

3. Drag and drop a Data Flow Task from the SSIS Toolbox onto the Control Flow surface and double-click it.
   

   

4. Inside the Data Flow, drag and drop an ODBC Source component.
   

   

5. Double-click the ODBC Source to configure it.

6. Click New to open the Configure ODBC Connection Manager window, then click New again to create a new ODBC connection.
   

   

7. Select the DuckDB ODBC data source you created earlier, click Test Connection, then click OK to finish the configuration.
   

   

8. Choose the Data access mode (Table name or SQL command).

   • Select a table if available, or
   • Enter a SQL query (for example, a read_parquet query).
     Click Preview to validate the results, then click OK.
     
     

9. Drag and drop a destination component (in this case, trash destination), connect it to the ODBC Source, and run the package to view the results.
   

   

Conclusion

By integrating Parquet files with SSIS using the ZappySys ODBC JDBC (DuckDB) Driver, you can efficiently query Parquet data using SQL and process it within SSIS data flows. This solution provides a flexible and powerful way to read Parquet files, automate data extraction, and export results to various destinations, including databases, flat files, and Excel.

References

• Parquet (DuckDB) Connector
• Parquet (DuckDB) Connector – Documentation
• Blog articles
• ODBC PowerPack

Contact us

If you encounter any issues or have specific questions, reach out to our support team via live chat or support ticket using our email support@zappysys.com.