Using Transformations but it's only importing 10,000 rows

Yellowfin FAQ shared this question 11 days ago
Answered

We are trying to import over 400,000 rows in a Transformation Flow, but when we look at our Database we are only seeing 10,000 rows. We have tried changing the Max Row Returned setting on the Data Source but we are still only seeing 10,000 rows imported.

Comments (1)

photo
1

If you are seeing this behavior you are most likely using the "Run in Design Mode" option the in the Transformation Flow designer. The designer is set up to handle smaller sets of data, just to test that the import does work as intended and has a default limit of 10,000 rows. This can be overridden but it is recommended to run the Transformation manually from the Browse screen or via a schedule. The following wiki entry will guide you on the full transformation flow execution.


If you need to increase the rows imported by the Design Mode for further testing before completing a full transformation, click on Settings and enter the number of rows you require into the Preview Row Limit setting as shown below:


0ca53d99b703df5a2e56c10e439051ac


In order for the Preview Row Limit changes to apply, you may need to Publish the Transformation and then re-enter the designer.