Data Transformation Table Creation Can't Identify Integers

Stephen Johnson shared this problem ago

Resolved

I am trying to use Yellowfin as a simple data warehouse ETL manager. I'm trying to use the Data Transformation "Output to DB step", which provides the option of creating a table to output to if one does not already exist:

/fZJZ3cJicXyGzrIXA6ePHj3Ljxo1tgZbbHNoWuwj8Ayx59o9rnREIAAAAAElFTkSuQmCC

The problem is that this process is failing to identify integers. My input data has a primary key which is an integer but the auto-generated table instead has a decimal with only 6 digits of capacity:

Not only is this the wrong data format, but if my primary key grows to a 7th digit I'm going to start getting errors when I migrate data.

Yellowfin's auto-generated tables should have columns the same format as the input data set.

I am migrating data from MS SQL Server 2012 to MySQL.

The same problem 0

Replies (2)

Guest ●

Hi Stephen,

I have been able to replicate this issue over here and so have raised product defect YFN-11059 for it to be fixed.

Unfortunately I couldn't find any workaround for the bug except to alter the MySQL table post-Transformation.

Thanks for alerting us to this bug and apologies for the inconvenience caused by it.

regards,

David

Reply URL

Guest ●

Hi Stephen,

the developers have rejected my defect with the following explanation:

The output step converts all "numeric" types to decimal. The output step determines target column-size based on the data it processes. It will set the target width to the longest number in the 200 it receives from the input (default limit in preview mode).

To increase column width using the step's functions, consider increasing the number of rows processed (Process settings) or running the entire volume through (run from browse page).

The table creation functions in the output to DB step is not recommended for production use as it may not have enough information to guess the myriad of datatypes in the target DB and field widths. A DBA is expected to create the target table, therefore setting up data types, widths, precisions and indices. Yellowfin ETL can then be used to populate it.

I hope that clears things up.

regards,

David

Reply URL

Stephen Johnson ●

It does....there is a feature in your software "not intended for production use".

There needs to be big signs, flags, warnings, etc. explaining that to people. One of the most important parts of user experience is setting expectations. It's a negative customer experience to learn after you've gone through the trouble of contacting support to learn that a feature is "not intended for production use".

URL

Guest ●

Hi Stephen,

it is already mentioned in the wiki article on the Output step that the db admin will create the table, however, I will put in a request to the technical writing team that it needs to stand out more.

regards,

David

URL