Does Yellowfin reserve memory based on metadata?
I'm creating a new table for some Google Analytics data that I'm extracting from Bigquery.
When testing using transformation flows, I saw that the table generated was using the extract to determine the varchar length for some of the columns
Thing is, I can't really know ahead of time whether certain values will exceed this limit
I was thinking of using the max length supported by my DWH (Snowflake) so we never have truncated values but on researching it, Snowflake have mentioned that some applications may reserve memory based on the metadata.
I don't want Yellowfin reserving 16MB of memory for values that may be a fraction of that as it could quickly consume a lot of the memory available on the server