Importing CSVs from folder

Edgar Kautzner shared this question 7 months ago
Answered

Hi guys,

I have a question about implementing a production system allowing CSVs in a standard format to be uploaded to a data table for analysis by a group of users.

I can see two ways of doing this in Yellowfin:

CSV import.

This works well in that it allows a user to append data easily using the YF interface, however if a user accidentally selects 'overwrite' instead of 'append', all data will be overwritten which is an issue. Is it possible to restrict access to only allow specified users access to append and not overwrite data when importing CSVs? Another point about the solution above is that it was mentioned in another post as not being a good long-term solution https://community.yellowfinbi.com/topic/can-you-refresh-data-imported-from-a-csv-file

Data Transformation

This allows more flexibility but needs the ability for CSVs with varying file names to be picked up and loaded. In testing the 'network share' option, it doesn't seem possible to include a wildcard in the file path (e.g. C:\share\Test\*.csv) so that it picks up all CSVs in the folder. Is there any way to achieve this?

I could select 'local file' and run the DT each time a new file is selected however I see this as being cumbersome and not really suited for end users.

Do you have any other suggestions for this scenario?

Many thanks,

Edgar

Comments (1)

photo
1

Hi Edgar,

CSV import. - no it is not possible to do so from within Yellowfin. However, tackling the issue from outside of YF, I'm wondering if you removed the DROP permission from your Yellowfin Account in your data source db maybe that would do the trick?

And Nathan's advice in that community topic seems very sound to me - do it from within the DBMS itself, wouldn't take much work and would give you more flexibility.

Data Transformation - no, currently there is no way to achieve this, however, another client has already requested the very same thing so I've added you to that idea.


If you chose the batch file for SQL Server (or other DB) suggestion as described in the above first link then I imagine you could sneak a wildcard in there as well! regards,

David