csv export changes in latest build?

Further to this, for some reason if you open a csv file created in 7.4 in excel and then try to save it excel thinks its a UTF text file rather than a CSV file. CSV files created in 7.35 imported into excel save as CSV files. Looks like something has changed between 7.35 and 7.4.

Hello Peter,

Thanks for reaching out with your question. It would be helpful if you could provide some additional information on the error you're seeing. In addition, can you confirm which version you have upgrade from?

Thanks,

Ryan

Hi Ryan, we dont see an error as such - but clearly the actual structure or encoding of the file has changed from YF 7.35 to YF 7.4. The easiest way to see the impact of the difference is to open a CSV file produced in YF7.35 or earlier in Excel. With YF7.35 and earlier you can open the file, Excel recognises it as a CSV file.You can save it in Excel and Excel treats it as a CSV file.

If you produce a CSV file in YF7.4 Excel will open it as a CSV file, but if you go to save it Excel thinks its a UTF encoded text file (and in fact if you save it it strips out the comma field seperators which makes your CSV file all but useless).

We have other internal apps that expect a straight CSV file which produce error messages now when they attempt to import CSV files produced from YF7.4. The reports that produce these CSV files worked perfectly prior to YF7.4

Our hypothesis is that the format of the YF7.4 CSV file is different to the format of the YF7.35 CSV file.

Hopefully this helps explain the situation.

Hi Peter,

Thanks for the additional information. We did make a change in our November releases of Yellowfin. In this change we added the missing Byte Order Mark when exporting CSV files in UTF-8 character set.

When exporting CSV files from Yellowfin, I am not experiencing any issues using them in other programs including Excel.

I would verify that you are still exporting them in the Character Set your applications are expecting. This can be done at time of export:

Or at a global default level under 'Administration' > 'Configuration' > Globe Icon > 'Text Formats' > 'CSV Character Set' :

If you find that adjusting these settings to your desired Character Set still produces issues, please attach an example CSV here to demonstrate your issue.

Thanks,

Ryan

Hi Ryan, have you followed the steps I outlined above ie importing the CSV file into Excel and then selecting File - Save As in Excel and seeing the default is now UTF Text File rather than CSV, saving the file and then seeing that the comma delimiters are stripped from the file?

Hi Peter,

In looking at the exported file itself, the format is being outputted as expected. I also didn't experience this behavior with Libre Office. After working with Excel directly, I did note that 'Save As' does try to save the file as a .txt. In letting Excel do this, the delimiters are naturally stripped from the file.

This is likely the result of a change that happened in our November releases related to the BOM, as mentioned. The good news here is that as of our upcoming January end of month releases, you will be able to choose UTF-8 with BOM or UTF-8 without BOM. Choosing the latter should resolve the behavior you're seeing once this is released.

I'll update you here when I have an official release date confirmation. In the meantime, as long as the file isn't saved as a .txt accidentally, hopefully you can still utilize the files.

Thanks,

Ryan

Thanks Ryan, we can work around the issue but it means we need to import every CSV file that comes out of Yellowfin into something like Excel and then save it from Excel as a CSV file to strip out the BOM that YF is inserting so we can use the file - which is a lot of double handling. Do you have a rough ETA on the next release?

Hi Peter,

I do not at present have a release date. I can say that this configuration change will be part of our next release, which should be within the next couple of weeks. I'm going to move this over to an Issue so I can set this as Defect Logged.

Thanks,

Ryan

Hi Peter,

I wanted to reach out on this and let you know that this is now fixed in our current release. You now have the option of UTF-8 with BOM or UTF-8 without BOM. Choosing UTF-8 without BOM should resolve this issue.

Let us know how it goes.

Thanks,

Ryan