Seperate Data Refresh and Version History Schedules

Asif Bhat shared this idea 6 months ago
Idea Logged

Currently we have clients that require a daily version history of the report, so their managers can look at older versions, going up to 2 months back. They only need one version per day.

However, we also have the requirement that the data in these reports is refreshed minutely, so that anyone visiting the report through the UI will see current data.

Currently in Yellowfin, these two are linked. So everytime they run a minutely refresh on the report data, it creates a new record in the reportinstance, documentitem, and documentdata tables, and these will stay there until the version history limit is reached.

This results in > 60k records/ day being added to these tables which is causing congestion on the server.

This enhancement is to separate these two processes into different schedules so:

Data refresh - creates a single record in the reportinstance/document tables. The next time this schedule is run, the previous record created by this schedule is deleted (as it is no longer useful).

Version history - creates a record everytime this schedule is run. If this is scheduled for "daily" there will be a single record for everyday until the version history time limit (or size limit) is reached.