BigQuery (BQ) database view is still visible in YF even after it is removed from BQ database

Vefa Gulecyuz shared this question 4 months ago
Answered

Hey guys, we are connecting to a Big Query (BQ) data source. We have database views in Big Query and we use those db views in Yellowfin to create YF views.

We needed to remove a db view from BQ and created a new one. However, after we removed the db view from BQ, it is still available in YF.

What I expected is that once we remove the db view from BQ, YF view would give us a warning saying that db view is not available anymore and if we want to replace it with another one. YF still thinks that BQ db view is there. When I try to access the data of non-existing BQ db view, YF does not show any data, naturally. (Because there is no such a db view)

I think something went wrong in YF repository db and the non-existing db views are still considered as existing.

Attached is a screenshot showing YF repository db's reportview table. viewid 1940064 is the new view I that want to use and 1914445 is the db view that does not exist anymore in BQ. However this db view still has a status OPEN. I assume that this status should not be OPEN, right?

I did this process in Posgresql db and it worked as expected. It might be related to BQ data source connector.

How can I get over with this problem? Is there somehow caching in YF repo db that did not detect deletion on BQ db view? Can I change simply the status of non existing db view to something other than OPEN without breaking other things?

Comments (6)

photo
1

Hey guys, any updates on this?

photo
1

Hi Vefa,


Thanks for your patience on this one, I'm aware that the UK team is out on a national holiday and hasn't been able to get back to you.


At this stage, we would need to spin up a Google Cloud instance and test this behaviour in order for us to record the defective behaviour, which may take us a little while longer.


Again, apologies for the delay and we will reach out with an update shortly.


Kind regards,

Simon

photo
1

Hi Vefa,


I'm sorry for the delay in this one, I'm just reaching out to let you know that I am looking after this one from Australia. I'm in the process of acquiring a GCP account to test this on, which shouldn't take too much longer.


Apologies for the delay.


Kind regards,

Simon

photo
1

Hi Vefa,


Thanks for your patience on this one, and sorry for the delay in getting back to you. I've finally managed to get some movement on getting it tested.


I've gone ahead and attempted replicating the behaviour that you reported, without much luck. I'm currently using version 9.6, and I'm using the JDBC 4.1 Google Big Query Driver - where I believe the behaviour is what you'd expect. I created a connection to the Big Query Data Source, creating a Yellowfin View with a dataset 'beer' and a view of 'beer-v2'.


I then removed the Big Query View, where upon editing the Yellowfin View I was not able to view the results of the Big Query View 'beer-v2'.


/e1e9c86f2cb2102a995b0fe40af93a30


I then added the View again, where the namespace for the Big Query View is saved in Yellowfin, but the fields are added again.


/91818bdbfc860ac97355d335c8198f0d


Is this similar to the behaviour you experienced, or is there something that I am missing?


Which version of Yellowfin are you currently using, which JDBC Driver are you using and which Connection method with Yellowfin are you using to make a connection to Big Query? For example, I am using Yellowfin version 9.6, using the Simba JDBC 4.1 Driver and using the Generic JDBC Data Source Connection method within Yellowfin.


Kind regards,

Simon

photo
1

Hi Vefa,


I hope you're well!


I just wanted to check in with you to see how you are traveling with this issue. I haven't been able to replicate your exact behaviour, and I just wanted to check the Connection settings that you were using so I can confirm I am replicating it correctly. Otherwise, if this is no longer an issue let me know and I can go ahead and mark this as closed for the moment.


Kind regards,

Simon

photo
1

Hi Vefa,


I'm just messaging to let you know that I'm going to be marking this as closed, due to inactivity and due to the fact that I was unable to replicate this in house.


Please feel free to reach out to re-open this issue if you have any additional information that may help us to replicate this and I'll get back to you shortly.


Kind regards,

Simon