BigQuery driver upload problem

Hi Kerem,

Thanks for reaching out. Are you attempting this with the JDBC 4.2-compatible BigQuery drivers? If so, this is a known defect and I recommend you instead try one of the JDBC 4.1-compatible JDBC drivers instead. You can find these in this Google article.

Simply expand the Previous JDBC Versions option:

And select the JDBC 4.1-compatible option instead.

If you are already attempting this with the 4.1 driver then please re-attempt replication by getting back to the error and taking note of the timestamp for when the error occurred and subsequently provide a compressed copy of your entire logs folder, located at <YellowfinInstall>/appserver/logs, along with said timestamp, so we can investigate further.

Please let me know how goes.

Regards,

Mike

Hi Mike,

It seems the BQ JBDC 4.1 driver is currently not available. When trying to download it we receive:

Is there another way to get the 4.2 version running or retrieve the 4.1 version?

Thanks!

Kr. Machiel

Hi Machiel,

I thought I would jump in here, I have a working plugin file that I've been using in my Google Big Query instance in version 9.4.2, you can find it at this link.

Let me know how you go!

Kind regards,

Simon

Hello,

Our main problem is driver size I guess, If a try to upload random jar or yfx files lower than 1mm it doesn't give "Submit error 413" error. How can we configure max file size for a plugin files?

Regards,

Hi Simon,

We eventually managed to upload your file by unpacking it and uploading the `GoogleBigQueryJDBC41.jar`.

Could you please inform us on when you expect to fix the issue with the 4.2 versioned driver? I checked in with Google and they advice against using the 4.1 driver as JDBC 4.1 support has been dropped since September 2019 and we miss out on quite some bug fixes.

https://storage.googleapis.com/simba-bq-release/jdbc/release-notes.txt

Regards, Machiel

Hi All,

I'm glad to hear that you managed to upload the 4.1 Big Query Driver.

I have gone ahead and added you to an existing defect task relating to the 4.2 version of the Driver, and I will reach out as soon as it has been assessed by Developers and there is something to update you on.

In the meantime, please don't hesitate to reach out with any further questions.

Kind regards,

Simon

Hi guys,

We do have an additional question: just uploading "GoogleBigQueryJDBC41.jar" does not entirely seem to cut it as we now run into following error when testing the connection:

java.lang.NoClassDefFoundError: com/google/api/client/json/JsonFactory

It seems we have trouble uploading any file larger than 1MB through the plugin manager.

Do you know if there is a property anywhere in YF's config which limit's uploads to 1MB?

Thanks!

Hi guys,

Please ignore my question above, I managed to fix the 413 error by increasing the max allowed payload on our Nginx ingress.

Unfortunately this brings us to the following issue: the package now seems to upload properly, after which the status changes too "Processing file". This process takes about 60 sec. after which it is terminated with a "Submut error(504)".

Any change you are familiar with this issue and how can troubleshoot it? It seems the Yellowfin instance takes a little while (>60s) to install the package, but that causes a time out.

Ps. if there is any backdoor we can use to install these drivers (instead of using the plugin manager) that might be very helpful.

Hi all,

Please ignore my last 2 comments. Both were caused by configuration on our infrastructure.

We managed to upload the 4.1 driver and setup the connection to BigQuery.

Thanks for your support.

Kind regards,

Machiel

Hi Machiel,

That's great to hear! Thanks for letting us know. This considered, I'll go ahead and close this case out, but please don't hesitate to reach out with any other questions or concerns.

Regards,

Mike

Hi Mike,

Sure no problem! Is there any way we can track progress on the defect task relating to the 4.2 version of the Driver?

Regards, Machiel

Hi Machiel,

Yes, sorry about that! I'll set this back to Defect Logged instead so I can keep you posted on any progress made with this task.

Regards,

Mike

Hi Mike,

Any updates on the defect regarding the 4.2 version of the driver? A.t.m. we are running into the first issues where booleans are not properly converted and we expect such issues to be solved when using a more recent driver.

Thanks!

Kind regards,

Machiel

Hi Machiel,

This is actively being looked at by the dev team. A fix was created and is currently in testing and as such, is targeted for next release 9.6. I'll let you know once testing is completed and that's officially, however.

Regards,

Mike