BigQuery driver upload problem

kerem shared this problem 20 days ago
Defect Logged

Hi,

I am trying to upload bigquery drivers but getting following error:

Any suggestions?


/b239be093cab0c5853d6d6bc80bbe235

Comments (12)

photo
1

Hi Kerem,

Thanks for reaching out. Are you attempting this with the JDBC 4.2-compatible BigQuery drivers? If so, this is a known defect and I recommend you instead try one of the JDBC 4.1-compatible JDBC drivers instead. You can find these in this Google article.

Simply expand the Previous JDBC Versions option:

/8171670cd23cb754ee4ca8e250a08214

And select the JDBC 4.1-compatible option instead.

If you are already attempting this with the 4.1 driver then please re-attempt replication by getting back to the error and taking note of the timestamp for when the error occurred and subsequently provide a compressed copy of your entire logs folder, located at <YellowfinInstall>/appserver/logs, along with said timestamp, so we can investigate further.

Please let me know how goes.

Regards,

Mike

photo
1

Hi Mike,

It seems the BQ JBDC 4.1 driver is currently not available. When trying to download it we receive:


/bdd45e9ba0e651eb6c73c266cf8052f5

Is there another way to get the 4.2 version running or retrieve the 4.1 version?

Thanks!

Kr. Machiel

photo
1

Hi Machiel,


I thought I would jump in here, I have a working plugin file that I've been using in my Google Big Query instance in version 9.4.2, you can find it at this link.


Let me know how you go!


Kind regards,

Simon

photo
1

Hello,


Our main problem is driver size I guess, If a try to upload random jar or yfx files lower than 1mm it doesn't give "Submit error 413" error. How can we configure max file size for a plugin files?

Regards,

photo
1

Hi Simon,

We eventually managed to upload your file by unpacking it and uploading the `GoogleBigQueryJDBC41.jar`.


Could you please inform us on when you expect to fix the issue with the 4.2 versioned driver? I checked in with Google and they advice against using the 4.1 driver as JDBC 4.1 support has been dropped since September 2019 and we miss out on quite some bug fixes.


https://storage.googleapis.com/simba-bq-release/jdbc/release-notes.txt


Regards, Machiel

photo
1

Hi All,


I'm glad to hear that you managed to upload the 4.1 Big Query Driver.


I have gone ahead and added you to an existing defect task relating to the 4.2 version of the Driver, and I will reach out as soon as it has been assessed by Developers and there is something to update you on.


In the meantime, please don't hesitate to reach out with any further questions.


Kind regards,

Simon

photo
1

Hi guys,

We do have an additional question: just uploading "GoogleBigQueryJDBC41.jar" does not entirely seem to cut it as we now run into following error when testing the connection:


java.lang.NoClassDefFoundError: com/google/api/client/json/JsonFactory
Probably because we are missing all the other libs from the SimbaJDBCDriverforGoogleBigQuery41_1.2.1.1001.yfp package.


It seems we have trouble uploading any file larger than 1MB through the plugin manager.


Do you know if there is a property anywhere in YF's config which limit's uploads to 1MB?


Thanks!

photo
1

Hi guys,

Please ignore my question above, I managed to fix the 413 error by increasing the max allowed payload on our Nginx ingress.


Unfortunately this brings us to the following issue: the package now seems to upload properly, after which the status changes too "Processing file". This process takes about 60 sec. after which it is terminated with a "Submut error(504)".


Any change you are familiar with this issue and how can troubleshoot it? It seems the Yellowfin instance takes a little while (>60s) to install the package, but that causes a time out.

Ps. if there is any backdoor we can use to install these drivers (instead of using the plugin manager) that might be very helpful.


/3bb12d52a5666a2ac02a5dd8fca26a9d

photo
1

Hi all,

Please ignore my last 2 comments. Both were caused by configuration on our infrastructure.

We managed to upload the 4.1 driver and setup the connection to BigQuery.

Thanks for your support.


Kind regards,

Machiel

photo
1

Hi Machiel,

That's great to hear! Thanks for letting us know. This considered, I'll go ahead and close this case out, but please don't hesitate to reach out with any other questions or concerns.

Regards,

Mike

photo
1

Hi Mike,

Sure no problem! Is there any way we can track progress on the defect task relating to the 4.2 version of the Driver?

Regards, Machiel

photo
1

Hi Machiel,

Yes, sorry about that! I'll set this back to Defect Logged instead so I can keep you posted on any progress made with this task.

Regards,

Mike