Yellowfin issues - disconnection troubles

Ivan Barinov shared this problem 2 years ago
Resolved

Hi,

I am a part of the Sample6 team and we faced a few problems on our yellowfin(YF) server.

We use AWS EC2(m4.large) and docker as a wrapper for YF.


1. After some time(about ~24 hours or less, It was about 5-6 times), We cannot login into YF.

Login page works but after the login operation - a white screen is appeared and we are not able to check reports.

I've attached some logs from our server and the log file has a lot of different exceptions.

We have to restart tomcat which is not a good solution for us.

Now, it has been working for four days without any issues, however we would to know what might be ajusted to be able to avoid the problem in the future.


2. Chrome browser had some errors in the browser console and endless title 'Creating reports.. please wait' is appeared.

It was after restart YF server twice.

A few screenshots with errors were attached.

3. We also have some performance issues. For the issue we have increased -xmx parameter from 1024mb to 4096mb. What else we can adjust just to be able to use YF without performance issues?

Thanks in advance

Comments (12)

photo
1

Hello Ivan,


AWS installations can be tricky to troubleshoot this type of issue, however, I think we have a couple of changes that will help here.


I assume that your database is either on a different AWS server or on a different host entirely, this in turn meas that the connection between the database server and the yellowfin server may have timed out, gone bad or been closed.


the links below have instructions on how to enable "Volatile Sources" and "JDBC Verify"

These settings will check for an active connection and re-establish contact if it's not successful.


Some of our other AWS customers have had good success with these settings, so would be worth trying.

http://community.yellowfinbi.com/knowledge-base/article/jdbc-verify

http://community.yellowfinbi.com/knowledge-base/article/volatile-data-sources-what-is-it-and-when-do-i-need-it


I hope this helps,

Please let us know if this resolves your issue

Best regards,

Pete

photo
1

Hi Pete,


Thank you for the useful information.

We've applied these settings on our server. I will let you know after a few days or a week about our results.


Regards,

Ivan

photo
1

Thanks for the update Ivan!


I'll keep this ticket in "Awaiting reply" until you give me another update.


Best regards,

Pete

photo
1

Hi Peter,


We have restarted YF server about 4 times during last 12 days.

Looks like we have the same exceptions in the logs(attached).

What else might be changed in our configuration?

photo
2

Hello Ivan,


Thanks for getting back to me. Sorry to see that you are still having issues with the install on AWS.


I am escalating this to my colleagues in support who may have seen this issue before.

As soon as we have some further information for you, we'll update the ticket.


Thanks for your patience,

Best regards,

Pete

photo
1

Hello Ivan,


Thanks for being so patient on waiting for a reply!


One of my colleagues got back to me with this:


Hey Pete,

I looked through their 2 catalina.out files and in both of them I saw that before they had to restart tomcat there were lots of strange exceptions that I've never seen before, and they all seemed to stem from the following one:

org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header

Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.

java.lang.IllegalStateException: Unexpected state: headers already parsed. Buffer not recycled?

which led to many more exceptions like the following:

SEVERE [http-nio-8080-exec-1] org.apache.coyote.http11.Http11Processor.service Error processing request

java.lang.NullPointerException

There are plenty of forum posts on the 'net about these exceptions. I've read through a few of them and it gets a little confusing because the Apache Tomcat support raise a bug for it, then they say it's fixed, but then there's a later bug raised for the same exception. Here are some of the Apache Tomcat bug IDs I found that were raised for that exception: 58646, 59649, 60409, 60918

Suffice to say, it is not a Yellowfin bug, it is an Apache Tomcat bug, and going by all those notes for the various bugs, it looks like it's something they're aware of and keep trying to fix, so I guess all we can say is to upgrade Tomcat to the latest build and hope for the best. I notice the build they're using is 1 year old so it wouldn't do any harm to get the latest anyway.


So, the next step would be to looking into upgrading Tomcat to the latest version - We have a guide for this!

The guide covers an older version, but the process is the same.

https://community.yellowfinbi.com/knowledge-base/article/how-to-upgrade-tomcat


Please let me know how you get on,

Best regards,

Pete

photo
1

Hi Pere,


We're going to try out the guide. I've read it and I found that item 9 is unclear for me.

Open the \Yellowfin\conf\server.xml file and remove the lines below. Then save.


There are no lines below the text.


Regrads,

Ivan

photo
1

Hi Ivan,


Sorry looks like the lines are stripped out of the article cause of the tags.

Here are the 2 lines to remove from the server.xml (until I can figure out how to update that article)


  <!--Initialize Jasper prior to webapps are loaded. Documentation at /docs/jasper-howto.html -->
  <Listener className="org.apache.catalina.core.JasperListener" />
Please let us know how it all goes.


Regards,

David

photo
1

Peter, David,


Thank you for your support. We've upgraded our tomcat to 8.5.16.

We will continue to monitor the issues and let you know after a few days or a week about our results


Regards,

Ivan

photo
1

Hello,


That's great news Ivan!


I'll await an update :)


Have a great day,

Pete

photo
1

Hello,


It’s been over 14 days since you updated this ticket so we are closing it down now.


If you still have an issue that is relevant to this ticket, please respond here and the ticket will re-open.

If you have any other issues, please raise a new case.


Best regards,

Yellowfin Support

photo
1

Hi Peter,

Sorry for that. It has been working for 2 month without any problems. Thanks you.

Regards,

Ivan

Comments have been locked on this page!