Yellowfin issues - disconnection troubles
Hi,
I am a part of the Sample6 team and we faced a few problems on our yellowfin(YF) server.
We use AWS EC2(m4.large) and docker as a wrapper for YF.
1. After some time(about ~24 hours or less, It was about 5-6 times), We cannot login into YF.
Login page works but after the login operation - a white screen is appeared and we are not able to check reports.
I've attached some logs from our server and the log file has a lot of different exceptions.
We have to restart tomcat which is not a good solution for us.
Now, it has been working for four days without any issues, however we would to know what might be ajusted to be able to avoid the problem in the future.
2. Chrome browser had some errors in the browser console and endless title 'Creating reports.. please wait' is appeared.
It was after restart YF server twice.
A few screenshots with errors were attached.
3. We also have some performance issues. For the issue we have increased -xmx parameter from 1024mb to 4096mb. What else we can adjust just to be able to use YF without performance issues?
Thanks in advance
The same problem 
Hello Ivan,
AWS installations can be tricky to troubleshoot this type of issue, however, I think we have a couple of changes that will help here.
I assume that your database is either on a different AWS server or on a different host entirely, this in turn meas that the connection between the database server and the yellowfin server may have timed out, gone bad or been closed.
the links below have instructions on how to enable "Volatile Sources" and "JDBC Verify"
These settings will check for an active connection and re-establish contact if it's not successful.
Some of our other AWS customers have had good success with these settings, so would be worth trying.
http://community.yellowfinbi.com/knowledge-base/article/jdbc-verify
http://community.yellowfinbi.com/knowledge-base/article/volatile-data-sources-what-is-it-and-when-do-i-need-it
I hope this helps,
Please let us know if this resolves your issue
Best regards,
Pete
Hello Ivan,
AWS installations can be tricky to troubleshoot this type of issue, however, I think we have a couple of changes that will help here.
I assume that your database is either on a different AWS server or on a different host entirely, this in turn meas that the connection between the database server and the yellowfin server may have timed out, gone bad or been closed.
the links below have instructions on how to enable "Volatile Sources" and "JDBC Verify"
These settings will check for an active connection and re-establish contact if it's not successful.
Some of our other AWS customers have had good success with these settings, so would be worth trying.
http://community.yellowfinbi.com/knowledge-base/article/jdbc-verify
http://community.yellowfinbi.com/knowledge-base/article/volatile-data-sources-what-is-it-and-when-do-i-need-it
I hope this helps,
Please let us know if this resolves your issue
Best regards,
Pete
Hi Pete,
Thank you for the useful information.
We've applied these settings on our server. I will let you know after a few days or a week about our results.
Regards,
Ivan
Hi Pete,
Thank you for the useful information.
We've applied these settings on our server. I will let you know after a few days or a week about our results.
Regards,
Ivan
Thanks for the update Ivan!
I'll keep this ticket in "Awaiting reply" until you give me another update.
Best regards,
Pete
Thanks for the update Ivan!
I'll keep this ticket in "Awaiting reply" until you give me another update.
Best regards,
Pete
Hi Peter,
We have restarted YF server about 4 times during last 12 days.
Looks like we have the same exceptions in the logs(attached).
What else might be changed in our configuration?
Hi Peter,
We have restarted YF server about 4 times during last 12 days.
Looks like we have the same exceptions in the logs(attached).
What else might be changed in our configuration?
Hello Ivan,
Thanks for getting back to me. Sorry to see that you are still having issues with the install on AWS.
I am escalating this to my colleagues in support who may have seen this issue before.
As soon as we have some further information for you, we'll update the ticket.
Thanks for your patience,
Best regards,
Pete
Hello Ivan,
Thanks for getting back to me. Sorry to see that you are still having issues with the install on AWS.
I am escalating this to my colleagues in support who may have seen this issue before.
As soon as we have some further information for you, we'll update the ticket.
Thanks for your patience,
Best regards,
Pete
Hello Ivan,
Thanks for being so patient on waiting for a reply!
One of my colleagues got back to me with this:
Hey Pete,
I looked through their 2 catalina.out files and in both of them I saw that before they had to restart tomcat there were lots of strange exceptions that I've never seen before, and they all seemed to stem from the following one:
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalStateException: Unexpected state: headers already parsed. Buffer not recycled?
which led to many more exceptions like the following:
SEVERE [http-nio-8080-exec-1] org.apache.coyote.http11.Http11Processor.service Error processing request
java.lang.NullPointerException
There are plenty of forum posts on the 'net about these exceptions. I've read through a few of them and it gets a little confusing because the Apache Tomcat support raise a bug for it, then they say it's fixed, but then there's a later bug raised for the same exception. Here are some of the Apache Tomcat bug IDs I found that were raised for that exception: 58646, 59649, 60409, 60918
Suffice to say, it is not a Yellowfin bug, it is an Apache Tomcat bug, and going by all those notes for the various bugs, it looks like it's something they're aware of and keep trying to fix, so I guess all we can say is to upgrade Tomcat to the latest build and hope for the best. I notice the build they're using is 1 year old so it wouldn't do any harm to get the latest anyway.
So, the next step would be to looking into upgrading Tomcat to the latest version - We have a guide for this!
The guide covers an older version, but the process is the same.
https://community.yellowfinbi.com/knowledge-base/article/how-to-upgrade-tomcat
Please let me know how you get on,
Best regards,
Pete
Hello Ivan,
Thanks for being so patient on waiting for a reply!
One of my colleagues got back to me with this:
Hey Pete,
I looked through their 2 catalina.out files and in both of them I saw that before they had to restart tomcat there were lots of strange exceptions that I've never seen before, and they all seemed to stem from the following one:
org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level.
java.lang.IllegalStateException: Unexpected state: headers already parsed. Buffer not recycled?
which led to many more exceptions like the following:
SEVERE [http-nio-8080-exec-1] org.apache.coyote.http11.Http11Processor.service Error processing request
java.lang.NullPointerException
There are plenty of forum posts on the 'net about these exceptions. I've read through a few of them and it gets a little confusing because the Apache Tomcat support raise a bug for it, then they say it's fixed, but then there's a later bug raised for the same exception. Here are some of the Apache Tomcat bug IDs I found that were raised for that exception: 58646, 59649, 60409, 60918
Suffice to say, it is not a Yellowfin bug, it is an Apache Tomcat bug, and going by all those notes for the various bugs, it looks like it's something they're aware of and keep trying to fix, so I guess all we can say is to upgrade Tomcat to the latest build and hope for the best. I notice the build they're using is 1 year old so it wouldn't do any harm to get the latest anyway.
So, the next step would be to looking into upgrading Tomcat to the latest version - We have a guide for this!
The guide covers an older version, but the process is the same.
https://community.yellowfinbi.com/knowledge-base/article/how-to-upgrade-tomcat
Please let me know how you get on,
Best regards,
Pete
Hi Pere,
We're going to try out the guide. I've read it and I found that item 9 is unclear for me.
Open the \Yellowfin\conf\server.xml file and remove the lines below. Then save.
There are no lines below the text.
Regrads,
Ivan
Hi Pere,
We're going to try out the guide. I've read it and I found that item 9 is unclear for me.
Open the \Yellowfin\conf\server.xml file and remove the lines below. Then save.
There are no lines below the text.
Regrads,
Ivan
Hi Ivan,
Sorry looks like the lines are stripped out of the article cause of the tags.
Here are the 2 lines to remove from the server.xml (until I can figure out how to update that article)
Please let us know how it all goes.
Regards,
David
Hi Ivan,
Sorry looks like the lines are stripped out of the article cause of the tags.
Here are the 2 lines to remove from the server.xml (until I can figure out how to update that article)
Please let us know how it all goes.
Regards,
David
Peter, David,
Thank you for your support. We've upgraded our tomcat to 8.5.16.
We will continue to monitor the issues and let you know after a few days or a week about our results
Regards,
Ivan
Peter, David,
Thank you for your support. We've upgraded our tomcat to 8.5.16.
We will continue to monitor the issues and let you know after a few days or a week about our results
Regards,
Ivan
Hello,
That's great news Ivan!
I'll await an update :)
Have a great day,
Pete
Hello,
That's great news Ivan!
I'll await an update :)
Have a great day,
Pete
Hello,
It’s been over 14 days since you updated this ticket so we are closing it down now.
If you still have an issue that is relevant to this ticket, please respond here and the ticket will re-open.
If you have any other issues, please raise a new case.
Best regards,
Yellowfin Support
Hello,
It’s been over 14 days since you updated this ticket so we are closing it down now.
If you still have an issue that is relevant to this ticket, please respond here and the ticket will re-open.
If you have any other issues, please raise a new case.
Best regards,
Yellowfin Support
Hi Peter,
Sorry for that. It has been working for 2 month without any problems. Thanks you.
Regards,
Ivan
Hi Peter,
Sorry for that. It has been working for 2 month without any problems. Thanks you.
Regards,
Ivan
Replies have been locked on this page!