YF instance slow and unresponsive

Stephen Van Rooyen shared this problem 6 months ago
Resolved

Hi Guys,


We have a customer where there YF instance becomes unresponsive, they are running v7.4 build 20180515.

They previously updated to v7.4.6 and was told to rollback to the v7.4 build 20180515 build due to a bug in v7.4.6

Yellowfin instance hanging

There are no errors in the log files, apart from the memory leak warnings (see info pages and logs attached).

Previously I logged a similar call on the same version and I was told that the memory leaks warnings is not related.

YF instance unresponsive

Regards,

Stephen

Comments (9)

photo
1

Hi Stephen,


Apologies for not getting back you on this as yet, we will get to you in the next 24hrs.


Thanks,

David

photo
1

Hi David


Thanks for letting me know


Regards,

Stephen

photo
1

Hi Stephen,

Thanks for getting in touch. I have reviewed the previous tickets and I had mentioned that we should firstly consider upgrading the JVM size if there is still an issue with performance. Can you advise if this has been done as yet to see if this improves the performance?

Regards,

Paul

photo
1

Hi Paul,


This is 2 different enviroments, I just pointed out the similarities.

As you can see in this instance (by the included info.jsp page) there is almost 9GB of ram available to the JVM.


Regards,

Stephen

photo
1

Hi Stephen,

I have just been looking through your source jdbc logs and notice the following, which seems to point to connection issue with the JDBC database where it does not have sufficient connections available which I would suggest looking at, but perhaps increasing the number of connections to this data source. If Yellowfin is waiting for connections it will appear to look like it has hung.

2018-08-13 07:08:00 WARN: [70025] [352546] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:00 WARN: [70025] [352542] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:01 WARN: [70025] [352547] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:03 WARN: [70025] [352543] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:05 WARN: [70025] [352545] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:28 WARN: [70025] [352543] No connections available. Waiting for a connection to be returned.

2018-08-13 07:08:28 WARN: [70025] [352570] No connections available. Waiting for a connection to be returned.

2018-08-13 07:09:00 WARN: [70025] All connections are checked out and 1 threads are waiting!

2018-08-13 07:09:00 INFO: [70025] Connection[0] was checked out at 2018-08-13 07:08:29

2018-08-13 07:09:00 INFO: [70025] at com.hof.pool.DBConnectionPool.checkOutConnection(DBConnectionPool.java:1414)

2018-08-13 07:09:00 INFO: [70025] at com.hof.pool.DBConnectionManager.getConnection(DBConnectionManager.java:312)

2018-08-13 07:09:00 INFO: [70025] at com.hof.util.DBAction.<init>(DBAction.java:352)

2018-08-13 07:09:00 INFO: [70025] at com.hof.util.DBAction.<init>(DBAction.java:465)


2018-04-17 06:52:10 NOTICE: [70025] sourceName: Redshift (testdb)

2018-04-17 06:52:10 NOTICE: [70025] connectionMethod: JDBC

2018-04-17 06:52:10 NOTICE: [70025] min connections: 1

2018-04-17 06:52:10 NOTICE: [70025] max connections: 5


I have also noticed the following in the same source log, for which we have a KB on which you can find here.

2018-08-10 16:34:59 ERROR: [70025] Connection[3] is closed

2018-08-10 16:35:00 INFO: [70025] Connection[3] successfully reopened

2018-08-10 16:35:59 WARN: [70025] Execution problem occurred when clearing warnings: java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: This connection has been closed.

java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: This connection has been closed.

at java.util.concurrent.FutureTask.report(FutureTask.java:122)

at java.util.concurrent.FutureTask.get(FutureTask.java:206)

at com.hof.pool.JDBCConnection.C(JDBCConnection.java:391)

at com.hof.pool.JDBCConnection.D(JDBCConnection.java:371)

at com.hof.pool.DBConnectionPool.E(DBConnectionPool.java:1680)

at com.hof.pool.DBConnectionPool.A(DBConnectionPool.java:238)

at com.hof.pool.DBConnectionPool$_A.run(DBConnectionPool.java:2104)

at java.util.TimerThread.mainLoop(Timer.java:555)

at java.util.TimerThread.run(Timer.java:505)

Caused by: org.postgresql.util.PSQLException: This connection has been closed.

at org.postgresql.jdbc.PgConnection.checkClosed(PgConnection.java:803)

at org.postgresql.jdbc.PgConnection.clearWarnings(PgConnection.java:721)

at com.hof.pool.JDBCConnection$3.call(JDBCConnection.java:384)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)


Can I suggest you look at this jdbc data source and look at increasing the maximum connections and go through the JDBC Verify link provided above to see if this fixes the problem.

Thanks you,

Paul

photo
1

Hi Paul,


Thank for the feedback and suggestions, I have forwarded it on to the client.


Regards,

Stephen

photo
1

Hi Stephen,

Just touching base again as I have not heard back from you in regards to this and checking in to see how things are going and if you need any further help?

Regards,

Paul

photo
1

Hi Paul,


I think you can close it for now.


I sent through your suggestions to the customer, and have not heard and further complaints


Regards,

Stephen

photo
1

Thanks Stephen for getting back to me. Closing for now.