yellowfin silent installation mysql db getting overwritten

Hi Ravindra,

Thanks for reaching out to Yellowfin support.

Based on the configuration provided in the install.properties file, it looks like the CreateYellowfinDB parameter is set to false. This parameter specifies whether to create a new Yellowfin database during the installation process. Setting this parameter to false means that the installation process will not create a new database and will only connect to the existing one.

However, it is possible that the data in the existing database is being wiped off because the CreateYellowfinDBUser parameter is set to true. This parameter specifies whether to create a new Yellowfin database user during the installation process. If this parameter is set to true, the installation process will create a new user with the username and password specified in the DatabaseUser and DatabasePassword parameters, respectively. If a user with the same username already exists in the database, the installation process may overwrite it and remove any associated data.

To avoid overwriting the existing user and data, set the CreateYellowfinDBUser parameter to false. This will instruct the installation process to use the existing user and password specified in the DatabaseUser and DatabasePassword parameters, respectively.

In summary, make sure the following parameters are set as below in the install.properties file:

CreateYellowfinDB=false
CreateYellowfinDBUser=false

This should ensure that Yellowfin only connects to the existing MySQL database and does not overwrite any data.

Regards,

Sri Vamsi

I have shared the install.properties file, which has both the parameters, as mentioned by you, set to false, even then db is getting overwritten. can you check;

CreateYellowfinDB=false

CreateYellowfinDBUser=false

I have shared the install.properties file, which has both the parameters, as mentioned by you, set to false, even then db is getting overwritten. can you check;

CreateYellowfinDB=false

CreateYellowfinDBUser=false

Hi Ravindra,

If the CreateYellowfinDB and CreateYellowfinDBUser parameters are set to false in the install.properties file, it should not create a new database or user during the Yellowfin installation process. However, if you are still seeing the database being overwritten, it is possible that the issue is not with the installation process itself, but with the script or process that is being used to run the installation.

It's also possible that the issue is related to the existing MySQL database configuration. For example, if the existing database has been configured to allow DROP TABLE commands or other destructive SQL statements, it could be causing the data to be wiped out during the Yellowfin installation process.

To troubleshoot this issue, I would recommend reviewing the MySQL server logs to see if there are any errors or warnings related to the Yellowfin installation process and share it with us. And also can you try running the installation process with verbose logging enabled to see if there are any clues as to what is causing the issue.

Yes, to enable verbose logging during the Yellowfin installation process, you can add the -debug flag to the command you are using to run the installation. For example:

java -jar yellowfin-8.0.11-20220223-full.jar -silent install.properties -debug

This will provide additional output in the console during the installation process, which may help to identify any issues that are causing the database to be overwritten.

You can also specify a log file to capture the output by adding the -logfile flag to the command, followed by the path and name of the log file. For example:

java -jar yellowfin-8.0.11-20220223-full.jar -silent install.properties -debug -logfile /var/log/yellowfin_install.log

This will create a log file at /var/log/yellowfin_install.log and write all output to that file, in addition to the console output.

In addition, you may want to consider using a database backup solution to create a backup of the existing MySQL database before running the Yellowfin installation process. This will allow you to restore the database to its previous state if the Yellowfin installation process does end up overwriting the data.

Regards,

Sri Vamsi

I tried all the options provided above, but the DB is still getting overwritten.

Below is the console log:

[root@ct-cp-yellowfin-prod-407812 tmp]# java -jar yellowfin-8.0.11-20220223-full.jar -silent install.properties -debug -logfile /var/log/yellowfin_install.log
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
Feb 17, 2023 5:11:01 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Feb 17, 2023 5:11:02 PM liquibase.changelog
INFO: Reading from yellowfin.DATABASECHANGELOG
Feb 17, 2023 5:11:02 PM liquibase.lockservice
INFO: Successfully released change log lock
Feb 17, 2023 5:11:02 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Skipping auto-registration
Feb 17, 2023 5:11:02 PM liquibase.hub
WARNING: Skipping auto-registration
Feb 17, 2023 5:11:02 PM liquibase.changelog
INFO: Continuing past: com/hof/patch/db-schema-changelog-8.0.xml::YFN-21641 Increase FormatCode to fit encrypted passwords for SQL Server::pd despite precondition failure due to onFail='CONTINUE':
com/hof/patch/db-schema-changelog-master.xml : DBMS Precondition failed: expected mssql, got mysql

Feb 17, 2023 5:11:02 PM liquibase.lockservice
INFO: Successfully released change log lock
Feb 17, 2023 5:11:21 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Feb 17, 2023 5:11:21 PM liquibase.changelog
INFO: Reading from yellowfin.DATABASECHANGELOG
Feb 17, 2023 5:11:21 PM liquibase.lockservice
INFO: Successfully released change log lock
Feb 17, 2023 5:11:21 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Skipping auto-registration
Feb 17, 2023 5:11:21 PM liquibase.hub
WARNING: Skipping auto-registration
Feb 17, 2023 5:11:21 PM liquibase.lockservice
INFO: Successfully released change log lock
Feb 17, 2023 5:11:23 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Feb 17, 2023 5:11:23 PM liquibase.changelog
INFO: Reading from yellowfin.DATABASECHANGELOG
Feb 17, 2023 5:11:23 PM liquibase.lockservice
INFO: Successfully released change log lock
Feb 17, 2023 5:11:23 PM liquibase.lockservice
INFO: Successfully acquired change log lock
Skipping auto-registration
Feb 17, 2023 5:11:23 PM liquibase.hub
WARNING: Skipping auto-registration
Feb 17, 2023 5:11:23 PM liquibase.changelog
INFO: null
Feb 17, 2023 5:11:23 PM liquibase.changelog
INFO: ChangeSet com/hof/util/liquibase/db-runalways.xml::YFN-9269 Insert schema version and schema release in Configuration::cesar ran successfully in 34ms
Feb 17, 2023 5:11:23 PM liquibase.changelog
INFO: Continuing past: com/hof/util/liquibase/db-runalways.xml::YFN-9269 Update schema version and schema release in Configuration::cesar despite precondition failure due to onFail='CONTINUE':
com/hof/installer/process/job/liquibase/db-customclasses-installer-master.xml : Expected changelog property 'installer.type' to have a value of 'patcher'. Got 'installer'

Feb 17, 2023 5:11:23 PM liquibase.lockservice
INFO: Successfully released change log lock

I dont see any /var/log/yellowfin_install.log file getting created with above command.

[root@ct-cp-yellowfin-prod-407812 tmp]# ls /var/log | grep yellowfin
[root@ct-cp-yellowfin-prod-407812 tmp]#

Hi Ravindra,

Thanks for your reply.

Can you please share the Yellowfin logs and jdbc logs. I would like to check if this error related to YF or your config DB.

Based on the console log you provided, it seems that the Liquibase changelog is not properly configured for your MySQL database. The log shows a warning about a deprecated MySQL driver and a precondition failure due to the expected DBMS of 'mssql' but getting 'mysql' instead.

Regards,

Sri Vamsi

Hi Vamsi

I had cleaned up the directory before running install command. So, when I run the command I get the above console log. and do not see anything in logs directory:

[root@ct-cp-yellowfin-prod-407812 ~]# cd /opt/yellowfin/appserver/logs/
[root@ct-cp-yellowfin-prod-407812 logs]# ls -lrta
total 4
-rw-------. 1 root root 0 Feb 17 17:15 safeToDelete.tmp
drwx------. 2 root root 30 Feb 17 17:15 .
drwx------. 8 root root 4096 Feb 17 17:16 ..

[root@ct-cp-yellowfin-prod-407812 logs]# pwd
/opt/yellowfin/appserver/logs

Hi Ravindra,

I would like you to try the below steps based on your console log errors and let me know :

Error 1:

. INFO: Continuing past: com/hof/patch/db-schema-changelog-8.0.xml::YFN-21641 Increase FormatCode to fit encrypted passwords for SQL Server::pd despite precondition failure due to onFail='CONTINUE':

com/hof/patch/db-schema-changelog-master.xml : DBMS Precondition failed: expected mssql, got mysql

-- Based on the console log you provided below, the Liquibase update process is failing due to a DBMS precondition failure. The log indicates that the expected DBMS (Database Management System) is "mssql", but the actual DBMS is "mysql".

This error typically occurs when there is a mismatch between the expected database type and the actual database type. In your case, it seems that the Liquibase changelog is configured for a Microsoft SQL Server database, but you are running the update on a MySQL database.

Here are the steps you can follow to fix this issue:

1. Open the Liquibase changelog file that is causing the issue (in this case, it is com/hof/patch/db-schema-changelog-master.xml).
2. Look for the DBMS precondition that is causing the failure. In this case, it is the precondition for "mssql".
3. Update the DBMS precondition to match your MySQL database. You can do this by changing the precondition from "mssql" to "mysql".
4. Save the changelog file.
5. Run the Liquibase update process again and verify that it completes without any errors.

By updating the DBMS precondition to match your MySQL database, you should be able to resolve the issue and successfully update your Yellowfin installation.

Error 2:

INFO: Continuing past: com/hof/util/liquibase/db-runalways.xml::YFN-9269 Update schema version and schema release in Configuration::cesar despite precondition failure due to onFail='CONTINUE':
com/hof/installer/process/job/liquibase/db-customclasses-installer-master.xml : Expected changelog property 'installer.type' to have a value of 'patcher'. Got 'installer'

Based on the console log you provided, the Liquibase update process is failing due to a precondition failure. The log indicates that the expected value for the "installer.type" property is "patcher", but the actual value is "installer".

This error typically occurs when there is a mismatch between the expected value and the actual value for a property or configuration parameter in the Liquibase changelog.

To resolve this issue, you need to update the Liquibase changelog to set the correct value for the "installer.type" property. This property is used to specify the type of installer being used, and it needs to be set to "patcher" in order for the Liquibase update process to work correctly.

Here are the steps you can follow to fix this issue:

1. Open the Liquibase changelog file that is causing the issue (in this case, it is com/hof/installer/process/job/liquibase/db-customclasses-installer-master.xml).
2. Look for the "installer.type" property and verify that its value is set to "patcher".
3. If the value is not set to "patcher", update the value to "patcher" and save the changelog file.
4. Run the Liquibase update process again and verify that it completes without any errors.

Could you please try these steps to resolve the errors in console log and try it again. If possible please share the recorded video and share it in our FTP server https://ftp.yellowfin.bi/files/

Regards,

Sri Vamsi

Hi Ravindra,

Hope things are good on your end. Just wanted to check in to make sure you had what you needed here.

Regards,

Sri Vamsi

Hi Vamsi, it is tough to replicate so many steps while doing automation. I think yellowfin should support the mysql connector and java versions out of the box.

To proceed with automation and remove all these efforts, I have taken the workaround way of using a dummy db while creating and then updating web.xml to connect to the existing DBs.

Ideally, updating CreateYellowfinDB and CreateYellowfinDBUser parameters to false in the install.properties file, should have been enough and these issues should not have cropped up.

Hi Ravindra,

Thank you for getting back to me.

That's great to hear! If that's all good, I'll go ahead and mark this as complete?

Regards,

Sri Vamsi

Hi Ravindra,

I'm going to go ahead and mark this ticket as Completed due to inactivity at this time. Feel welcome to reach out in the future.

Regards,

Sri Vamsi