Ability to execute host machine scripts from within ETL tasks

Gerhard Mitterlechner shared this idea 10 months ago
Idea Logged

I achieved to call a shell-script (starting our ETL) from a custom JSP (via "Runtime.getRuntime().exec"), but it is not really usable, as the webapp runs under a different OS user as out shell-script on the OS must be started, and I found no way to "switch user" via JSP and especially to provide the password for the ETL-user.

We now did a workaround: a periodic controller ETL polls the "etlrunlog" table and "listens" for any entries there. If a special dummy transformation on the YF gets executed, the controller ETL checks the transformation name and timestamps and starts the actual ETL when the dummy transformation was executed and therefore triggers an insert in "etlrunlog" table.

However, this is far from elegant. The advantage is that the YF user does not have to switch to a customized JSP/HTML and that we do not have to use the VALIDATEUSER function (which I was not able to understand from the examples).

A very helpful solution would be that yellowfin would provide a transformation-flow step like "call OS script" or similar, which would allow to execute a batch/shell script on the underlying machine. Ideally, crendentials of the OS user could be configured, in case the shell-script should not get started as the same user as the YF server runs under. Are there any plans / is there any possibility to obtain such a step?

Replies (1)

photo
1

Hi Gerhard,

I've created this public Idea post so the community can view and vote for your idea. If it proves popular, that might encourage our developer team to prioritise it! You should also be able to track any updates and follow its progress from here.

Kind regards,

Chris

photo
1

Thanks a lot, greatly appreciate your effort.

photo
1

No worries. I'll move this over to Idea Logged while we wait to hear more.

photo
Leave a Comment
 
Attach a file