Oozie with saved Hive query: custom parameters doesn't get passed

20 views
Skip to first unread message

Saul

unread,
Aug 2, 2017, 8:19:39 AM8/2/17
to Hue-Users
Hello all,

I have a saved Hive query with several external parameters specified. The parameters are recognized at Hive Editor and I'm able to run the query there without any problems. 

Example of such query: 
 
INSERT INTO TABLE custom_table SELECT * FROM table WHERE date = ${custom_date}
 
where custom_date is the parameter passed.  

However, I'm not able to pass that custom parameter when I try to schedule the query with Oozie workflow. 

The approach I've tried:
1. Created a new workflow and selected the saved hive query to be ran.
2. In the Parameters section I'm able to add, for example, "custom_date=2017.01.01" as the parameter (the custom_date= is actually suggested when pressed on a blank field).
3. Save the workflow, both the workflow and the query finishes successfully. 

Problem: the set custom_date parameter is fixed to what was specified in the workflow. For example, submitting the same workflow, but changing the custom date in the pop-up window still runs with the parameter that was specified when creating the workflow. Therefore, "custom_date=" passes empty string as the parameter, even though in the pop-up submit window I specify some date. Additionally, "custom_date = ${another_parameter}" passes the string "${another_parameter}" to the query, but not the value of the other parameter. Again,  it is unaffected by changes in the pop-up submit workflow window.

Note: I've tried both HUE v3.12 and v4.0.1 with the same results, so I'm guessing I'm not using the correct syntax.

Question: what am I missing? What is the correct way of dynamically sending the parameters to hive from oozie? 

Thanks!

Saul

unread,
Aug 2, 2017, 10:00:09 AM8/2/17
to Hue-Users
Edit: apparently it works successfully when specifying an additional dummy parameter in the Parameters section when creating a workflow, i.e.:

custom_date=${dummy_custom}
dummy_custom
=dummy_custom

Now specifying custom value for dummy_custom sends the correct value to custom_date.

Is this a bug or is this intended as such?
Reply all
Reply to author
Forward
0 new messages