What's the right way to implement and export new custom hook?

1,328 views
Skip to first unread message

Brenda Bell

unread,
Feb 22, 2016, 12:03:07 PM2/22/16
to Airflow
I've implemented a new hook as follows (first source file for the hook and second one for the plugin):

import configparser
import logging

from airflow.utils import AirflowException
from airflow.hooks.base_hook import BaseHook

import snowflake.connector


def _parse_snowflake_config(config_file_name, profile=None):
    Config = configparser.ConfigParser()
    [snip]

class SnowflakeHook(BaseHook):
    def __init__(
            self,
            snowflake_conn_id='snowflake_default'):
        super(SnowflakeHook, self).__init__()
        [snip]

    def get_conn(self):
        snowflake_config = _parse_snowflake_config(self.snowflake_config_file, self.profile)
        connection = snowflake.connector.Connect(**snowflake_config)


from airflow.executors.base_executor import BaseExecutor
from airflow.hooks.base_hook import BaseHook
from airflow.models import  BaseOperator
from airflow.plugins_manager import AirflowPlugin

from flask import Blueprint
from flask.ext.admin import BaseView, expose
from flask.ext.admin.base import MenuLink

from snowflake_hook import SnowflakeHook

class SnowflakeOperator(BaseOperator):
    pass

class SnowflakePlugin(AirflowPlugin):
    name = "snowflake_plugin"
    operators = []
    flask_blueprints = []
    hooks = [SnowflakeHook]
    executors = []
    admin_views = []
    menu_links = []


Both files are in ${AIRFLOW_HOME}/plugins with permissions set to 0755. I relaunched airflow to pick up the new plugins and the both compiled (I see the .pyc's and there are no compilation errors in the console). I'm pretty sure they're being loaded because I did see syntax errors in the console before I corrected the bugs.

When I try to create a new connection, I don't see my hook listed. Am I misunderstanding how this works?

Reply all
Reply to author
Forward
0 new messages