AWS Lambda Executor?

[Maximilian Roos]

Today Amazon announced the availability of AWS Lambda to run Python functions: https://aws.amazon.com/blogs/aws/aws-lambda-update-python-vpc-increased-function-duration-scheduling-and-more/

Would managing the running of these be within scope for Airflow? By managing, I think that would be having a LambdaExecutor, although I don't know Airflow well enough to know that for sure.

[Craig Kimerer]

What would you have the LambdaExecutor do?  You can still only invoke Lambda's with specific triggers (hit a URL, fetch data from stream, new file in s3, etc).

It could in theory trigger your lambda at a given time (using say the http endpoint option), but I am not sure what you're trying to gain from it.

[Maxime Beauchemin]

It seems like it would be easy to trigger an AWS Lambda from Airflow, either from a python or bash operator. Perhaps an AwsLambdaOperator would be useful for authentication and monitoring for the end of the execution. It would have to be community contributed as we don't have immediate use cases for it...

Max