Reducing Jar file size for AWS Lambda

[Jose Trigueros]

Hi all,

I've been using AWS Lambda for a small project. I've been leveraging the lein-clj-lambda plugin to build and upload the compiled Jar to S3 which is then used to update my Lambda function. Aside from the Jar generation, a lot of the time is spent uploading the Jar up to S3. The resulting Jar file is only about 12mb but does considerably slow down the feedback loop.

I have the following questions:

• Does anyone have any tips for reducing the size of a jar even more?
• I've tried excluding dependencies until the Jar broke, saved a few KBs
• I briefly looked into ProGuard, but the obfuscating process seemed to take some time which would negate any time saved on upload.
• Am I doing this wrong? Is there a way to mimic AWS Lambda locally?
• To mitigate this loop, I use the REPL to play with the individual functions until they feel right, but at some point you gotta test on the server, being far from perfect, I make a lot of mistakes so that's when the iteration happens.

Here's a link to the project.clj in question for reference.

I will say that using the lein-clj-lambda is already a huge win (before plugin it was `lein uberjar`, manually upload jar to S3, manually update AWS Lambda), but I'm wondering if there's an even better way.

-jvt

[Moe Aboulkheir]

Jose,

Leaving aside shrinking the jar, you could try:

 - Structuring your application so it's easier to construct plausible inputs in the tests / REPL

 - Using https://github.com/lambci/docker-lambda locally, to get an environment closer to the deployment target

 - Deploying to a geographically closer region, or triggering deployments on a build slave with better network connectivity

 - Deploying manually with awscli, as your package is small enough to be uploaded directly to lambda, rather than using S3

(With a strong bias for the first one - even if the your package was a third of the size, it's still going to be frustrating to deploy it and then get unexpected errors)

Take care,

Moe 

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jose Trigueros]

Hi Moe,

Thanks for your suggestions, yes, I've been mostly practicing the first option suggested. docker-lambda is a good option as well. It does help that Lambda provides test messages, which is what I've been using to test locally. Seems like the best way to go about it is to test as much as possible locally before uploading.

-jvt

clojure+u...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

[Kimmo Koskinen]

Hi!

Although still at early stages, check out portkey https://github.com/cgrand/portkey.

Portkey does tree-shaking by starting from a closure to minimize resulting jar size. Uses kryo and implementation of var tracing started in powderkeg project (https://github.com/HCADatalab/powderkeg)
.

pk/deploy can lift (an anonymous) function with [in out ctx] args as a Lambda function. pk/mount can expose a defn through API gateway with query params mapped to function arguments.

Christophe Grand and I have been working on portkey for now, but we'd welcome additional help on making it more awesome :)

- Kimmo

[Colin Fleming]

portkey looks great, I wasn't aware of that - thanks!

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to

clojure+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.

[Jose Trigueros]

This looks like a viable solution! Is Portkey on Clojars or will I have to build myself? Not a problem either way, just making sure I'm doing it right. Thanks Kimmo.

[Kimmo Koskinen]

Hi!

Not yet Clojars -ready so lein install for now. There is also #portkey on Clojurians Slack for chatting. Happy hacking! :)

- Kimmo