Hashes, Repeatability, and You
7 views
Skip to first unread message
Eric Myhre
Jan 10, 2014, 1:13:44 AM1/10/14
to Practical Agility
Hello practitioners of agility!
I had the honor of doing a lightning talk at the Practical Agility meetup this week, and talked briefly (read: horribly unpreparedly!) about repeatability and tools to help drive it. Some folks asked for links after the talk, and Matt graciously invited me to leave them here on the list, so thanks, and here goes :)
If there's a meta-concept I could drive, it's that repeatability brings power, automation is the first step towards repeatability, and a hash brings a true guarantee of repeatability.
Towards that end, I introduced one possible solution: MDM[1], a dependency management tool.
To rehash the talk a bit, in case you've forgotten the name but are googling the mailing list for keywords later: you've probably used some sort of dependency manager before... maven is one, or any+ivy, npm for nodejs, gems and bundler for ruby, examples abound. In using these, you've probably stubbed your toe before when discovering that these tools very much want to grab something from http, and this has a ton of downers: can't work offline, sometimes builds break randomly, your hotel connection at the conference lies about DNS and your tool can't tell (!), someone in your office poisons DNS so artifactory.mycorp.net points to wuhahaha.evilcorp.cn... wouldn't it be great if we lived in a world with distributed version control? Ahha! MDM uses git and submodules to manage your dependencies in a way that's decentralized, provably repeatable, and pretty darn secure. Binaries can come too; MDM uses a history layout that makes it possible to only download the bytes you need.
My occasional partner-in-crime nath...@kofalt.com also gave an off-the-cuff talk about a project called Hroot[2] -- the name is an abbreviation for "historic root", reminiscing of 'chroot'.
Hroot has a value proposition very similar to MDM, using hashes as technical guarantees of repeatability. Hroot applies specifically in collaboration with Docker[3] (a lightweight containerization system -- think virtual machines, but spin up in .2 sec and don't need preallocated memory). Docker provides isolation of processes and a chroot-style filesystem; Hroot transports the filesystem images, keeping an audit log of their history and assuring integrity via commit hashes.
MDM and hroot are both language-agnostic (in the last two years, I've written javascript, golang, java, python, and ruby, and I'm *tired* of relearning fundamentally the same toolchains over and over in a language-specific way!), but they are both git-emphatic. Git's inherent understanding of hashes makes it an excellent ground to build up on.
MDM and hroot are both under the Apache 2 license: riff, fork, and be merry!
Now, questions for others on this list! Do you know of other hash/guarantee oriented tools that can help drive CI? Can you think of other ways to apply existing tools to get stronger guarantees of repeatability? Are there limitations you can see in these approaches?
---
[1] http://github.com/polydawn/mdm
[2] http://github.com/polydawn/hroot
[3] http://github.com/dotcloud/docker
I had the honor of doing a lightning talk at the Practical Agility meetup this week, and talked briefly (read: horribly unpreparedly!) about repeatability and tools to help drive it. Some folks asked for links after the talk, and Matt graciously invited me to leave them here on the list, so thanks, and here goes :)
If there's a meta-concept I could drive, it's that repeatability brings power, automation is the first step towards repeatability, and a hash brings a true guarantee of repeatability.
Towards that end, I introduced one possible solution: MDM[1], a dependency management tool.
To rehash the talk a bit, in case you've forgotten the name but are googling the mailing list for keywords later: you've probably used some sort of dependency manager before... maven is one, or any+ivy, npm for nodejs, gems and bundler for ruby, examples abound. In using these, you've probably stubbed your toe before when discovering that these tools very much want to grab something from http, and this has a ton of downers: can't work offline, sometimes builds break randomly, your hotel connection at the conference lies about DNS and your tool can't tell (!), someone in your office poisons DNS so artifactory.mycorp.net points to wuhahaha.evilcorp.cn... wouldn't it be great if we lived in a world with distributed version control? Ahha! MDM uses git and submodules to manage your dependencies in a way that's decentralized, provably repeatable, and pretty darn secure. Binaries can come too; MDM uses a history layout that makes it possible to only download the bytes you need.
My occasional partner-in-crime nath...@kofalt.com also gave an off-the-cuff talk about a project called Hroot[2] -- the name is an abbreviation for "historic root", reminiscing of 'chroot'.
Hroot has a value proposition very similar to MDM, using hashes as technical guarantees of repeatability. Hroot applies specifically in collaboration with Docker[3] (a lightweight containerization system -- think virtual machines, but spin up in .2 sec and don't need preallocated memory). Docker provides isolation of processes and a chroot-style filesystem; Hroot transports the filesystem images, keeping an audit log of their history and assuring integrity via commit hashes.
MDM and hroot are both language-agnostic (in the last two years, I've written javascript, golang, java, python, and ruby, and I'm *tired* of relearning fundamentally the same toolchains over and over in a language-specific way!), but they are both git-emphatic. Git's inherent understanding of hashes makes it an excellent ground to build up on.
MDM and hroot are both under the Apache 2 license: riff, fork, and be merry!
Now, questions for others on this list! Do you know of other hash/guarantee oriented tools that can help drive CI? Can you think of other ways to apply existing tools to get stronger guarantees of repeatability? Are there limitations you can see in these approaches?
---
[1] http://github.com/polydawn/mdm
[2] http://github.com/polydawn/hroot
[3] http://github.com/dotcloud/docker
bill turner
Jan 10, 2014, 10:03:13 AM1/10/14
to practic...@googlegroups.com
I am hoping this is a trend, that speakers will post references to the group as they often come to fast to note. Thanks, Eric!
--
--
To Post: email to practic...@googlegroups.com
To Read Posts: http://groups.google.com/group/practical-agile?hl=en?hl=en
To The Website: https://sites.google.com/site/twincitiespracticalagility/
To Unsubscribe: email topractical-ag...@googlegroups.com
---
You received this message because you are subscribed to the Google Groups "Practical Agility" group.
To unsubscribe from this group and stop receiving emails from it, send an email to practical-agi...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
--
Bill Turner
GR8 in the US conference - http://gr8conf.us/
Blog : Groovy/Grails Talk - http://www.changent.com
Twitter : bill_turner - http://twitter.com/bill_turner
LinkedIn : wltiii - http://www.linkedin.com/in/wltiii
Phone : +1.612.276.2135
An innovative Software Development Professional, Bill Turner is the co-founder and Principal Organizer for GR8 in the US, a conference on Groovy, Grails, Griffon and related technologies. He has worked in companies from emerging startups to Fortune 500s, and has consulted and lived internationally. The majority of his 30 plus years in the software development industry has been spent as a consultant, with 75% of the business being repeat/referral business. He has collaborated closely with individuals at every level of the organization and has performed a wide variety of roles. He knows software development. He understands business. He helps companies succeed.
Reply all
Reply to author
Forward
0 new messages