Chef and knife ec2, basic questions.

[Tim Walker]

Howdy,

I'm hoping someone can help me with some basic questions that I'm struggling to understand. Trust me, the search results marked as "read" are voluminous. I've read and watched tutorials and am writing code to test my assumptions, including the excellent TDI by a group member. I'm hoping Stephen will read this as fodder for clarification in the next release of that. Despite considerable effort, I remain confused about some basic things. 

The use case is: spinning up (n) development environments in the EC equipped as developer instances. 

Can someone please verify: 

1. You need a chef server with knife ec2, hosted chef is a good intermediary solution. 

2. That implies several components in the mix: My management client running knife ec2, hosted chef for organizations, an AWS account

Here's how far I've gotten: 

bundle exec knife ec2 server create --verbose -I ami-1e299d7e -r "role[student]" -f t2.micro -S knife-ec2 -i ~/.ssh/knife-ec2.pem --ssh-user xxxxxxx  --region us-west-2 

Instance ID: i-065884ad6fb27ce16

Flavor: t2.micro

Image: ami-1e299d7e

Region: us-west-2

Availability Zone: us-west-2b

Security Groups: default

Tags: Name: i-065884ad6fb27ce16

SSH Key: knife-ec2

Waiting for instance......

Public DNS Name: ec2-52-11-40-222.us-west-2.compute.amazonaws.com

Public IP Address: 52.11.40.222

Private DNS Name: ip-172-31-18-55.us-west-2.compute.internal

Private IP Address: 172.31.18.55

....done

Bootstrapping Chef on ec2-52-11-40-222.us-west-2.compute.amazonaws.com

Failed to authenticate xxxxx - trying password auth

Enter your password: 

Confused at this step and have tried several things. Mostly I'm confused around the knife-ec2.pem and my ssh user. Is it using this to authenticate the chef session and the user installing chef during the bootstrap? 

Any help is sincerely appreciated. 

Thank you, 

Tim

[Thom May]

Hi Tim,

I'm sort of curious how you stumbled over this list; it's pretty well dead and most Chef discussion happens on discourse (https://discourse.chef.io) or Slack (https://community-slack.chef.io). Anyway, welcome!

You're correct that the most typical way to use a node with ec2 is with a chef server, and Hosted Chef is certainly one of the easiest ways to get a chef server.  

Now, looking at your logs, I assume that `knife-ec2.pem` is the ssh key file that you've downloaded from EC2, and which you've called `knife-ec2` in the EC2 management console. Once the node has started, can you do: `ssh -i  ~/.ssh/knife-ec2.pem xxxxxxx@ec2-52-11-40-222.us-west-2.compute.amazonaws.com ` and connect to the node? If not, then you're not using the correct ssh key, and you'll need to fix that.

Knife only uses the ssh key to connect to the node, and uses the key you should have downloaded from hosted chef to talk to the Chef API. I'd recommend working through https://learn.chef.io/tutorials/manage-a-node/ if you haven't already, since that's a good run through of the technology stack and a typical workflow.

Hope this helps,

-Thom

--
You received this message because you are subscribed to the Google Groups "chef-testing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chef-testing+unsubscribe@googlegroups.com.
To post to this group, send email to chef-t...@googlegroups.com.
Visit this group at https://groups.google.com/group/chef-testing.
For more options, visit https://groups.google.com/d/optout.

[Tim Walker]

Hi Thom! 

Response inline below. 

On Wednesday, January 18, 2017 at 8:30:15 AM UTC-6, Thom May wrote:

Hi Tim,

I'm sort of curious how you stumbled over this list; it's pretty well dead and most Chef discussion happens on discourse (https://discourse.chef.io) or Slack (https://community-slack.chef.io). Anyway, welcome!

Thank you very, very much. I didn't know this was a dying group. Given the BDD/Infrastructure bend I assumed it was the current excellence. I am a BDD instructor and have been teaching, for example, Executable Requirements with FIT and Cucumber/BDD/TDD for a decade and a half. I'm using BDD to drive this process and to learn Chef.  

You're correct that the most typical way to use a node with ec2 is with a chef server, and Hosted Chef is certainly one of the easiest ways to get a chef server.  

Now, looking at your logs, I assume that `knife-ec2.pem` is the ssh key file that you've downloaded from EC2, and which you've called `knife-ec2` in the EC2 management console. Once the node has started, can you do: `ssh -i  ~/.ssh/knife-ec2.pem xxxxxxx@ec2-52-11-40-222.us-west-2.compute.amazonaws.com ` and connect to the node? If not, then you're not using the correct ssh key, and you'll need to fix that.

Knife only uses the ssh key to connect to the node, and uses the key you should have downloaded from hosted chef to talk to the Chef API. I'd recommend working through https://learn.chef.io/tutorials/manage-a-node/ if you haven't already, since that's a good run through of the technology stack and a typical workflow.

This helps considerably! Just to give me the confidence I'm on the right track and to point out a couple of basic, essential things. I recognize that, in some groups, people would not have treated me as kindly as you have and I am grateful. I will take those clues, read that tutorial and press on! Thank you very, very much. Sincerely, Tim 

Hope this helps,

-Thom

To unsubscribe from this group and stop receiving emails from it, send an email to chef-testing...@googlegroups.com.

[Thom May]

On Thu, Jan 19, 2017 at 3:40 AM, Tim Walker <t...@theagiletester.com> wrote:

Hi Thom! 

Response inline below. 

On Wednesday, January 18, 2017 at 8:30:15 AM UTC-6, Thom May wrote:

Hi Tim,

I'm sort of curious how you stumbled over this list; it's pretty well dead and most Chef discussion happens on discourse (https://discourse.chef.io) or Slack (https://community-slack.chef.io). Anyway, welcome!

Thank you very, very much. I didn't know this was a dying group. Given the BDD/Infrastructure bend I assumed it was the current excellence. I am a BDD instructor and have been teaching, for example, Executable Requirements with FIT and Cucumber/BDD/TDD for a decade and a half. I'm using BDD to drive this process and to learn Chef.  

I'd say that TDD in general is so deeply embedded in Chef's culture at this point that we don't really need a specialized list - ChefSpec, Inspec, and Test Kitchen provide a default workflow that we ship in the ChefDK, the cookbook metrics we're developing for supermarket have integration tests as one of their measurements (https://github.com/chef-cookbooks/cookbook-quality-metrics/tree/master/quality-metrics), etc.

You're correct that the most typical way to use a node with ec2 is with a chef server, and Hosted Chef is certainly one of the easiest ways to get a chef server.  

Now, looking at your logs, I assume that `knife-ec2.pem` is the ssh key file that you've downloaded from EC2, and which you've called `knife-ec2` in the EC2 management console. Once the node has started, can you do: `ssh -i  ~/.ssh/knife-ec2.pem xxxxxxx@ec2-52-11-40-222.us-west-2.compute.amazonaws.com ` and connect to the node? If not, then you're not using the correct ssh key, and you'll need to fix that.

Knife only uses the ssh key to connect to the node, and uses the key you should have downloaded from hosted chef to talk to the Chef API. I'd recommend working through https://learn.chef.io/tutorials/manage-a-node/ if you haven't already, since that's a good run through of the technology stack and a typical workflow.

This helps considerably! Just to give me the confidence I'm on the right track and to point out a couple of basic, essential things. I recognize that, in some groups, people would not have treated me as kindly as you have and I am grateful. I will take those clues, read that tutorial and press on! Thank you very, very much. Sincerely, Tim 

No worries; welcome to the Chef community!

-Thom

[Tim Walker]

Thom - just for closure and to thank you. I am past this step. The basic problem was the ssh user was wrong and should have been ec2-user. I can never tell in the examples if the user is literal or an example! 8) 

Now, I have more specific things to run down (below) but well in to the chef bootstrap. Appreciate all your help. Tim

ec2-35-165-83-114.us-west-2.compute.amazonaws.com [2017-01-21T18:35:05+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out

ec2-35-165-83-114.us-west-2.compute.amazonaws.com Chef Client failed. 0 resources updated

ec2-35-165-83-114.us-west-2.compute.amazonaws.com [2017-01-21T18:35:05+00:00] ERROR: Connection refused - Connection refused connecting to https://localhost/clients, giving up

ec2-35-165-83-114.us-west-2.compute.amazonaws.com [2017-01-21T18:35:05+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

[Tim Walker]

I guess I'm sad this group is gone. Just got this working and trying to start a run list from my Cucumber. From a "test driven" experience, I have a good story to tell here now. Getting that first step to pass: 

Scenario: Build an AMI
In order to be able to take full advantage of AWS virtualization
As a course instructor for The Agile Tester
I'd like to be able to create a "base" image to be used to create virtual instances

  Given an empty AWS Linux Server running chef

Was huge for me in understanding how Chef is all wired together and I can leverage that basic step over and over. Having a fully automated hosted chef instance controlling my AWS instance just rocks so flippen' hard. 

Thom, and Chef dudes generally, Thanks man! 

Tim