Packer Azure failed to continue running the template after the PowerShell domain join command ran

722 views
Skip to first unread message

Sim

unread,
Apr 30, 2018, 12:35:16 AM4/30/18
to Packer

Packer failed to continue running the template after the PowerShell domain join command was executed. 

This issue has been here for a while, I did raise this issue as a bug but @Mathew suggested me to update the plugins which I already did, and also to wait for the latest release which could possibly fix my issue, but the issue still has not been resolved and I believe this is a bug and hopefully there would be someone to investigate this bug. Thanks so much.


As I need to add the temporary VM which is generated by Packer during the build into the domain, I have added a Domain join PowerShell script into the template which looks like this:


$domain = "xxxxx" 

$password = "xxxxx" | ConvertTo-SecureString -asPlainText -Force 

$username = "$domain\aaa.bbb" 

$credential = New-Object System.Management.Automation.PSCredential($username,$password) 

Add-Computer -DomainName $domain -Credential $credential


and then in the provisioners part of the template, I have placed the domain join script before the windows-restart command.

{ "type": "powershell",

 "script": "PackerUpload/join_domain.ps1" }, 

{ "type": "windows-restart",

 "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\"" }


Technically, Packer is supposed to restart and continue the running other components in the template. However, I don't know why the packer build failed to restart or even run another script (I did attempt to replace the windows-restart with another PowerShell script, but it also failed to run). I might suspect that the domain join script has somehow disconnected the temporary VM with Packer or maybe Packer failed to reconnect the VM after the domain join script had been successfully executed. I guess this may be a bug in Packer. I have attached the debug_log below. There is no problem if I run other PowerShell scripts (not domain join one), the connection is fine and windows-restart works fine. Thank you.

Info:

Alvaro Miranda Aguilera

unread,
Apr 30, 2018, 3:35:56 AM4/30/18
to packe...@googlegroups.com
can you put a minimal repro case?

in that way will be easier to just run packer and see the issue.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/76298b3c-1137-4eed-bb1d-dc7d253dd191%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Alvaro

Gabo Kete

unread,
Apr 30, 2018, 6:14:47 PM4/30/18
to Packer
In your Packer logs you are getting the following message

2018/04/07 10:04:40 packer.exe: 2018/04/07 10:04:40 Retryable error: http response error: 401 - invalid content type
2018/04/07 10:04:40 packer.exe: 2018/04/07 10:04:40 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:04:40 packer.exe: 2018/04/07 10:04:40 [INFO] 0 bytes written for 'stdout'

Http 401 error are credential related issues.. I will suggest first in your script to hardcode the username/password and see how it goes

If you want to troubleshoot your script, will suggest to print out what is the value you get in the variable $password and make sure your script is passing valid username/password values..


$domain = "xxxxx" 

$password = "xxxxx" | ConvertTo-SecureString -asPlainText -Force 

$username = "$domain\aaa.bbb" 

$credential = New-Object System.Management.Automation.PSCredential($username,$password) 


To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.



--
Alvaro

Sim

unread,
Apr 30, 2018, 9:18:35 PM4/30/18
to Packer
Hi @Gabo, 

I don't think it's about my credential issue as I did run this PowerShell domain join with this credential in the new machine and it worked (added the new machine to the domain). Could you help me try to put the domain join Powershell script in the template like the above order and build it to see if you get the same issue as me (couldnt complete the packer build - everything after domain join execution just doesnt work). 

This is the Packer JSON file that I use, feel free to replace anything after the powershell domain join (running other powershell script, restart VM,... ) as Packer just didnt do anything after the domain join thing. 

{
  "variables": {
    "client_id": "{{env `AZURE_CLIENT_ID`}}",
    "client_secret": "{{env `AZURE_CLIENT_SECRET`}}",
    "tenant_id": "{{env `AZURE_TENANT_ID`}}",
    "subscription_id": "{{env `AZURE_SUBSCRIPTION_ID`}}",
    "object_id": "{{env `AZURE_OBJECT_ID`}}"
    },
  "builders": [{
    "type": "azure-arm",

    "client_id": "{{user `client_id`}}",
    "client_secret": "{{user `client_secret`}}",
    "tenant_id": "{{user `tenant_id`}}",
    "subscription_id": "{{user `subscription_id`}}",
    "object_id": "{{user `object_id`}}",

    "managed_image_resource_group_name": "MyPacker",
    "managed_image_name": "myPackerImage",


    "virtual_network_name": "------",               
    "virtual_network_subnet_name": "-----",       
    "virtual_network_resource_group_name": "Networking",

    "os_type": "Windows",
    "image_publisher": "MicrosoftWindowsServer",
    "image_offer": "WindowsServer",
    "image_sku": "2016-Datacenter",

    "disk_additional_size": "12",

    
    "communicator": "winrm",
    "winrm_use_ssl": "true",
    "winrm_insecure": "true",
    "winrm_timeout": "10m",
    "winrm_username": "packer",

    "azure_tags": {
        "dept": "Engineering",
        "task": "Image deployment"
    },

    "location": "australiasoutheast",
    "vm_size": "Standard_DS2_v2"
  }],
  "provisioners": [
    {
      "type": "powershell",
      "inline": ["mkdir c:\\Packer"]
    },
    {
      "type": "file",
      "source": "ToUpload/",
      "destination": "c:\\Packer"
    },
    {
      "type": "windows-restart",
      "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\""
    },
    {
      "type": "windows-restart",
      "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\""
    },
    {
      "type": "powershell",
      "script": 
        "ToUpload/join_domain.ps1"
    },
    {
      "type": "windows-restart",
      "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\""
    },
    {
    "type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
      "& $Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit"
    ]
  }]
}



Thanks so much, 

Gabo Kete

unread,
Apr 30, 2018, 10:10:01 PM4/30/18
to Packer
HI Sim

I do not have an Azure account to help you to test this.. but I went through the documentation and to authorize Packer to build in Azure there are 6 configuration options you will need ( Check this link --> Azure Setup ) 

I compared it with your JSON file and seems there are at least 2 missing ( resource_group_name and storage_account ) 














Also the link I shared above, tells you how to setup your packer environment so can work with Azure plus gives you a couple of examples how to do some testings .If you haven't installed the AZURE Cli, is part of what you need to setup prior using packer..

I will recommend to have a look at the link above and make sure you have the environment setup prior using packer to deploy your VMs in AZURE..

Let us know how it goes.. 

Regards


Gabo Kete

unread,
Apr 30, 2018, 10:36:32 PM4/30/18
to Packer
Also I have gone through the logs and seems everything is executing just fine till the below part of the JSON

   {
      "type": "windows-restart",
      "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\""
    },


Which matches with this part of the log

2018/04/07 09:59:58 [INFO] (telemetry) ending powershell
2018/04/07 09:59:58 [INFO] (telemetry) Starting provisioner windows-restart
2018/04/07 09:59:58 ui: ==> azure-arm: Restarting Machine
2018/04/07 10:00:02 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:00:02 [INFO] 0 bytes written for 'stdout'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 [INFO] 0 bytes written for 'stdout'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 Retryable error: http response error: 401 - invalid content type
2018/04/07 10:00:08 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:00:08 [INFO] 0 bytes written for 'stdout'

Then I am assuming packer is looking to perform the following and last bit of your JSON file

"type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
      "& $Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit"
    ]
  }]

And this is where the error of  "Retryable error: http response error: 401 - invalid content type" shows in the logs

I reckon the issue is with Add-WindowsFeature which seems replaced by Install-WindowsFeature ( Source Link --> Install-WindowsFeature Documentation

Seems the Add-WindowsFeature content was used in Windows Server 2008 R2... But not being an expert in Windows, I'd recommend you remove the last bit of your JSON file where you try to install/enable the windows server role... see whether packer fails or not, if that does not fail, then run the command shown below and see what is the output..


"type": "powershell",
    "inline": [
      "Add-WindowsFeature Web-Server",
      "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml ){ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}",
      "& $Env:SystemRoot\\System32\\Sysprep\\Sysprep.exe /oobe /generalize /quiet /quit"
    ]
  }]


Gabo Kete

unread,
Apr 30, 2018, 11:23:03 PM4/30/18
to Packer
Another suggestion as part of the troubleshooting will be , as soon as the provisioner "windows-restart" kicks off, just log in into the Wndows VM built and see whether it reboots 

Looks like the "restart_check_command" is not happening and this could be other reason why you are getting the "Retryable error: http response error: 401 - invalid content type" error 

If the VM restarts but still see the same error, then I will suggest try to log in with the credentials you have setup and see whether that works.. 

I reckon packer is trying to log in to run the command "powershell -command \"& {Write-Output 'restarted.'}\"" and is getting an authentication error 

I'd recommend to do this first before moving forward with more troubleshooting 

Sim

unread,
May 6, 2018, 9:43:21 PM5/6/18
to Packer
Hi @Gabo, 

Thanks for your suggestion and sorry for the late reply. I think you might be right that after joining the domain, Packer failed to run "window-restart" command in the temporary VM. But there is no way log into that temporary VM to check as I don't know the admin credentials which Packer created for the temporary VM, and cannot also use my domain credentials to log in as the VM hasn't been configured for remote access. So what should I do now in terms of troubleshooting, surely feel like this is the packer bug but seems like no one could really figure out what is wrong with it as Packer is supposed to log back to the VM after domain join and run the rest of the template. Regarding the "Add-WindowsFeature" that you mentioned in the above post, I believe that wasn't the issue as Packer couldn't reach it if it couldn't pass the windows-restart step which it failed to do so. Do you know the master admin user that Packer created to generate the temporary VM in Azure? I hope you could help? Have been with this issue for a month and not many Packer maintainers could help me out :( 

Thanks, 

Gabo Kete

unread,
May 6, 2018, 10:29:15 PM5/6/18
to Packer
I do not think this is a packers bug, well at least I can't tell based on the information you have shared 

I reckon at the time packers uses the windows-restart provisioner, it starts using the credentials you have setup for the VM using your script and right after the windows-restart provisioner is where you start to have problems 


Have a look at this logs

2018/04/07 09:59:58 [INFO] (telemetry) Starting provisioner windows-restart
2018/04/07 09:59:58 ui: ==> azure-arm: Restarting Machine
2018/04/07 10:00:02 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:00:02 [INFO] 0 bytes written for 'stdout'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 [INFO] 0 bytes written for 'stdout'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 [INFO] 0 bytes written for 'stderr'
2018/04/07 10:00:02 packer.exe: 2018/04/07 10:00:02 Retryable error: http response error: 401 - invalid content type

I'd suggest verifying your PS script and make sure your variables are working as expected... I did try your script and the outcome I got was not consistent... 

I would recommend to review this bit of your PS CODE and verify the variables produce the right username and password

Below is a capture of my screen showing a bit of your code where it shows I am not getting the right password value

Gabo Kete

unread,
May 6, 2018, 10:47:37 PM5/6/18
to Packer
again do not take my word for granted as I am not a PS/Windows expert.. but I reckon you can do the troubleshooting to rule out this issue... 

Probably another recomendation I'd suggest is to remove the windows-restart provisioner and verify if you packer provisioning moves forward with the next part of your JSON file

Sim

unread,
May 7, 2018, 12:35:35 AM5/7/18
to Packer

Hi Gabo, 

I have double checked my PS script and even tested that again on a unjoin-domained VM, it worked fine as shown in the photo I have attached.
I reckon the reason why the password value you mentioned did not match as PS converted the plain text string to secure.string in which it will keep you string secured and hidden so when you do write-host I will not display the whole readable string to the console. 
And just wondering how could Packer use the credential which I specified in the domain join script to log into the VM after it finished doing the domain join as that domain join script was called separately. 


Regarding your last suggestion about replacing the window-restart part with another script, I have tried that several times and still the same issue, Packer could not run anything after executing the domain join script. The Packer build will be fine and successfully build an image if I remove the PS domain join script from the template and just run normal PS scripts. Thus, I reckon there is something to do with the PS domain join script which I hardly find any errors and it just works totally fine when I run it in my local VMs. 

Thanks 


Reply all
Reply to author
Forward
0 new messages