mrlesmithjr / packer-templates Goto Github PK

View Code? Open in Web Editor NEW

136.0 15.0 30.0 825 KB

License: MIT License

Shell 61.71% PowerShell 20.96% Python 17.33%

packer virtualbox packer-template vagrant ubuntu debian fedora redhat opensuse

packer-templates's Introduction

Table of Contents generated with DocToc

packer-templates

packer-templates

Purpose

This repository is for maintaining my personal Vagrant Box Templates using Packer.

Information

All builds are based on the following providers:

virtualbox
vmware_desktop
You can find my collection of builds here

NOTE: All builds are base builds and follow the Vagrant guidelines of how a Vagrant box should be built.

Requirements

All of my Packer templates are configured to upload to Vagrant Cloud after a successful build has been executed. In order to upload a box version to Vagrant Cloud, you will need to create a private_vars.json file in the root of this repo with the following info:

{
  "vagrant_cloud_token": "Your Vagrant Cloud private API token",
  "vagrant_cloud_username": "Your Vagrant Cloud username"
}

If you do not want this functionality, you will need to edit the respective template within the distro folder and remove the following:

{
  "type": "vagrant-cloud",
  "box_tag": "{{ user `box_tag` }}",
  "access_token": "{{ user `vagrant_cloud_token` }}",
  "version": "{{ timestamp }}"
}

Software

Usage

Building a box

To build a Vagrant box with Packer for Virtualbox:

Select distro

Choose which distro you are interested in building.

Build distro

NOTE: This example we will have chosen Ubuntu Xenial

cd Ubuntu/xenial64/server
packer build -var-file=../../../private_vars.json -var-file=ubuntu1604.json ../../ubuntu-server.json

Now watch your build kick off and run through the building process. Once it has completed you will be ready to test it out.

Testing a box

Once your build has completed you are ready to test it out.

Add box to Vagrant

Note: The number at the end is the epoch time of the build. Replace this accordingly.

cd Ubuntu/xenial64/server
vagrant box add xenial64-server-packer-template-virtualbox-1542509766 xenial64-server-packer-template-virtualbox-1542509766.box

Create Vagrantfile

cd ~
mkdir -p packer/vagrant/xenial64-server
cd packer/vagrant/xenial64-server
vagrant init xenial64-server-packer-template-virtualbox-1542509766

Spin it up

vagrant up

Test it out

vagrant ssh

Now do some basic tests to validate all is good.

Tear it down

vagrant destroy -f

Cleaning up

When you need to clean up any of the lingering files/folers generated during building, you can execute the cleanup_builds.sh script.

Using pre-built and ready for consumption Vagrant templates

The majority of these templates are used to populate my vagrant-box-templates repo. I would highly recommend leveraging this repo for testing and etc.

License

MIT

Author Information

Larry Smith Jr.

packer-templates's People

Contributors

Stargazers

Watchers

packer-templates's Issues

Add Ubuntu 19.10 Desktop

The previous issue #23 only addressed the server version. Need to add the desktop version now.

Add Ubuntu 20.04 Beta

Ubuntu 20.04 will be released this month. Need to prepare using the beta version for now.

Add Fedora 31 release

Fedora 31 has been released and needs to be added

cannot install packages with Arch... perhaps re-spin the image?

Having a problem installing any packages using pacman. Perhaps re-spin/update the box? Maybe???

I've tried following the steps from https://bbs.archlinux.org/viewtopic.php?id=193382 but was not having any luck. I also reviewed https://wiki.archlinux.org/index.php/Pacman/Package_signing

Here's my log:

mikekinneymbp:arch mikekinney$ rm Vagrantfile
mikekinneymbp:arch mikekinney$ vagrant init mrlesmithjr/arch
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
mikekinneymbp:arch mikekinney$ vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Box 'mrlesmithjr/arch' could not be found. Attempting to find and install...
    default: Box Provider: vmware_desktop, vmware_fusion, vmware_workstation
    default: Box Version: >= 0
==> default: Loading metadata for box 'mrlesmithjr/arch'
    default: URL: https://vagrantcloud.com/mrlesmithjr/arch
==> default: Adding box 'mrlesmithjr/arch' (v1561616456) for provider: vmware_desktop
    default: Downloading: https://vagrantcloud.com/mrlesmithjr/boxes/arch/versions/1561616456/providers/vmware_desktop.box
    default: Download redirected to host: vagrantcloud-files-production.s3.amazonaws.com
==> default: Successfully added box 'mrlesmithjr/arch' (v1561616456) for 'vmware_desktop'!
==> default: Cloning VMware VM: 'mrlesmithjr/arch'. This can take some time...
==> default: Checking if box 'mrlesmithjr/arch' version '1561616456' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Starting the VMware VM...
==> default: Waiting for the VM to receive an address...
==> default: Forwarding ports...
    default: -- 22 => 2222
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Configuring network adapters within the VM...
==> default: Waiting for HGFS to become available...
==> default: Enabling and configuring shared folders...
    default: -- /Users/mikekinney/vagrant/arch: /vagrant
mikekinneymbp:arch mikekinney$ vagrant ssh
[vagrant@archlinux ~]$ sudo su -
[root@archlinux ~]# pacman --noconfirm -S archlinux-keyring vim
warning: archlinux-keyring-20190123-2 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (4) gpm-1.20.7.r27.g1fd1941-1  vim-runtime-8.1.1467-2  archlinux-keyring-20190123-2
             vim-8.1.1467-2

Total Download Size:    7.82 MiB
Total Installed Size:  33.73 MiB
Net Upgrade Size:      32.80 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
error: failed retrieving file 'archlinux-keyring-20190123-2-any.pkg.tar.xz' from mirrors.ocf.berkeley.edu : The requested URL returned error: 404
error: failed retrieving file 'archlinux-keyring-20190123-2-any.pkg.tar.xz' from mirror.mia11.us.leaseweb.net : The requested URL returned error: 404
error: failed retrieving file 'archlinux-keyring-20190123-2-any.pkg.tar.xz' from mirrors.rit.edu : The requested URL returned error: 404
error: failed retrieving file 'archlinux-keyring-20190123-2-any.pkg.tar.xz' from il.us.mirror.archlinux-br.org : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
error: failed retrieving file 'archlinux-keyring-20190123-2-any.pkg.tar.xz' from ftp.osuosl.org : The requested URL returned error: 404
 archlinux-keyring-20190123-2-any    685.1 KiB   486K/s 00:01 [#################################] 100%
error: failed retrieving file 'gpm-1.20.7.r27.g1fd1941-1-x86_64.pkg.tar.xz' from mirrors.ocf.berkeley.edu : The requested URL returned error: 404
error: failed retrieving file 'gpm-1.20.7.r27.g1fd1941-1-x86_64.pkg.tar.xz' from mirror.mia11.us.leaseweb.net : The requested URL returned error: 404
error: failed retrieving file 'gpm-1.20.7.r27.g1fd1941-1-x86_64.pkg.tar.xz' from mirrors.rit.edu : The requested URL returned error: 404
error: failed retrieving file 'gpm-1.20.7.r27.g1fd1941-1-x86_64.pkg.tar.xz' from il.us.mirror.archlinux-br.org : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
error: failed retrieving file 'gpm-1.20.7.r27.g1fd1941-1-x86_64.pkg.tar.xz' from ftp.osuosl.org : The requested URL returned error: 404
 gpm-1.20.7.r27.g1fd1941-1-x86_64    131.8 KiB   283K/s 00:00 [#################################] 100%
error: failed retrieving file 'vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz' from mirrors.ocf.berkeley.edu : The requested URL returned error: 404
error: failed retrieving file 'vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz' from mirror.mia11.us.leaseweb.net : The requested URL returned error: 404
error: failed retrieving file 'vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz' from mirrors.rit.edu : The requested URL returned error: 404
error: failed retrieving file 'vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz' from il.us.mirror.archlinux-br.org : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
error: failed retrieving file 'vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz' from ftp.osuosl.org : The requested URL returned error: 404
 vim-runtime-8.1.1467-2-x86_64         5.6 MiB  1291K/s 00:04 [#################################] 100%
error: failed retrieving file 'vim-8.1.1467-2-x86_64.pkg.tar.xz' from mirrors.ocf.berkeley.edu : The requested URL returned error: 404
error: failed retrieving file 'vim-8.1.1467-2-x86_64.pkg.tar.xz' from mirror.mia11.us.leaseweb.net : The requested URL returned error: 404
error: failed retrieving file 'vim-8.1.1467-2-x86_64.pkg.tar.xz' from mirrors.rit.edu : The requested URL returned error: 404
error: failed retrieving file 'vim-8.1.1467-2-x86_64.pkg.tar.xz' from il.us.mirror.archlinux-br.org : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
error: failed retrieving file 'vim-8.1.1467-2-x86_64.pkg.tar.xz' from ftp.osuosl.org : The requested URL returned error: 404
 vim-8.1.1467-2-x86_64              1468.5 KiB   709K/s 00:02 [#################################] 100%
(4/4) checking keys in keyring                                [#################################] 100%
(4/4) checking package integrity                              [#################################] 100%
error: vim-runtime: signature from "Levente Polyak (anthraxx) <[email protected]>" is unknown trust
:: File /var/cache/pacman/pkg/vim-runtime-8.1.1467-2-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: vim: signature from "Levente Polyak (anthraxx) <[email protected]>" is unknown trust
:: File /var/cache/pacman/pkg/vim-8.1.1467-2-x86_64.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n]
error: failed to commit transaction (invalid or corrupted package (PGP signature))
Errors occurred, no packages were upgraded.
[root@archlinux ~]#

raising vm_vcpus fails suse builds

Hi,

I'm preparing a patch that will clean up some parts of the SUSE builds, since a few of them are broken (some just need new ISO urls, but in 99% the problems are because of facter). This is a huge time sink, so I wanted to speed up the installs.
I try to raise vm_vcpus in opensuse-server.json, it's mentioned twice, supposedly one setting for the build time and one for the generated VM. I noticed, if I change the value from 1 to any other number, the typing boot command goes wrong and the VM will sit at Booting from local disk forever.

If you have any advice, I'd appreciate, otherwise this is going to take many more hours for nothing...

Ubuntu 19.10 has been released

Need to add Ubuntu 19.10(Eoan Ermine)

base.sh rework

Hi,

could you have a look at this gist?
https://gist.github.com/FlorianHeigl/cd4f36acafab02ee6370b8b75452ed96

tried to unify & sort things a bit.
pretends to be naive and uses os_name for all main decisions, I'm aware that that will need some testing.
there are notes where I found missing paths in the original (i.e. OS version dead or one that will not have any choices made for itself)
no functional / package changes except for facter in some SLES variants & wiping machine-id whereever applicable

it seems there are many cases where facter doesn't get involved - at least in this script. i think base.sh can work without (relevant for reproducible builds off ISO since facter is often fetched elsewhere). but since many other scripts also rely on it, the choice is not so easy.

Update Python requirements

Need to ensure that all Python requirements are updated and functional.

global packer_cache directory

Could we also symlink the packer_cache subdirectory to ../../../packer_cache?

At least in my case it made sense to have it in a different location, and with a symlink that change can be done more easily.

Add Debian Stretch Distro

Need to add Debian Stretch x64 distro.

Virtualbox Additions Not Working On Debian Stretch

Currently the Virtualbox additions do not work properly on Debian Stretch.

Alpine image hangs at sharing folder

Looks the Alpine image with vmware desktop hangs at sharing the folder as part of the "vagrant up" command.

I think it is because open-vm-tools needs to be installed and started.

I have this environment variable set:
VAGRANT_DEFAULT_PROVIDER=vmware_desktop

Here is what happens:

$ vagrant init mrlesmithjr/alpine311
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.


$ vagrant up
Bringing machine 'default' up with 'vmware_desktop' provider...
==> default: Box 'mrlesmithjr/alpine311' could not be found. Attempting to find and install...
    default: Box Provider: vmware_desktop, vmware_fusion, vmware_workstation
    default: Box Version: >= 0
==> default: Loading metadata for box 'mrlesmithjr/alpine311'
    default: URL: https://vagrantcloud.com/mrlesmithjr/alpine311
==> default: Adding box 'mrlesmithjr/alpine311' (v1576823149) for provider: vmware_desktop
    default: Downloading: https://vagrantcloud.com/mrlesmithjr/boxes/alpine311/versions/1576823149/providers/vmware_desktop.box
    default: Download redirected to host: vagrantcloud-files-production.s3.amazonaws.com
==> default: Successfully added box 'mrlesmithjr/alpine311' (v1576823149) for 'vmware_desktop'!
==> default: Cloning VMware VM: 'mrlesmithjr/alpine311'. This can take some time...
==> default: Checking if box 'mrlesmithjr/alpine311' version '1576823149' is up to date...
==> default: Verifying vmnet devices are healthy...
==> default: Preparing network adapters...
==> default: Starting the VMware VM...
==> default: Waiting for the VM to receive an address...
==> default: Forwarding ports...
    default: -- 22 => 2222
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Configuring network adapters within the VM...
==> default: Waiting for HGFS to become available...
==> default: Enabling and configuring shared folders...

and it just "hangs" there.

Add CentOS 8 desktop

Need to add CentOS 8 desktop and fix any provisioning issues.

SUSE templates: cleanup needs to be extended

There's one issue in these with packer builds.

You need to clean up the following file

linux:~ # find /var -name "*rand*"
/var/lib/misc/random-seed
/var/lib/systemd/random-seed

especially since the first of the two has ugly permissions:

linux:~ # find /var -name "*rand*" -ls
 70973    4 -rw-r--r--   1 root     root         4096 Nov 23 23:43 /var/lib/misc/random-seed
 60356    4 -rw-------   1 root     root          512 Dec  2 04:52 /var/lib/systemd/random-seed

neither should be on-filesystem while the system is running, but the first one is packed with the image(!)

Update README.md, add LICENSE.md, and generate TOC

Need to update README.md, add LICENSE.md, and generate table of contents.

`ssh_wait_timeout` issues with latest Packer versions

1 error occurred:
	* unknown configuration key: "ssh_wait_timeout"; raws is []interface {}{map[string]interface {}{"boot_command":[]interface {}{"{{ user `boot_command_prefix` }}", "/install/vmlinuz", " initrd=/install/initrd.gz", " auto=true", " priority=critical", " url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ubuntu/preseed.cfg <wait>", "<enter>"}, "cpus":"1", "disk_adapter_type":"scsi", "disk_size":"36864", "disk_type_id":0, "guest_os_type":"ubuntu-64", "headless":true, "http_directory":"http", "iso_checksum":"e2ecdace33c939527cbc9e8d23576381c493b071107207d2040af72595f8990b", "iso_checksum_type":"sha256", "iso_url":"http://cdimage.ubuntu.com/releases/18.04/release/ubuntu-18.04.4-server-amd64.iso", "memory":"512", "output_directory":"output-bionic64-server-packer-template-vmware-iso-1586122792", "shutdown_command":"echo '/sbin/halt -h -p' > shutdown.sh; echo 'packer'|sudo -S bash 'shutdown.sh'", "ssh_password":"vagrant", "ssh_username":"vagrant", "ssh_wait_timeout":"60m", "vm_name":"bionic64-server-packer-template-1586122792", "vmx_data":map[interface {}]interface {}{"ethernet0.pciSlotNumber":"32"}, "vmx_remove_ethernet_interfaces":true}, map[string]interface {}{"packer_build_name":"vmware-iso", "packer_builder_type":"vmware-iso", "packer_debug":false, "packer_force":false, "packer_on_error":"", "packer_template_path":"/Users/larrysmithjr/Git_Projects/Personal/GitHub/mrlesmithjr/packer-templates/Ubuntu/ubuntu-server.json", "packer_user_variables":map[interface {}]interface {}{"":"", "auto_build":"True", "boot_command_prefix":"<enter><wait><f6><esc><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>", "box_name":"bionic64", "box_tag":"mrlesmithjr/bionic64", "compression_level":"6", "description":"Ubuntu 18.04 (Bionic Beaver) Server", "iso_checksum":"e2ecdace33c939527cbc9e8d23576381c493b071107207d2040af72595f8990b", "iso_checksum_type":"sha256", "iso_url":"http://cdimage.ubuntu.com/releases/18.04/release/ubuntu-18.04.4-server-amd64.iso", "private":"False", "qemu_accelerator":"kvm", "short_description":"Ubuntu Bionic Server", "vagrant_cloud_token":"ZlOqTLUHYB8P5g.atlasv1.iyHHQr7CbhylZIFqKbTI1cfQJsEqzePISGFGsAvz0Q61xwqv2krr8bgruapznh4dJ5U", "vagrant_cloud_username":"mrlesmithjr", "vm_disk_adapter_type":"scsi", "vm_disk_size":"36864", "vm_memory":"512", "vm_name":"bionic64-server-packer-template", "vm_ssh_password":"vagrant", "vm_ssh_username":"vagrant", "vm_vcpu":"1"}}}

 and ctx data is map[interface {}]interface {}(nil)

Add ssh_timeout back

After #59 was merged, we now need to add ssh_timeout back as some initial testing without is having issues with you guessed it, ssh timeouts with default timing.

Issue with time check in utils.py

Traceback (most recent call last):
  File "utils.py", line 596, in <module>
    main()
  File "utils.py", line 35, in main
    decide_action(args, username, vagrant_cloud_token)
  File "utils.py", line 76, in decide_action
    build_all(username, vagrant_cloud_token)
  File "utils.py", line 235, in build_all
    build_image = get_box(box_info, username, vagrant_cloud_token)
  File "utils.py", line 265, in get_box
    last_updated_str, '%Y-%m-%dT%H:%M:%S.%fZ')
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_strptime.py", line 577, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7/lib/python3.7/_strptime.py", line 359, in _strptime
    (data_string, format))
ValueError: time data '2019-08-07T21:32:21.134-04:00' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Add Fedora 32

Fedora 32 has been released and needs to be added.

Add Ubuntu 20.04

Ubuntu 20.04 was released today and needs to be added.