churchers / vm-bhyve Goto Github PK
View Code? Open in Web Editor NEWShell based, minimal dependency bhyve manager
License: BSD 2-Clause "Simplified" License
Shell based, minimal dependency bhyve manager
License: BSD 2-Clause "Simplified" License
Hello.
I have issues with CentOS installation with hostname set in installer. It that case, root partition is named as /dev/mapper/centos_$HOSTNAME-root, resulting unbootable system. As boot.cmd is regenerated every time, this can be a problem :)
Suggestions
Is it possible to use NAT to allow guests to share the host wireless internet? I followed the instructions to enable the NAT with 'vm switch nat public on' and also started dnsmasq service, but DHCP did not resolve inside the VMs I tested. I did not add the physical wlan0 to the bridge, as I've read this will not work. Thanks for your time.
need a new options to set the tap address so guest can communicate with hosts.
when a vm start, a new tap device will add automatically.
(when tap device is not specified by config)
currently, there is no way to set the ip address of that tap device.
it would be nice to have such feature.
such as by adding the following to vm config
network0_tap_ifconfig="inet 10.0.0.1 netmask 255.255.255.0"
vm-bhyve will set the newly created tap device properly.
I know for other fs than zfs, snapshot is not easy for vm-bhyve to do.
but when using zfs as backend, it is possible.
I know when I use
root@qnas:~/vm-bhyve # vm clone win2012 win2013
vm-bhyve will snapshot the source vm, and clone to dest vm.
root@qnas:~/vm-bhyve # zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank/vm 10.7G 2.62T 3.93G /vm
tank/vm/win2012 6.76G 2.62T 104K /vm/win2012
tank/vm/win2012/disk0 6.76G 2.62T 6.76G -
tank/vm/win2013 76K 2.62T 100K /vm/win2013
tank/vm/win2013/disk0 8K 2.62T 6.76G -
root@qnas:~/vm-bhyve # zfs list -t snapshot
NAME USED AVAIL REFER MOUNTPOINT
tank/vm/win2012@342a3d90 0 - 104K -
tank/vm/win2012/disk0@342a3d90 0 - 6.76G -
Base on this, I think we can have snapshot
and rollback
command.
It seems that Linux guests (Centos at least) get stuck in grub-bhyve
on boot. Unfortunately it also seems to use 100% CPU while it's stuck.
In man page. BASIC SETUP.
dollar mark is not needed when syntax add to rc.conf
this will not work when somebody copy & paste that to rc.conf.
BASIC SETUP
Once vm is installed, create the directory which will store your virtual
machine configuration and data. This directory will be referred to as
$vm_dir throughput this man page.
Add the following into /etc/rc.conf
$vm_enable="YES"
$vm_dir="/your/vm/path"
$vm_list=""
$vm_delay="5"
I've got vm-bhyve 0.9.13 (??? there's no tag for this in the repo?) installed from packages on 10.2-RELEASE. I copied this VM out of VirtualBox into a flat file. It's mostly working, but I can't get it to boot automatically in vm-bhyve. The VM is currently running Ubuntu 15.04, but it's been around a while and it's upgraded from whatever the original install was, something between 12.04 and 13.10 probably.
The three issues I'm having:
guest="ubuntu"
, grub-bhyve doesn't seem to pick up /grub/grub.cfg
and boot automatically. I think this is because it's looking for /boot/grub.cfg
instead, but I'm not sure.configfile /grub/grub.cfg
I have in there. It then hangs spinning when its redirected stdio hits EOF.vm start
redirects to null and doesn't have a debug choice to redirect to a log file. So I have to use vm _run
by hand (not the worst thing, but I had to figure this all out).So currently I have to do vm start linux-apps
, then vm console linux-apps
and then manually type configfile /grub/grub.cfg
, and everything takes off from there.
Diffs between the install vm-bhyve and github master don't seem to show anything relevant. Anything else to try before I start seriously hacking on this?
As an alternative to installing from ISO images.
Examples:
# vm image import /path/to/FreeBSD-10.2-RELEASE-amd64.raw
Image of importXXXXXXXX created with UUID ed0badc7-e84a-4ca2-b3a2-c38b3c8d6156
Successfully imported FreeBSD-10.2-RELEASE-amd64.raw
# vm image provision ed0badc7-e84a-4ca2-b3a2-c38b3c8d6156 cloudvictim0
Then after some other process takes care of creating an OpenStack config drive ISO:
# vm configure
...edit disk1_name variable to point to ISO...
# vm start cloudvictim0
I don't know yet if an additional top-level vm
command would be welcome to choose the ISO; if it is then I can open another issue.
I know I can do it by myself. just wondering if this feature will be add in the future?
like following
vm rename vm_old_name vm_new_name
It appears that in some circumstances, grub-bhyve will open /dev/tap0, which creates this device if it doesn't already exist. I'm not sure of the exact logic, but this seems to only happen during an install.
Because grub-bhyve is opening this itself, vm-bhyve doesn't know about it and does not remove this tap device when finished. The end result is something like this -
It would be great if there was a way to create VMs in a different data store than the default vm_dir. Perhaps this could be a command line switch that is then added to the global config ? Or, all your data stores are in rc.conf and only those can be selected on the cmd line ?
Scenario: I have an ssd as the main boot drive and some VMs are living there. But, I'd like to create file storage and caching VMs on a spiny disk.
In my configuration my VMs share the host LAN interface (igb0). I have a vm switch configured (lan) which creates the bridge0 interface and binds to igb0:
switch_list="lan wan"
ports_lan="igb0"
ports_wan="igb3"
My rc.conf initially looked like this:
ifconfig_igb0="inet xx.xx.xx.5 netmask 255.255.255.0"
ifconfig_igb0_ipv6="inet6 xxxx::5 accept_rtadv"
After creating a test fbsd10 VM bound to 'lan', on boot that VM is unable to communicate with the host on IPv6 unless the host first sends a packet to the VM.
After some troubleshooting I was able to narrow this down to the bridge configuration. Per the FreeBSD handbook, the bridge0 interface must be the addressed interface. So, this is my 2nd configuration:
# new config moving IP to the bridge
cloned_interfaces="bridge0"
ifconfig_bridge0="addm igb0 up"
ifconfig_bridge0_alias0="inet xx.xx.xx.5 netmask 255.255.255.0"
ifconfig_bridge0_ipv6="inet6 xxxx::5 accept_rtadv"
ifconfig_igb0="up"
ifconfig_igb3="up"
The system comes up properly, but the vm script would not properly bind the tap interfaces to the bridges because they didn't have the vm script bridge identifiers. So, I added the bridge identifiers and manually specified both bridges:
# new config moving IP to the bridge
cloned_interfaces="bridge0 bridge1"
ifconfig_bridge0="addm igb0 up description vm-lan"
ifconfig_bridge1="addm igb3 up description vm-wan"
ifconfig_bridge0_alias0="inet xx.xx.xx.5 netmask 255.255.255.0"
ifconfig_bridge0_ipv6="inet6 xxxx::5 accept_rtadv auto_linklocal"
ifconfig_igb0="up"
ifconfig_igb3="up"
The vm script now binds tap interfaces to the bridges properly, and vm's work as expected. (I added auto_linklocal as because link-local addresses are not automatically added to bridge interfaces even with sysctl -w net.inet6.ip6.auto_linklocal=1)
However, because the vm switch configuration automatically configures bridge interfaces there is potential for breakage if switch configuration is modified using 'vm switch' commands.
It would be ideal if in the switch configuration supported manually specified bridge interfaces as well as bridge autoconfiguration. The bridge interface description could be configured by 'vm switch' to preserve existing behavior (in effect, vm would 'adopt' a manually configured bridge). In such case 'vm switch destroy' should not be able to destroy a bridge that was manually configured.
The resulting switch configuration for this feature might look something like this:
switch_list="lan wan"
bridge_lan="bridge0"
bridge_wan="bridge1"
Doing my first Bhyve install and trying to use vm-bhyve.
Don't see a mailing list. What is the best/appropriate place to ask a question?
# vm stop ns1
/usr/local/sbin/vm: ERROR: unable to locate process id for this virtual machine
root@bhyve1:/usr/local/bhyve # vm list
NAME GUEST CPU MEMORY AUTOSTART STATE
ns1 freebsd 2 512M No Bootloader (11691)
ns2 freebsd 1 256M No Stopped
ns_master freebsd 1 512M Yes [1] Running (10604)
owncloud_demo freebsd 2 1024M Yes [3] Running (6013)
pg freebsd 4 4096M Yes [2] Running (2012)
root@bhyve1:/usr/local/bhyve # ps aux |grep 11691
root 11691 100.0 0.0 553440 3884 3 R 1:13AM 0:41.21 bhyveload -c /dev/nmdm3A -m 512M -d /usr/local/bhyve/vm/ns1/ns1 ns1
Hi,
I have an FreeBSD 10.2-STABLE #0 r292683 amd64 with zfs compiled into the kernel.
On "vm init" I get "ERROR: ZFS support requested but ZFS not available".
Looking at vm-zfs, I noticed that the the script only checks, if the kernel module is loaded:
# check zfs running
kldstat -n zfs >/dev/null 2>&1
[ $? -ne 0 ] && __err "ZFS support requested but ZFS not available"
Adding "-m zfs" to the line seems to work, and should also find a loaded kernel module, but I can't test it.
Should I also (or instead) open an issue for the port on bugzilla?
Patch attached
vm-zfs.patch.txt
Would also be extra handy to support pci pass thru (including -S parameter for bhyveload/bhyve for wiring memory)
when create a vm from template or add disk to an existing vm. if disk type is zvol/zvol-sparse, there is no way to set the volblocksize when creating. while this parameter is a little importance, affect the guest's disk performance.
here is the compare result.
using file backend.
using zvol with 8k volblocksize
using zvol with 128k volblocksize
Hello,
I am running vm: Bhyve virtual machine managament v0.9.13
I have a switch which I have set to NAT mode:
# vm switch list
NAME TYPE IDENT VLAN NAT PORTS
public auto bridge0 - - igb0 tap0 tap1 tap2 tap3
natswitch auto bridge2 - yes -
The VM I am using is a Debian guest and it is set to use the natswitch
switch. It gets a private IP address fine but I am not able to ping out onto the internet.
I have multiple IPs on my physical interface, so I am not sure how pf's NAT will handle the NAT instruction in pf.conf. I tried changing the line
nat on igb0 from {172.16.2.0/24} to any -> (igb0)
to
nat on igb0 from {172.16.2.0/24} to any -> $EXT_IF
then loaded the ruleset into pf, but I am still not able to access the internet.
In tcpdump
I see the traffic on the bridge2
interface but I do not see any traffic leaving igb0
to the ping
command's destination.
Any tips would be appreciated.
Thanks,
Manas
I experienced a problem when running 'vm init', the file '.config/switch' doesn't exist and vm-bhyve does not create it. I manually created the file with 'touch .config/switch' and 'vm init' completed succesfully.
thanks
if I use
vm clone guest guest1
vm clone guest guest2
vm-bhyve will create snapshot of guest first, then
clone that snapshot to guest1/guest2
I think it would be nice to have a feature such as
vm clone guest@snapshot guest1
for this command, clone will not create snapshot first, using the snapshot from first argc.
It will void creating useless snapshot every clone.
Hi, thank you for you your scripts - exactly what I needed.
Q: Is anyone working on adding support for zvols?
It seems like a small amount (hehe) of rework around the $_disk bits for zvols ...
Would anyone else like to use tmux and/or screens for console in addition to nmdm?
I'm trying to use a configuration with dnsmasq. When I reboot the machine dnsmasq fails to start up because the vm service has not created the interfaces yet. Any idea how I could make this work?
Jan 11 01:27:33 aukaost dnsmasq[594]: unknown interface bridge0
Jan 11 01:27:33 aukaost dnsmasq[594]: FAILED to start up
Jan 11 01:27:33 aukaost root: /etc/rc: WARNING: failed to start dnsmasq
As of commit 4292fee 24/10, it's possible to run a SmartOS guest. The vm-bhyve
configuration is the same as windows.conf
, with the following changes:
guest="generic"
uefi="csm"
hostbridge="none"
comports="com1 com2"
Seeing as it's a UEFI guest and requires no specific loader, there's no point me adding another guest support function for it when generic
can be used.
Note that you'll need a copy of the UEFI CSM firmware which can be obtained from the same place as the normal UEFI firmware. The other important difference, that is only possible since the commit mentioned, is hostbridge="none"
. At the moment the hostbridge causes the virtio-net
network adapter to be presented to SmartOS in a way that causes it to panic. I believe this is down to a VirtIO bug in Illumos which has already been fixed in development versions.
The other issue with SmartOS is that it is designed to never boot from the hard disk. It is designed to run from a CD, or more commonly a network location. The hard disk is only used as persistent storage. Because of this, the guest always needs to be started in install mode. The first time SmartOS will enter a configuration menu to set up the system and store configuration on the hard disk. The next time it boots off the ISO it will boot straight to a login prompt.
vm install smartosguest smartos-latest.iso
If you use the normal vm start
command, it will fail to boot, as there's not actually any boot code or kernel installed to the hard disk. Unfortunately this does mean that you can't use the autostart (start on boot) option with SmartOS, as that uses the start
command.
If there are any requests for this to be supported, comment here. If people actually want it supported properly, I can add a bootiso="isofilename.iso"
configuration option, which can be used to specify an ISO file which should be used for every boot of a guest. That way you can vm start smartos
, and it will still boot off the ISO. It's not worth adding this if there's no requests for it though as SmartOS is a bit of a special case.
centos os type tries to boot centos 6 with something like:
linux /vmlinuz-2.6.32-573.el6.x86_64 root=/dev/mapper/centos-root
However in standard linux fashion CentOS has decided to arbitrarily rename foundational elements for no particular reason.
For CentOS 6 default, something like
root=/dev/mapper/VolGroup-lv_root
is needed instead.
I don't know how you wish to handle this as custom grub is already supported, but it was a bit of a surprise when CentOS 6 didn't work.
Currently, vm's NAT feature assumes that vm is the only user of dnsmasq on the host. This is not necessarily true - I currently use dnsmasq to provide local DNS and DHCP services. The current behavior of vm would overwrite my dnsmasq configuration if I were to inadvertently enable NAT functionality.
Rather than assuming that the current dnsmasq.conf may be overwritten vm should instead write a sample vm-specific dnsmasq configuration file that might be included by the administrator.
Here is the test result using iperf.
iperf server in FreeBSD host, and from Debian 8.0/Windows 2012 iperf guest, both guests are in same bridge.
testing in TCP mode, first result comes from Debian. follows by Windows 2012.
root@gateway:~ # iperf -s -i 1 -w 256k
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 256 KByte
------------------------------------------------------------
[ 4] local 10.40.0.254 port 5001 connected with 10.40.0.88 port 39817
[ ID] Interval Transfer Bandwidth
[ 4] 0.0- 1.0 sec 125 MBytes 1.05 Gbits/sec
[ 4] 1.0- 2.0 sec 119 MBytes 998 Mbits/sec
[ 4] 2.0- 3.0 sec 123 MBytes 1.03 Gbits/sec
[ 4] 3.0- 4.0 sec 120 MBytes 1.00 Gbits/sec
[ 4] 4.0- 5.0 sec 122 MBytes 1.02 Gbits/sec
[ 4] 5.0- 6.0 sec 120 MBytes 1.01 Gbits/sec
[ 4] 6.0- 7.0 sec 116 MBytes 976 Mbits/sec
[ 4] 7.0- 8.0 sec 121 MBytes 1.01 Gbits/sec
[ 4] 8.0- 9.0 sec 120 MBytes 1.01 Gbits/sec
[ 4] 9.0-10.0 sec 118 MBytes 989 Mbits/sec
[ 4] 0.0-10.0 sec 1.18 GBytes 1.01 Gbits/sec
[ 5] local 10.40.0.254 port 5001 connected with 10.40.0.168 port 49831
[ 5] 0.0- 1.0 sec 80.9 MBytes 679 Mbits/sec
[ 5] 1.0- 2.0 sec 52.1 MBytes 437 Mbits/sec
[ 5] 2.0- 3.0 sec 81.4 MBytes 683 Mbits/sec
[ 5] 3.0- 4.0 sec 88.2 MBytes 739 Mbits/sec
[ 5] 4.0- 5.0 sec 86.2 MBytes 723 Mbits/sec
[ 5] 5.0- 6.0 sec 82.8 MBytes 694 Mbits/sec
[ 5] 6.0- 7.0 sec 78.5 MBytes 658 Mbits/sec
[ 5] 7.0- 8.0 sec 69.6 MBytes 584 Mbits/sec
[ 5] 8.0- 9.0 sec 95.8 MBytes 803 Mbits/sec
[ 5] 0.0-10.0 sec 805 MBytes 676 Mbits/sec
in UDP mode
root@gateway:~ # iperf -u -s -i 1
------------------------------------------------------------
Server listening on UDP port 5001
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 10.40.0.254 port 5001 connected with 10.40.0.88 port 44112
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 93.3 MBytes 782 Mbits/sec 0.014 ms 1805/68344 (2.6%)
[ 3] 1.0- 2.0 sec 93.2 MBytes 782 Mbits/sec 0.010 ms 1739/68218 (2.5%)
[ 3] 2.0- 3.0 sec 92.6 MBytes 777 Mbits/sec 0.011 ms 2104/68135 (3.1%)
[ 3] 3.0- 4.0 sec 93.3 MBytes 783 Mbits/sec 0.010 ms 1675/68221 (2.5%)
[ 3] 4.0- 5.0 sec 92.3 MBytes 774 Mbits/sec 0.011 ms 2389/68217 (3.5%)
[ 3] 5.0- 6.0 sec 93.0 MBytes 780 Mbits/sec 0.008 ms 1838/68141 (2.7%)
[ 3] 6.0- 7.0 sec 93.0 MBytes 780 Mbits/sec 0.011 ms 1815/68156 (2.7%)
[ 3] 7.0- 8.0 sec 93.9 MBytes 788 Mbits/sec 0.013 ms 1251/68252 (1.8%)
[ 3] 8.0- 9.0 sec 92.5 MBytes 776 Mbits/sec 0.007 ms 2102/68052 (3.1%)
[ 3] 0.0-10.0 sec 929 MBytes 780 Mbits/sec 0.012 ms 18411/681408 (2.7%)
[ 3] 0.0-10.0 sec 1 datagrams received out-of-order
[ 4] local 10.40.0.254 port 5001 connected with 10.40.0.168 port 54637
[ 4] 0.0- 1.0 sec 20.2 MBytes 169 Mbits/sec 0.049 ms 0/14386 (0%)
[ 4] 1.0- 2.0 sec 25.1 MBytes 210 Mbits/sec 0.490 ms 0/17897 (0%)
[ 4] 2.0- 3.0 sec 21.7 MBytes 182 Mbits/sec 0.277 ms 0/15459 (0%)
[ 4] 3.0- 4.0 sec 21.7 MBytes 182 Mbits/sec 0.097 ms 0/15500 (0%)
[ 4] 4.0- 5.0 sec 23.8 MBytes 199 Mbits/sec 0.056 ms 0/16945 (0%)
[ 4] 5.0- 6.0 sec 21.8 MBytes 183 Mbits/sec 0.060 ms 0/15544 (0%)
[ 4] 6.0- 7.0 sec 22.1 MBytes 185 Mbits/sec 0.057 ms 0/15755 (0%)
[ 4] 7.0- 8.0 sec 22.8 MBytes 191 Mbits/sec 0.050 ms 0/16249 (0%)
[ 4] 8.0- 9.0 sec 22.9 MBytes 192 Mbits/sec 0.049 ms 0/16337 (0%)
[ 4] 0.0-10.0 sec 223 MBytes 187 Mbits/sec 1.065 ms 0/159175 (0%)
basically, windows 2012's result is very bad.
search for some tuning tips, but all does not working.
windows driver also update to virtio-win-0.1.110.
I don't know this is relate to Windows or FreeBSD or bhyve or virtio-net driver.
any performance tuning tips?
Here is the story...
root@bhyve:~ # vm snapshot vm01@test
root@bhyve:~ # vm rename vm01 newvm01
root@bhyve:~ # vm clone newvm01@test newvm02
mv: rename /zones/vm/newvm02/newvm01.conf to /zones/vm/newvm02/newvm02.conf: No such file or directory
tail: /zones/vm/newvm02/newvm02.conf: No such file or directory
Maybe for vm, the config name just a fixed name such as vm.config
?
vm-bhyve happily allows multiple guest disks. For UEFI guests this is currently limited to 3 although bhyveload|grub-bhyve
guests can have as many as bhyve will support.
However, while the first disk is created during vm create
, additional disks must be created by hand. I have considered moving the disk provisioning into _run
, so that disks are created at runtime if they don't already exist. If a new disk is added to the guest configuration, it will be automatically created next time the guest runs. The problem with this is that the configuration doesn't currently contain the disk size.
There are two possible solutions to this that I can see:
Add the disk size to the configuration.
When creating a new guest I will need to take the size given by the user and add that to the configuration. It may also make sense to update the sample templates to include a default size. The problem with this is if the user changes the size in the template. Either we end up out of sync with the real image, or we try and resize (dangerous). Also the user resizing the disk manually will make it out of sync with the configuration. All in all it seems messy to include the disk sizes in configuration.
Provide a vm add
command
Provide a command which can be used to add disks (and possibly other devices) to the configuration. This would be most useful for disks as it can take care of creating the image file or zvol in the correct place, as well as updating the configuration.
While I aim to keep the vm
command as simple as possible, I think it may be worth adding the second option. In most cases users can edit the configuration file (vm configure guest
) to make changes, but the extra complication of having to create the image file or zvol will make this command useful.
VM lockfiles (run.lock) are left behind on host reboot, preventing vm autostart from occuring on host reboot. This happens whether or not host is booted gracefully.
Manually removing lock file allows VM startup.
Suggested fix: Lock should be removed or ignored if VM isn't running on start (perhaps using /dev/vmm check?).
Since SVN r295124, FreeBSD 10 stable support booting Windows via UEFI.
the UEFI version checking should change from
1100000
to
1002509(FreeBSD 10.3 Beta1)
Hi!
I am planning of building a server with FreeBSD, ZFS and bhyve.
Aside of a debian VM, I want to create a Windows (Server 2012 R2) VM as well, according to the latest news this is now officially supported in bhyve!
Do you have any plans of creating a Windows template?
root@bhyve:/zones/vm/.templates # vm
vm: Bhyve virtual machine managament v0.9.16
...
root@bhyve:/zones/vm/.templates # vm create -t debian 170.vm02
/usr/local/sbin/vm: ERROR: invalid virtual machine name '170.vm02'
I think dot is a valid character for vm name.
Hi,
First of all, thanks for vm-bhyve! It makes bhyve a breeze.
I would like to get DHCP/DNS up and running for my VMs, but I am having some difficulties. (No DHCP)
dnsmasq is running
Config: http://pastebin.com/yZDwRdcL
proxy.conf:
# guest (required)
# The type of operating system this virtual machine will use.
# Different systems often require specific steps to load the
# virtual machine, or specific bhyve options. The generic option
# does no specific loading and can be used for standard uefi guests.
#
# Valid Options: freebsd,netbsd,openbsd,centos,centos-grub,alpine
# ubuntu,debian,windows,generic
guest="debian"
# uefi
# Tells bhyve that it should load the UEFI firmware. To use UEFI
# this can be any value other than [empty]/off/false/no/0. If
# it contains the string 'csm', the UEFI BIOS compatability (CSM)
# firmware will be used
#
# Default: no
#
uefi=""
# cpu (required)
# specify the number of cpu cores to give to the guest
#
cpu="1"
# memory (required)
# specify the amount of ram to give to the guest. This can be
# followed by M or G.
#
memory="512M"
# uuid
# This is set automatically by vm-bhyve when creating a new guest. Normally
# bhyve assigns a UUID at runtime based on host and guest name. This
# option allows you to specify a fixed UUID that will always be used. Remove
# this or leave blank to return to the normal bhyve behaviour.
#
uuid="529f0770-a973-11e5-b0eb-10bf487fe811"
# disk0_type (required)
# This specifies the emulation type for disk0. Please note that each disk requires
# at least a type and name.
#
# Valid Options: virtio-blk,ahci-hd
#
disk0_type="virtio-blk"
# disk0_dev
# The type of device used as the backing store for this disk. The default is 'file',
# which means a sparse file is used. This file is stored in the guest's directory.
# For the zvol options, the zvol must be directly under the guest dataset.
# There is also a 'custom' option, in which case the disk name should be the full path
# to the file or device you want to use.
#
# Default: file
# Valid Options: file,zvol,sparse-zvol,custom
#
disk0_dev="sparse-zvol"
# disk0_name (required)
# The name of the file or zvol for this disk. If the device type is 'custom', it should
# be the full path to whichever device or file you want to use
#
# This value is translated to a path as follows, based on disk0_dev
#
# DEVICE TYPE DISK NAME BHYVE PATH USED
# file 'disk0.img' -> '$vm_dir/$name/disk0.img'
# zvol|sparse-zvol 'disk0' -> '/dev/zvol/pool/dataset/path/guest/disk0'
# custom '/dev/da10' -> '/dev/da10'
#
disk0_name="zvol-proxy"
# network0_type
# This specifies the emulation type to use for the first network interface.
# Networking is not required, although this field is mandatory if you do want
# to add an interface
#
# Valid Options: virtio-net
#
network0_type="virtio-net"
network1_type="virtio-net"
# network0_switch
# The name of the virtual switch to connect this interface to. When starting the
# guest, if this switch cannot be found, or no switch is specified, the interface
# is still created but will not be connected to anything.
#
# All default templates use a switch called 'public', although it's perfectly
# reasonable to use other switch names that make sense in your environment
#
network0_switch="public"
network1_switch="external"
# network0_device
# If you do not want vm-bhyve to create a new interface, but use an existing
# one, enter the interface name here. This allows you to preconfigure the network
# device in a custom configuration, then instruct vm-bhyve to use that rather
# than create all interfaces dynamically at run time.
#
network0_device=""
network1+device=""
# network0_mac
# This allows you to specify a fixed mac address for this interface inside the guest.
# Without this option, bhyve will automatically assign a mac address to the
# interface.
#
network0_mac="58:9c:fc:00:7a:25"
network1_mac="58:9c:fc:00:e8:92"
Other block dev should be able to be used eg /dev/zvol/zroot/demo
disk0_name="/dev/zvol/zroot/owncloud-demo"
fails to boot because it appends that string to the vm_dir.
changing to:
disk0_name="owncloud-demo"
and creating a soft link to /dev/zvol/zroot/owncloud-demo works.
Probably there should be some sort of mechanism to pass extra params to bhyve too. Then some vm's needing some obtuse configuration can still be used in the framework.
I think it is nice to have "this" keyword as a vm name when the current directory is in the vm directory.
and also have a cd command.
such as
root@bhyve:~ # vm list
NAME GUEST CPU MEMORY AUTOSTART STATE
vm01 windows 2 4G No Stopped
root@bhyve:~ # vm cd vm01
root@bhyve:/zones/vm/vm01 #
root@bhyve:/zones/vm/vm01 # vm start this
root@bhyve:/zones/vm/vm01 #
it will be easier to type command, especially when vm name is very long and hard to remember.
It seems that Windows can be fairly intermittent about responding to vm stop
or vm stopall
. Most critically, this may cause problems shutting down if a Windows guest is running; I'm not sure whether FreeBSD will wait forever or eventually kill the process.
As with most other hypervisors, I would recommend that guests are safely stopped before trying to shutdown the host, although obviously the ideal would be to have vm stopall
work correctly. For Windows the best solution is to initiate a shutdown from inside the guest.
I don't believe this is an error with vm-bhyve
, as we merely run a kill BHYVE-PID
command. If for any reason we can't find the correct PID
, we output an error, so no output from vm-bhyve suggests it is finding the process ID and sending the kill
command.
I will need to see if any other bhyve users are experiencing this issue.
Up until now vm-bhyve
has worked like most other bhyve managers, and lets bhyve assign mac addresses to virtual interfaces (unless manually specified). According to documentation, bhyve uses the slot & function number, and device name to create a 'unique' mac address. There are a few problems with this:
As such I believe the best solution is to assign a fixed mac address to each interface. Once set, the guest will see the exact same MAC permanently, regardless of device/slot/host/etc.
freebsdguest was installed from a FreeBSD-10.2 ISO as per the instructions in the vm man page.
[root@hesseth /vm/vm-bhyve]# vm image create -d freebsdimage freebsdguest
Creating a compressed image, this may take some time...
Image of freebsdguest created with UUID c6feaf0d-ba26-11e5-9985-000c29330a8b
[root@hesseth /vm/vm-bhyve]# vm image provision c6feaf0d-ba26-11e5-9985-000c29330a8b
[root@hesseth /vm/vm-bhyve]# echo freebsd*/freebsd*.conf
freebsdclone/freebsdclone.conf freebsdguest/freebsdguest.conf
[root@hesseth /vm/vm-bhyve]# grep -h uuid freebsd*/freebsd*.conf
uuid="987971a8-ba23-11e5-9985-000c29330a8b"
uuid="987971a8-ba23-11e5-9985-000c29330a8b"
Should the UUID of the newly-created guest really be identical to that of the guest from which it was created? I realize that this probably doesn't have a lot of material effects on the operation of each individual guest, but the concept of uniqueness of UUIDs seems a bit abused here.
bhyve may limit the guest OS in the number of CPU cores and RAM. But sometimes it is necessary to limit the guest not only by the number of cores. You can use the limitation of resources through RCTL: https://wiki.freebsd.org/SummerOfCode2012/CPULimits
Also restrictions will be added soon to the disk subsystem: https://wiki.freebsd.org/RudolfTomori/IOLimits
Can you add a resource limit function? This will require a new settings in the config file and transfer bhyve's PID as a parameter for the RCTL.
relevant ifconfig:
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vm-public
ether 02:b9:60:11:33:00
nd6 options=1<PERFORMNUD>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 14 priority 128 path cost 2000000
member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 13 priority 128 path cost 2000000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 12 priority 128 path cost 2000000
member: bxe2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 3 priority 128 path cost 2000
bridge2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
description: vm-iscsi
ether 02:b9:60:11:33:02
nd6 options=1<PERFORMNUD>
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 5 priority 128 path cost 2000
tap3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
description: vmnet-demo-1-iscsi
options=80000<LINKSTATE>
ether 00:bd:28:85:26:03
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: active
Opened by PID 2538
# vm switch list
NAME IDENT VLAN NAT PORTS
public bridge0 - - bxe2
internal bridge1 4092 - bxe2
iscsi bridge2 - - ix0
relevant portion of conf file:
network0_type="virtio-net"
network0_switch="public"
network1_type="virtio-net"
network1_switch="iscsi"
relevant portion of debug file:
Oct 30 23:14:34: created network device tap2
Oct 30 23:14:34: created network device tap3
Oct 30 23:14:34: booting
Oct 30 23:14:34: bhyveload -c /dev/nmdm3A -m 1024M -d /usr/local/bhyve/vm/demo/demo demo
Oct 30 23:14:44: [bhyve options: -c 2 -m 1024M -AHP -U 17be9253-7e88-11e5-83f9-b82a72d9c6db]
Oct 30 23:14:44: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/usr/local/bhyve/vm/demo/demo -s 5:0,virtio-net,tap2 -s 5:1,virtio-net,tap3]
Oct 30 23:14:44: [bhyve console: -l com1,/dev/nmdm3A]
Oct 30 23:14:44: starting bhyve
Same as in issue #53 : kldstat -m instead of kldstat -n in vm-common fixes it.
Also, could you pretty please update the port?
Thanks.
When using vm _run guest
, the bhyve error message can be displayed in console like the following
root@bhyve:/zones/vm/debian # vm _run debian
Unable to setup memory (17)
while in vm start guest
, this error message cannot be captured in vm-bhyve.log
It will be nice to display that kind of log in vm-bhyve.log.
Oct 22 05:35:19: create file /zones/vm/debian/device.map
Oct 22 05:35:19: -> (hd0) /dev/zvol/zones/vm/debian/disk0
Oct 22 05:35:19: grub-bhyve -c /dev/nmdm1A -r hd0,msdos1 -m /zones/vm/debian/device.map -M 512M debian
Oct 22 05:35:25: [bhyve options: -c 1 -m 512M -AHP -U 2c208d8f-77f0-11e5-b01a-0021cc60a290 -S]
Oct 22 05:35:25: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/zones/vm/debian/disk0 -s 5:0,virtio-net,tap1 -s 6:0,passthru,0/22/0]
Oct 22 05:35:25: [bhyve console: -l com1,/dev/nmdm1A]
Oct 22 05:35:25: starting bhyve
Oct 22 05:35:25: bhyve exited with status 1
Oct 22 05:35:25: destroying network device tap1
Oct 22 05:35:25: stopped
On 10.2 RELEASE:
# vm init
cat: /usr/local/bhyve/vm/.config/switch: No such file or directory
when add tap device to vm switch, currently only act as a normal member. by
ifconifg bridge0 addm tap0
command. while tap device can be a span member
by using ifconifg bridge0 span tap0
we can specify this to vm config file. such as
network1_type="virtio-net"
network1_switch="private"
network1_switch_member_type="span"
and then when execute vm start command, vm bhyve will add this tap device to bridge as a span member.
here is the log.
root@bhyve:~/vm-bhyve # make install
/bin/mkdir -p /usr/local/sbin
/bin/mkdir -p /usr/local/lib/vm-bhyve
/bin/mkdir -p /usr/local/share/examples/vm-bhyve
install -m 0500 vm /usr/local/sbin/
install lib/* /usr/local/lib/vm-bhyve/
install sample-templates/* /usr/local/share/examples/vm-bhyve/
install rc.d/* /usr/local/etc/rc.d/
install: /usr/local/etc/rc.d/: No such file or directory
*** Error code 71
Stop.
make: stopped in /root/vm-bhyve
root@bhyve:~/vm-bhyve # mkdir -p /usr/local/etc/rc.d
root@bhyve:~/vm-bhyve # make install
/bin/mkdir -p /usr/local/sbin
/bin/mkdir -p /usr/local/lib/vm-bhyve
/bin/mkdir -p /usr/local/share/examples/vm-bhyve
install -m 0500 vm /usr/local/sbin/
install lib/* /usr/local/lib/vm-bhyve/
install sample-templates/* /usr/local/share/examples/vm-bhyve/
install rc.d/* /usr/local/etc/rc.d/
rm -f vm.8.gz
gzip -k vm.8
install vm.8.gz /usr/local/man/man8/
install: /usr/local/man/man8/: No such file or directory
*** Error code 71
Stop.
make: stopped in /root/vm-bhyve
root@bhyve:~/vm-bhyve # mkdir -p /usr/local/man/man8
root@bhyve:~/vm-bhyve # make install
/bin/mkdir -p /usr/local/sbin
/bin/mkdir -p /usr/local/lib/vm-bhyve
/bin/mkdir -p /usr/local/share/examples/vm-bhyve
install -m 0500 vm /usr/local/sbin/
install lib/* /usr/local/lib/vm-bhyve/
install sample-templates/* /usr/local/share/examples/vm-bhyve/
install rc.d/* /usr/local/etc/rc.d/
rm -f vm.8.gz
gzip -k vm.8
install vm.8.gz /usr/local/man/man8/
I have a problem with pf firewall. After reboot, my virtual hosts have no access to the world. I have to manually run pfctl -f /etc/pf.conf
I think this is because bridges and tap devices are not created at the time pf initializes itself. Any tips to work around the issue?
So at this point I have a number of VM's managed by this tool and given the diversity in the networking setup it's getting rather difficult to easily figure out what is doing what on the networking side.
It would be great if functionality like:
vm list <name> / <all>
printed out the vm details / all vm details.
And/or if there was some built-in method of listing out vm nic(s)/switch(s)
Much of the interface here reminds of the Xen/xm days and in that vein I'm asking for:
xm network-list
I think Linux distribution is a mess.... so many distributions.
and since bhyve does't support vga output yet, it is better to have some HOW-TOs to particular Linux Distributions.
and here is some tips on How to install Redhat Enterprise Linux 6.5 under vm-bhyve.
Any Linux Virtual-Host will be started by the following command where the partition "msdos1" is hard coded.
grub-bhyve -r hd0,msdos1 -m "${vm_dir}/${_name}/device.map" -M "${_memory}" "${_name}
this should not be hard coded to msdos1, you cant expect msdos1 to be the boot-partition. There should be a config option for this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.