| *** deadalnix has joined #openvswitch | 00:35 | |
| *** deadalnix_ has quit IRC | 00:36 | |
| *** deadalnix has quit IRC | 02:05 | |
| *** acidfu has quit IRC | 03:57 | |
| *** acidfu has joined #openvswitch | 04:03 | |
| *** thaller has quit IRC | 04:07 | |
| *** thaller has joined #openvswitch | 04:07 | |
| *** acidfu has quit IRC | 04:07 | |
| *** yamamoto has joined #openvswitch | 04:56 | |
| *** yamamoto has quit IRC | 05:20 | |
| *** yamamoto has joined #openvswitch | 05:20 | |
| *** yamamoto has quit IRC | 06:25 | |
| *** yamamoto has joined #openvswitch | 07:30 | |
| *** fdangelo has quit IRC | 07:42 | |
| *** fdangelo has joined #openvswitch | 07:42 | |
| *** slaweq has joined #openvswitch | 07:53 | |
| *** yamamoto has quit IRC | 07:56 | |
| *** yamamoto has joined #openvswitch | 08:00 | |
| *** yamamoto has quit IRC | 08:02 | |
| *** amorenoz_ has joined #openvswitch | 08:12 | |
| *** amorenoz has quit IRC | 08:14 | |
| *** yamamoto has joined #openvswitch | 08:23 | |
| *** slaweq has quit IRC | 08:25 | |
| *** dobson has quit IRC | 08:32 | |
| *** dobson has joined #openvswitch | 08:32 | |
| *** yamamoto has quit IRC | 09:01 | |
| *** yamamoto has joined #openvswitch | 09:09 | |
| *** yamamoto has quit IRC | 09:19 | |
| *** thaller has quit IRC | 09:51 | |
| *** yamamoto has joined #openvswitch | 10:00 | |
| *** yamamoto_ has joined #openvswitch | 10:14 | |
| *** yamamoto has quit IRC | 10:14 | |
| *** mdgray has joined #openvswitch | 10:17 | |
| *** mdgray has quit IRC | 11:10 | |
| *** amorenoz_ has quit IRC | 11:19 | |
| *** slaweq has joined #openvswitch | 11:32 | |
| *** thaller has joined #openvswitch | 11:57 | |
| tryauuum | bdonnahue2: "how to build a kvm vm and connect it to an overlay" — I don't know if you have promlems with connecting a VM to a openvswitch network, but if you do, simply use libvirt | 12:48 |
|---|---|---|
| tryauuum | if you put "<virtualport type='openvswitch'/>" in a VM's xml, libvirt will create an OVS port when VM is started and delete it when VM is stopped, pretty neat | 12:50 |
| tryauuum | Buy the way, has anyone succeed in running https://github.com/openstack/neutron/tree/master/vagrant/ovn/sparse in vagrant? I have some weird errors with ovn-vtep instance | 12:51 |
| tryauuum | are there maybe some hidden steps? e.g. maybe I shouldn't use master branch? | 12:52 |
| tryauuum | maybe I should use a specific version vagrant? Or run vagrant on a specific OS? | 13:02 |
| *** oanson has joined #openvswitch | 13:02 | |
| *** oanson_ has quit IRC | 13:03 | |
| *** acidfu has joined #openvswitch | 13:23 | |
| *** yamamoto_ has quit IRC | 13:44 | |
| *** yamamoto has joined #openvswitch | 13:47 | |
| *** yamamoto has quit IRC | 13:52 | |
| bdonnahue2 | tryauuum, good morning | 13:54 |
| bdonnahue2 | new to the spaces... does virt-install use libvirt? | 13:54 |
| bdonnahue2 | oops i mean virsh net-define... looks like it does | 13:55 |
| tryauuum | bdonnahue2 yeah, virsh connects to libvirt daemon and is a part of libvirt | 13:58 |
| tryauuum | but I think virsh net-define doesn't define openvswitch networks.. | 13:59 |
| tryauuum | you have to use "virsh edit" and manually change interface type to libvirt — https://docs.openvswitch.org/en/latest/howto/libvirt/ | 14:00 |
| bdonnahue2 | tryauuum, thanks, taking a look at that link | 14:00 |
| tryauuum | I think all "net-*" virsh commands are pointless when using openvswitch. Since they are mostly for IP accounting and dhcpd.. and libvirt can't do those things with openvswitch | 14:03 |
| bdonnahue2 | good to know, thanks | 14:05 |
| larsks | bdonnahue2: tryauuum: you can absolutely use virsh net-* commands with openvswitch. Create an OVS network using virsh net-create makes it super easy to attach new virtual machines to the associated OVS bridge, although even without the libvirt network it's still pretty easy. | 14:31 |
| larsks | bdonnahue2: I put together a complete walkthough of getting a pair of virtual machines talking over a vxlan overlay network, you can find it here: https://blog.oddbit.com/post/2021-04-17-vm-ovs-vxlan/ | 14:34 |
| larsks | If you have any questions or want more detail in parts, let me know. | 14:34 |
| bdonnahue2 | larsks, awesome! taking a look now!! | 14:36 |
| bdonnahue2 | larsks, is there any sanity check we can do after setting up the overlay network but before setting up the vm? how would i know the commands worked and a connection was made? | 14:40 |
| larsks | You could assign an address to the `br0` interface on both systems and verify connectivity that way. | 14:41 |
| bdonnahue2 | so the address would be some arbitrary address for the subnet i want to run through the tunnel? | 14:42 |
| larsks | E.g for the configuration in that post, `ip addr add 10.0.0.20/24 dev br0` on node0 and `ip addr add 10.0.0.21/24 dev br0` on node1, and then from node0 `ping 10.0.0.21' | 14:42 |
| larsks | The tcpdump output for that should look pretty much the same as what's in the post. | 14:43 |
| bdonnahue2 | ahh ok, thanks, still reading | 14:43 |
| bdonnahue2 | so one thing that is confusing me, in some other guides I see people connecting their hosts's eth0 to a bridge, but here i dont see that happening... what is the magic that would get the ICMP packats from the br0 to the eth0? is it some kind of lookup based on the routing table and remote ip for the tunnel? | 14:45 |
| larsks | There is no magic. VXLAN is just a UDP protocol; getting vxlan packets from one host to another is just like getting any other UDP/TCP traffic to connect: look at your routes, send it to the remote system. | 14:46 |
| *** yamamoto has joined #openvswitch | 14:47 | |
| bdonnahue2 | larsks, ok makes sense | 14:49 |
| bdonnahue2 | im seeing an odd issue getting br0 up: https://pastebin.com/Dw33Y8aD | 14:49 |
| * larsks looks | 14:49 | |
| larsks | That is odd. If you create a new ovs bridge with a different name, do you see the same issue? E.g., if you run `ovs-vsctl add-br br1`, does that show up in `ip link`? | 14:50 |
| bdonnahue2 | looks like there is an error creating the device (after deleting everything) i think i missed a package install step my bad | 14:50 |
| bdonnahue2 | ovs-vsctl: Error detected while setting up 'br0'. See ovs-vswitchd log for details. | 14:50 |
| bdonnahue2 | ovs-vsctl: The default log directory is "/var/log/openvswitch". | 14:50 |
| bdonnahue2 | let me rerun the package installs and see if that fixes things... maybe i borked a copy/paste | 14:51 |
| bdonnahue2 | my sencos 8 box is saying theres no openvswitch2.15 (im on 2.12, looks like i could go to 2.13) | 14:53 |
| bdonnahue2 | is that an issue? | 14:53 |
| larsks | I don't know. openvswitch2.15 is what's in the centos-release-openstack-victoria repository I used in the blog post. | 14:55 |
| larsks | I think this should all work with earlier versions. | 14:56 |
| bdonnahue2 | ok, updating the packages etc cleared up the error creating br0... back on track | 14:58 |
| bdonnahue2 | still on 2.13 ovs but so far no issue | 14:58 |
| *** yamamoto has quit IRC | 14:58 | |
| *** deadalnix has joined #openvswitch | 15:00 | |
| bdonnahue2 | larsks, my test of the br0 was successful. here are the commands incase you think it would be worth adding to the article you wrote: https://pastebin.com/M6K9uvqF | 15:05 |
| larsks | Thanks! | 15:06 |
| larsks | bdonnahue2: I have to take off shortly, but I was curious if everything is working out? | 15:21 |
| bdonnahue2 | still working, but so far so good | 15:22 |
| bdonnahue2 | the vm creation step is what i am on now | 15:23 |
| bdonnahue2 | almost got things working | 15:23 |
| bdonnahue2 | just man-paging the command, on the last subcommand now for the virt-install | 15:24 |
| larsks | +1 | 15:24 |
| bdonnahue2 | seeing this error: ERROR Couldn't create storage volume 'test1.qcow2': 'inaccessible backing store volume centos-8-stream.qcow2: No such file or directory' | 15:27 |
| bdonnahue2 | test1 is my vm name | 15:27 |
| bdonnahue2 | in previous experiences i had always created a storage disk first, and then installed the os | 15:28 |
| bdonnahue2 | looking at the man, i inferred it would be created for me | 15:28 |
| bdonnahue2 | as part of the storage pool | 15:28 |
| bdonnahue2 | ahhh, ok i missed a step, this is a base image that needs to be downloaded | 15:29 |
| bdonnahue2 | sorry, new to libvirt | 15:29 |
| *** deadalnix has quit IRC | 15:55 | |
| bdonnahue2 | larsks, i am stuck at the section "Configure networking for vm0.0" | 16:05 |
| bdonnahue2 | so you say to get the address for the new vm | 16:05 |
| bdonnahue2 | (note i did not use a base image but installed a vm from iso) | 16:06 |
| bdonnahue2 | and i do not see the vnet2 interface you mentioned (i see no interface) | 16:06 |
| bdonnahue2 | on my system i do see vnet devices | 16:06 |
| bdonnahue2 | so i am assuming i will need to configure the networking inside the vm and then vnet5 would show up for example? | 16:06 |
| larsks | bdonnahue2: I'm back :). Just so we're clear, when you run "virsh domifaddr vm0.0" you don't see anything in the output? | 16:09 |
| larsks | (you're running that on the host, not in the vm) | 16:09 |
| larsks | *inside* the vm, you'll have "eth0" and "eth1" | 16:09 |
| bdonnahue2 | yes running on the host, not the vm, i see nothing | 16:09 |
| bdonnahue2 | inside the vm I have two interfaces | 16:09 |
| larsks | I guess you logged in on the vm console? | 16:10 |
| bdonnahue2 | looking at the mac, they match vnet5 and 6 on my host | 16:10 |
| bdonnahue2 | yes, vm console through cockpit | 16:10 |
| larsks | Does eth0 in the vm have an address? | 16:10 |
| bdonnahue2 | no | 16:14 |
| bdonnahue2 | but my host network is also not having a dhcp server right now | 16:14 |
| bdonnahue2 | (idk if that factors in) | 16:14 |
| larsks | It definitely does. Those instructions assume you're using the standard libvirt "default" network, which includes a DHCP server. If you've modified your libvirt config somehow, all bets are off. | 16:14 |
| larsks | Fortunately, that interface isn't important if you're happy to log in on the console instead. | 16:15 |
| larsks | You can still set up eth1 and take advantage of the overlay network. | 16:15 |
| larsks | Without a dhcp server, eth0 won't get an address so "virsh domifaddr" isn't going to return anything (because there's nothing to return). | 16:15 |
| bdonnahue2 | larsks, makes sense | 16:15 |
| bdonnahue2 | so i was ale to get the vms to ping each other but not the outside world | 16:16 |
| bdonnahue2 | i think that "other" interface is still borked | 16:16 |
| larsks | Right, that's expected. Because of your network config they don't have a route to the outside world. | 16:16 |
| larsks | You need to provide them with a default route, which normally they would get via the libvirt default network. | 16:16 |
| larsks | With an appropriate configuration that route could be via the overlay network (give your hosts an address on the overlay network, set your default route to the host ip, configure masquerading on the host, etc) | 16:17 |
| larsks | But it's much easier just to let libvirt take care of that for you. | 16:17 |
| bdonnahue2 | ok, so if I am understanding you correctly, i need to run some command on my host so that is assignes an IP to the vnet device and creates the appropriate route on the host? | 16:19 |
| bdonnahue2 | my vm's routing table looks correct | 16:20 |
| larsks | Not exactly, no. The vnet device on the host doesn't get an address. | 16:20 |
| larsks | Your vm routing table *can't* look correct because it won't have a default route. | 16:20 |
| larsks | Ideally, just use the standard libvirt default network config, which will create a virtual bridge and a dhcp server for your virtual machines. | 16:20 |
| larsks | Otherwise, you need to assign an appropriate address to eth0 in the vm, and configure a default route to an appropriate gateway. | 16:21 |
| bdonnahue2 | larsks, i modified the vm's ifcfg scripts so that the "eth0" interface is the defroute | 16:22 |
| bdonnahue2 | maybe that was a mistake ... | 16:22 |
| larsks | No, that's correct, as long as your default route points to an appropriate gateway. | 16:23 |
| larsks | I mean, I expect your default route to be out eth0. If you actually configured eth0's address *as the gateway* that would be incorrect. | 16:23 |
| bdonnahue2 | hmmm... looking at the vm's routing tables and ip scripts, things look correct | 16:24 |
| larsks | Do you want to post your network config somewhere? At least the outputs of `ip route` and `ip addr` inside the vm, and also the output of `virsh dumpxml vm0.0` on the host. | 16:25 |
| bdonnahue2 | sure... trying to figure out how to get it in text format, but since its not networked i cant. would a screenshot be ok? | 16:27 |
| bdonnahue2 | https://imagebin.ca/v/5ylZnpzT5TxM | 16:28 |
| bdonnahue2 | we see that enp1s0 is configured to have an ip consistent with my lan subnet 15.0.0.0/8 | 16:29 |
| larsks | If you could post the output of the specific commands I mentioned that would help somewhat. I'm particularly interested in the output of the `virsh dumpxml` command, because that will show us how the vms are connected on the host (and would give us something to which we can apply tcpdump for diagnostics). | 16:32 |
| larsks | Are you able to ping the gateway address itself (15.1.1.1) from inside the vm? | 16:33 |
| bdonnahue2 | ahh im sorry ill send those commands now | 16:35 |
| bdonnahue2 | and no, i cannot ping the gateway | 16:35 |
| bdonnahue2 | here are the commands from inside the vm: https://imagebin.ca/v/5ylcRRKXSbZV | 16:37 |
| bdonnahue2 | larsks, bahahah... patebin is telling me the xml has been detected as "offensive content" ... i guess i really messed up the config hahaha | 16:39 |
| larsks | That's why nobody uses pastebin. | 16:39 |
| larsks | That, and all the ads. | 16:40 |
| bdonnahue2 | https://paste.centos.org/view/35a2ba83 | 16:40 |
| bdonnahue2 | thats the dumpxml^^ | 16:40 |
| larsks | Yup, looking. | 16:40 |
| larsks | On your host, can you run 'ip addr show virbr0' and 'brctl show' (or 'bridge link' if 'brctl' isn't available) | 16:41 |
| larsks | ?\ | 16:41 |
| bdonnahue2 | https://paste.centos.org/view/c2cb5d50 | 16:43 |
| bdonnahue2 | i see an odd subnet there on virtbr0... must have been created by default or something? | 16:44 |
| larsks | That's the default libvirt network. | 16:44 |
| larsks | You've misconfigured your vm networking. Your virtual machines are attached to the 15.0.0.0/8 network. | 16:44 |
| larsks | They're attached to the libvirt 192.168.122.0/24 network. | 16:45 |
| larsks | They need addresses on that network, and they should be setting 192.168.122.1 as their default gateway. | 16:45 |
| larsks | If you want them connected to your 15.0.0.0/8 network, you would need to create a different bridge (one that includes a physical interface on that network) and attach them to that instead of the default network. | 16:45 |
| larsks | Sorry, typo above: "Your virtual machines are attached to the 15.0.0.0/8 network." --> "Your virtual machines are **NOT** attached to the 15.0.0.0/8 network." | 16:46 |
| bdonnahue2 | larsks, what is a "libvirt network" is that some type of internal network? | 16:47 |
| bdonnahue2 | like a docker network? something NAT'ed from the LAN? | 16:47 |
| larsks | Your virtual machines are attached to a bridge named "virbr0", which has address 192.168.122.1. There is no physical device attached to that bridge. | 16:51 |
| larsks | When libvirt creates the default network, it creates (a) the bridge, and (b) sets up NAT rules so that your virtual machines have outbound connectivity. | 16:51 |
| larsks | See e.g. https://wiki.libvirt.org/page/Networking#NAT_forwarding_.28aka_.22virtual_networks.22.29 | 16:51 |
| bdonnahue2 | ahh, so if I wanted them connected to the lan, I would create a br1, add the host's NIC to the br1, and then create a vnet off br1? | 16:53 |
| larsks | pretty much, yes. You wouldn't create the vnet yourself; that would get created automatically if you started a vm with --network bridge=br1, for example. | 16:55 |
| larsks | I have to run off again, but I'll poke my head in later today to see how things are going. | 16:55 |
| bdonnahue2 | ok sounds good. thanks so much larsks | 16:58 |
| *** moldorcoder7 has quit IRC | 17:15 | |
| *** deadalnix has joined #openvswitch | 17:17 | |
| *** moldorcoder7 has joined #openvswitch | 17:28 | |
| *** deadalnix has quit IRC | 17:34 | |
| *** slaweq has quit IRC | 19:32 | |
| *** yamamoto has joined #openvswitch | 19:52 | |
| *** yamamoto has quit IRC | 19:58 | |
| *** deadalnix has joined #openvswitch | 21:23 | |
| larsks | bdonnahue2: hey, how did things work out? | 21:29 |
| bdonnahue2 | larsks, i got things working thanks! | 21:29 |
| bdonnahue2 | very happy | 21:29 |
| larsks | Awesome! | 21:29 |
| bdonnahue2 | i am now looking at this project: https://plvision.eu/rd-lab/open-vmonitor | 21:30 |
| bdonnahue2 | i was originally considering openstack,... but it's so hard to setup compared to other solutions | 21:30 |
| larsks | I guess it depends what you want to do: if you're just looking for a simple virtual machine management solution, openstack isn't it. But for a multi-tenant, API-based cloud resource scheduler, it's a reasonable (if complex to configure) solution. | 21:31 |
| larsks | And you always need to ask yourself if you even really need virtual machines, or if a container-based solution will suit you as well or better. | 21:31 |
| *** deadalnix has quit IRC | 21:33 | |
| *** fdangelo_ has joined #openvswitch | 22:04 | |
| *** fdangelo has quit IRC | 22:06 | |
| *** fdangelo__ has joined #openvswitch | 22:33 | |
| *** fdangelo_ has quit IRC | 22:35 | |
| bdonnahue2 | larsks, true. looking into k8 as well | 22:55 |
| bdonnahue2 | one thing i wanted to ask you about the vxlan: so i need a port for each host? eg. a tunnel spanning three hosts, each host neets two connections | 22:56 |
| bdonnahue2 | options:remote_ip= maybe accepts multiple ips? | 22:56 |
| larsks | Sort of. You'll have more reliable connectivity that way. Given three hosts, you could have A<->B and C<->B, but then if B goes down A and C no longer have connectivity with each other. | 22:57 |
| larsks | remote_ip does not accept multiple ips, you will need to create a new vxlan interface for each connection. | 22:58 |
| bdonnahue2 | larsks, ok thanks! | 22:58 |
| larsks | I think you don't actually need two connections -- in the config we created today, I think you could just have A->B and things would work fine. Worth trying out! | 22:58 |
| larsks | I might try that later and maybe update the post. Time to eat dinner now :) | 22:59 |
| *** moldorcoder7 has quit IRC | 22:59 | |
| bdonnahue2 | bon apetite | 23:05 |
| *** moldorcoder7 has joined #openvswitch | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!