| tasker | crap. | 00:03 |
|---|---|---|
| tasker | SSH timeout during the run of a playbook. | 00:03 |
| tasker | that'S how long it took. | 00:03 |
| tasker | /sigh | 00:03 |
| *** jwitko has quit IRC | 00:04 | |
| *** jwitko has joined #openstack-ansible | 00:04 | |
| tasker | I'm going home. I hope someone can shed some light on what OSA uses to determine the need to reinstall the virtual environments. | 00:05 |
| *** acormier has joined #openstack-ansible | 00:06 | |
| cloudnull | tasker: sorry. was not watching | 00:06 |
| tasker | no worries. | 00:07 |
| SamYaple | in the openstack-ansible gates, do you take advantage of the prebuilt wheels in infra? or just use it as a pypi mirror | 00:07 |
| cloudnull | the venvs are created on the repo server | 00:07 |
| cloudnull | and downloaded within the role. | 00:07 |
| tasker | i'm a "stream of conciousness" debugger. | 00:07 |
| cloudnull | SamYaple: we use the wheels from infra | 00:07 |
| SamYaple | cloudnull: any issues with that? | 00:07 |
| cloudnull | ** when they're available | 00:07 |
| SamYaple | heh | 00:07 |
| cloudnull | none that i'm aware of? | 00:08 |
| cloudnull | tasker: was it hanging connecting? | 00:08 |
| cloudnull | the setup-openstack playbook is just a meta playbook | 00:08 |
| tasker | no. the playbook was chugging along. it was my ssh to the staging host that timed-out. | 00:08 |
| SamYaple | im considering consuming those and publishing the resulting artifact to dockerhub, seem like a bad idea? | 00:08 |
| cloudnull | which calls the various os-SERVICE-install.yml playbooks | 00:08 |
| cloudnull | oh. | 00:08 |
| tasker | right. what I'm hoping to find an answer to is: OSA is largely idempotent. but, how does OSA determine that it needs to redo some work, like reinstall a virtual environment? | 00:09 |
| cloudnull | tasker: what version ? | 00:09 |
| tasker | 14.2.8 | 00:09 |
| *** weezS has joined #openstack-ansible | 00:10 | |
| cloudnull | the venvs themselves should only download when the version changes. | 00:10 |
| tasker | because this last run it looked and acted like it was reinstalling _all_ of openstack. | 00:10 |
| cloudnull | its is going to churn through a lot of those tasks, but it should not be reinstalling everything | 00:10 |
| cloudnull | unless the containers were destroyed? | 00:11 |
| cloudnull | SamYaple: they seem to work well enough | 00:11 |
| cloudnull | and make a fairly large impact in the speed of the gate | 00:11 |
| DimGR | cloudnull when should we expect pike to be stable enough in prod ? | 00:11 |
| cloudnull | DimGR: it should be stabalish now | 00:11 |
| tasker | odd, that. because I pushed 14.2.8 a while ago. then I started the opertaing system upgrades. i took out one-third of the containers, but it was doing _alot_ of work on the other two thirds. i don't know for sure but it looked like it was reinstalling the venv on everything. | 00:12 |
| cloudnull | but you might want to give it a week or two | 00:12 |
| DimGR | ok | 00:12 |
| tasker | especially considering that it took over two hours to do. | 00:12 |
| cloudnull | with all the zuul shinanigans going on there may be issues not getting fixed atm | 00:12 |
| DimGR | i have time to wait | 00:12 |
| SamYaple | cloudnull: awesome. thanks man! | 00:12 |
| cloudnull | anytime ! | 00:12 |
| tasker | let me see if I can find some evidence to support my theory. | 00:13 |
| cloudnull | ++ | 00:13 |
| cloudnull | is developer_mode set anywhere? | 00:13 |
| tasker | sure -- external to the keyboard | 00:14 |
| cloudnull | haha | 00:14 |
| tasker | something in a user_config? | 00:14 |
| cloudnull | no. it'd be a var set in something user_variables.yml etc. | 00:14 |
| cloudnull | that said, we did a lot of work for rolling upgrades which can be noticeably slow. | 00:15 |
| cloudnull | maybe your seeing that? | 00:15 |
| tasker | ok .. here's neutron on compute-1, "Oct 2 19:43 neutron-14.2.8". the 2 is when I first pushed 14.2.8. and here's nova on the same host, "Oct 4 20:15 nova-14.2.8" | 00:18 |
| tasker | it did recreate the venv there. | 00:19 |
| tasker | I'll grant you that I ran the playbook without a --limit | 00:19 |
| tasker | I was hoping to take advantage of OSA's idempotency. | 00:20 |
| *** acormier has quit IRC | 00:20 | |
| cloudnull | have you rebuilt the venvs since you pushed to 14.2.8? | 00:21 |
| tasker | is it because I rebuilt the repo containers and the packages there are newer than what was originally used to install 14.2.8? | 00:21 |
| tasker | yes | 00:22 |
| tasker | because I'm starting the operating system upgrades in the control-plane. | 00:22 |
| cloudnull | each venv is built and stored along side the sha256 (it might be md5) of the constructed venv . | 00:22 |
| tasker | and I needed the xenial tarballs | 00:22 |
| cloudnull | if that changed it would update to whatever is latest. | 00:22 |
| tasker | of course it did. | 00:22 |
| cloudnull | ah. ok | 00:23 |
| tasker | and I started at the primary repo container, repo_all[0] | 00:23 |
| * tasker deflates | 00:23 | |
| tasker | i need scotch. | 00:23 |
| tasker | or pool. | 00:23 |
| tasker | or a dragon to stab. | 00:23 |
| cloudnull | why not both ! | 00:23 |
| tasker | inverse proportinalities. | 00:23 |
| tasker | ok. well, mystery solved. | 00:24 |
| cloudnull | sorry? | 00:24 |
| tasker | of why OSA determined that it needed to reinstall the venvs. | 00:25 |
| cloudnull | now, re-installing the venvs should be pretty quick. it just grabs a tarball from the repo server and extracts it in place. | 00:26 |
| *** dxiri has quit IRC | 00:26 | |
| cloudnull | were you seeing that take a long time too? | 00:26 |
| tasker | using some prior information, I used "openstack-ansible repo-install --list-hosts --limit 'repo_all[0]'" to figure out which third of the control plane to start on. so I started on controller-2. this required destroying all the lxcs. so it took out the "master" repo container. after a rebuild, everything would have been different. | 00:27 |
| tasker | kinda sorta. | 00:27 |
| tasker | especially during nova. it did each piece in turn. it didn't glob or parallelize anything during the install of nova. | 00:27 |
| cloudnull | yea, thats a result of the rolling upgrades. | 00:28 |
| cloudnull | compute should be faster than control plane but its largely been serialized. | 00:28 |
| cloudnull | nova specifically that is | 00:29 |
| tasker | ok. so, lesson learned, don't use "setup-openstack". | 00:30 |
| tasker | or at least increase the SSH timeout if I do. | 00:30 |
| cloudnull | I normally fire up tmux | 00:31 |
| cloudnull | and let it do its thing | 00:31 |
| tasker | i have not used that. | 00:31 |
| cloudnull | its like screen | 00:32 |
| tasker | oh yeah | 00:32 |
| tasker | eh. i didn't think about it. | 00:32 |
| tasker | (but, shhh. don't tell my boss. he's a screen junkie) | 00:32 |
| tasker | so that makes TWO lessons learned. | 00:32 |
| cloudnull | http://paste.openstack.org/show/622708/ | 00:33 |
| cloudnull | that just fires it off into the background | 00:33 |
| cloudnull | then `tmux attach` to check in on it | 00:33 |
| tasker | neat | 00:33 |
| *** poopcat has quit IRC | 00:33 | |
| tasker | ^M vs [enter]? | 00:34 |
| cloudnull | https://tmuxcheatsheet.com/ | 00:34 |
| cloudnull | ^ a good reference on general commands | 00:34 |
| *** jwitko has quit IRC | 00:35 | |
| *** poopcat has joined #openstack-ansible | 00:35 | |
| tasker | thanks. | 00:35 |
| *** jwitko has joined #openstack-ansible | 00:35 | |
| tasker | I'm off. I'll pick up the pieces tomorrow. | 00:35 |
| tasker | thanks for your help, cloudnull. | 00:35 |
| tasker | thank you, so very much. | 00:35 |
| cloudnull | take care | 00:35 |
| cloudnull | have much scotch | 00:35 |
| cloudnull | :D | 00:35 |
| tasker | . ) | 00:35 |
| cloudnull | cloud runs on scotch | 00:35 |
| *** albertcard1 has joined #openstack-ansible | 00:40 | |
| *** poopcat has quit IRC | 00:40 | |
| spotz | cloudnull: Can it also run on Tequilla? | 00:48 |
| *** woodard has joined #openstack-ansible | 00:57 | |
| *** woodard_ has quit IRC | 00:57 | |
| *** weezS has quit IRC | 01:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 01:08 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 01:08 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 01:08 |
| *** acormier has joined #openstack-ansible | 01:10 | |
| *** acormier has quit IRC | 01:20 | |
| *** acormier has joined #openstack-ansible | 01:21 | |
| *** acormier_ has joined #openstack-ansible | 01:26 | |
| *** acormier has quit IRC | 01:29 | |
| *** acormier_ has quit IRC | 01:36 | |
| *** jhesketh has quit IRC | 01:51 | |
| *** jhesketh has joined #openstack-ansible | 01:51 | |
| *** dave-mcc_ has quit IRC | 01:57 | |
| *** adreznec has quit IRC | 02:00 | |
| *** adreznec has joined #openstack-ansible | 02:02 | |
| *** mpranjic has quit IRC | 02:03 | |
| *** woodard has quit IRC | 02:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 02:40 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 02:40 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 02:41 |
| *** vishwanathj has quit IRC | 02:44 | |
| cloudnull | spotz: absolutely | 02:53 |
| cloudnull | Tequilla makes the world go round | 02:53 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 02:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 02:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 02:57 |
| *** germs has quit IRC | 02:57 | |
| *** mpranjic has joined #openstack-ansible | 02:58 | |
| *** germs has joined #openstack-ansible | 02:58 | |
| *** germs1 has joined #openstack-ansible | 03:01 | |
| *** germs has quit IRC | 03:04 | |
| *** dxiri has joined #openstack-ansible | 03:04 | |
| *** dxiri_ has joined #openstack-ansible | 03:08 | |
| *** dxiri has quit IRC | 03:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 03:12 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 03:12 |
| *** lbragstad has joined #openstack-ansible | 03:12 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 03:12 |
| *** udesale has joined #openstack-ansible | 03:31 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 03:32 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 03:32 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 03:33 |
| *** markvoelker has quit IRC | 03:36 | |
| *** acormier has joined #openstack-ansible | 03:36 | |
| *** yifei has joined #openstack-ansible | 03:37 | |
| *** gkadam has joined #openstack-ansible | 03:37 | |
| *** yifei has quit IRC | 03:38 | |
| *** acormier has quit IRC | 03:41 | |
| *** lbragstad has quit IRC | 03:50 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 04:02 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 04:02 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 04:02 |
| *** albertcard1 has quit IRC | 04:05 | |
| *** jamielennox has quit IRC | 04:14 | |
| *** jamielennox has joined #openstack-ansible | 04:18 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 04:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 04:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 04:19 |
| *** germs1 has quit IRC | 04:21 | |
| *** germs has joined #openstack-ansible | 04:35 | |
| cloudnull | if folks have time to review https://review.openstack.org/#/c/499396 it be appreciated. | 04:41 |
| cloudnull | should knock of quite a bit of time | 04:41 |
| *** yifei has joined #openstack-ansible | 04:53 | |
| *** yifei has quit IRC | 04:54 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts master: Create an LXC create template for machinectl https://review.openstack.org/501054 | 05:01 |
| *** chyka has joined #openstack-ansible | 05:04 | |
| *** gouthamr has quit IRC | 05:06 | |
| *** chyka has quit IRC | 05:08 | |
| *** eumel8 has joined #openstack-ansible | 05:13 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: [WIP] Add machinectl backend store options https://review.openstack.org/501059 | 05:15 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: [WIP] Add machinectl backend store options https://review.openstack.org/501059 | 05:16 |
| *** germs has quit IRC | 05:16 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [DNM] Testing integrated repo with machinectl template https://review.openstack.org/501090 | 05:19 |
| *** chyka has joined #openstack-ansible | 05:35 | |
| *** markvoelker has joined #openstack-ansible | 05:37 | |
| *** chyka has quit IRC | 05:39 | |
| *** udesale__ has joined #openstack-ansible | 05:42 | |
| *** udesale has quit IRC | 05:45 | |
| *** threestrands has quit IRC | 05:45 | |
| *** drifterza has joined #openstack-ansible | 05:51 | |
| *** vishwanathj has joined #openstack-ansible | 05:54 | |
| *** dxiri has joined #openstack-ansible | 06:01 | |
| *** dxiri_ has quit IRC | 06:02 | |
| *** Oku_OS-away is now known as Oku_OS | 06:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 06:10 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 06:10 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 06:10 |
| *** markvoelker has quit IRC | 06:11 | |
| *** vishwana_ has joined #openstack-ansible | 06:19 | |
| *** jbadiapa has joined #openstack-ansible | 06:19 | |
| *** jvidal has joined #openstack-ansible | 06:19 | |
| *** vishwanathj has quit IRC | 06:21 | |
| jrosser | cloudnull: yes a loop device is fine | 06:22 |
| *** hachi_ has joined #openstack-ansible | 06:28 | |
| *** dxiri has quit IRC | 06:29 | |
| *** drifterza has quit IRC | 06:35 | |
| *** DanyC has quit IRC | 06:38 | |
| *** DanyC has joined #openstack-ansible | 06:39 | |
| *** DanyC has quit IRC | 06:43 | |
| nwonknu | ivve: yes, if you have examples for that would be great | 06:44 |
| nwonknu | i'm trying right now by assigning multiple mgmt ip's to a single host and basically having it appear to ansible as multiple hosts, but that's surely not the best/cleanest approach | 06:44 |
| nwonknu | so i'll take a look at the os_cinder role if i can figure out what you meant | 06:46 |
| *** udesale has joined #openstack-ansible | 06:46 | |
| *** udesale__ has quit IRC | 06:48 | |
| *** vishwana_ has quit IRC | 06:48 | |
| *** vishwanathj has joined #openstack-ansible | 07:04 | |
| *** drifterza has joined #openstack-ansible | 07:05 | |
| *** woodard has joined #openstack-ansible | 07:06 | |
| *** markvoelker has joined #openstack-ansible | 07:08 | |
| *** pcaruana has joined #openstack-ansible | 07:08 | |
| *** woodard has quit IRC | 07:10 | |
| *** TomMc has joined #openstack-ansible | 07:23 | |
| *** dxiri has joined #openstack-ansible | 07:24 | |
| *** chyka has joined #openstack-ansible | 07:30 | |
| *** drifterza has quit IRC | 07:32 | |
| *** drifterza has joined #openstack-ansible | 07:33 | |
| *** chyka has quit IRC | 07:35 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Fix zuul clonemap https://review.openstack.org/509701 | 07:36 |
| *** drifterza has quit IRC | 07:37 | |
| *** markvoelker has quit IRC | 07:42 | |
| *** DanyC has joined #openstack-ansible | 07:50 | |
| *** DanyC has quit IRC | 07:50 | |
| *** DanyC has joined #openstack-ansible | 07:50 | |
| *** armaan has quit IRC | 08:00 | |
| *** armaan has joined #openstack-ansible | 08:00 | |
| *** DanyC has quit IRC | 08:01 | |
| *** OldTom has joined #openstack-ansible | 08:04 | |
| *** TomMc has quit IRC | 08:07 | |
| *** OldTom has quit IRC | 08:08 | |
| *** pbandark has joined #openstack-ansible | 08:10 | |
| *** pbandark has quit IRC | 08:11 | |
| *** pbandark has joined #openstack-ansible | 08:13 | |
| *** br0k3n has joined #openstack-ansible | 08:14 | |
| *** mbuil has joined #openstack-ansible | 08:15 | |
| *** shardy has joined #openstack-ansible | 08:19 | |
| *** LiterateHawk_ has quit IRC | 08:22 | |
| *** LiterateHawk_ has joined #openstack-ansible | 08:23 | |
| *** TomMc has joined #openstack-ansible | 08:27 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Force the installation of qemu packages https://review.openstack.org/509711 | 08:28 |
| evrardjp | TIL installing qemu qemu-common or qemu-system doesn't automatically provide qemu-kvm | 08:31 |
| evrardjp | under ubuntu16 | 08:31 |
| *** pylot has joined #openstack-ansible | 08:34 | |
| nwonknu | having some issues with keepalived now; it keeps restarting continuously; if i run it manually with -R i see it crashes sometimes, other times it will continue running fine | 08:37 |
| evrardjp | mmm | 08:37 |
| evrardjp | version? | 08:37 |
| nwonknu | only thing that appears in the logs before it happens is 'VRRP child process(7053) died: Exiting' | 08:37 |
| nwonknu | pike | 08:37 |
| evrardjp | keepalived version I mean | 08:37 |
| nwonknu | Keepalived v1.2.19 | 08:37 |
| nwonknu | Ubuntu 16.04.3 LTS | 08:38 |
| evrardjp | just a sec | 08:38 |
| nwonknu | yup | 08:38 |
| evrardjp | on ipv6? | 08:38 |
| *** markvoelker has joined #openstack-ansible | 08:38 | |
| nwonknu | no, ipv4 | 08:39 |
| nwonknu | i'll pastebin the config | 08:39 |
| evrardjp | yeah that would be great | 08:39 |
| evrardjp | I will check the version in uca | 08:40 |
| evrardjp | because there are updates | 08:40 |
| nwonknu | https://pastebin.com/1zpwxUH2 | 08:40 |
| evrardjp | I know there are regularily vrrp fixes | 08:40 |
| evrardjp | are you sure you don't have two daemons running? | 08:40 |
| evrardjp | looks simple enough | 08:41 |
| nwonknu | yeah, i can reproduce the issue consistently even when everything is killed (ps aux | grep keepalived is empty) and i run manually with -R (don't respawn) | 08:41 |
| nwonknu | i'll give a full debug log too | 08:41 |
| *** hachi_ has quit IRC | 08:42 | |
| pylot | every time i install a new environment -> spice is throwing 503 until the nova_console container is restarted... is there a way to fix that? | 08:42 |
| evrardjp | nwonknu: 1.2.19 is the official version of ubuntu 1604 | 08:45 |
| evrardjp | they don't provide any updated version in UCA | 08:45 |
| nwonknu | correct | 08:45 |
| nwonknu | 1:1.2.19-1ubuntu0.2 | 08:45 |
| nwonknu | i can try the zesty version to see if it will install/run | 08:46 |
| evrardjp | nah | 08:46 |
| evrardjp | bad idea | 08:46 |
| evrardjp | better use the ppa | 08:46 |
| nwonknu | ok | 08:46 |
| evrardjp | just a sec | 08:47 |
| evrardjp | I will check if the ppa can work for 16.04 for you | 08:47 |
| evrardjp | but I will first find the root cause | 08:47 |
| *** drifterza has joined #openstack-ansible | 08:47 | |
| evrardjp | did you try stracing it? | 08:48 |
| *** epalper has joined #openstack-ansible | 08:49 | |
| evrardjp | when you run manually, you don't forget to pass the config file, do you? | 08:49 |
| evrardjp | oh my bad I misread your log | 08:50 |
| nwonknu | hrm i don't specify it, assumed it was using default config location | 08:50 |
| nwonknu | i can try an strace | 08:51 |
| evrardjp | it looks like it loads | 08:51 |
| evrardjp | but give it in the cli, just in case | 08:52 |
| evrardjp | but I think it's good | 08:52 |
| evrardjp | however I see don't see something like | 08:52 |
| evrardjp | VRRP sockpool: [ifindex(10), proto(112), unicast(0), fd(10,11)] | 08:52 |
| evrardjp | so it can't access your interface | 08:53 |
| evrardjp | did you change permissions/namespaces or anything? | 08:53 |
| nwonknu | nope, new setup | 08:53 |
| evrardjp | let me check if there is an open bug for that | 08:53 |
| evrardjp | ok | 08:53 |
| evrardjp | so not an upgrade | 08:53 |
| nwonknu | nope | 08:53 |
| nwonknu | strace doesn't seem that useful, as it's the child process that dies | 08:54 |
| nwonknu | but will pastebin it anyway | 08:54 |
| evrardjp | pylot: I don't know, you should file a bug | 08:54 |
| evrardjp | nwonknu: strace -f | 08:54 |
| nwonknu | ahhh ok | 08:54 |
| nwonknu | right | 08:54 |
| nwonknu | [pid 9015] +++ killed by SIGSEGV (core dumped) +++ | 08:56 |
| evrardjp | ahah | 08:56 |
| evrardjp | seg faults! | 08:56 |
| nwonknu | yup | 08:57 |
| nwonknu | but i don't have --core-dump parameter :[ | 08:57 |
| *** fxpester has joined #openstack-ansible | 08:57 | |
| evrardjp | Resource temporarily unavailable | 08:57 |
| *** TomMc has quit IRC | 08:57 | |
| evrardjp | is the nic you're using passed to something else? | 08:58 |
| evrardjp | could you try with a different nic, dedicated for it, to see if it works better? | 08:58 |
| evrardjp | I think it's a configuration problem | 08:58 |
| evrardjp | gut feeling | 08:58 |
| nwonknu | eno1.1489 is not, but eno1 is part of br-mgmt | 08:58 |
| nwonknu | it's a new setup, so very possible | 08:59 |
| nwonknu | eno1 has native vlan with mgmt iprange, but also has 1489 vlan for public range | 08:59 |
| nwonknu | i'll remove one of the two keepalived floating ip configs | 08:59 |
| nwonknu | and see if it still happens then | 08:59 |
| evrardjp | could you add another nic with vlan 1489 as native ? | 08:59 |
| nwonknu | still happens with only br-mgmt | 09:00 |
| evrardjp | or try lo0 temporarily | 09:00 |
| nwonknu | yup will try a few different configs | 09:00 |
| *** DanyC has joined #openstack-ansible | 09:01 | |
| nwonknu | happens with only eno1.1489 as well | 09:01 |
| evrardjp | if you're looking for an updated package, give me a few minutes | 09:01 |
| nwonknu | sure np | 09:02 |
| evrardjp | nwonknu: if you want an updated version coming from ppa | 09:02 |
| evrardjp | you can simply do keepalived_ubuntu_src: "ppa" | 09:02 |
| nwonknu | ok, easy enough | 09:03 |
| *** jbadiapa has quit IRC | 09:03 | |
| evrardjp | and then re-run the keepalived role, by running haproxy playbook | 09:03 |
| evrardjp | I doubt there is a need though | 09:03 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.1 https://review.openstack.org/508142 | 09:06 |
| nwonknu | same issue when running natively on only eno1 in mgmt range | 09:06 |
| nwonknu | and lo doesn't work for some reason, it assumes i don't have any interface specified then so it skips vrrp (Unknown interface !) | 09:07 |
| evrardjp | k | 09:08 |
| evrardjp | I can't spend more time on this ... if nothing works, try upgrading. But I think it's a network configuration issue | 09:09 |
| evrardjp | you configured your nics and then ran the keepalived role by running the haproxy playbook, right? | 09:10 |
| nwonknu | i configured the nics and went through the whole setup procedure as documented (setup-hosts, setup-infrastructure) | 09:10 |
| nwonknu | this failed at a pip install on galera container | 09:11 |
| nwonknu | so i started looking, noticed keepalived ip being down, checked keepalived logs and saw it continuously respawning child process | 09:11 |
| *** markvoelker has quit IRC | 09:12 | |
| *** br0k3n has quit IRC | 09:13 | |
| nwonknu | installed ppa and trying latest, appears to have more output at least (pid 10232 exited due to segmentation fault (SIGSEGV).) | 09:14 |
| nwonknu | Keepalived v1.2.23 fyi | 09:14 |
| nwonknu | this one has the core-dump option | 09:14 |
| *** br0k3n has joined #openstack-ansible | 09:14 | |
| *** jbadiapa has joined #openstack-ansible | 09:17 | |
| nwonknu | not too useful without a debug binary i guess | 09:17 |
| nwonknu | https://pastebin.com/XbEWvmt6 | 09:17 |
| evrardjp | 1.2.23 is the latest stable, or 1.2.24 I don't recall | 09:18 |
| nwonknu | lets try the daily then | 09:18 |
| nwonknu | thats 1.2.19 | 09:18 |
| evrardjp | you shouldn't | 09:18 |
| evrardjp | I mean from keepalived | 09:19 |
| nwonknu | yeah i know | 09:19 |
| nwonknu | figured i'll just try latest available from the ppa, if that doesn't work i'll compile from source/latest? | 09:19 |
| evrardjp | nah, don't go that far | 09:20 |
| evrardjp | ppa is fine | 09:20 |
| evrardjp | I think you should start with simple configuration on your host | 09:20 |
| evrardjp | removing the track scripts | 09:20 |
| evrardjp | oh check if you can load the ip_vs module | 09:20 |
| nwonknu | same issue with v1.2.23 | 09:21 |
| evrardjp | ok | 09:21 |
| evrardjp | so it's a config issue | 09:21 |
| evrardjp | pretty sure of that | 09:21 |
| nwonknu | ip_vs loads ok | 09:21 |
| nwonknu | alright, will try reducing configuration more | 09:21 |
| nwonknu | the core dump is generated in the config dir | 09:21 |
| nwonknu | so seems to indicate you are right | 09:21 |
| nwonknu | but still very strange it works sometimes | 09:22 |
| evrardjp | that's why I think it's the nic that's dedicated to something else | 09:22 |
| evrardjp | cannot bind or something | 09:23 |
| evrardjp | how many hosts do you have ? | 09:23 |
| evrardjp | in your haproxy group | 09:23 |
| nwonknu | 3 controller nodes, so should be 3, but i'll doublecheck | 09:25 |
| *** drifterza has quit IRC | 09:25 | |
| nwonknu | yup 3 | 09:26 |
| *** TomMc has joined #openstack-ansible | 09:27 | |
| *** armaan has quit IRC | 09:28 | |
| *** armaan has joined #openstack-ansible | 09:29 | |
| *** mrtenio has quit IRC | 09:34 | |
| nwonknu | hrm, just tried with this config https://pastebin.com/iPiJ24sT on "eno4" and keepalived 1.2.23, an interface that has 0 configuration and i literally just did "ifconfig eno4 up" -> still segfaults | 09:35 |
| *** br0k3n has quit IRC | 09:36 | |
| *** br0k3n has joined #openstack-ansible | 09:37 | |
| epalper | hi, i am trying to run osa to install openstack with ovs with additional configuration in user_variables.yml | 09:38 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Add ZFS backing store to the container create role https://review.openstack.org/509608 | 09:38 |
| epalper | # Ensure the openvswitch kernel module is loaded | 09:38 |
| epalper | openstack_host_specific_kernel_modules: | 09:38 |
| epalper | - name: "openvswitch" | 09:38 |
| epalper | pattern: "CONFIG_OPENVSWITCH" | 09:38 |
| epalper | group: "network_hosts" | 09:38 |
| epalper | but i don't see ovs being installed in both neutron server container and compute nodes | 09:39 |
| *** DanyC has quit IRC | 09:39 | |
| *** mrch has joined #openstack-ansible | 09:49 | |
| epalper | can someone help me why it's not getting installed ? | 09:50 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_container_create master: Update the LXC container create role https://review.openstack.org/509530 | 09:51 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Allow container creation on ZFS backing storage https://review.openstack.org/509737 | 09:52 |
| *** chyka has joined #openstack-ansible | 09:54 | |
| eumel8 | epalper: some ansible output would be helpful to analyze | 09:54 |
| epalper | enumel8: here it is https://paste.ubuntu.com/25678659/ | 09:58 |
| epalper | this is being run from opnfv's releng-xci | 09:58 |
| *** chyka has quit IRC | 09:59 | |
| *** drifterza has joined #openstack-ansible | 10:03 | |
| epalper | enumel8: afk for another 3 hrs. Please send your updates | 10:08 |
| *** markvoelker has joined #openstack-ansible | 10:09 | |
| evrardjp | epalper: I haven't looked at that code for a while | 10:10 |
| evrardjp | hwoarang: is there a plan for OPNFV to add a openvswitch OSA scenario? | 10:12 |
| evrardjp | that would be good | 10:12 |
| hwoarang | evrardjp: yes | 10:12 |
| evrardjp | great | 10:12 |
| evrardjp | I think it should be in our gates | 10:12 |
| hwoarang | yeah i agree | 10:12 |
| hwoarang | epalper: the kernel module alone will not force ovs to get installed | 10:13 |
| *** epalper has quit IRC | 10:13 | |
| hwoarang | you also need to install the userspace package | 10:13 |
| hwoarang | otherwise you only load the module and nothing really uses it | 10:13 |
| jrosser | odyssey4me: to test this zfs stuff, where in the tests to i start wiring in creating a block device and setting it up? | 10:18 |
| evrardjp | jrosser: https://github.com/openstack/openstack-ansible/tree/master/tests/roles/bootstrap-host/tasks ? | 10:20 |
| jrosser | hmm, does that get consumed when i run the role tests? | 10:24 |
| jrosser | sorry just finding my way in all this | 10:25 |
| *** br0k3n1 has joined #openstack-ansible | 10:26 | |
| *** br0k3n has quit IRC | 10:26 | |
| odyssey4me | jrosser as far as I remember, there are no tests for different devices in that role - so you'd be creating a new set ;) | 10:29 |
| eumel8 | epalper: additional there are different errors and warnings to investigate: line 281, 2520, 2736 (again) ... dunno, maybe this documentation is helpful: http://trumant.github.io/openstack-ansible-openvswitch.html | 10:29 |
| odyssey4me | jrosser the tests are relatively simply done in the roles - the test.yml playbook from the tests directory is run, and the inventory and vars are all there: https://github.com/openstack/openstack-ansible-lxc_container_create/tree/master/tests | 10:30 |
| odyssey4me | so I'd suggest implementing perhaps a specific backend for a new container in the host_vars, then implementing something later to verify that it was done right | 10:30 |
| odyssey4me | does that make sense? | 10:31 |
| jrosser | sort of - i see piles of stuff in openstack-ansible-tests too | 10:32 |
| jrosser | at a very basic level i need to find the right place to create a loop device and format it in some way in the environment that the test runs, before the test runs | 10:32 |
| jrosser | and yes then pass variables in via the vars to point at that storage | 10:33 |
| *** br0k3n has joined #openstack-ansible | 10:35 | |
| *** br0k3n1 has quit IRC | 10:35 | |
| *** br0k3n has quit IRC | 10:35 | |
| *** br0k3n has joined #openstack-ansible | 10:36 | |
| *** udesale has quit IRC | 10:38 | |
| *** markvoelker has quit IRC | 10:42 | |
| *** chyka has joined #openstack-ansible | 10:45 | |
| odyssey4me | jrosser as this only needs exercising in the lxc-container-create role, changes only need to be made there ... all overrides, inventory changes, etc | 10:48 |
| odyssey4me | the only reason to implement anything in the tests repo would be if it's used in tests across multiple roles | 10:48 |
| jrosser | oh well you see i've proposed a change to lxc_host as well | 10:48 |
| jrosser | becasue it needs to create the base image on zfs, then create the container from there | 10:49 |
| jrosser | https://review.openstack.org/509737 | 10:49 |
| *** chyka has quit IRC | 10:50 | |
| *** Lirt has joined #openstack-ansible | 10:58 | |
| odyssey4me | jrosser ah ok, yeah that will make sense - a bit of cross-repo fun then :/ | 11:00 |
| *** jwitko has quit IRC | 11:03 | |
| *** dave-mccowan has joined #openstack-ansible | 11:04 | |
| *** drifterza has quit IRC | 11:06 | |
| *** drifterza has joined #openstack-ansible | 11:07 | |
| ArchiFleKs | Hi, does someone encountered since on cinder-master ? I don't think it is related to OSA but: http://paste.openstack.org/show/622741/ | 11:11 |
| nwonknu | evrardjp: found the problem | 11:25 |
| nwonknu | "auth_pass" option didn't have an argument in the keepalived.conf | 11:26 |
| *** br0k3n has quit IRC | 11:26 | |
| nwonknu | so it may be useful to auto-generate one, or set a mandatory filter on that variable somewhere | 11:26 |
| nwonknu | quite frustrating to see it will work sometimes, but fail most of the times in this particular case | 11:27 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Allow container creation on ZFS backing storage https://review.openstack.org/509737 | 11:27 |
| *** smatzek has joined #openstack-ansible | 11:37 | |
| *** markvoelker has joined #openstack-ansible | 11:39 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Fix zuul clonemap https://review.openstack.org/509426 | 11:40 |
| *** chyka has joined #openstack-ansible | 11:41 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix zuul clonemap https://review.openstack.org/509701 | 11:44 |
| *** chyka has quit IRC | 11:45 | |
| *** ianychoi has quit IRC | 11:46 | |
| *** ansibler has quit IRC | 11:53 | |
| *** br0k3n has joined #openstack-ansible | 11:58 | |
| *** br0k3n has left #openstack-ansible | 12:09 | |
| *** markvoelker has quit IRC | 12:12 | |
| *** armaan has quit IRC | 12:16 | |
| *** armaan has joined #openstack-ansible | 12:16 | |
| mhayden | morning! | 12:18 |
| eumel8 | morning! | 12:21 |
| *** acormier has joined #openstack-ansible | 12:23 | |
| *** acormier has quit IRC | 12:23 | |
| *** acormier has joined #openstack-ansible | 12:23 | |
| *** markvoelker has joined #openstack-ansible | 12:28 | |
| *** TomMc has quit IRC | 12:34 | |
| *** TomMc has joined #openstack-ansible | 12:35 | |
| *** woodard has joined #openstack-ansible | 12:43 | |
| *** woodard has quit IRC | 12:43 | |
| *** woodard has joined #openstack-ansible | 12:44 | |
| *** gokhan has quit IRC | 12:46 | |
| *** acormier has quit IRC | 12:48 | |
| *** gkadam has quit IRC | 12:54 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_cinder master: Workaround bad cinder defaults https://review.openstack.org/509768 | 12:55 |
| *** kylek3h has joined #openstack-ansible | 12:55 | |
| *** gkadam has joined #openstack-ansible | 12:57 | |
| *** TomMc has quit IRC | 12:57 | |
| *** udesale has joined #openstack-ansible | 12:58 | |
| *** gkadam has quit IRC | 13:02 | |
| *** lbragstad has joined #openstack-ansible | 13:06 | |
| *** esberglu has joined #openstack-ansible | 13:09 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create master: SUSE: Remove lxc-1.X.X workarounds https://review.openstack.org/509771 | 13:15 |
| *** eumel8 has quit IRC | 13:15 | |
| mhayden | successfully upgraded a CentOS 7 Ocata cloud to pike! :) | 13:16 |
| mhayden | and there was much rejoicing | 13:16 |
| evrardjp | congrats | 13:16 |
| evrardjp | now we'll maybe have proper centos support on Pike and below? :p | 13:16 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: SUSE: Remove lxc-1.X.X workarounds https://review.openstack.org/509772 | 13:18 |
| evrardjp | ^ woot | 13:19 |
| ArchiFleKs | evrardjp: do you know about this ? https://bugs.launchpad.net/nova/+bug/1718295 I have it on Pike | 13:20 |
| openstack | Launchpad bug 1718295 in OpenStack Compute (nova) "Live migration fails with qemu-img >= 2.10: "Failed to get shared "write" lock\nIs another process using the image?"" [High,Fix released] - Assigned to Sean Dague (sdague) | 13:20 |
| evrardjp | hwoarang: could you add this as a topic? I remember there was the destroy -nf and other similar things | 13:20 |
| hwoarang | evrardjp: what? | 13:21 |
| *** smatzek has quit IRC | 13:21 | |
| *** smatzek has joined #openstack-ansible | 13:22 | |
| evrardjp | the lxc 1 to 2 | 13:22 |
| evrardjp | ArchiFleKs: I was not aware of that, but give me a sec to confirm | 13:23 |
| hwoarang | evrardjp: add it as a topic where? | 13:24 |
| evrardjp | in the review :p | 13:24 |
| evrardjp | let me fix that for you | 13:24 |
| *** smatzek_ has joined #openstack-ansible | 13:25 | |
| *** smatzek_ has quit IRC | 13:25 | |
| *** gouthamr has joined #openstack-ansible | 13:25 | |
| *** smatzek_ has joined #openstack-ansible | 13:25 | |
| *** smatzek has quit IRC | 13:29 | |
| evrardjp | ArchiFleKs: so, we have still a few things pending on releases. When these will get merge, this can merge: https://review.openstack.org/#/c/508142 | 13:29 |
| *** gokhan has joined #openstack-ansible | 13:29 | |
| evrardjp | it should have everything you need | 13:29 |
| *** smatzek_ has quit IRC | 13:30 | |
| ArchiFleKs | evrardjp: thanks, but it does not seem to have been backport to pike yet, for the nova bug | 13:33 |
| *** jwitko has joined #openstack-ansible | 13:33 | |
| cloudnull | mornings | 13:33 |
| evrardjp | ArchiFleKs: according to Matt Riedmann https://review.openstack.org/#/c/505674/ should be the fix | 13:34 |
| evrardjp | so because we have that, it should be alright | 13:35 |
| evrardjp | well that's my understanding | 13:35 |
| evrardjp | with a quick check | 13:35 |
| evrardjp | but don't take my word for it | 13:35 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: playbooks: os-lxc-container-setup: Use the 'reboot' option for lxc restart https://review.openstack.org/509775 | 13:36 |
| ArchiFleKs | I have to admit I don't get all of it, from my understanding, it is independant from nova-version, and because OSA install nova from source, and for example on my compute node, qemu has already been updated to 2.10 so unless we have to new nova commit with the --force-share flag I don't know how it is going to solve the issue | 13:37 |
| *** TomMc has joined #openstack-ansible | 13:38 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Update the LXC container create role https://review.openstack.org/509530 | 13:41 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Converge neutron agents onto Baremetal https://review.openstack.org/454450 | 13:46 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Converge neutron agents onto Baremetal https://review.openstack.org/454450 | 13:47 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 13:47 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 13:48 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 13:48 |
| evrardjp | ArchiFleKs: https://launchpadlibrarian.net/337670248/qemu-2.10-compat.patch is the patch applied in UCA | 13:52 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create master: tasks: lxc_container_create: Retry LXC start on failures https://review.openstack.org/509786 | 13:53 |
| evrardjp | ArchiFleKs: I am still reading | 13:54 |
| *** epalper has joined #openstack-ansible | 13:57 | |
| *** rodolof has joined #openstack-ansible | 13:57 | |
| cloudnull | looks like we're seeing Zuul failures again, is there another transition in the works? | 13:58 |
| *** smatzek has joined #openstack-ansible | 13:59 | |
| *** shardy has quit IRC | 14:00 | |
| epalper | mwe1^ | 14:00 |
| ArchiFleKs | evrardjp: Yes i really don't get all of it, if nova adapts to qemu or if it is the other way around | 14:00 |
| *** shardy has joined #openstack-ansible | 14:00 | |
| *** smatzek has quit IRC | 14:00 | |
| evrardjp | cloudnull: we have zuul and jenkins for now | 14:00 |
| mwe1 | epalper: I'm here :D | 14:01 |
| evrardjp | cloudnull: jenkins is doing the gates | 14:01 |
| *** smatzek has joined #openstack-ansible | 14:01 | |
| ArchiFleKs | evrardjp: this patch is applied to nova ? by the ubuntu maintainer right ? but we are not using nova ubuntu cloud archive package ? | 14:02 |
| epalper | mwe1: any chance to look at logs for ovs issue ? | 14:02 |
| evrardjp | ArchiFleKs: never mind my ping :p | 14:02 |
| evrardjp | ok releases patch merged, now we need to get our releases done | 14:03 |
| *** mwe1 has quit IRC | 14:04 | |
| cloudnull | evrardjp: is zuul creating blocking reviews ? | 14:04 |
| *** mwe1 has joined #openstack-ansible | 14:04 | |
| evrardjp | no | 14:05 |
| evrardjp | just giving votes, like an external ci would | 14:05 |
| evrardjp | the situation seems unblocked, there is just still large queues | 14:05 |
| ArchiFleKs | You can follow here : https://review.openstack.org/#/c/509774/1 in fact the fix is only in Ubuntu UCA | 14:05 |
| jrosser | this step takes several minutes ==> ubuntu1604: functional develop-inst-nodeps: /vagrant | 14:06 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_monasca master: Updated from OpenStack Ansible Tests https://review.openstack.org/502607 | 14:06 |
| jrosser | is there a shortcut i can take whilst hacking on a role to go straight to running the ansible again | 14:06 |
| evrardjp | ArchiFleKs: yes, I've seen this patch and your ping in #openstack-nova | 14:07 |
| evrardjp | :p | 14:07 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_aodh master: Standardize services list and package installation https://review.openstack.org/492580 | 14:07 |
| evrardjp | they have nova meeting right now, maybe time to discuss it? | 14:08 |
| evrardjp | cloudnull: if you want to help unblock master: https://review.openstack.org/#/c/509768/ | 14:11 |
| evrardjp | if you want to help on full metal: https://review.openstack.org/#/c/509494/ | 14:11 |
| evrardjp | if you want to help pike ceph: https://review.openstack.org/#/c/509108/ | 14:12 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_magnum master: Fix cinder_service_region undefined https://review.openstack.org/507788 | 14:13 |
| evrardjp | you can also vote on https://review.openstack.org/#/c/504279/ :) | 14:14 |
| mhayden | is anyone else having a hard time reviewing logs from CI jobs? | 14:14 |
| mhayden | i'm getting a bunch of 404's | 14:14 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_swift master: Updated from OpenStack Ansible Tests https://review.openstack.org/502614 | 14:15 |
| *** jheroux has joined #openstack-ansible | 14:16 | |
| evrardjp | mhayden: review number? | 14:16 |
| mhayden | probably about 10-20 different reviews | 14:16 |
| mhayden | http://logs.openstack.org/08/509108/1/check/legacy-ansible-func/3d96319/ | 14:16 |
| mhayden | that just sits and spins for me | 14:16 |
| mhayden | or well it finally loaded | 14:16 |
| mhayden | just took a long time | 14:17 |
| jrosser | if i'm locally testing a role in vagrant which depends on a proposed patch in gerrit, is there a special incantation i can put in ansible-role-requirements.yml or similar to use the patch | 14:17 |
| evrardjp | dear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccollum, darrenc, JRobinson__, asettle, colinm | 14:17 |
| evrardjp | cnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen: the openstack-ansible community meeting will start in 1h43 minutes in #openstack-meeting-4 | 14:17 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Allow container creation on ZFS backing storage https://review.openstack.org/509737 | 14:17 |
| evrardjp | mhayden: yeah I've seen slowness | 14:18 |
| mhayden | okay, glad i'm not crazy | 14:18 |
| evrardjp | no 404 though | 14:18 |
| cloudnull | evrardjp: I asked in infra | 14:18 |
| evrardjp | ? | 14:18 |
| evrardjp | what did you ask? | 14:19 |
| cloudnull | re zuul + jenkins reviewing. | 14:19 |
| cloudnull | zuul is bound to fail. | 14:19 |
| cloudnull | but is that blocking ? | 14:19 |
| evrardjp | no it's not :p | 14:19 |
| evrardjp | I just told you, like an external CI :p- | 14:19 |
| evrardjp | nothing is blocking us right now | 14:20 |
| *** cshen has quit IRC | 14:20 | |
| evrardjp | I mean, nothing computer related | 14:20 |
| evrardjp | now it's humans | 14:20 |
| evrardjp | :D | 14:20 |
| cloudnull | ah i missed your comment about external ci | 14:20 |
| evrardjp | haha no problem. | 14:21 |
| cloudnull | I read "they have nova meeting right now, maybe time to discuss it?" | 14:21 |
| cloudnull | and thought, well i'll go ask anyway | 14:21 |
| evrardjp | that was for ArchiFleKs | 14:21 |
| cloudnull | lots of confusion | 14:21 |
| cloudnull | :D | 14:21 |
| evrardjp | cloudnull: I should start all I say with a nick. Understood | 14:21 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client stable/pike: Make the ceph mon test more robust https://review.openstack.org/509108 | 14:21 |
| *** acormier has joined #openstack-ansible | 14:25 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-lxc_container_create master: Add ZFS backing store to the container create role https://review.openstack.org/509608 | 14:30 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_keystone master: Update upgrade role for Pike https://review.openstack.org/496678 | 14:35 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_neutron stable/pike: Update vars and test tooling for Pike https://review.openstack.org/496622 | 14:35 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_nova master: Update upgrade role for Pike https://review.openstack.org/496679 | 14:36 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_neutron master: Update upgrade role for Pike https://review.openstack.org/496680 | 14:36 |
| *** woodard has quit IRC | 14:36 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create master: SUSE: Remove lxc-1.X.X workarounds https://review.openstack.org/509771 | 14:38 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible master: SUSE: Add OpenStack cloud repository when bootstrapping the host https://review.openstack.org/503722 | 14:39 |
| evrardjp | holy moly | 14:39 |
| logan- | heh evrardjp I was just pushing a patch for that qemu-system bug and saw you did one this morning | 14:40 |
| logan- | comment for you on https://review.openstack.org/#/c/509711 | 14:40 |
| evrardjp | logan-: qemu-system got pulled over, so no need to do it, to what I've tested | 14:41 |
| evrardjp | I can add the others indeed | 14:41 |
| evrardjp | fine for me | 14:42 |
| logan- | i think it is best to list it so that it is upgraded in line with the other qemu packages over time | 14:42 |
| logan- | it is important that all of the qemu package versions stay in sync | 14:43 |
| mgariepy | morning everyone | 14:43 |
| evrardjp | logan-: should I add block? | 14:43 |
| evrardjp | I mean qemu-block-extra | 14:44 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Force the installation of qemu packages https://review.openstack.org/509711 | 14:44 |
| logan- | yeah i wouldnt be opposed to also adding -block-extra, -user, and -utils | 14:44 |
| evrardjp | logan-: ^ updated, added qemu-block-extra | 14:44 |
| logan- | thats probably prudent | 14:44 |
| logan- | thanks | 14:44 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Force the installation of qemu packages https://review.openstack.org/509711 | 14:45 |
| evrardjp | updated again | 14:46 |
| evrardjp | with the user and utils | 14:46 |
| evrardjp | I think we are good | 14:46 |
| logan- | qemu-kvm is still in the list | 14:46 |
| logan- | that shouldn't be installed | 14:46 |
| evrardjp | why? | 14:46 |
| logan- | https://wiki.qemu.org/Features/KVM | 14:46 |
| logan- | "The KVM project used to maintain a fork of QEMU called qemu-kvm. All feature differences have been merged into QEMU upstream and the development of the fork suspended." | 14:47 |
| logan- | none of the qemu packages we install list it as a dependency and normally it is not present | 14:47 |
| evrardjp | thanks | 14:47 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Force the installation of qemu packages https://review.openstack.org/509711 | 14:47 |
| evrardjp | done | 14:48 |
| evrardjp | that's fast feedback loop | 14:48 |
| evrardjp | :D | 14:48 |
| logan- | :) thanks | 14:48 |
| *** TomMc has quit IRC | 14:50 | |
| *** weezS has joined #openstack-ansible | 14:52 | |
| *** epalper has quit IRC | 14:53 | |
| *** Lirt has quit IRC | 14:54 | |
| *** jvidal has quit IRC | 14:55 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible master: Better CI detection in bootstrap-hosts https://review.openstack.org/507092 | 15:00 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible master: Run the ceph-mons deploy in serial https://review.openstack.org/508503 | 15:00 |
| jrosser | cloudnull: tried hacking on zfs container role test today, something like this needs to slot in somewhere http://paste.openstack.org/show/622772/ | 15:02 |
| *** gouthamr_ has joined #openstack-ansible | 15:03 | |
| *** gouthamr has quit IRC | 15:04 | |
| *** gouthamr_ is now known as gouthamr | 15:04 | |
| *** DanyC has joined #openstack-ansible | 15:04 | |
| *** Oku_OS is now known as Oku_OS-away | 15:05 | |
| odyssey4me | logan- evrardjp hmm, I wonder if we shouldn't submit a patch upstream with a flag to also include deps when doing upgrades? | 15:10 |
| logan- | yeah idk where the problem lies, is it the python-apt module where that change would need to be made? | 15:10 |
| odyssey4me | or alternatively perhaps we should have some sort of task which does exactly that - upgrade all packages which are deps for things in our distro package list | 15:10 |
| logan- | but i noticed even direct dependencies are not updated, if I include qemu-system in the package list and specify state: latest, none of the qemu-system-* packages get updated even though they are dependencies of qemu-system | 15:11 |
| odyssey4me | I expect we could probably slim down our distro package lists if we do that, because many of the packages there were added for the same reason | 15:11 |
| logan- | i was testing for the least intrusive way to fix it this morning but there did not seem to be an alternative to explicitly listing all of the packages | 15:11 |
| logan- | gotcha | 15:12 |
| odyssey4me | logan- yep, that's why I think we should have something that figures out all deps and ensures they're in the list too | 15:12 |
| logan- | right | 15:12 |
| odyssey4me | ideally our list should be minimal, and we derive all deps, then that list gets installed/upgraded | 15:12 |
| odyssey4me | this would work well for an action module, or perhaps ansible upstream | 15:13 |
| odyssey4me | but isn't there a flag to give apt that tells it to update deps too? | 15:13 |
| odyssey4me | heh, we aren't the first to wish for it: https://askubuntu.com/questions/179160/is-there-a-command-to-upgrade-a-package-and-all-its-dependencies-without-upgradi | 15:14 |
| odyssey4me | https://serverfault.com/questions/577942/install-only-dependencies-of-a-given-package-in-debian-or-ubuntu-apt | 15:15 |
| logan- | :/ | 15:16 |
| odyssey4me | so I guess we'd have to loop through the list, putting a list of deps together - then add that to the package list to install | 15:18 |
| odyssey4me | what a 'mare | 15:18 |
| *** galstrom_zzz is now known as galstrom | 15:18 | |
| odyssey4me | it'd be super-nice to have that in the apt module | 15:18 |
| odyssey4me | install_depends = yes | 15:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Workaround bad cinder defaults https://review.openstack.org/509768 | 15:19 |
| cloudnull | jrosser: we should be able to add that into the lxc_container_create tests role | 15:19 |
| *** woodard has joined #openstack-ansible | 15:19 | |
| logan- | nice work on the container create stuff cloudnull, reading thru the patch atm | 15:19 |
| cloudnull | thanks! | 15:20 |
| *** rodolof has quit IRC | 15:29 | |
| *** rodolof has joined #openstack-ansible | 15:30 | |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-repo_server master: apt-cacher-ng was moved to epel. https://review.openstack.org/509493 | 15:35 |
| mgariepy | mhayden, ^^ | 15:35 |
| jrosser | cloudnull: ive spent most of the day fighting that in virtualbox, tbh i'm a bit swamped in the test setup | 15:36 |
| jrosser | i could realld do with seeing another role that tests particular os/$thing combinations and how they are set up | 15:39 |
| *** pcaruana has quit IRC | 15:39 | |
| *** jbadiapa has quit IRC | 15:45 | |
| *** chyka has joined #openstack-ansible | 15:47 | |
| jrosser | /win 22 | 15:51 |
| *** acormier has quit IRC | 15:53 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible stable/pike: Update all SHAs for 16.0.1 https://review.openstack.org/508142 | 15:53 |
| *** eumel8 has joined #openstack-ansible | 15:56 | |
| *** mrtenio has joined #openstack-ansible | 15:57 | |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-repo_server master: apt-cacher-ng was moved to epel. https://review.openstack.org/509493 | 15:58 |
| evrardjp | sorry I am a little late for the meeting | 16:08 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix zuul clonemap https://review.openstack.org/509701 | 16:09 |
| evrardjp | community meeting in #openstack-meeting-4 ... cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccoll | 16:10 |
| evrardjp | um, darrenc, JRobinson__, asettle, colinmcnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen | 16:10 |
| *** drifterza has quit IRC | 16:12 | |
| *** kylek3h has quit IRC | 16:12 | |
| *** kylek3h has joined #openstack-ansible | 16:12 | |
| evrardjp | guys I am doing the meeting alone. I'd be happy to have some presence to validate discussions together. | 16:15 |
| cloudnull | p/ | 16:18 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add machinectl backend store options https://review.openstack.org/501059 | 16:24 |
| *** gouthamr_ has joined #openstack-ansible | 16:32 | |
| *** gouthamr has quit IRC | 16:35 | |
| openstackgerrit | Manuel Buil proposed openstack/openstack-ansible-openstack_openrc master: Add tacker to the list of aliases https://review.openstack.org/509831 | 16:39 |
| *** rodolof has quit IRC | 16:46 | |
| dmsimard | evrardjp, cloudnull, odyssey4me: came across this tool from the openshift-ansible guys that might be of interest https://github.com/mtnbikenc/ansible-repo-grapher | 16:49 |
| dmsimard | not sure if it's generic enough to be used in openstack-ansible, haven't tried it myself | 16:49 |
| dmsimard | let me get you an example screenshot.. | 16:49 |
| *** udesale has quit IRC | 16:49 | |
| dmsimard | https://i.imgur.com/QY1eHTF.jpg | 16:51 |
| logan- | looks kind of like ansigenome | 16:53 |
| evrardjp | yes that was the name I was looking for | 16:53 |
| evrardjp | ansigenome is for roles dependencies though | 16:53 |
| evrardjp | but it's fancy. | 16:53 |
| evrardjp | :D | 16:53 |
| dmsimard | never heard of ansigenome | 16:54 |
| * dmsimard looks | 16:54 | |
| evrardjp | wow | 16:55 |
| evrardjp | you have to try it :p | 16:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 17:01 |
| cloudnull | jrosser: ^ | 17:01 |
| jrosser | oooh i will look later, not goingto get back on this seriously till monday though | 17:04 |
| jrosser | thanks btw! | 17:05 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [DNM] Build everything on metal https://review.openstack.org/504224 | 17:05 |
| odyssey4me | cloudnull why not rather implement each type as a container in a single test - why require multiple tests? | 17:06 |
| odyssey4me | ie setup base containers for each type, and create a cow container for each from the base using host_vars | 17:07 |
| cloudnull | odyssey4me: that's not a bad idea | 17:07 |
| * cloudnull refactoring | 17:07 | |
| odyssey4me | awesome, thanks | 17:08 |
| logan- | that's neat though dmsimard.. I think it would come in handy for some of the task file includes in some roles where you're wondering what the tree looks like | 17:09 |
| logan- | thanks | 17:09 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: tasks: lxc_container_create: Retry LXC start on failures https://review.openstack.org/509786 | 17:10 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: SUSE: Remove lxc-1.X.X workarounds https://review.openstack.org/509771 | 17:11 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 17:22 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 17:23 |
| jrosser | cloudnull: does that have several different COW types exist in the same host at the same time? | 17:24 |
| cloudnull | it would. | 17:25 |
| jrosser | well, zfs dataset pool/lxc could well mount at /var/lib/lxc (or wherever the filesystem view of the containers is) | 17:26 |
| jrosser | so would own the whole of that tree | 17:26 |
| cloudnull | that should be ok. i think | 17:27 |
| cloudnull | - /var/lib/machines is btrfs | 17:27 |
| odyssey4me | cloudnull heh, see PS2 | 17:27 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 17:28 |
| odyssey4me | also, why not add a btrfs one too? ;) | 17:28 |
| jrosser | oh maybe i misunderstan but i thought the base image was meddled with directly at /var/lib/lxc? | 17:28 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 17:28 |
| cloudnull | jrosser: it would be, for zfs and overlayfs | 17:28 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 17:28 |
| cloudnull | odyssey4me: zfs and btrfs would occupy the same path. | 17:29 |
| jrosser | can they though? thats my question | 17:29 |
| cloudnull | in that case ,no | 17:29 |
| jrosser | becasue certinaly in the zfs case it would become a mount | 17:29 |
| cloudnull | machinectl is using btrfs but its under /var/lib/machines | 17:30 |
| cloudnull | Thats why I didn't make a fifth contianer type | 17:30 |
| cloudnull | if we wanted to test all of the backends possible they'd have to be different tests | 17:31 |
| cloudnull | we would be able to combine them like this | 17:31 |
| odyssey4me | cloudnull is that just because of the base container used? | 17:31 |
| odyssey4me | or is it due to machinectl being used? | 17:31 |
| cloudnull | no. | 17:32 |
| odyssey4me | I'm not following - each container has its own file system, so why can't they each do their own thing? | 17:32 |
| cloudnull | lxc supports btrfs naively, like zfs | 17:32 |
| cloudnull | however both btrfs and zfs have to use the path /var/lib/lxc | 17:32 |
| cloudnull | we can't change that | 17:32 |
| odyssey4me | oh, I see | 17:33 |
| odyssey4me | ok, makes sense then | 17:33 |
| cloudnull | machinectl is its own thing | 17:33 |
| cloudnull | so we "should" be able to do dir, overlayfs, zfs and machinectl together. | 17:33 |
| cloudnull | lvm and btrfs are missing | 17:34 |
| *** jwitko is now known as Swifty | 17:34 | |
| *** Swifty is now known as SwiftyB | 17:34 | |
| odyssey4me | however, it is possible to specify the location of the rootfs for each container yourself... and thus you can use different file systems I imaging... but the configs would still all be in /var/lib/lxc | 17:34 |
| cloudnull | unless we want to set it up as separate tests. | 17:34 |
| cloudnull | only dir supports changing the directory path | 17:34 |
| *** SwiftyB is now known as jwitko | 17:35 | |
| jrosser | for cow filesystems /var/lib/lxc becomes a mount point for that fs | 17:35 |
| *** germs has joined #openstack-ansible | 17:35 | |
| cloudnull | ^ | 17:35 |
| odyssey4me | --lxcpat ? | 17:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Force the installation of qemu packages https://review.openstack.org/509711 | 17:35 |
| odyssey4me | http://manpages.ubuntu.com/manpages/xenial/man1/lxc-create.1.html | 17:35 |
| jrosser | becasue the snapshots are created within the fs, and appear in the same space | 17:35 |
| odyssey4me | --lxcpath I mean | 17:35 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_nova stable/pike: Force the installation of qemu packages https://review.openstack.org/509848 | 17:36 |
| cloudnull | yes we can set that, but I don't this its supported by things like clone, etc | 17:36 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-os_nova stable/ocata: Force the installation of qemu packages https://review.openstack.org/509849 | 17:36 |
| cloudnull | I guess we could try? | 17:37 |
| odyssey4me | ok, interesting | 17:37 |
| odyssey4me | worth a try for the role test I reckon | 17:37 |
| *** lbragstad has quit IRC | 17:37 | |
| *** germs1 has joined #openstack-ansible | 17:37 | |
| jrosser | theres some hardwiring to negotiate too https://github.com/openstack/openstack-ansible-lxc_hosts/blob/dce34c0b872d14c97edc311ef1b5e341e4b85a46/tasks/lxc_post_install.yml#L22 | 17:38 |
| odyssey4me | oh yeah | 17:38 |
| odyssey4me | fun times | 17:38 |
| mgariepy | If you guys find something weird on LXC + zfs that is not working correctly poke me I can probably make stgraber fix it quite fast | 17:38 |
| odyssey4me | ok, I'm out for the night - cheers all | 17:39 |
| mgariepy | cya odyssey4me | 17:39 |
| mhayden | could someone give me a looksee on this backport? https://review.openstack.org/505618 | 17:40 |
| cloudnull | later odyssey4me | 17:40 |
| *** germs has quit IRC | 17:40 | |
| cloudnull | jrosser: yea. that might get tricky | 17:41 |
| cloudnull | mgariepy: nice! | 17:41 |
| cloudnull | well hopefully soon we'll have a zfs option tested in the gate | 17:41 |
| *** jwitko is now known as SwiftyBot | 17:42 | |
| jrosser | thanks for pushing that forward - still crawling up the learing curve here! | 17:42 |
| *** SwiftyBot is now known as jwitko | 17:42 | |
| *** jwitko is now known as SwiftyBot | 17:42 | |
| *** SwiftyBot is now known as jwitko | 17:42 | |
| cloudnull | if the combined things dont work its simple enough to set it up as seperate tests. | 17:42 |
| mgariepy | mhayden, what's the -1 zull ? | 17:43 |
| mgariepy | zuul** | 17:43 |
| mhayden | mgariepy: i cannot figure it out | 17:43 |
| mhayden | but it sounds like we're to treat it like a third party CI from what i saw in infra | 17:43 |
| mgariepy | http://logs.openstack.org/18/505618/2/check/legacy-ansible-func-centos-7/7cc1c1d/ara/result/affe2919-51b1-45ef-a88a-8e5ba1893e86/ | 17:44 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 17:44 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 17:44 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 17:44 |
| cloudnull | example using machinectl | 17:46 |
| cloudnull | http://paste.openstack.org/show/622784/ | 17:46 |
| cloudnull | I suspect the ZFS option would be similar in size | 17:46 |
| cloudnull | mgariepy: do you know if zfs is supported on cent/rhel? | 17:47 |
| cloudnull | mgariepy: mhayden: i asked about that this morning too | 17:47 |
| cloudnull | its expected | 17:47 |
| cloudnull | zuulv3 is running, but its like an external ci atm | 17:48 |
| mgariepy | we usualy install it with dkms | 17:48 |
| cloudnull | on cent? | 17:48 |
| mgariepy | yep | 17:48 |
| cloudnull | cool | 17:48 |
| cloudnull | so this is potentially something we could test in the gate? | 17:48 |
| mgariepy | https://github.com/zfsonlinux/zfs/wiki/RHEL-%26-CentOS | 17:49 |
| cloudnull | hwoarang: ^ same question but for suse ? | 17:49 |
| mgariepy | it be nice to test it. but we would need to build it. | 17:49 |
| cloudnull | I see the lzfse is available but nothing else in the default tumbleweed repos . | 17:50 |
| mhayden | mgariepy: since we're talking centos... https://review.openstack.org/#/c/506363/ ;) | 17:50 |
| cloudnull | mgariepy: maybe we could build it and host it on the infra mirrors, or is hosting that package still a no-no? | 17:50 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 17:53 |
| mgariepy | cloudnull, we can try to make the zfs test pass on centos, then ask infra to mirror it ? | 17:54 |
| mgariepy | we need to build the kernel modules for centos not sure how easy it would be to pkg this.. | 17:55 |
| cloudnull | yea. it may be more pain than its worth. | 17:55 |
| * cloudnull thinking about how the images are built for the infra tests | 17:56 | |
| mgariepy | maybe centos 8 will have zfs by default :D | 17:56 |
| cloudnull | maybe | 17:56 |
| *** hamzy has quit IRC | 17:57 | |
| mgariepy | also not sure if we need to build lxc with zfs support or it's magically included. | 17:57 |
| cloudnull | its magically included in the ubuntu packages. I believe suse too? | 17:58 |
| *** germs1 has quit IRC | 17:58 | |
| *** galstrom is now known as galstrom_zzz | 17:59 | |
| *** germs has joined #openstack-ansible | 18:01 | |
| cloudnull | jrosser: https://review.openstack.org/#/c/509838/4/tests/test-create-zfs-dev.yml@43 | 18:01 |
| cloudnull | do we need to specify the mount point | 18:01 |
| cloudnull | w/ -o mountpoint=/var/lib/lxc | 18:02 |
| cloudnull | IE: zpool create -o mountpoint=/var/lib/lxc {{ zfs_losetup.stdout }} | 18:04 |
| cloudnull | or is that not needed | 18:04 |
| *** ivve has quit IRC | 18:04 | |
| *** galstrom_zzz is now known as galstrom | 18:05 | |
| *** galstrom is now known as galstrom_zzz | 18:05 | |
| *** galstrom_zzz is now known as galstrom | 18:08 | |
| *** lbragstad has joined #openstack-ansible | 18:10 | |
| openstackgerrit | Merged openstack/ansible-hardening master: Update to RHEL 7 V1R2 STIG https://review.openstack.org/506363 | 18:11 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 18:14 |
| *** DanyC has quit IRC | 18:16 | |
| *** DanyC has joined #openstack-ansible | 18:16 | |
| *** ivve has joined #openstack-ansible | 18:17 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible master: [Docs] Add the bump of the upstream repo https://review.openstack.org/502502 | 18:20 |
| *** electrofelix has quit IRC | 18:20 | |
| *** DanyC has quit IRC | 18:21 | |
| mgariepy | cloudnull, added a comment in https://review.openstack.org/#/c/509838/ | 18:23 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add machinectl backend store options https://review.openstack.org/501059 | 18:23 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: [DNM] - Testing machinectl containers https://review.openstack.org/509866 | 18:24 |
| mgariepy | cloudnull, want me to edit the test-create-zfs-dev task? | 18:24 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 18:25 |
| cloudnull | mgariepy: cool ! | 18:26 |
| cloudnull | so it may be perfectly do-able with minimal change? | 18:26 |
| mgariepy | yep i guess so. | 18:26 |
| mgariepy | install zfs from the kmod repo and then you should be good to go :) | 18:27 |
| cloudnull | we probably want to do that within the lxc-hosts role | 18:27 |
| cloudnull | maybe no... | 18:28 |
| cloudnull | maybe we just make it part of the zfs test | 18:28 |
| * cloudnull thinking our lou d | 18:28 | |
| mgariepy | I would be in favor of having zfs by default | 18:28 |
| mgariepy | It saved me once during my newton trusty to xenial migration :) | 18:28 |
| mgariepy | i love snapshots | 18:29 |
| cloudnull | we'd have to figure out a way to not break existing environments if we did that. | 18:29 |
| cloudnull | maybe we put it all the packages in place | 18:30 |
| mgariepy | Yeah I know | 18:30 |
| logan- | do we build volgroups in lxc_hosts currently | 18:30 |
| cloudnull | I dont think so ? | 18:30 |
| mgariepy | maybe we could just detect if a certain zpool is there and use it. | 18:31 |
| cloudnull | i think if we put all the tools in place by default, but not specifically create a zfs pool and mount it we should be ok? | 18:31 |
| openstackgerrit | Merged openstack/openstack-ansible master: SUSE: Remove lxc-1.X.X workarounds https://review.openstack.org/509772 | 18:31 |
| mgariepy | like we do with the lxc -lvm. | 18:31 |
| logan- | yeah thats what im thinking mgariepy | 18:31 |
| cloudnull | lxc lvm is a little different as it uses the vg and really leaves /var/lib/lxc un-touched | 18:32 |
| cloudnull | where btrfs and zfs would mount over /var/lib/lxc | 18:32 |
| *** DanyC has joined #openstack-ansible | 18:33 | |
| mgariepy | could be prereq like the network config and so on. | 18:33 |
| *** galstrom is now known as galstrom_zzz | 18:33 | |
| cloudnull | ++ | 18:33 |
| mgariepy | if you put your containers in zfs, I'm pretty sure you do want to have /openstack and /var/lib/lxc in zfs. | 18:33 |
| mgariepy | at least I do. | 18:33 |
| cloudnull | ++ | 18:33 |
| logan- | yeah i wouldnt mind seeing some prep work in lxc_hosts. im thinkin let the deployers install and load the module, create the zpool, and feed that to osa | 18:34 |
| mgariepy | osa could detect it i guess. | 18:34 |
| logan- | and then carve the zfs (is zvol the correct term?) from there | 18:34 |
| logan- | from the specified zpool* | 18:34 |
| cloudnull | with the new changes to lxc-container-create now merged we can do that. | 18:34 |
| cloudnull | the backend store can be set to whatever is needed on the host | 18:35 |
| mgariepy | anyone uses btrfs ? | 18:35 |
| logan- | all of my stuff is on overlayfs currently | 18:35 |
| cloudnull | and once https://review.openstack.org/#/c/509608/ goes in we should have zfs support | 18:35 |
| cloudnull | mgariepy: I'm using btrfs but with machinectl | 18:35 |
| cloudnull | https://review.openstack.org/#/c/501059/ | 18:36 |
| cloudnull | effectively that | 18:36 |
| cloudnull | that way I can export import etc and have it all managed by machinectl | 18:37 |
| *** DanyC_ has joined #openstack-ansible | 18:38 | |
| cloudnull | from a redeployment i did today on a rack of gear | 18:39 |
| cloudnull | lxc_container_create : Create container (machinectl) ------------------- 38.15s | 18:39 |
| cloudnull | obviously the rest of the role took more time but the provisioning side of is quick | 18:40 |
| mgariepy | hehe | 18:40 |
| *** hamzy has joined #openstack-ansible | 18:41 | |
| *** DanyC has quit IRC | 18:42 | |
| *** hachi_ has joined #openstack-ansible | 18:44 | |
| *** DanyC_ has quit IRC | 18:44 | |
| jrosser | cloudnull: yes we do need to specify the mountpoint | 18:44 |
| cloudnull | i added it in the last patch | 18:44 |
| *** Taseer has quit IRC | 18:44 | |
| cloudnull | I figured you'd yell at me if i was wrong :D | 18:45 |
| ggnel | qtop | 18:45 |
| jrosser | and then zfs create <blah> or use the ansible zfs module for any other datasets you want, like /openstack or whatever | 18:46 |
| cloudnull | ggnel: https://raw.githubusercontent.com/qtop/qtop/master/qtop_py/contrib/qtop_demo.gif | 18:46 |
| cloudnull | :) | 18:46 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_openrc master: Add tacker to the list of aliases https://review.openstack.org/509831 | 18:48 |
| jrosser | cloudnull: if i were you i'd mount pool/lxc at /var/lib/lxc | 18:55 |
| jrosser | then you've got pool/anything-else for other stuff | 18:55 |
| jrosser | i made that mistake with lxd+zfs and its a royal PITA to suffle it all round afterwards | 18:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 18:57 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 18:57 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 18:57 |
| openstackgerrit | Merged openstack/openstack-ansible master: Fix Aodh's health checks https://review.openstack.org/507842 | 19:00 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Create an LXC create template for machinectl https://review.openstack.org/501054 | 19:07 |
| *** Taseer has joined #openstack-ansible | 19:08 | |
| *** DanyC has joined #openstack-ansible | 19:09 | |
| *** MasterOfBugs has joined #openstack-ansible | 19:11 | |
| *** markvoelker_ has joined #openstack-ansible | 19:15 | |
| *** markvoelker has quit IRC | 19:15 | |
| *** armaan has quit IRC | 19:16 | |
| *** armaan has joined #openstack-ansible | 19:17 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Add ZFS backing store to the container create role https://review.openstack.org/509608 | 19:20 |
| openstackgerrit | Merged openstack/openstack-ansible master: SUSE: Add OpenStack cloud repository when bootstrapping the host https://review.openstack.org/503722 | 19:23 |
| openstackgerrit | Merged openstack/openstack-ansible master: Run the ceph-mons deploy in serial https://review.openstack.org/508503 | 19:23 |
| *** pbandark has quit IRC | 19:27 | |
| *** pbandark has joined #openstack-ansible | 19:38 | |
| *** woodard has quit IRC | 19:42 | |
| *** mbuil has quit IRC | 19:56 | |
| *** smatzek has quit IRC | 19:59 | |
| *** tomtomtom has quit IRC | 19:59 | |
| *** smatzek has joined #openstack-ansible | 20:00 | |
| *** smatzek has quit IRC | 20:00 | |
| *** ianw|pto is now known as ianw | 20:00 | |
| *** smatzek has joined #openstack-ansible | 20:02 | |
| *** smatzek has quit IRC | 20:06 | |
| *** smatzek has joined #openstack-ansible | 20:08 | |
| cloudnull | jrosser: whats the best way to change that ? | 20:09 |
| jrosser | i found an example, just a mo..... | 20:10 |
| mhayden | mgariepy: so far, pike looks good on a prod CentOS 7 cloud | 20:11 |
| cloudnull | woot! | 20:11 |
| cloudnull | mhayden: ^ | 20:11 |
| jrosser | cloudnull: https://scotte.org/2016/07/lxc-containers-on-zfs | 20:11 |
| jrosser | like that | 20:12 |
| mhayden | other than me being an idiot, the O>P upgrade on CentOS was cake | 20:12 |
| cloudnull | about the loopback file . it would seem we dont need it, which is awesome | 20:12 |
| jrosser | yes thst is super cool if it works | 20:12 |
| mhayden | jrosser: nice writeup | 20:12 |
| cloudnull | jrosser: is pool0/... the normal ? | 20:12 |
| jrosser | you can choose | 20:12 |
| cloudnull | what do you do? | 20:13 |
| jrosser | pool | 20:13 |
| cloudnull | mgariepy: ^ | 20:13 |
| cloudnull | same question | 20:13 |
| jrosser | others choose tank | 20:13 |
| cloudnull | I'm good with whatever. its all net new to me :D | 20:13 |
| jrosser | but its a pool, sooooo | 20:13 |
| cloudnull | sounds good to m e | 20:13 |
| cloudnull | do you use pool/lxc | 20:15 |
| jrosser | not exactly - pool/lxd :) | 20:16 |
| cloudnull | ah yea | 20:17 |
| cloudnull | so... | 20:17 |
| cloudnull | we truncate the img file. | 20:17 |
| cloudnull | then `zpool create -m /var/lib/lxc -f lxc /openstack/lxc-zfs.img` | 20:17 |
| jrosser | but ykwim. and the you can out oher stuff i there too as ithers have suggested | 20:17 |
| jrosser | i would just create the pool | 20:18 |
| jrosser | thats "all the storage" | 20:18 |
| cloudnull | cool | 20:18 |
| cloudnull | updating that pr | 20:18 |
| jrosser | then you create a dataset in the pool | 20:18 |
| jrosser | but i have a suspicion lxc will do that for you | 20:18 |
| jrosser | courtesy of passing it pool/lxc | 20:19 |
| jrosser | but im unusre of how the mount is going to work | 20:19 |
| jrosser | because in lxd this is all magic and happens for you | 20:19 |
| jrosser | cloudnull: there was an earlier suggestion that /openstack was also a good thing to put in zfs, maybe the img should be elsewhere? | 20:21 |
| *** smatzek has quit IRC | 20:24 | |
| cloudnull | indeed. I think that makes a lot sense. | 20:25 |
| cloudnull | in prod that is | 20:25 |
| cloudnull | in test we're putting other image files in the openstack path | 20:25 |
| cloudnull | https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/tasks/prepare_loopback_cinder.yml#L32 | 20:25 |
| jrosser | a sure yes ths is just test of course | 20:25 |
| cloudnull | so for now I think its best to keep it in line, but we can always adjust later. | 20:26 |
| *** galstrom_zzz is now known as galstrom | 20:26 | |
| jrosser | that paste from earlier i had it lashed into the role test in vagrant | 20:26 |
| jrosser | could ssh in and muck around with the created pool, probably spin a container by hand and see how it all looks | 20:27 |
| jrosser | but i had to leave so didnt get as further than seeing the pool ok with sudo zfs list | 20:27 |
| cloudnull | I have this on an infra node now. its cool | 20:28 |
| gunix | guys, i have a question regarding 3 types of openstack deployment | 20:30 |
| *** lbragstad has quit IRC | 20:30 | |
| gunix | 1. you install everything with scripts and you cluste everything by hand | 20:30 |
| gunix | 2. you install everything by scripts and you use pacemaker for cluster | 20:30 |
| gunix | 3. you install everything with ANSIBLE but you don't use pacemaker | 20:31 |
| gunix | 4. you install everything with ansible and you configure all clustering via pacemaker | 20:31 |
| gunix | which ideea do you find best? | 20:31 |
| mhayden | well there are four types there ;) | 20:31 |
| gunix | yes, i figured along the way | 20:32 |
| gunix | i am about to go for 3 | 20:32 |
| gunix | i already did 1 | 20:32 |
| mhayden | OSA's stance is probably #3 | 20:32 |
| mhayden | i've done 1, 2, and 4 before | 20:32 |
| gunix | so do you think pacemaker is worth the effort of learning it? | 20:32 |
| mhayden | pacemaker is awesome software, but it's not entirely necessary for an openstack deployment | 20:32 |
| mhayden | i know red hat prefers it, and i understand their reasons | 20:33 |
| mhayden | using ansible (or any other config mgmt software) is helpful since you can ensure the right changes were made at the right times | 20:33 |
| mhayden | and you can rely on the automation rather than hand-configuring things | 20:33 |
| mhayden | custom configs done *outside* of config management leads to problems | 20:34 |
| mhayden | which has always burned me on #1 and #2 | 20:34 |
| mhayden | also pacemaker requires some more networking knowledge (especially around managing those corosync rings) | 20:36 |
| mhayden | you'll need two network paths for corosync to use for sending heartbeats back and forth | 20:36 |
| mhayden | gunix: does that help? | 20:36 |
| gunix | mhayden: network knowledge is not a problem, it's just that i never used pacemaker before. and i am just starting with ansible. i did only option #1 | 20:39 |
| mhayden | ah okay | 20:39 |
| mhayden | pacemaker is good stuff, don't get me wrong, but it can be challenging to get right | 20:39 |
| gunix | now, this should go into testing within the company and i need to build the poc, and i want to do it from now with ansible so i can scale it later | 20:39 |
| mhayden | and you'll need a fencing/STONITH capability with pacemaker, which isn't always easy | 20:39 |
| gunix | galera doesn't have stonith by default? | 20:40 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 20:40 |
| *** lbragstad has joined #openstack-ansible | 20:40 | |
| cloudnull | jrosser: ^ | 20:40 |
| mhayden | if you find people doing pacemaker without fencing (I've seen it done!), i worry about them | 20:40 |
| mhayden | well, galera helps out with mariadb clustering by making it easy to write to any of the nodes | 20:40 |
| mhayden | and it keeps the data in sync | 20:40 |
| gunix | so you can't get proper stonith without pacemaker? | 20:40 |
| mhayden | no no -- i mean you cannot have pacemaker without STONITH/fencing capabilities | 20:41 |
| gunix | oh | 20:41 |
| mhayden | well, you can, but it's a bad idea ;) | 20:41 |
| mhayden | consider two nodes sharing a filesystem -- if one of those starts misbehaving and you have no fencing, then you may end up with a corrupted fs | 20:41 |
| mhayden | or a split-brain situation | 20:41 |
| * mhayden has been in awful DRBD split brain scenarios before | 20:41 | |
| mhayden | or two machines might argue over who can hold the floating IP address | 20:42 |
| gunix | i can only think in a situation where somehow two computes try to power on the same VM and it somehow tries to write the same logical volume from cinder | 20:42 |
| gunix | do you need shared filesystem for anything else except cinder? | 20:43 |
| gunix | i mean, for the backing storage solution ... which could be some ceph, glusterfs or some dedicated storage (which i would avoid, but my company might force me to that) | 20:44 |
| mhayden | well i don't think you need clustering for nova compute | 20:44 |
| mhayden | if the hypervisor that runs nova compute goes offline, then your VMs and nova compute are down ;) | 20:44 |
| *** gouthamr_ has quit IRC | 20:44 | |
| mhayden | gunix: best bet i've seen so far for storage is nova + cinder + ceph | 20:44 |
| jrosser | cloudnull: looks good - have you tried that pool direct from file thing? | 20:44 |
| gunix | and i guess the nove-scheduler should take care of that, if a compute node goes down | 20:44 |
| cloudnull | jrosser: yes it works well | 20:45 |
| jrosser | great | 20:45 |
| gunix | i don't know ceph, i find gluster A LOT easier to use | 20:45 |
| gunix | afaik proxmox has some great tutorials for ceph. they have ceph support integrated. in should deploy that once to get a feeling of it | 20:46 |
| jrosser | cloudnull: so in theory with the patches that went in today all the pieces are in place? | 20:46 |
| mhayden | gunix: gluster isn't bad, but it seems people are moving to ceph more often now | 20:47 |
| *** TomMc has joined #openstack-ansible | 20:47 | |
| mhayden | gunix: our philosophy is to run more than one of each service, and use a LB to talk to them | 20:47 |
| gunix | mhayden: yes, they are, ceph is the more complex/enterprise solution, as far as i see it. gluster is a bit more plug and play | 20:47 |
| mhayden | so instead of a complex pacemaker setup for nova api, just run 2-3 of them | 20:47 |
| mhayden | and put a LB in front | 20:47 |
| mhayden | gunix: but ceph would give more performance (does require more hardware/knowledge investment, though) | 20:48 |
| gunix | mhayden: yea, that's exactly what i want to aim for. but i wanted to talk with someone to clear out the downside of NOT using pacemaker... redhat is pushing it so hard that i feel like i am missing on something big here | 20:48 |
| gunix | but i think i could do in production on a large scale without it | 20:48 |
| mhayden | keep in mind that red hat sells solutions around it ;) | 20:49 |
| mhayden | that could be a reason | 20:49 |
| * mhayden is not disparaging pacemaker, either ;) | 20:49 | |
| cloudnull | jrosser: i think so | 20:49 |
| gunix | redhat always had their own way of doing stuff. i am mostly a debian guy. i had a lot of redhat at work and i always had the "but why?" feeling. apart from that, their systems are incredebly stable | 20:50 |
| *** DanyC has quit IRC | 20:50 | |
| *** hamzy has quit IRC | 20:52 | |
| jrosser | cloudnull: looks like time to let it rip and see what happens :) | 20:52 |
| cloudnull | jrosser: once https://review.openstack.org/#/c/501059/ and https://review.openstack.org/#/c/509838/ go in we're done and have tests for the various backends | 20:52 |
| *** smatzek has joined #openstack-ansible | 20:53 | |
| *** smatzek has quit IRC | 20:58 | |
| *** perniciouscaffei has quit IRC | 20:59 | |
| gunix | mhayden: thank you for the chat! i will go /away now to sleep | 21:03 |
| *** TomMc has quit IRC | 21:03 | |
| *** TomMc has joined #openstack-ansible | 21:04 | |
| mhayden | gunix: anytime ;) | 21:04 |
| mhayden | i'm always happy to let people know how i've failed horribly in the past ;) | 21:04 |
| SamYaple | im disparaging pacemaker | 21:04 |
| SamYaple | i really dont like it | 21:05 |
| mhayden | but you're too late! :) | 21:05 |
| gunix | mhayden: if you failed a lot, write a blog or even better, start a youtube blog. | 21:05 |
| * mhayden prefers ol' keepalived | 21:05 | |
| SamYaple | indeed | 21:05 |
| SamYaple | though that doesnt have fencing, meh | 21:05 |
| mhayden | gunix: i do blog from time to time -> https://major.io/ | 21:05 |
| gunix | haha | 21:06 |
| gunix | your name is really major | 21:06 |
| gunix | nice | 21:06 |
| SamYaple | he was born to lead | 21:07 |
| *** jheroux has quit IRC | 21:11 | |
| *** eumel8 has quit IRC | 21:11 | |
| chyka | oh damn. i just ran os-nova-install.yml on a working environment, and it looks like it updated some libvirt and qemu packages, and broke instance creation and snapshots | 21:12 |
| chyka | looks like the playbook didnt update qemu-system-x86, which i had to update manually on the compute nodes | 21:12 |
| chyka | is there a "don't force any upgrades ever" variable? | 21:13 |
| cloudnull | chyka: yes. you can set the package state. https://github.com/openstack/openstack-ansible-os_nova/blob/master/defaults/main.yml#L29-L32 | 21:14 |
| *** armaan has quit IRC | 21:15 | |
| chyka | is there any catchall across all the playbooks? or just variables for individual services | 21:16 |
| cloudnull | chyka: if you're on newton+ you can set the global "package_state" option | 21:16 |
| chyka | on ocata, nice thats perfect thanks | 21:16 |
| cloudnull | https://github.com/openstack/openstack-ansible/blob/stable/newton/playbooks/inventory/group_vars/all.yml#L25-L27 | 21:17 |
| chyka | ahh i didnt see it in the docs or user variables samples, didnt think to look in there. ill see if there are any other useful options in there, thanks again | 21:20 |
| cloudnull | yea there's lots of tunables within group_vars which help at a global level . | 21:21 |
| cloudnull | but its a bit cryptic to get at | 21:21 |
| *** mrtenio has quit IRC | 21:23 | |
| *** TomMc has quit IRC | 21:29 | |
| *** esberglu has quit IRC | 21:36 | |
| tasker | i need to apply some patches to nova before they hit my hosts and containers. someone suggested setting "nova_git_repo" in /etc/openstack_deploy/user_varibale.yml | 21:41 |
| tasker | I thought I had, but OSA wasn't pulling my repo in. I looked and the setting is gone. | 21:41 |
| tasker | does OSA ever overwrite that file? | 21:41 |
| tasker | -- sorry: user_varibales.yml | 21:41 |
| tasker | say, during bootstrap? | 21:42 |
| *** hachi_ has quit IRC | 21:42 | |
| *** gouthamr has joined #openstack-ansible | 21:43 | |
| cloudnull | no that file is user controlled. | 21:45 |
| tasker | ok. must be my memory then. | 21:45 |
| *** MasterOfBugs has quit IRC | 21:46 | |
| *** MasterOfBugs has joined #openstack-ansible | 21:46 | |
| *** armaan has joined #openstack-ansible | 21:52 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create stable/pike: tasks: lxc_container_create: Retry LXC start on failures https://review.openstack.org/509910 | 21:54 |
| *** perniciouscaffei has joined #openstack-ansible | 21:56 | |
| kylek3h | chyka: Still around? | 21:59 |
| chyka | yup | 22:00 |
| kylek3h | Regarding that issue yesterday with cloud init and ssh keys. | 22:00 |
| kylek3h | I fixed my install. | 22:00 |
| kylek3h | I had to add: enable_isolated_metadata = True to /etc/neutron/dhcp_agent.ini | 22:00 |
| *** perniciouscaffei has quit IRC | 22:01 | |
| chyka | ohh interesting, i'll give that a shot thanks! 23.111.76.22 | 22:02 |
| chyka | whoops * kylek3h | 22:03 |
| *** woodard has joined #openstack-ansible | 22:03 | |
| kylek3h | sure. :-) | 22:03 |
| *** pramodrj07 has joined #openstack-ansible | 22:04 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 22:04 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create master: tasks: lxc_container_create: Check for 'lxc-info' failures and retry https://review.openstack.org/509915 | 22:06 |
| chyka | ah damn i already have that in my dhcp.ini | 22:06 |
| *** MasterOfBugs has quit IRC | 22:07 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_hosts master: The ZFS root name needs a default https://review.openstack.org/509916 | 22:08 |
| *** threestrands has joined #openstack-ansible | 22:12 | |
| cloudnull | any cores around want to give this a push https://review.openstack.org/#/c/501059/ | 22:16 |
| *** galstrom is now known as galstrom_zzz | 22:32 | |
| *** lbragstad has quit IRC | 22:38 | |
| *** pbandark has quit IRC | 22:38 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.10 https://review.openstack.org/508152 | 22:48 |
| *** weezS has quit IRC | 22:54 | |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server stable/pike: Optimize CentOS Erlang installation https://review.openstack.org/505618 | 22:58 |
| masber | good morning, I am reading OSA documentation, I would like to give it a go. The docs says bonding network is recommended. Which type of bonding would you recommend? https://docs.openstack.org/project-deploy-guide/openstack-ansible/pike/overview-requirements.html | 23:00 |
| cloudnull | I use LACP but it really does not matter. | 23:02 |
| cloudnull | its totally up to you | 23:02 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 23:03 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 23:06 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 23:07 |
| *** perniciouscaffei has joined #openstack-ansible | 23:11 | |
| masber | we internally used XOR as recommended by storage provider but it links mac address with NIC so vm to vm bandwidth will be restricted to only 1 nic, not sure if you can get full capability using LACP? | 23:12 |
| masber | cloudnull, ^^^ | 23:12 |
| masber | is there a plan to move from LSC to docker? | 23:14 |
| masber | *LXC | 23:15 |
| *** galstrom_zzz is now known as galstrom | 23:15 | |
| masber | also, would it be possible to deploy the control plane in VMWare, compute and ceph on baremetal and still be able to use ironic for deployment? | 23:19 |
| *** germs has quit IRC | 23:20 | |
| cloudnull | masber: w/ LACP the bond is active/active so yes it will use both nics at the same time which should provide better performance. that said the switch has to support LACP specifically. | 23:26 |
| cloudnull | there's no plan to go to docker. our container usage is geared toward machine containers. | 23:27 |
| cloudnull | and all of our roles support both physical hosts and containers | 23:27 |
| cloudnull | as for VMware it sure is. i think jamesdenton is using vmware for the control plane | 23:28 |
| cloudnull | I use KVM w/ physical hosts for test infra all the time. | 23:29 |
| *** germs has joined #openstack-ansible | 23:31 | |
| cloudnull | Ironic simply needs to be able to communicate with the physical hosts out of band management. we use ipmi its simple and almost universally supportable. | 23:33 |
| cloudnull | the ironic "compute" nodes, api and scheduler would all go in your vmware control plane, and you'd have baemental hosts enrolled into ironic. | 23:34 |
| *** chyka has quit IRC | 23:36 | |
| *** perniciouscaffei has quit IRC | 23:36 | |
| masber | yes | 23:36 |
| *** chyka has joined #openstack-ansible | 23:36 | |
| masber | ok, also documentation says "Because of a limitation with container iSCSI connectivity, you must deploy the cinder-volume service directly on a physical host (not into a container) when using storage back ends that connect via iSCSI. This includes the LVMVolumeDriver and many of the drivers for commercial storage devices." I am guessing having cinder service in vm should not be an issue, is that right? | 23:37 |
| masber | I am double checking because it says "you must deploy the cinder-volume service directly on a physical host" | 23:38 |
| masber | but I guess a vm should also work | 23:38 |
| *** jwitko has quit IRC | 23:39 | |
| *** jwitko has joined #openstack-ansible | 23:39 | |
| cloudnull | in a vm is totally fine. | 23:39 |
| cloudnull | that issue is largely due to AF_NETLINK not being namespace aware. | 23:40 |
| cloudnull | that causes the isci initiator to fail. | 23:40 |
| cloudnull | on the compute side. | 23:40 |
| cloudnull | putting everything in a VM works great. | 23:41 |
| *** chyka has quit IRC | 23:41 | |
| cloudnull | if you've got a host with a lot of ram/storage/procs have a look at https://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio | 23:42 |
| cloudnull | builds everything in VMs to simulate a multi-node environment. | 23:42 |
| cloudnull | useful for folks who dont have a rack of gear laying around. | 23:42 |
| masber | cloudnull, nice! | 23:43 |
| *** hamzy has joined #openstack-ansible | 23:43 | |
| masber | our case is a little but different, we use VMWare to deploy our control plane for HPC, we recently added new nodes to out VMWare cluster so I better use it to deploy openstack control plane | 23:45 |
| cloudnull | cool. | 23:45 |
| cloudnull | that works fine | 23:45 |
| cloudnull | make the vm, setup networking, profit! | 23:46 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add functional tests for our different backendstores https://review.openstack.org/509838 | 23:49 |
| *** markvoelker_ has quit IRC | 23:51 | |
| cloudnull | any cores around? wanna give me a review https://review.openstack.org/#/c/501059 ? | 23:55 |
| *** pramodrj07 has quit IRC | 23:56 | |
| cloudnull | this one too https://review.openstack.org/#/c/509916/ | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
