Sunday, 2018-10-07

*** Adri2000 has joined #openstack-ansible00:06
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839300:06
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Add cloudkitty dashboard  https://review.openstack.org/60301300:08
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839301:02
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839301:04
*** markvoelker has joined #openstack-ansible01:34
*** sawblade6 has joined #openstack-ansible01:37
*** sawblade6 has quit IRC01:43
*** mugsie has quit IRC02:01
*** markvoelker has quit IRC02:06
openstackgerritzhulingjie proposed openstack/openstack-ansible-os_horizon master: Add watcher dashboard into horizon  https://review.openstack.org/60315602:46
openstackgerritzhulingjie proposed openstack/openstack-ansible-os_horizon master: Add watcher dashboard into horizon  https://review.openstack.org/60315602:53
openstackgerritzhulingjie proposed openstack/openstack-ansible-os_horizon master: Add watcher dashboard into horizon  https://review.openstack.org/60315602:59
*** sawblade6 has joined #openstack-ansible03:01
*** dave-mccowan has quit IRC03:02
*** markvoelker has joined #openstack-ansible03:03
*** sawblade6 has quit IRC03:05
*** sep has quit IRC03:06
*** sep has joined #openstack-ansible03:12
*** markvoelker has quit IRC03:37
*** bradm has quit IRC03:41
*** weezS has joined #openstack-ansible04:36
*** weezS has quit IRC04:41
*** gkadam has joined #openstack-ansible05:19
*** gkadam has quit IRC05:23
*** markvoelker has joined #openstack-ansible05:34
*** dariko has joined #openstack-ansible05:35
*** dariko has quit IRC05:44
*** DanyC has joined #openstack-ansible06:03
*** DanyC_ has joined #openstack-ansible06:07
*** markvoelker has quit IRC06:07
*** DanyC has quit IRC06:10
*** gkadam has joined #openstack-ansible06:13
*** DanyC_ has quit IRC06:32
*** gkadam has quit IRC06:43
*** markvoelker has joined #openstack-ansible07:04
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible master: Fix healthcheck-hosts to work when behind a proxy  https://review.openstack.org/60544007:31
*** markvoelker has quit IRC07:37
*** pcaruana has joined #openstack-ansible07:51
*** spatel has joined #openstack-ansible08:16
*** spatel has quit IRC08:20
*** DanyC has joined #openstack-ansible08:33
*** DanyC has quit IRC08:39
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: [DNM] debug slow apt installs  https://review.openstack.org/60842308:51
*** sawblade6 has joined #openstack-ansible09:05
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: [DNM] debug slow apt installs  https://review.openstack.org/60842309:08
*** sawblade6 has quit IRC09:08
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: [DNM] test with repo server package cache disabled  https://review.openstack.org/60844509:31
*** markvoelker has joined #openstack-ansible09:34
*** DanyC has joined #openstack-ansible09:35
*** sabomia` has quit IRC09:35
*** DanyC has quit IRC09:41
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: [DNM] debug slow apt installs  https://review.openstack.org/60842309:44
benkohlHi! Any idea what this means and how to fix it? "ceph-volume lvm: error: unrecognized arguments: batch --filestore --yes /dev/sdb /dev/sdd /dev/sde" in "TASK [ceph-osd : use ceph-volume lvm batch to create filestore osds]".10:06
benkohlI use osa rocky and try to deploy ceph together with osa.10:06
*** markvoelker has quit IRC10:07
*** dariko has joined #openstack-ansible10:30
*** dariko has quit IRC10:31
*** markvoelker has joined #openstack-ansible11:04
*** Emine has quit IRC11:05
*** tosky has joined #openstack-ansible11:22
*** dave-mccowan has joined #openstack-ansible11:29
*** faizy98 has joined #openstack-ansible11:29
*** faizy_ has quit IRC11:32
*** markvoelker has quit IRC11:37
*** sawblade6 has joined #openstack-ansible12:02
*** sawblade6 has quit IRC12:06
*** spatel has joined #openstack-ansible12:17
*** spatel has quit IRC12:22
*** sawblade6 has joined #openstack-ansible12:36
*** sawblade6 has quit IRC12:41
*** hamzaachi has joined #openstack-ansible12:44
*** faizy_ has joined #openstack-ansible13:04
*** faizy98 has quit IRC13:07
*** Darcidride has joined #openstack-ansible13:13
*** sawblade6 has joined #openstack-ansible13:19
*** hamzaachi has quit IRC13:22
*** sawblade6 has quit IRC13:23
*** hamzaachi has joined #openstack-ansible13:30
*** markvoelker has joined #openstack-ansible13:35
*** hamzaachi has quit IRC13:47
*** ioni has quit IRC13:52
*** markvoelker has quit IRC14:07
*** sawblade6 has joined #openstack-ansible14:08
*** sawblade6 has quit IRC14:13
*** udesale has joined #openstack-ansible14:20
*** tosky has quit IRC14:35
*** hamzaachi has joined #openstack-ansible14:38
*** tosky has joined #openstack-ansible14:40
*** hamzaachi has quit IRC14:58
*** sawblade6 has joined #openstack-ansible15:08
*** ansmith has joined #openstack-ansible15:18
mnaserlogan-: looks like we're getting kvm failures on limestone15:24
mnaserspecifically happening to centos15:24
logan-like nested virt type failures?15:25
mnaserhttp://logs.openstack.org/27/608427/2/check/openstack-ansible-deploy-aio_metal-centos-7/878d24e/logs/host/libvirt/qemu/instance-00000001.log.txt.gz15:25
mnaseryeah15:25
mnaserKVM: entry failed, hardware error 0x015:25
logan-fun15:25
mnaserlogan-: do you mind if i push up a patch to drop it?15:26
mnaseri think its affecting opensuse too15:26
mnaserdrop using nested virt that is15:26
mnaserand just always use qemu15:26
logan-i thought we had that already15:26
mnaseronly for obh15:26
mnaserovh*15:26
logan-xenial nested virt is also broken15:26
mnaserbleh15:26
logan-you sure? odyssey4me reverted the limestone one a few weeks ago i thought15:26
mnaserso just switch to qemu always15:26
logan-but heres the thing15:26
*** sawblade6 has quit IRC15:26
logan-xenial base kernel on the hv = broken nested virt for xenial nodepool guests right now15:27
logan-xenial hwe on the hv = working nested virt in nodepool guests15:27
logan-i wonder if the same is true for centos15:27
logan-hwe is the 4.15 kernel instead of the base 4.4 xenial kernel15:27
*** udesale has quit IRC15:28
logan-let me check where this test ran15:28
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Disable nested virt and set SUSE+Bionic jobs to non-voting  https://review.openstack.org/60842715:29
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Restore Bionic jobs to voting  https://review.openstack.org/60842815:29
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Restore OpenSUSE jobs  https://review.openstack.org/60842915:29
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Enable nspawn jobs  https://review.openstack.org/60843315:29
mnaserlogan-: ok cool, in the meantime, above ^15:29
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Add one test case to the TestMergeDictUnit for same key dict merge  https://review.openstack.org/60016015:30
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Build translated docs in tox  https://review.openstack.org/60456715:30
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Fix order for restarts  https://review.openstack.org/59303215:30
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Set default glance image tag  https://review.openstack.org/60803715:31
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Generalize package setup across distros  https://review.openstack.org/60717815:31
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: scripts: sources-branch-updater: Make regexp for master branch stricter  https://review.openstack.org/60830115:31
*** Darcidride has quit IRC15:31
*** hamzaachi has joined #openstack-ansible15:41
logan-it ran on one of the ones that did not have an hwe kernel yet15:42
logan-ive been putting off upgrading the rest due to the zuul queue backlog the last few weeks.15:42
logan-im going to remove them from scheduling now and roll thru the rest for upgrades today so hopefully that will get us working nested virt again :)15:43
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_octavia master: Stop arbitrary shutting down services  https://review.openstack.org/60622115:45
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_octavia master: Convert role to use a common systemd service role  https://review.openstack.org/60622215:45
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_octavia master: Automatically create and import keypair if it doesn't exist  https://review.openstack.org/60614315:45
*** hamzaachi has quit IRC16:33
benkohlI'm getting "The conditional check 'physical_host in hostvars['localhost']['unbound_physical_hosts']' failed. The error was: template error while templating string: unexpected '|'" and "The error appears to have been in '/opt/openstack-ansible/playbooks/common-tasks/unbound-clients.yml': line 18, column 3, [...]" in TASK [Apply resolver sorting].17:02
benkohlAny Idea what that means?17:02
jrossermnaser: this doesnt seem to give those apt-get problems https://review.openstack.org/#/c/608445/17:07
mnaserjrosser: darn17:07
mnaserso we know its apt-cacher-ng thats the cause17:07
jrossermnaser: starts to point to things between the repo server and upstream mirror17:07
jrosseri guess theres haproxy in there too17:08
mnaserjrosser: btw i have a held timed out job17:08
mnaserwith a vm if you need access to it17:08
mnaseri wasnt able to troubleshoot much, would you like access to it?17:08
jrosserwhy not..... its very curious17:09
mnaserjrosser: can i have ssh keys?17:09
mnaserjrosser: so i dunno17:19
mnaseri dont think its an haproxy issue imho17:19
jrosserthe connect() that apt-get makes would be to haproxy though?17:20
mnaserwe would have seen other things go wrong17:20
mnaserhaproxy logs didnt show anything weird17:20
mnaserjrosser: maybe the next test could be skipping haproxy17:21
mnaserlets set the mirror directly to the container that runs it and avoid haproxy ?17:21
mnaserand see if that issue happens?17:21
jrosseryeah, sounds good plan17:21
mnaserthat will narrow it down further17:22
*** tosky has quit IRC17:38
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: bypass haproxy for apt-cacher-ng  https://review.openstack.org/60845917:44
*** hamzaachi has joined #openstack-ansible17:46
mnasercloudnull: so im debugging nspawn jobs17:55
mnaserit looks like it fails because it cannot ssh17:55
mnasernow based on my checking it looks like `mv-container` does not actually get an ip address17:56
mnaseroh i think i found whats wrong17:56
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Enable passing nspawn jobs  https://review.openstack.org/60843317:59
mnasercloudnull: ^ if you have a minute to check it out, ill hack some more, landing shortly17:59
logan-np18:34
logan-wrong window18:34
logan-let me know if you see more nested virt fails mnaser. i disabled those old kernel hvs earlier, so hopefully that will resolve it.18:35
logan-ideally we can avoid disabling it because it makes tempest so much slower and prone to job timeouts18:35
*** hamzaachi has quit IRC18:55
*** sawblade6 has joined #openstack-ansible18:59
*** aedc has quit IRC19:02
*** sawblade6 has quit IRC19:03
*** hamzaachi has joined #openstack-ansible19:27
*** sawblade6 has joined #openstack-ansible19:35
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839319:43
*** sawblade6 has quit IRC19:45
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: [DNM] bypass haproxy for apt-cacher-ng  https://review.openstack.org/60845920:13
*** tosky has joined #openstack-ansible20:15
prometheanfiremnaser: I may need to lengthen the lxc cache prep timeout20:24
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839320:30
*** Emine has joined #openstack-ansible20:32
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839320:39
*** sawblade6 has joined #openstack-ansible20:41
*** sawblade6 has quit IRC20:45
*** sawblade6 has joined #openstack-ansible20:55
*** sawblade6 has quit IRC21:00
*** hamzaachi has quit IRC21:01
*** Emine has quit IRC21:03
*** aludwar has quit IRC21:15
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839321:21
openstackgerritMohammed Naser proposed openstack/openstack-ansible master: Enable passing nspawn jobs  https://review.openstack.org/60843321:26
*** sawblade6 has joined #openstack-ansible21:37
*** sawblade6 has quit IRC21:41
*** sawblade6 has joined #openstack-ansible21:50
*** sawblade6 has quit IRC21:55
*** aludwar has joined #openstack-ansible22:06
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839322:07
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839322:58
*** sawblade6 has joined #openstack-ansible23:13
*** sawblade6 has quit IRC23:18
*** tosky has quit IRC23:21
*** sawblade6 has joined #openstack-ansible23:34
*** sawblade6 has quit IRC23:39
openstackgerritMatthew Thode proposed openstack/openstack-ansible-lxc_hosts master: add gentoo support  https://review.openstack.org/60839323:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!