Monday, 2019-12-02

*** gokhani has quit IRC00:07
*** tosky has quit IRC00:26
*** xjared has left #openstack-ansible00:47
*** spsurya has joined #openstack-ansible01:28
*** dave-mccowan has joined #openstack-ansible01:47
*** udesale has joined #openstack-ansible04:37
*** dave-mccowan has quit IRC04:40
*** gokhani has joined #openstack-ansible04:45
*** raukadah is now known as chkumar|rover05:54
*** pcaruana has joined #openstack-ansible06:32
*** miloa has joined #openstack-ansible06:48
*** chkumar|rover is now known as chkumar|ruck06:51
*** cshen has joined #openstack-ansible07:03
*** kmadac has joined #openstack-ansible07:11
cshenmorning07:12
*** kmadac_ has quit IRC07:14
*** jbadiapa has joined #openstack-ansible07:47
*** tosky has joined #openstack-ansible08:31
*** rpittau|afk is now known as rpittau08:31
*** rgogunskiy has joined #openstack-ansible08:44
*** DanyC has joined #openstack-ansible09:01
*** DanyC has quit IRC09:05
*** cshen has quit IRC09:06
*** cshen has joined #openstack-ansible09:06
*** DanyC has joined #openstack-ansible09:07
*** trident has quit IRC09:07
*** DanyC has quit IRC09:09
*** trident has joined #openstack-ansible09:09
*** DanyC has joined #openstack-ansible09:23
*** rgogunskiy has quit IRC09:24
*** rgogunskiy has joined #openstack-ansible09:25
*** rgogunskiy has quit IRC09:28
*** rgogunskiy has joined #openstack-ansible09:29
*** DanyC has quit IRC09:43
*** DanyC has joined #openstack-ansible09:44
*** DanyC has quit IRC09:46
*** DanyC has joined #openstack-ansible09:47
jrossernoonedeadpunk: i think we need to drop our debian jobs on master, or fix buster "2019-12-01 14:32:29.586 7984 ERROR oslo_service.service nova.exception.InternalError: Nova requires libvirt version 4.0.0 or greater."09:49
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Move debian jobs to buster  https://review.opendev.org/68177709:50
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Disable debian jobs on master branch  https://review.opendev.org/69684810:05
openstackgerritDirk Mueller proposed openstack/openstack-ansible-os_horizon master: opensuse: adjust for Python 3.x switch  https://review.opendev.org/69183310:08
*** ivve has joined #openstack-ansible10:08
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Move debian jobs to buster  https://review.opendev.org/68177710:12
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/69685610:18
*** tosky_ has joined #openstack-ansible10:26
*** tosky has quit IRC10:28
*** gchamoul has joined #openstack-ansible10:30
*** tosky_ is now known as tosky10:38
noonedeadpunkdmsimard: I dunno how safe it is to update upper-constraints though.. I think lots of stuff relies on django10:41
noonedeadpunkjrosser: I didn;t have a chance to look into that, but I'd prefer fixing this10:43
jrossernoonedeadpunk: i've just rebased my patch to move stretch->buster so we should work on that10:59
jrosserbut from memory there were some problems that were quite difficult11:00
openstackgerritMerged openstack/openstack-ansible master: Imported Translations from Zanata  https://review.opendev.org/69685611:01
noonedeadpunkActually I don't fully understand why tempest failed on stretch just on single test...11:04
noonedeadpunkBut anyway I guess it's good to move to buster11:04
*** DanyC has quit IRC11:21
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/train: Install galera client for placement migration  https://review.opendev.org/69686511:22
*** DanyC has joined #openstack-ansible11:22
*** ivve has quit IRC11:26
*** ansmith has quit IRC11:43
*** nicolasbock has joined #openstack-ansible12:07
admin0hi all .. how do I troubleshoot\fix garbage in the console12:11
noonedeadpunkNot really sure I got what you mean...12:13
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Make log socket available to chrooted filesystem  https://review.opendev.org/69673812:18
*** ivve has joined #openstack-ansible12:19
jrossernoonedeadpunk: oh right i remember whats wrong on the debian buster jobs, https://e79e37680ebfee3a2bcd-315b60fae67ecb2da54e93ac8154187b.ssl.cf5.rackcdn.com/681777/16/check/openstack-ansible-deploy-aio_metal-debian-stable/204dbfe/logs/openstack/aio1-utility/tempest_run.log.txt.gz12:32
jrossersomething to do with neutron is just broken12:32
jrosserand also LXC<>apparmor is not happy12:32
*** ansmith has joined #openstack-ansible12:33
noonedeadpunkand stretch fails on the same test but because of nova https://6014179e6be7e2acc289-7fb9678ffa261b44cc17a96761baa1dd.ssl.cf5.rackcdn.com/696798/1/check/openstack-ansible-deploy-aio_metal-debian-stable/9a8a4ae/logs/openstack/aio1-utility/stestr_results.html12:34
*** udesale has quit IRC12:36
*** udesale has joined #openstack-ansible12:37
jrossernoonedeadpunk: yes and if you look in the nova compute log got stretch you'll see "Nova requires libvirt version 4.0.0 or greater"12:37
noonedeadpunkyep...12:38
noonedeadpunkso we probably can't support strretch in U as I don't think they will backport libvirt https://packages.debian.org/search?searchon=sourcenames&keywords=libvirt12:40
noonedeadpunkRegarding neutron: "Policy DROP not allowed for user defined chains" https://e79e37680ebfee3a2bcd-315b60fae67ecb2da54e93ac8154187b.ssl.cf5.rackcdn.com/681777/16/check/openstack-ansible-deploy-aio_metal-debian-stable/204dbfe/logs/host/neutron-linuxbridge-agent.service.journal-11-29-05.log.txt.gz12:41
noonedeadpunkMaybe we should try this https://ask.openstack.org/en/question/120060/neutron-failing-to-deploy-with-policy-drop-not-allowed-for-user-defined-chains/12:43
noonedeadpunkjust not sure at what role this should be done... Neutron or somewhere during setup_hosts...12:44
noonedeadpunkWill try to reproduce in sandbox12:45
admin0noonedeadpunk, when i open console in openstack .. the output is garbaled .. cannot read .. when i type something, i can see what i type .. but the output is like garbage12:55
mgariepyhttps://review.opendev.org/#/c/696720/ any core avalaible to push this one ?12:59
mgariepysmall backport for 18.0412:59
*** nicolasbock1 has joined #openstack-ansible12:59
*** nicolasbock has quit IRC13:01
*** nicolasbock1 is now known as nicolasbock13:01
jrosserzigo: did you have to do anything for this https://ask.openstack.org/en/question/120060/neutron-failing-to-deploy-with-policy-drop-not-allowed-for-user-defined-chains/13:10
jrossermgariepy: done13:13
mgariepythanks jrosser13:13
*** dave-mccowan has joined #openstack-ansible13:15
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Make log socket available to chrooted filesystem  https://review.opendev.org/69673813:17
noonedeadpunkjrosser: I was wondering if removal of this empty dir is going to work as well?13:18
noonedeadpunkOr it's created during mount?13:18
jrosserit's a bit confusing13:18
jrosseri have a centos AIO here and it made /var/lib/haproxy/dev/ dir13:19
jrosserbut that seems missing on the zuul run13:19
*** dave-mccowan has quit IRC13:21
jrosserah no i've reproduced the zuul centos failure now13:21
noonedeadpunkIt was on patch1 isn't it?13:21
noonedeadpunkhttps://zuul.opendev.org/t/openstack/build/645bb5edaef1464f9592abbf199fcb92/log/job-output.txt#331313:21
jrosserthats made a directory i think13:22
jrosser /var/lib/haproxy/dev/log/13:22
jrosserwhen the ansible mount module runs, that looks very much like a bug13:23
noonedeadpunkAnd it's just for centos, which makes situation even more weird13:26
jrosseryeah, there is something odd going on with centos there13:29
*** nicolasbock has quit IRC13:46
*** jroll has quit IRC13:57
*** schwicht has joined #openstack-ansible13:59
*** jroll has joined #openstack-ansible13:59
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-haproxy_server master: Make log socket available to chrooted filesystem  https://review.opendev.org/69673814:03
openstackgerritMerged openstack/openstack-ansible-galera_client master: Install mariadb-client for mysqldump command  https://review.opendev.org/69539614:07
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-galera_client stable/train: Install mariadb-client for mysqldump command  https://review.opendev.org/69688614:07
*** nicolasbock has joined #openstack-ansible14:07
*** goldenfri has joined #openstack-ansible14:18
jrossernoonedeadpunk: this works on centos now https://review.opendev.org/#/c/696738/414:28
jrosserwe can merge as-is or do a [DNM] patch to double check on the integrated repo that the logs actually appear?14:28
noonedeadpunkI think we can merge it and update master bump to check this afterwards14:33
*** gixx has joined #openstack-ansible14:34
*** fyx has joined #openstack-ansible14:35
*** fyx has quit IRC14:36
*** fyx_ has joined #openstack-ansible14:36
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Don't require provider_networks with no_containers  https://review.opendev.org/69671314:38
*** fyx_ is now known as fyx14:41
cjloadermorning all15:03
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-galera_client stable/train: Install mariadb-client for mysqldump command  https://review.opendev.org/69688615:06
noonedeadpunko/15:06
cjloadernoonedeadpunk: making changes, should have it updated shortly fyi15:10
dmsimardnoonedeadpunk: django is in upper-constraints probably because of horizon ? unless you install horizon in the ansible venv I'm not sure it would be a problem15:11
noonedeadpunkupper-constraints is smth under releases team governance. I guess it's at least horizon and it's dashboards, but maybe smth else...15:19
dmsimarddjango v3 just got released today, 2.2 has been the LTS for a good amount of time already.. hopefully they can bump the constraints :p15:22
*** ansmith has quit IRC15:22
*** ansmith has joined #openstack-ansible15:23
*** cshen has quit IRC15:23
cjloadernoonedeadpunk: i'm confused with what you mean by making the db/service_users like the ironic_service_list15:30
*** df_sbr has joined #openstack-ansible15:30
cjloaderlike this? http://paste.openstack.org/show/786987/15:31
noonedeadpunkdmsimard: maybe we can offer this at least...15:31
zigojrosser: No, not me... However, there's the issue that Neutron doesn't play well with iptables-nft.15:31
zigoI've learned this the hard way !15:31
jrosserwe get "Dec 02 11:23:28 aio1 neutron-linuxbridge-agent[13228]: 2019-12-02 11:23:28.805 13228 ERROR neutron.agent.linux.utils [req-768e19a4-a930-4e34-a4f7-d33c67fcd89b - - - - -] Exit code: 255; Stdin: ; Stdout: ; Stderr: Policy DROP not allowed for user defined chains."15:32
jrosserthough if you use openvswitch then i think you don't see that problem15:33
noonedeadpunkcjloader: almost, but you can use {%   set _ = db_list.extend(ironic_db) %}15:33
cjloaderthen define ironic_db in defaults/main.yml?15:34
noonedeadpunkor {%   set _ = db_list.append({'username': {{ ironic_inspector_galera_user }}, 'password': {{ ironic_inspector_container_mysql_password }} }) %}15:36
cjloaderok15:37
*** rgogunskiy has quit IRC15:37
cjloadersounds good15:37
cjloadermakes sense15:38
noonedeadpunksry {%   set _ = db_list.append({'username': ironic_inspector_galera_user, 'password': ironic_inspector_container_mysql_password })15:38
noonedeadpunkyou don't need "{{ .. }}" inside jinja stuff15:38
cjloaderand we just do that style for all 4 areas you suggested15:39
noonedeadpunkI think yes15:39
cjloaderok15:39
cjloaderi'll try and do that15:39
*** cshen has joined #openstack-ansible15:40
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use legacy ebtables on Debian  https://review.opendev.org/69690315:41
noonedeadpunkjrosser: maybe we need this ^ only for buster as for now ?15:41
jrossernoonedeadpunk: you mean it should be more specific than ansible_distribution == 'Debian' ?15:42
noonedeadpunkyeah15:42
jrosserbecasue i don't think that task will run on ubuntu15:43
noonedeadpunkbut it will probably fail for stretch...15:43
noonedeadpunkIt's failing anyway but still15:43
*** cshen has quit IRC15:45
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use legacy ebtables on Debian  https://review.opendev.org/69690315:47
jrossermaybe that - bit of a guess though15:47
noonedeadpunkhttp://paste.openstack.org/show/786988/15:48
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Move debian jobs to buster  https://review.opendev.org/68177715:49
noonedeadpunkI think we can use is version15:49
jrosseransible_distribution_version": "buster/sid"15:49
noonedeadpunkhmm15:49
jrosseri have a VM here just looking for the best var to use15:49
noonedeadpunkIt's from aio vm as well....15:50
noonedeadpunkfor me it looks like http://paste.openstack.org/show/786990/15:51
jrosseroh dear https://zuul.opendev.org/t/openstack/build/990f49f0cf4a435cab1b86fa0c8d305a/log/job-output.txt#6674-668015:51
noonedeadpunkyeah15:51
noonedeadpunklets use ansible_distribution_major_version?:)15:52
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use legacy ebtables on Debian  https://review.opendev.org/69690315:52
jrosseri think i need to delete my buster image and get a newer one http://paste.openstack.org/show/786991/15:53
noonedeadpunkthat's really confusing....15:53
noonedeadpunkwhat do you have in /etc/os-release ? Just out of interest15:54
jrosserhttp://paste.openstack.org/show/786992/15:55
noonedeadpunkit looks like this for me http://paste.openstack.org/show/786993/15:56
*** macz has joined #openstack-ansible15:58
noonedeadpunkjrosser: So it looks like that we need to backport buster support to train, as it will be release for migration16:11
noonedeadpunk*upgrade16:11
jrosseryes, i've had the buster patch around for a long time, ideally this would have been merged already16:12
cjloadernoonedeadpunk: like this http://paste.openstack.org/show/786996/?16:13
noonedeadpunklooks good. hope it works :p16:14
cjloaderlet me do similar to the service users/endpoints/catalog16:14
cjloaderand then i'll push up16:15
cjloaderi'd like to get this in today if we can lol16:15
*** chkumar|ruck is now known as raukadah16:17
*** ivve has quit IRC16:26
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Use legacy ebtables on Debian  https://review.opendev.org/69690316:27
*** gyee has joined #openstack-ansible16:29
*** lbragsta_ has joined #openstack-ansible16:36
*** gyee has quit IRC16:36
*** rpittau is now known as rpittau|afk16:38
*** udesale has quit IRC16:41
*** rgogunskiy has joined #openstack-ansible16:50
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028416:50
*** miloa has quit IRC16:50
cjloadernoonedeadpunk: updated changes ^16:51
*** gyee has joined #openstack-ansible16:51
*** trident has quit IRC16:51
*** trident has joined #openstack-ansible16:53
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028416:53
*** rgogunskiy has quit IRC16:55
*** sshnaidm is now known as sshnaidm|afk17:13
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova master: Readd some QEMU distro packages  https://review.opendev.org/69693217:15
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova master: Readd some distro packages  https://review.opendev.org/69693217:16
noonedeadpunkcjloader: let it pass CI first:)17:16
*** KeithMnemonic has joined #openstack-ansible17:18
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028417:23
*** macz has quit IRC17:32
cjloadermirror failed | W: Failed to fetch http://mirror.regionone.fortnebula.opendev.org/ubuntu/dists/bionic/InRelease  Could not connect to mirror.regionone.fortnebula.opendev.org:8017:42
jrosseryes i see that in a different job too17:44
*** lbragsta_ has quit IRC17:44
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028417:48
*** DanyC_ has joined #openstack-ansible17:51
*** macz has joined #openstack-ansible17:52
*** DanyC_ has quit IRC17:54
*** DanyC has quit IRC17:54
*** Darcidride has joined #openstack-ansible18:07
*** Darcidride has quit IRC18:07
*** ivve has joined #openstack-ansible18:11
*** macz has quit IRC18:23
*** spsurya has quit IRC18:25
*** macz has joined #openstack-ansible18:27
*** macz has joined #openstack-ansible18:28
*** jtagcat has joined #openstack-ansible18:38
jrossernoonedeadpunk: looks like buster/metal passes https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_e39/681777/17/check/openstack-ansible-deploy-aio_metal-debian-stable/e39b26e/18:39
*** nicolasbock has quit IRC18:44
*** nicolasbock has joined #openstack-ansible18:44
*** macz has quit IRC19:01
*** Daemoen has joined #openstack-ansible19:30
*** macz has joined #openstack-ansible19:33
*** rgogunskiy has joined #openstack-ansible19:38
*** rgogunskiy has quit IRC19:43
*** donnyd_ has joined #openstack-ansible19:54
*** fyx_ has joined #openstack-ansible19:54
*** jrosser_ has joined #openstack-ansible19:55
*** persia_ has joined #openstack-ansible19:56
*** tinwood_ has joined #openstack-ansible19:56
*** mugsie_ has joined #openstack-ansible19:56
*** macz has quit IRC20:02
*** fyx has quit IRC20:02
*** pcaruana has quit IRC20:02
*** Jeffrey4l_ has quit IRC20:02
*** tinwood has quit IRC20:02
*** persia has quit IRC20:02
*** mugsie has quit IRC20:02
*** johanssone has quit IRC20:02
*** jrosser has quit IRC20:02
*** donnyd has quit IRC20:02
*** fyx_ is now known as fyx20:02
*** donnyd_ is now known as donnyd20:02
*** jrosser_ is now known as jrosser20:02
*** Jeffrey4l has joined #openstack-ansible20:02
*** johanssone has joined #openstack-ansible20:03
*** pcaruana has joined #openstack-ansible20:05
*** gyee has quit IRC20:09
*** DanyC has joined #openstack-ansible20:09
*** sc has joined #openstack-ansible20:15
*** sc is now known as Guest6278620:16
*** macz has joined #openstack-ansible20:31
*** gyee has joined #openstack-ansible20:36
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028420:59
*** rgogunskiy has joined #openstack-ansible21:15
*** ansmith has quit IRC21:22
*** mugsie_ is now known as mugsie21:59
openstackgerritCam J. Loader (cjloader) proposed openstack/openstack-ansible-os_ironic master: Adds configuration for the Ironic Inspector  https://review.opendev.org/68028422:05
*** gshippey has quit IRC22:10
*** pcaruana has quit IRC22:11
openstackgerritTom Jose Kalapura proposed openstack/openstack-ansible-os_mistral stable/stein: Add meta to mistral  https://review.opendev.org/69698122:12
*** jbadiapa has quit IRC22:14
*** schwicht has quit IRC22:25
*** DanyC has quit IRC22:38
*** rgogunskiy has quit IRC22:44
*** Adri2000 has quit IRC23:12
openstackgerritMerged openstack/openstack-ansible-ceph_client master: Importing keyrings from files rather than from mons  https://review.opendev.org/67176223:34
*** tosky has quit IRC23:41
*** nicolasbock has quit IRC23:52

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!