Friday, 2020-09-04

*** k_mouza has joined #openstack-kolla00:09
*** k_mouza has quit IRC00:14
*** zhanglong has joined #openstack-kolla00:26
*** zhanglong has quit IRC01:06
*** zhanglong has joined #openstack-kolla01:08
*** zzzeek has quit IRC03:17
*** zzzeek has joined #openstack-kolla03:18
*** JamesBenson has quit IRC03:48
*** skramaja has joined #openstack-kolla03:57
*** zhanglong has quit IRC04:02
*** brinzhang_ has quit IRC04:03
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-kolla04:33
*** dave-mccowan has quit IRC04:42
*** shyamb has joined #openstack-kolla04:52
*** shyamb has quit IRC05:14
*** shyamb has joined #openstack-kolla05:21
*** shyam89 has joined #openstack-kolla05:24
oklhostMorning05:24
*** zhanglong has joined #openstack-kolla05:25
*** shyamb has quit IRC05:27
*** zhanglong has quit IRC05:32
*** hjensas has joined #openstack-kolla05:32
*** zhanglong has joined #openstack-kolla05:34
*** brinzhang_ has joined #openstack-kolla05:34
*** shyam89 has quit IRC05:49
*** stingrayza has joined #openstack-kolla06:05
*** vishalmanchanda has joined #openstack-kolla06:06
*** also_stingrayza has quit IRC06:06
*** zhanglong has quit IRC06:07
*** zhanglong has joined #openstack-kolla06:13
*** cah_link has joined #openstack-kolla06:17
*** gfidente has joined #openstack-kolla06:20
*** shyamb has joined #openstack-kolla06:35
yoctozeptomorning06:58
*** bengates has joined #openstack-kolla07:02
*** bengates has quit IRC07:05
*** bengates has joined #openstack-kolla07:06
*** shyamb has quit IRC07:10
openstackgerritRadosław Piliszek proposed openstack/kolla stable/train: Use upstream Erlang on CentOS 8  https://review.opendev.org/73747807:13
*** shyamb has joined #openstack-kolla07:14
*** shyamb has quit IRC07:16
*** shyamb has joined #openstack-kolla07:16
*** brinzhang0 has joined #openstack-kolla07:16
*** shyamb has quit IRC07:18
*** shyamb has joined #openstack-kolla07:19
*** brinzhang_ has quit IRC07:20
openstackgerritRadosław Piliszek proposed openstack/kolla stable/rocky: Fix ironic build issues - LANG related  https://review.opendev.org/74988107:22
mnasiadkamorning07:24
mnasiadkayoctozepto: so where are we, all green? :)07:25
*** shyamb has quit IRC07:29
yoctozeptomnasiadka: yup, patching rocky now07:35
yoctozeptomnasiadka: the k-a bifrost jobs are non-voting and red07:35
yoctozeptomnasiadka: and bifrost is unbuildable07:35
yoctozeptobut it's really up to the upstream07:35
mnasiadkayoctozepto: great, thanks for driving it forward - I think it will end up that bifrost project will EOL bifrost/stein and earlier, looking at the mail thread ;-)07:36
openstackgerritRadosław Piliszek proposed openstack/kolla stable/rocky: Add /var/lib/kolla/venv/bin to Zun exec_dirs  https://review.opendev.org/73719807:37
yoctozeptomnasiadka: yeah, that's essentialy why I was saying the above07:38
*** mgoddard has joined #openstack-kolla07:38
yoctozeptousers need to keeping using the old one07:38
yoctozeptokeep*07:38
yoctozeptoand bikolla needs to get prioritised to avoid this hell in future07:39
*** dougsz has joined #openstack-kolla07:39
yoctozeptostein and ussuri queues for kolla are clean07:39
yoctozeptotrain is being merged07:39
yoctozeptowell, gated07:39
yoctozeptomnasiadka: https://review.opendev.org/747696 - to clean train queue too07:40
patchbotpatch 747696 - kolla (stable/train) - Add glusterfs-fuse package to manila-share container - 2 patch sets07:40
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/ussuri: Fix external mariadb documentation for database_user  https://review.opendev.org/74988807:41
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/train: Fix external mariadb documentation for database_user  https://review.opendev.org/74988907:41
openstackgerritMark Goddard proposed openstack/kolla-ansible stable/stein: Fix external mariadb documentation for database_user  https://review.opendev.org/74989007:41
*** zzzeek has quit IRC07:45
*** zzzeek has joined #openstack-kolla07:47
*** shyamb has joined #openstack-kolla07:47
*** shyamb has quit IRC07:48
*** shyamb has joined #openstack-kolla07:48
*** zhanglong has quit IRC07:49
*** shyam89 has joined #openstack-kolla07:49
*** zhanglong has joined #openstack-kolla07:53
*** shyamb has quit IRC07:53
*** jonaspaulo has joined #openstack-kolla07:59
*** e0ne has joined #openstack-kolla08:00
openstackgerritMerged openstack/kolla-ansible stable/stein: Fix external mariadb documentation for database_user  https://review.opendev.org/74989008:03
openstackgerritMerged openstack/kolla-ansible stable/train: Fix external mariadb documentation for database_user  https://review.opendev.org/74988908:05
*** bengates has quit IRC08:09
*** bengates has joined #openstack-kolla08:09
mnasiadkayoctozepto: what's the thing with 'kolla_internal_address' in bifrost CI?08:11
openstackgerritMerged openstack/kolla-ansible stable/ussuri: Fix external mariadb documentation for database_user  https://review.opendev.org/74988808:13
mgoddardmnasiadka: https://github.com/openstack/kolla-ansible/commit/b4603d927152f95848e56f6cf3ed4acd88baa377 fixed it on master08:16
mnasiadkamgoddard: then should we backport it to stein/rocky? :)08:17
mgoddardmnasiadka: can do. Not the deprecation but the default bit08:17
yoctozeptobut was not that a cosmetic thing though?08:22
yoctozeptobifrost is actively failing on it08:22
mnasiadkaso let's actively backport the default bit :)08:26
mnasiadkaI can look into this later today, now I'm a paper tiger08:26
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/ussuri: Add default value for kolla_internal_address variable  https://review.opendev.org/74989908:28
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/ussuri: Add default value for kolla_internal_address variable  https://review.opendev.org/74989908:28
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/ussuri: Add default value for kolla_internal_address variable  https://review.opendev.org/74989908:29
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/train: Add default value for kolla_internal_address variable  https://review.opendev.org/74990008:29
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/stein: Add default value for kolla_internal_address variable  https://review.opendev.org/74990108:29
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible stable/rocky: Add default value for kolla_internal_address variable  https://review.opendev.org/74990208:30
yoctozeptoit's08:37
yoctozeptoTASK [Starting bifrost deploy container] ***************************************08:37
yoctozeptotask path: /home/zuul/src/opendev.org/openstack/kolla-ansible/ansible/roles/bifrost/tasks/start.yml:208:37
yoctozeptofatal: [primary]: FAILED! => {08:37
yoctozepto    "msg": "The task includes an option with an undefined variable. The error was: 'kolla_internal_address' is undefined\n\nThe error appears to be in '/home/zuul/src/opendev.org/openstack/kolla-ansible/ansible/roles/bifrost/tasks/start.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending08:37
yoctozeptoline appears to be:\n\n---\n- name: Starting bifrost deploy container\n  ^ here\n"08:37
yoctozepto}08:37
yoctozeptoso it looks like it's trying to evaluate it for one of bifrost tasks08:37
yoctozeptoI wonder how it ended up there, let me see now08:37
mnasiadkaso what are we doing with this neutron bug crap?08:38
*** shyamb has joined #openstack-kolla08:40
*** shyam89 has quit IRC08:42
mgoddardyoctozepto, mnasiadka: reference to kolla_internal_address crept in when backporting bifrost proxy support08:51
mnasiadkamgoddard: mhm, then let's fix what we broke08:55
mnasiadkamgoddard: got a minute to look into https://review.opendev.org/#/c/748161/ ?08:55
patchbotpatch 748161 - kayobe - Move dnf.yml earlier in host configure workflow - 4 patch sets08:55
yoctozeptomgoddard, mnasiadka: true that, setting api address to empty string will just hide this bug08:55
yoctozeptoas the no_proxy syntax could well be wrong08:55
mnasiadkaoh boy :)_08:56
mgoddardmnasiadka: maybe later. client work all day08:56
mnasiadkamgoddard: sure08:56
yoctozeptomnasiadka: looking into it or me should fix it?08:57
*** zhanglong has quit IRC08:58
mnasiadkayoctozepto: if you have cycles, I only did a dumb backport, but it seems we should be smarter :)08:58
yoctozeptomnasiadka: I'll handle it08:59
*** zhanglong has joined #openstack-kolla08:59
mnasiadkahmm, ubuntu-source kolla master build failed in periodic - https://zuul.openstack.org/build/1453a98126734f3e8cef0cf2d26a4a78/log/kolla/build/000_FAILED_bifrost-base.log#1107009:00
mnasiadkacentos8-source the same09:01
mnasiadkahttps://zuul.openstack.org/build/27d53247643a4f8da313fb978c328de3/log/kolla/build/000_FAILED_bifrost-base.log#1509509:01
mnasiadkahttps://github.com/openstack/bifrost/commit/710e0db06885b13b398d6fe209893cc732a43d4109:02
mnasiadkaoh boy09:02
mnasiadkamaybe we should add some kolla build bifrost job in bifrost CI?09:02
mnasiadkaBecause those issues seem to multiply...09:03
yoctozeptomnasiadka: lol, we already disabled in stein and rocky09:04
mgoddarddtantsur is making some big changes to bifrost at the moment to strengthen the standalone ironic case09:05
yoctozeptomnasiadka: disable everywhere09:05
yoctozeptomgoddard: hopefully it's no longer scary to use afterwards09:05
mgoddardmaybe09:05
mnasiadkamgoddard: that is fantastic, but it seems those big changes are not really fit to work with our build :)09:05
yoctozeptomnasiadka: lesson learnt - don't do w+1 on non-trivial changes before they actually get tested09:06
*** brinzhang has joined #openstack-kolla09:06
yoctozeptomnasiadka: hah09:06
mnasiadkasystemd reload won't work in a container, come on :)09:06
*** zhanglong has quit IRC09:06
mnasiadkaespecially during docker build :)09:06
*** zhanglong has joined #openstack-kolla09:08
*** brinzhang0 has quit IRC09:09
*** zhanglong has quit IRC09:15
*** zhanglong has joined #openstack-kolla09:16
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Make no_proxy handling more robust  https://review.opendev.org/74990809:17
*** dougsz has quit IRC09:22
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Revert the default for kolla_internal_vip_address  https://review.opendev.org/74991009:23
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Make no_proxy handling more robust  https://review.opendev.org/74990809:23
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] test bifrost  https://review.opendev.org/74991109:25
yoctozeptomnasiadka: that should do it ^ let it test though09:26
*** dougsz has joined #openstack-kolla09:38
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] test bifrost  https://review.opendev.org/74991109:42
*** zhanglong has quit IRC09:46
*** zhanglong has joined #openstack-kolla09:48
*** shyamb has quit IRC09:54
*** shyamb has joined #openstack-kolla10:01
*** zhanglong has quit IRC10:13
*** dougsz has quit IRC10:14
yoctozeptomgoddard, mnasiadka: ok, we might want to do both in fact10:19
yoctozeptomgoddard, mnasiadka: default+more_robust due to how ansible evaluations work10:19
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: Make no_proxy handling more robust  https://review.opendev.org/74990810:20
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] test bifrost  https://review.opendev.org/74991110:20
mgoddardyoctozepto: belt+braces10:21
yoctozeptomgoddard: exactly :D10:21
mgoddardmnasiadka, yoctozepto, hrw, osmanlicilegi: I am on holiday next wednesday-friday10:22
yoctozeptomgoddard: have fun, it's only kolla meeting on wed then10:22
yoctozeptowe can skip it as well10:22
mgoddardyep. If someone is willing to chair, that would be appreciated. If not it can be skipped10:22
yoctozeptoI will probably be able to10:23
yoctozeptoit's sad to drive the monologue though10:23
mgoddardthanks. Can always be kept short10:23
mgoddardindeed10:23
yoctozeptoindeed10:23
mgoddardindeed10:23
yoctozepto:D10:23
mgoddardyou could just do the CI checklist, then ask everyone to spend the remaining 45 minutes reviewing a priority victoria patch10:24
mgoddardor do some group reviews10:25
mgoddardwhatever is efficient10:25
yoctozeptomgoddard: yup, I'm planning to spend even more time on those prios; especially octavia10:25
mgoddard+1 octavia10:25
mnasiadkaI'll try to update the kolla_docker heathchecks change, so maybe we could focus on pushing that through before freeze ;)10:25
yoctozeptomgoddard, mnasiadka: what are we doing about neutron on master?10:26
mgoddardhaven't been following10:26
yoctozeptomnasiadka: that would be cool10:26
mnasiadkayoctozepto: pin to commit id?10:26
mnasiadkaand wait for neutron stadium projects to fix it?10:26
yoctozeptomnasiadka: perhaps? or finally stop bootstrapping vpnaas?10:26
mnasiadkapinning is easy10:26
mnasiadkaI can work on fixing bootstrap10:26
mnasiadkanext week10:27
yoctozeptohmm, I'd prefer this week10:27
yoctozeptoI'll schedule it on me10:27
yoctozeptohonestly tired of breakages this cycle10:27
mnasiadkawell, temporarily we can disable vpnaas bootstrap on master10:27
mnasiadkaand bgpvpn10:27
yoctozeptoyup, it's only master that is affected10:27
yoctozeptodo we bagpipe though?10:27
yoctozeptothought not10:27
yoctozeptobbl10:28
openstackgerritMerged openstack/kolla stable/train: Upgrade Fluentd Monasca plugin  https://review.opendev.org/74392210:29
*** dougsz has joined #openstack-kolla10:30
mnasiadkayoctozepto: I don't know, ideally we should run bootstrap per enabled service :)10:32
openstackgerritMerged openstack/kolla stable/train: Add glusterfs-fuse package to manila-share container  https://review.opendev.org/74769610:40
*** kevko has joined #openstack-kolla11:06
openstackgerritMerged openstack/kolla stable/train: Use upstream Erlang on CentOS 8  https://review.opendev.org/73747811:06
openstackgerritMerged openstack/kolla stable/rocky: Fix ironic build issues - LANG related  https://review.opendev.org/74988111:06
openstackgerritMerged openstack/kolla stable/rocky: Add /var/lib/kolla/venv/bin to Zun exec_dirs  https://review.opendev.org/73719811:06
osmanlicilegimgoddard: enjoy your holiday11:17
*** zzzeek has quit IRC11:20
mgoddardthanks osmanlicilegi11:20
*** zzzeek has joined #openstack-kolla11:21
*** dave-mccowan has joined #openstack-kolla12:00
*** nikparasyr has joined #openstack-kolla12:15
*** shyamb has quit IRC12:24
*** skramaja has quit IRC13:04
*** cah_link has quit IRC13:22
openstackgerritPierre Riteau proposed openstack/kayobe master: Document that kolla_tag may need to be changed when upgrading  https://review.opendev.org/74995413:23
*** brinzhang_ has joined #openstack-kolla13:24
*** brinzhang has quit IRC13:28
*** vishalmanchanda has quit IRC13:29
kevkohi guys, is master kolla-ansible tested ? i'm failing on cluster-check for mariadb13:32
kevkohttp://paste.openstack.org/show/797476/13:32
kevkoclean testing deployment13:33
*** dave-mccowan has quit IRC14:08
*** zzzeek has quit IRC14:11
*** zzzeek has joined #openstack-kolla14:12
kplantyou shouldn't deploy from the master branch if you want a working deployment :-)14:17
*** dave-mccowan has joined #openstack-kolla14:25
*** priteau has joined #openstack-kolla14:43
mnasiadkakevko: well, if it's not synced - then sorry ;)14:45
mnasiadkakevko: it runs correctly in CI14:45
*** sluna has quit IRC14:52
*** sluna has joined #openstack-kolla14:54
*** sluna has quit IRC14:54
*** brinzhang0 has joined #openstack-kolla14:54
*** zzzeek has quit IRC14:56
*** brinzhang_ has quit IRC14:58
*** zzzeek has joined #openstack-kolla14:59
*** rpittau is now known as rpittau|afk15:18
kevkokplant: well, i'm writing feature for review ..so ..i really need to have master :)15:40
kplantwelp15:42
kplantthat makes too much sense15:42
kplant(-:15:42
openstackgerritMark Goddard proposed openstack/kolla master: Fix Masakari dashboard policy file location  https://review.opendev.org/74999015:44
kevkokplant: i've already destroyed infra ..but building new one15:49
kevkokplant: what is ENV in CI, deploy fresh OS , or apply changes to already built one ?15:49
kevkokplant: because i had feeling that it was because of haproxy user denied15:50
*** nikparasyr has left #openstack-kolla16:11
yoctozeptokevko: have you actually checked mariadb logs?16:25
kevkoyoctozepto: nope, but was trying telnet to 3306 ..and it worked16:26
yoctozeptokevko: that's not enough to pass clustercheck :-)16:27
yoctozeptocheck if galera (wsrep) is happy16:27
kevkoyoctozepto: well, what i've done was , that i checked script cluster-check16:28
kevkoyoctozepto: logged to ERR_LOG ..i got permission denied for user haproxy16:29
kevkoyoctozepto: will know more in few minutes ..now ceph is deploying16:29
yoctozeptokevko: ok; btw, what are you working on? :-)16:30
kevkoyoctozepto: custom_logging_conf - user will be able to define own logging.conf, instead of reading logs from file ..use native python3-fluent-logger to send to fluent16:32
*** dougsz has quit IRC16:33
kevkoyoctozepto: more clear, native, nice format , and also WITH os context16:33
yoctozeptomgoddard, mnasiadka, hrw: tripleo Victoria officially kolla-less16:33
kevkoyoctozepto: we have it deployed already in production on stein, but i want to put into master ..16:34
*** bengates has quit IRC16:34
*** bengates has joined #openstack-kolla16:35
*** gfidente is now known as gfidente|afk16:37
yoctozeptokevko: ack, you mean from openstack services directly to fluentd?16:38
yoctozeptokevko: i.e. skipping text file and its parsing via fluentd16:38
kevkoyoctozepto: yes16:38
*** bengates has quit IRC16:40
*** dosaboy has joined #openstack-kolla16:49
hrwyoctozepto: thanks for info. where is a patch to +2?17:02
*** kevko has quit IRC17:04
yoctozeptohrw: none yet, but can do; you mean dropping jobs, or more?17:25
hrwjobs for start17:25
*** lxkong has quit IRC17:33
openstackgerritRadosław Piliszek proposed openstack/kolla master: [CI] Drop the TripleO job  https://review.opendev.org/75000417:44
yoctozeptohrw: ^17:44
*** zzzeek has quit IRC18:19
*** zzzeek has joined #openstack-kolla18:20
openstackgerritMerged openstack/kolla master: [CI] Drop the TripleO job  https://review.opendev.org/75000418:24
yoctozeptohrw, mnasiadka, mgoddard: aaand it's gone18:41
kplantdid ooo dump kolla?18:41
yoctozeptoon one hand it is sad to see tripleo go, but on the bright side - less gate blockers ;d18:42
yoctozeptokplant: indeed18:42
kplant:-(18:42
yoctozeptothe way they used it, I imagine it could have been pain18:42
yoctozeptoas they literally override every aspect18:42
yoctozeptorepos, packages18:43
*** dave-mccowan has quit IRC18:46
*** priteau has quit IRC19:00
*** jonaspaulo has quit IRC19:17
*** cz3 is now known as cz3|hs20:05
*** cz3|hs is now known as cz320:05
*** zzzeek has quit IRC20:06
*** zzzeek has joined #openstack-kolla20:10
*** qqmber has joined #openstack-kolla20:14
qqmberhi.. If I want to install kolla-ansible with PIP, ans I want to install a specific version, I know I have to do it like this: "pip install kolla-ansible==7.2.2".. if I go to https://github.com/openstack/kolla-ansible/ and select "stable/rocky"... how can I know what version number is? exemaple: kolla-ansible stable/rocky == 7.2.220:16
yoctozeptoqqmber: if you e.g. run pip install on it, then it will show you some autogenerated version that's newer than any release from pypi, but that does not make much sense; when you work with git repos, it's best to learn basics of git; then you can navigate the commits based on their identifiers; some commits are tagged with versions (of releases)20:41
* yoctozepto going to sleep after busy day20:42
qqmberthanks!20:53
*** gfidente|afk has quit IRC21:01
*** hjensas has quit IRC22:10
*** dking has quit IRC22:36
*** zhanglong has joined #openstack-kolla23:44
*** zhanglong has quit IRC23:49
openstackgerritJames Kirsch proposed openstack/kolla master: Enable apache in Neutron server container  https://review.opendev.org/74960123:57

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!