| *** schwicht has quit IRC | 00:03 | |
| openstackgerrit | Merged openstack/kolla master: CI: Fix symlinks for failed build logs https://review.opendev.org/701911 | 00:07 |
|---|---|---|
| *** xinliang has joined #openstack-kolla | 00:33 | |
| *** threestrands has joined #openstack-kolla | 00:38 | |
| *** jistr has quit IRC | 00:45 | |
| *** jistr has joined #openstack-kolla | 00:48 | |
| openstackgerrit | Dincer Celik proposed openstack/kolla master: Fixes rally db commands https://review.opendev.org/702148 | 00:58 |
| *** jcmdln has quit IRC | 01:00 | |
| *** jcmdln has joined #openstack-kolla | 01:32 | |
| *** threestrands has quit IRC | 01:43 | |
| openstackgerrit | Dincer Celik proposed openstack/kolla master: Fixes rally db commands https://review.opendev.org/702148 | 01:54 |
| *** schwicht has joined #openstack-kolla | 01:59 | |
| *** schwicht has quit IRC | 02:06 | |
| *** threestrands has joined #openstack-kolla | 03:10 | |
| *** threestrands has quit IRC | 03:11 | |
| *** threestrands has joined #openstack-kolla | 03:11 | |
| *** xinliang has quit IRC | 04:04 | |
| *** skramaja has joined #openstack-kolla | 04:18 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-kolla | 05:34 | |
| *** threestrands has quit IRC | 05:38 | |
| *** zhanglong has joined #openstack-kolla | 06:00 | |
| *** zhanglong has quit IRC | 06:04 | |
| *** zhanglong has joined #openstack-kolla | 06:05 | |
| *** xinliang has joined #openstack-kolla | 06:16 | |
| *** pbing19 has joined #openstack-kolla | 06:27 | |
| *** Luzi has joined #openstack-kolla | 06:35 | |
| *** zhanglong has quit IRC | 06:36 | |
| *** zhanglong has joined #openstack-kolla | 06:44 | |
| *** zhanglong has quit IRC | 06:49 | |
| *** zhanglong has joined #openstack-kolla | 06:51 | |
| *** sri_ has joined #openstack-kolla | 06:55 | |
| sri_ | morning | 06:57 |
| *** pbing19 has quit IRC | 07:09 | |
| *** pbing19 has joined #openstack-kolla | 07:09 | |
| *** cah_link has joined #openstack-kolla | 07:11 | |
| *** zhanglong has quit IRC | 07:18 | |
| *** oncall-pokemon has quit IRC | 07:18 | |
| *** zhanglong has joined #openstack-kolla | 07:21 | |
| *** iniazi_ has joined #openstack-kolla | 07:28 | |
| *** iniazi has quit IRC | 07:30 | |
| egonzalez | morning | 07:39 |
| osmanlicilegi | morning | 07:42 |
| *** rgogunskiy has joined #openstack-kolla | 07:59 | |
| *** schwicht has joined #openstack-kolla | 08:02 | |
| hrw | elo | 08:06 |
| *** schwicht has quit IRC | 08:06 | |
| mnasiadka | morning | 08:15 |
| *** bjoernh has joined #openstack-kolla | 08:16 | |
| mnasiadka | osmanlicilegi: there is no need to recheck https://review.opendev.org/#/c/701904/ - it is there for yoctozepto to understand the failure :D | 08:16 |
| patchbot | patch 701904 - kolla-ansible - Fix fernet-node-sync error catching - 3 patch sets | 08:16 |
| hrw | something for morning coffee? https://review.opendev.org/701986 https://review.opendev.org/701990 https://review.opendev.org/702003 https://review.opendev.org/694104 | 08:20 |
| patchbot | patch 701986 - kolla - Use Debian mirror in CI jobs - 1 patch set | 08:20 |
| patchbot | patch 701990 - kolla - base: use install_packages() macro to install pack... - 2 patch sets | 08:20 |
| patchbot | patch 702003 - kolla - base: drop commented part about site-packages in U... - 1 patch set | 08:20 |
| patchbot | patch 694104 - kolla - Ubuntu: use local APT mirrors if possible - 4 patch sets | 08:20 |
| hrw | all are trivial, easy to review between sips of coffee | 08:20 |
| openstackgerrit | LEDUC Florian proposed openstack/kolla-ansible master: Add ability to use the neutron logging framework https://review.opendev.org/700895 | 08:21 |
| yoctozepto | morning | 08:30 |
| yoctozepto | mnasiadka, osmanlicilegi: yeah, no need :-) | 08:30 |
| hrw | mnasiadka: tx | 08:32 |
| sri_ | yoctozepto, sorry I missed your question on Friday, I've manually copied the "ceph.client.cinder.keyring" to /etc/kolla/nova-compute | 08:33 |
| *** rpittau|afk is now known as rpittau | 08:35 | |
| yoctozepto | sri_: hmm, should not be necessary with all your config | 08:35 |
| sri_ | yoctozepto, yea, It was copied in stain release | 08:35 |
| sri_ | yoctozepto, for some reason it's not copied in train :( | 08:36 |
| yoctozepto | sri_: mnasiadka glad to fix that ^^ | 08:37 |
| hrw | 14.2.5-3~bpo10+1 100 | 08:37 |
| hrw | 100 https://deb.debian.org/debian buster-backports/main amd64 Packages | 08:37 |
| hrw | ceph nautilus in Debian stable ;D | 08:37 |
| *** zbr is now known as zbr|rover | 08:37 | |
| osmanlicilegi | yoctozepto: https://review.opendev.org/#/c/701010/ looks good to me, worked well on my dev. I would like to test it on my prod environments too to be sure. this week I have control layer scaling work on tuesday and wednesday. wish luck to me. | 08:41 |
| patchbot | patch 701010 - kolla-ansible - Fix multiple issues with MariaDB handling - 17 patch sets | 08:41 |
| hrw | osmanlicilegi: wishing you luck I am | 08:41 |
| yoctozepto | osmanlicilegi: /me wishing luck too | 08:42 |
| yoctozepto | it really makes the process stabler | 08:43 |
| yoctozepto | I tried to break it locally but did not succeed | 08:43 |
| yoctozepto | (where on the old one the failure rate was high) | 08:43 |
| yoctozepto | I am more intrigued about that keystone | 08:44 |
| yoctozepto | time out on single-node job | 08:44 |
| yoctozepto | it worked and stopped | 08:44 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Ubuntu: use local APT mirrors if possible https://review.opendev.org/694104 | 08:45 |
| hrw | yoctozepto: releasenote added | 08:45 |
| *** gfidente has joined #openstack-kolla | 08:45 | |
| yoctozepto | hrw: thx | 08:46 |
| mnasiadka | sri_: is there a bug raised? | 08:48 |
| sri_ | mnasiadka, No | 08:49 |
| mnasiadka | sri_: please raise, describe the problem and I'm glad to fix that :) | 08:49 |
| sri_ | mnasiadka, sure, thanks :) | 08:49 |
| yoctozepto | mnasiadka: https://review.opendev.org/701010 | 08:52 |
| patchbot | patch 701010 - kolla-ansible - Fix multiple issues with MariaDB handling - 17 patch sets | 08:52 |
| yoctozepto | interesting case here | 08:52 |
| yoctozepto | fernet killed by running reconfigure | 08:52 |
| yoctozepto | at first I erred that it was singlenode but it is multinode | 08:53 |
| yoctozepto | and fernet kills off one keystone at least during reconfig | 08:53 |
| mnasiadka | yoctozepto: on centos as well, or only debian/ubuntu? :D | 08:54 |
| *** pbing19 has quit IRC | 08:55 | |
| *** tonythomas has joined #openstack-kolla | 08:59 | |
| *** bjoernh has quit IRC | 09:00 | |
| yoctozepto | mnasiadka: centos because it works (and hence breaks) | 09:03 |
| yoctozepto | after deeper debugging it all starts to make sense, mnasiadka ;-) | 09:03 |
| yoctozepto | work to fail, fail to work | 09:03 |
| mnasiadka | haha | 09:03 |
| mnasiadka | yoctozepto: you mean the TIMED_OUT centos upgrade job, or which one? | 09:04 |
| *** dougsz has joined #openstack-kolla | 09:04 | |
| yoctozepto | mnasiadka: FAILED cells | 09:04 |
| mnasiadka | yoctozepto: well, it failed on waiting for string MariaDB? :D | 09:05 |
| yoctozepto | mnasiadka: then TIMED_OUT, sorry, I have no idea what I'm clicking on | 09:06 |
| yoctozepto | lemme check again | 09:06 |
| yoctozepto | mnasiadka: no, cells | 09:07 |
| yoctozepto | Gateway Timeout (HTTP 504) | 09:07 |
| mnasiadka | yoctozepto: naah, you're right - I just looked into deploy and looked for fatal: :D | 09:07 |
| yoctozepto | don't trick me into mariadb issues | 09:07 |
| yoctozepto | gah | 09:07 |
| yoctozepto | ;p | 09:07 |
| mnasiadka | yoctozepto: well, the only errors in keystone logs are lost mysql connection during query :D | 09:08 |
| yoctozepto | mnasiadka: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bfd/701010/17/check/kolla-ansible-centos-source-cells/bfd76d3/primary/logs/kolla/keystone/keystone-apache-admin-error.txt | 09:09 |
| mnasiadka | script timed out before returning headers? come on, this happens all the time | 09:09 |
| yoctozepto | though this is after reconfigure touched fernet | 09:10 |
| yoctozepto | it worked until then | 09:10 |
| yoctozepto | just food for thought | 09:10 |
| mnasiadka | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_bfd/701010/17/check/kolla-ansible-centos-source-cells/bfd76d3/primary/logs/kolla/keystone/keystone.txt | 09:11 |
| mnasiadka | that's more interesting, those conflict errors :D | 09:11 |
| yoctozepto | mnasiadka: yeah, especially: | 09:12 |
| yoctozepto | Duplicate entry 'admin-<<null>>' for key 'ixu_role_name_domain_id' | 09:12 |
| yoctozepto | lolz | 09:12 |
| yoctozepto | as if reconfigure touched some bootstrap | 09:13 |
| yoctozepto | well, it being = deploy nowadays might have done such thing | 09:13 |
| yoctozepto | could be not fernet exactly | 09:14 |
| yoctozepto | point to monitor as well | 09:14 |
| mnasiadka | yoctozepto: well, one option is to merge docker healthchecks (I need to spend some minutes on release notes and other comments there), because now there's really nothing in the logs :) | 09:20 |
| yoctozepto | yeah, the logs are baddie bad | 09:21 |
| mnasiadka | it's failing on creating keystone services, which would point to those weird duplicate errors | 09:22 |
| mnasiadka | but the log says the endpoints where updated... argh | 09:23 |
| *** k_mouza has joined #openstack-kolla | 09:24 | |
| mnasiadka | yoctozepto: I wouldn't mind if we would have timestamps in the ansible/* logs... | 09:27 |
| mnasiadka | at least maybe we should again generate html from ara sqlite, so we would get it from there - it's hard to corellate logs to ansible log : | 09:29 |
| mnasiadka | :) | 09:29 |
| mnasiadka | yoctozepto: I have 3 +2s but no +w - https://review.opendev.org/#/c/701944/ :D | 09:30 |
| patchbot | patch 701944 - kolla-ansible - Use distro_python_version in fernet-node-sync - 2 patch sets | 09:30 |
| osmanlicilegi | mnasiadka: you got it | 09:30 |
| *** xinliang has quit IRC | 09:32 | |
| mnasiadka | osmanlicilegi: thanks | 09:40 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible stable/train: Use distro_python_version in fernet-node-sync https://review.opendev.org/702177 | 09:40 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Adapt fetch-fernet-tokens script to Python 3 https://review.opendev.org/701779 | 09:42 |
| *** xinliang has joined #openstack-kolla | 09:43 | |
| mgoddard | morning | 09:44 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Adapt fetch-fernet-tokens script to Python 3 https://review.opendev.org/701779 | 09:44 |
| *** pcaruana has joined #openstack-kolla | 09:49 | |
| *** pcaruana has quit IRC | 09:56 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Fix fernet-node-sync error catching https://review.opendev.org/701904 | 09:57 |
| stingrayza | hi all o/ | 10:00 |
| stingrayza | been a while :D | 10:00 |
| hrw | hi stingrayza | 10:01 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/train: CI: Fix symlinks for failed build logs https://review.opendev.org/702178 | 10:02 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/stein: CI: Fix symlinks for failed build logs https://review.opendev.org/702179 | 10:02 |
| mgoddard | hi stingrayza | 10:03 |
| *** zhanglong has quit IRC | 10:04 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/rocky: CI: Fix symlinks for failed build logs https://review.opendev.org/702180 | 10:04 |
| hrw | later today I will add patch to move us to buster-backports image. so newer ceph will be used and we will have nautilus in all distros | 10:12 |
| *** xinliang has quit IRC | 10:12 | |
| stingrayza | we just hit an issue adding a new node to our cluster. (when running prechecks with --limit for the new node) we're getting errors at "prechecks : Checking empty passwords in passwords.yml. Run kolla-genpwd if this task fails". Its moaning about some of the passwords.yml values being empty - but they're exactly as generated by a 'fresh' kolla-genpwd. any ideas? http://paste.openstack.org/show/788309/ | 10:14 |
| openstackgerrit | Merged openstack/kolla master: Use Debian mirror in CI jobs https://review.opendev.org/701986 | 10:15 |
| openstackgerrit | Merged openstack/kolla master: CI: use Debian/Ubuntu mirrors in base image https://review.opendev.org/701987 | 10:15 |
| openstackgerrit | Merged openstack/kolla master: base: drop commented part about site-packages in Ubuntu https://review.opendev.org/702003 | 10:15 |
| *** rgogunskiy has quit IRC | 10:16 | |
| sri_ | mnasiadka, https://bugs.launchpad.net/kolla-ansible/+bug/1859408 | 10:17 |
| openstack | Launchpad bug 1859408 in kolla-ansible "when we enable external ceph in train release "ceph.client.cinder.keyring" is not copied to /etc/kolla/nova-compute" [Undecided,New] | 10:17 |
| stingrayza | also fails (but on a different compute node) if dropping the --limit | 10:17 |
| *** rgogunskiy has joined #openstack-kolla | 10:18 | |
| *** rgogunskiy has quit IRC | 10:26 | |
| *** rgogunskiy has joined #openstack-kolla | 10:27 | |
| mgoddard | stingrayza: the task ignores ssh keys: result.stdout | regex_replace('(.*ssh_key.*)', '') is search(":") | 10:30 |
| *** rgogunskiy has quit IRC | 10:33 | |
| *** rgogunskiy has joined #openstack-kolla | 10:34 | |
| *** pbing19 has joined #openstack-kolla | 10:40 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Adapt fetch-fernet-tokens script to Python 3 https://review.opendev.org/701779 | 10:41 |
| mnasiadka | sri_: yeah, I see that - thanks | 10:42 |
| mnasiadka | question is does it affect stein and rocky as well | 10:43 |
| mnasiadka | but it won't hurt to backport it there as well | 10:43 |
| stingrayza | mgoddard: ah, doh. we put an extra struct in there with some of our internal passwords and it must be borking on that (but throwing all of the ssh_keys as additional errors just to confuse things ;) ) | 10:44 |
| sri_ | mnasiadka, No it's only affects in train | 10:44 |
| stingrayza | works perfectly if I clear that extra struct, thanks! | 10:44 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Some ansible-lint fixes https://review.opendev.org/694797 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: remove comparison with true or false https://review.opendev.org/702185 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: whitespace https://review.opendev.org/702186 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: task names https://review.opendev.org/702187 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: set -o pipefail when shell pipes used https://review.opendev.org/702188 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: version constraints for pip and docker https://review.opendev.org/702189 | 10:45 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Ansible lint: use command module instead of shell https://review.opendev.org/702190 | 10:45 |
| mgoddard | hrw: I split out some less contentious parts | 10:45 |
| stingrayza | (incidentally, uncovered a potentially unwanted issue where if you redirect your passwords.yml with the --passwords parameter, that particular check still validates the 'plain old' passwords.yml - I'll submit a patch for that... | 10:46 |
| mnasiadka | mgoddard: I was planning on picking that up later, but thanks for doing that :-) | 10:47 |
| mgoddard | mnasiadka: still some bits left in the original patch for you :) | 10:47 |
| mnasiadka | mgoddard: phew, thanks! | 10:48 |
| openstackgerrit | Merged openstack/kolla master: Fixes rally db commands https://review.opendev.org/702148 | 10:57 |
| openstackgerrit | Dincer Celik proposed openstack/kolla stable/train: Fixes rally db commands https://review.opendev.org/702192 | 11:00 |
| openstackgerrit | Dincer Celik proposed openstack/kolla stable/stein: Fixes rally db commands https://review.opendev.org/702193 | 11:00 |
| *** rpittau is now known as rpittau|bbl | 11:03 | |
| *** pbing19 has quit IRC | 11:04 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: CentOS 8: Don't force tag in build jobs https://review.opendev.org/702196 | 11:06 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: WIP: move to CentOS 8 in containers https://review.opendev.org/692368 | 11:08 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: External Ceph: copy also cinder keyring to nova services https://review.opendev.org/702198 | 11:25 |
| *** k_mouza_ has joined #openstack-kolla | 11:26 | |
| *** k_mouza has quit IRC | 11:26 | |
| hrw | mgoddard: ok. thanks | 11:27 |
| *** gfidente has quit IRC | 11:27 | |
| *** gfidente has joined #openstack-kolla | 11:28 | |
| *** shyamb has joined #openstack-kolla | 11:31 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Use distro_python_version in fernet-node-sync https://review.opendev.org/701944 | 11:36 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Debian: switch to 'buster-backports' by default https://review.opendev.org/702201 | 11:46 |
| hrw | Ceph nautilus :F | 11:46 |
| *** shyamb has quit IRC | 11:49 | |
| mnasiadka | hmm, ERROR: Package 'setuptools' requires a different Python: 2.7.17 not in '>=3.5' - in train | 11:51 |
| mnasiadka | do we need to pin setuptools? | 11:51 |
| mnasiadka | https://zuul.opendev.org/t/openstack/build/f06317a74ad847349d5f317c0e500571/console | 11:52 |
| *** zhanglong has joined #openstack-kolla | 11:56 | |
| hrw | mnasiadka: give links to build log not console | 11:56 |
| mnasiadka | hrw: I would be glad, but setup_gate.sh doesn't log to a file, and it's only visible on the console | 11:57 |
| hrw | k | 11:58 |
| mnasiadka | unless job-output.json is something you like to read :D | 11:58 |
| hrw | mnasiadka: which distro? | 11:58 |
| hrw | and can it be reproduced in local builds? | 11:59 |
| hrw | mnasiadka: once build finish == no console so give link to build. | 12:00 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: CentOS 8: Don't force tag in build jobs https://review.opendev.org/702196 | 12:01 |
| mnasiadka | hrw: on Ubuntu | 12:02 |
| hrw | mnasiadka: show me that json... | 12:04 |
| mnasiadka | hrw: you can see the error here: https://zuul.opendev.org/t/openstack/build/f06317a74ad847349d5f317c0e500571 - just wait for the Results to populate on the bottom :) | 12:05 |
| hrw | thanks | 12:05 |
| mnasiadka | hrw: or here: https://zuul.opendev.org/t/openstack/build/f06317a74ad847349d5f317c0e500571/log/job-output.json#11247 | 12:06 |
| hrw | mnasiadka: looks like requirements needs patch. | 12:07 |
| mnasiadka | hrw: that's what I'm thinking as well... | 12:07 |
| hrw | 44.0.0 was last py2 | 12:09 |
| hrw | or sth | 12:09 |
| mnasiadka | yeah, seems so - https://github.com/pypa/setuptools/commit/6449c5b745b2e664b05a4ac88ed8776d6d4f2a42 | 12:10 |
| mnasiadka | just after this they bumped to 45.0.0 | 12:10 |
| hrw | they restored py2 tests later | 12:11 |
| hrw | https://github.com/pypa/setuptools/commit/7bac1ed1f82199e968bd025d631d00a00f3f46b3 | 12:11 |
| mnasiadka | tests yes, but setup.cfg is still missing python2 :) | 12:13 |
| hrw | time for patch then... | 12:13 |
| mnasiadka | yup | 12:13 |
| hrw | 13:13 (0s) hrw@puchatek:~$ cd devel/linaro/openstack/requirements/ | 12:13 |
| hrw | 13:13 (0s) hrw@puchatek:requirements$ git up | 12:13 |
| hrw | but irc channel first as it gets tricky | 12:14 |
| *** zzzeek has quit IRC | 12:16 | |
| *** zzzeek has joined #openstack-kolla | 12:17 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Adapt fetch-fernet-tokens script to Python 3 https://review.opendev.org/701779 | 12:18 |
| *** kplant has joined #openstack-kolla | 12:24 | |
| hrw | 13:24 < openstackgerrit> Marcin Juszkiewicz proposed openstack/requirements master: setuptools 45.0.0 is Python 3.5+ https://review.opendev.org/702210 | 12:24 |
| patchbot | patch 702210 - requirements - setuptools 45.0.0 is Python 3.5+ - 1 patch set | 12:24 |
| hrw | mnasiadka: add that as Depends-on and check? | 12:28 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible stable/train: Use distro_python_version in fernet-node-sync https://review.opendev.org/702177 | 12:29 |
| mnasiadka | hrw: just did ^^ | 12:30 |
| hrw | mnasiadka: ah. train.. | 12:30 |
| mnasiadka | yeah, we don't use python2 in Ussuri :) | 12:30 |
| mnasiadka | but I guess it should merge the depends-on without problem | 12:31 |
| hrw | mnasiadka: last year I waited for 2020 to forget about py2. now it looks like I need to wait to 2022... | 12:33 |
| hrw | so we EOL train | 12:33 |
| yoctozepto | guys, the issue seems related to infra incomplete metadata on wheel index | 12:33 |
| mnasiadka | hrw: you can always move to Java project :D | 12:33 |
| yoctozepto | so unless that gets fixed soon | 12:33 |
| yoctozepto | we might want to disable using infra mirrors for now | 12:34 |
| hrw | mnasiadka: my machine has just 32GB ram ;( | 12:34 |
| yoctozepto | for pypi | 12:34 |
| yoctozepto | mgoddard, mnasiadka, hrw: ^ | 12:35 |
| hrw | yoctozepto: thanks. nice discussion on github. | 12:36 |
| yoctozepto | nice mess on reviews as well | 12:37 |
| yoctozepto | devstack, devstack-gate, project-config, zuul-jobs | 12:37 |
| yoctozepto | choose and match | 12:37 |
| hrw | yay | 12:37 |
| hrw | in other words: we can ignore it | 12:37 |
| hrw | it will get sorted out | 12:37 |
| *** schwicht has joined #openstack-kolla | 12:38 | |
| yoctozepto | hrw: yeah, though no idea when | 12:38 |
| yoctozepto | infra folks seem not around atm | 12:38 |
| hrw | in meantime I will play with Linaro Developer Cloud | 12:38 |
| yoctozepto | anyways, here is a little patch that passes: https://review.opendev.org/702034 | 12:38 |
| patchbot | patch 702034 - kolla-ansible - Fix Zun Docker runtime selection - 1 patch set | 12:39 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible stable/train: Use distro_python_version in fernet-node-sync https://review.opendev.org/702177 | 12:39 |
| yoctozepto | so you might want to w+1 | 12:39 |
| yoctozepto | this now | 12:39 |
| hrw | yoctozepto: ianw is in Australia, fungi iirc is US etc. | 12:39 |
| yoctozepto | yeah, ianw busy today ;/ | 12:39 |
| hrw | yoctozepto: done | 12:39 |
| yoctozepto | hrw: thx | 12:40 |
| hrw | uf. first thing to do in horizon: switch from Polish to any English. | 12:40 |
| yoctozepto | any English | 12:41 |
| hrw | I like the fact that OS and lot of stuff runs in Polish. but there are places where I prefer to stay away from names invented by people | 12:42 |
| yoctozepto | they tend to be great fun | 12:42 |
| hrw | 'Wpisz się' ('Sign in') looks far too bad for me | 12:42 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Fix fernet-node-sync error catching https://review.opendev.org/701904 | 12:43 |
| hrw | yoctozepto: I did some software translation in past. never again. | 12:43 |
| hrw | there are moments when I fix translation in services/apps but only moments | 12:43 |
| yoctozepto | hrw: "Sign out" as "wypisz się?" | 12:44 |
| yoctozepto | that would be misleading | 12:44 |
| hrw | yoctozepto: probably | 12:44 |
| hrw | yoctozepto: no. 'Wyloguj' | 12:45 |
| hrw | @$@#$! | 12:45 |
| hrw | "Modyfikacja ustawień tablicy dla użytkownika." | 12:45 |
| yoctozepto | DLA | 12:48 |
| yoctozepto | DLJA | 12:48 |
| mnasiadka | yoctozepto: mr release note master - don't we need a release note in that case - https://review.opendev.org/#/c/699312 ? ;-) | 12:49 |
| patchbot | patch 699312 - kolla-ansible - Configure services to use Certificate Authority - 10 patch sets | 12:49 |
| hrw | yoctozepto: yeah, translated by someone from Podlasie. clearly | 12:49 |
| yoctozepto | sounds like it | 12:49 |
| yoctozepto | though Mazovia usually adopts this errors as well | 12:49 |
| yoctozepto | and Lubelskie and Podkarpacie | 12:49 |
| hrw | yoctozepto: it's popular rusycyzm | 12:50 |
| hrw | took me a while to get rid of it | 12:50 |
| yoctozepto | yeah, it is indeed | 12:50 |
| yoctozepto | don't overdo do it | 12:50 |
| yoctozepto | I saw ppl trying to avoid "dla" in its perfectly fine usages | 12:50 |
| hrw | and there are several good jokes about it | 12:51 |
| yoctozepto | yeah, though I don't remember one atm | 12:52 |
| *** schwicht has quit IRC | 12:54 | |
| hrw | yoctozepto: https://baxu.pl/storage/posts/July2019/80F7E146-E0BA-4DCD-A580-30CE3B655B47.jpeg has nice text | 12:59 |
| *** skramaja has quit IRC | 13:00 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: Adapt fetch-fernet-tokens script to Python 3 https://review.opendev.org/701779 | 13:08 |
| yoctozepto | hrw: hahaha | 13:10 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: WIP: move to CentOS 8 in containers https://review.opendev.org/692368 | 13:16 |
| *** zhanglong has quit IRC | 13:22 | |
| yoctozepto | hrw: https://review.opendev.org/702166 - could you confirm my suspicions? | 13:26 |
| patchbot | patch 702166 - zuul/zuul-jobs - Disable wheel mirror - 1 patch set | 13:26 |
| yoctozepto | (mind blown) | 13:26 |
| *** rpittau|bbl is now known as rpittau | 13:32 | |
| mnasiadka | yoctozepto: mind blown or not, but now all non master changes are failing on this - judging by the traffic in my mailbox :) | 13:38 |
| yoctozepto | mnasiadka: I thought this through | 13:38 |
| yoctozepto | there is so much RED that I am losing my mind like a bull ;p | 13:39 |
| kplant | ole | 13:44 |
| mnasiadka | yoctozepto: not only you :) | 13:45 |
| mnasiadka | yoctozepto: so you want to say the requirements patch is the way to go? :D | 13:45 |
| yoctozepto | mnasiadka: no, the requirements will not pass | 13:45 |
| yoctozepto | the issue is in the mirror | 13:45 |
| yoctozepto | the workaround is to disable its usage | 13:45 |
| mnasiadka | or wait until it gets fixed | 13:49 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/train: WIP: Try fixing wheel mirror issue https://review.opendev.org/702220 | 13:50 |
| yoctozepto | mnasiadka: well, the current pending 'fix' is just the global workaround | 13:51 |
| yoctozepto | ;-) | 13:51 |
| yoctozepto | unless someone finds where and implements metadata generation properly | 13:51 |
| *** shyamb has joined #openstack-kolla | 13:58 | |
| *** KeithMnemonic has joined #openstack-kolla | 13:59 | |
| *** vmixor has joined #openstack-kolla | 14:04 | |
| *** spiette has joined #openstack-kolla | 14:09 | |
| yoctozepto | https://review.opendev.org/702220 seems to have helped | 14:09 |
| patchbot | patch 702220 - kolla-ansible (stable/train) - WIP: Try fixing wheel mirror issue - 1 patch set | 14:09 |
| yoctozepto | it got past the failure | 14:09 |
| *** Luzi has quit IRC | 14:12 | |
| yoctozepto | mnasiadka: https://review.opendev.org/702185 | 14:13 |
| patchbot | patch 702185 - kolla-ansible - Ansible lint: remove comparison with true or false - 1 patch set | 14:13 |
| yoctozepto | and https://review.opendev.org/#/c/702189/ | 14:13 |
| patchbot | patch 702189 - kolla-ansible - Ansible lint: version constraints for pip and docker - 1 patch set | 14:13 |
| yoctozepto | and off it goes | 14:13 |
| yoctozepto | all linter issues fixed | 14:13 |
| yoctozepto | (or at least from these patches) | 14:13 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix Zun Docker runtime selection https://review.opendev.org/702034 | 14:14 |
| *** spiette has quit IRC | 14:16 | |
| *** spiette has joined #openstack-kolla | 14:19 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/train: Fix Zun Docker runtime selection https://review.opendev.org/702225 | 14:24 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/stein: Fix Zun Docker runtime selection https://review.opendev.org/702226 | 14:26 |
| hrw | yoctozepto: soon you will join infra. | 14:31 |
| yoctozepto | hrw: fungi already proposed that before | 14:32 |
| hrw | ;D | 14:32 |
| yoctozepto | I run my infra so probably that's why I sympathize with openstack infra guys | 14:33 |
| *** pcaruana has joined #openstack-kolla | 14:34 | |
| hrw | I only complain about infra | 14:35 |
| *** schwicht has joined #openstack-kolla | 14:36 | |
| *** TrevorV has joined #openstack-kolla | 14:45 | |
| *** shyamb has quit IRC | 15:06 | |
| *** sri_ has quit IRC | 15:06 | |
| *** shyamb has joined #openstack-kolla | 15:09 | |
| *** pbing19 has joined #openstack-kolla | 15:15 | |
| *** rgogunskiy has quit IRC | 15:21 | |
| kplant | mnasiadka: i feel your pain with ceph-ansible | 15:37 |
| kplant | every commit they're just silently removing shit :-( | 15:37 |
| kplant | they don't copy keyrings to the fetch directory anymore | 15:37 |
| mnasiadka | kplant: yeah, they go to /etc/ceph | 15:38 |
| kplant | i feel like in stable-3.2 everything you could possibly want was in fetch/, now in stable-4.0 it's quite useless | 15:39 |
| *** shyamb has quit IRC | 15:39 | |
| kplant | /rage | 15:39 |
| *** damiandabrowski2 has joined #openstack-kolla | 15:44 | |
| *** dave-mccowan has joined #openstack-kolla | 15:46 | |
| damiandabrowski2 | Hello, In last few months I've had several problems with kolla ubuntu images and I'm wondering if centos images have better support and they're more stable or both of distributions have same support level? | 15:48 |
| *** dave-mccowan has quit IRC | 15:52 | |
| *** iniazi_ is now known as iniazi | 16:01 | |
| *** kplant has quit IRC | 16:01 | |
| *** kplant has joined #openstack-kolla | 16:06 | |
| *** pcaruana has quit IRC | 16:12 | |
| *** kplant has quit IRC | 16:23 | |
| *** kplant has joined #openstack-kolla | 16:28 | |
| osmanlicilegi | damiandabrowski2: if you are using binary images, try source images a chance. | 16:33 |
| osmanlicilegi | damiandabrowski2: source images install related services from source tarballs so you will be always up-to-date with the upstream. | 16:37 |
| mgoddard | damiandabrowski2: I think most of the core team use CentOS, although we try to keep ubuntu working as well | 16:45 |
| *** icarusfactor has quit IRC | 16:55 | |
| *** rpittau is now known as rpittau|afk | 17:02 | |
| damiandabrowski2 | ok thanks for the answers! I wanted to build everything on ubuntu because of much newer kernel(5.0 vs. 3.10) when comparing ubuntu 18.04 to centos 7. I think it may be faster in some cases(because of the modules responsible for network communication, kvm etc.). Do You think I'm right or both distros should be just as fast? | 17:13 |
| *** pcaruana has joined #openstack-kolla | 17:13 | |
| mgoddard | damiandabrowski2: it's possible that you are right - newer kernels may have some optimisations. RH often backports things it cares about though | 17:14 |
| damiandabrowski2 | ok I get it, thanks for the explanation! | 17:16 |
| *** rlljorge has joined #openstack-kolla | 17:28 | |
| rlljorge | hello, I have a problem when execute prechecks | 17:29 |
| rlljorge | TASK [nova-cell : Checking free port for Nova SSHfatal: [compute01]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ 'migration' | kolla_address }}'. Error was a <class 'kolla_ansible.exception.FilterError'>, original message: Interface 'eth0' not present on host 'compute01'"} | 17:29 |
| rlljorge | but my inventory file define the another interface for this host: | 17:30 |
| rlljorge | [compute] | 17:30 |
| *** evrardjp has quit IRC | 17:33 | |
| *** evrardjp has joined #openstack-kolla | 17:34 | |
| *** rlljorge has left #openstack-kolla | 17:35 | |
| *** rlljorge has joined #openstack-kolla | 17:35 | |
| mnasiadka | damiandabrowski2: usually UCA lacks a bit behind point releases of services, since they first go through testing repo, and then get released - there was a table with version comparison somewhere | 17:36 |
| *** primeministerp has quit IRC | 17:37 | |
| mnasiadka | damiandabrowski2: here are the version trackers: https://wiki.ubuntu.com/OpenStack/VersionTracking | 17:37 |
| *** primeministerp has joined #openstack-kolla | 17:37 | |
| *** k_mouza_ has quit IRC | 17:41 | |
| mnasiadka | yoctozepto, mgoddard: can I get some final reviews on https://review.opendev.org/701779 and https://review.opendev.org/701904? At least we would have the same broken-ish state of fernet on both CentOS and Ubuntu/Debian as a base for fixing it further :) | 17:49 |
| patchbot | patch 701779 - kolla - Adapt fetch-fernet-tokens script to Python 3 - 7 patch sets | 17:49 |
| patchbot | patch 701904 - kolla-ansible - Fix fernet-node-sync error catching - 5 patch sets | 17:49 |
| openstackgerrit | Dincer Celik proposed openstack/kolla-ansible master: Enable Glance to use iscsi backend https://review.opendev.org/699910 | 17:54 |
| *** gfidente is now known as gfidente|afk | 17:55 | |
| *** k_mouza has joined #openstack-kolla | 18:17 | |
| rlljorge | someone Can help-me ? | 18:19 |
| kplant | we can try | 18:19 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 18:19 |
| rlljorge | kpalnt: Thanks !!! I have a problem when execute prechecks ... | 18:20 |
| rlljorge | TASK [nova-cell : Checking free port for Nova SSHfatal: [compute01]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ 'migration' | kolla_address }}'. Error was a <class 'kolla_ansible.exception.FilterError'>, original message: Interface 'eth0' not present on host 'compute01'"} | 18:20 |
| rlljorge | But I defined in inventory file "compute01 network_interface=bond0.614" | 18:21 |
| *** k_mouza has quit IRC | 18:22 | |
| kplant | interesting | 18:23 |
| kplant | what about api_interface | 18:23 |
| rlljorge | kplant: I will try now .... | 18:25 |
| *** pbing19 has quit IRC | 18:28 | |
| rlljorge | kplant: I changed the inventory file to | 18:29 |
| rlljorge | compute01 network_interface=bond0.614 api_interface=bond0.614 | 18:29 |
| rlljorge | same error | 18:29 |
| kplant | i wouldn't think globals.yaml would take precendence over inventory | 18:30 |
| kplant | have you tried setting it in host_vars ? | 18:30 |
| kplant | do you have anything *_interface set to eth0 in globals.yaml? | 18:31 |
| rlljorge | Yes I have defined network_interface and neutron_external_interface in globals.yaml | 18:32 |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Configure services to use Certificate Authority https://review.opendev.org/699312 | 18:36 |
| mnasiadka | rlljorge: do you have kolla-ansible installed using pip? | 18:36 |
| rlljorge | mnasiadka: yes | 18:37 |
| mnasiadka | rlljorge: so what's left is to run kolla-ansible with -vvv and paste the output to paste.openstack.org :) | 18:37 |
| rlljorge | ok | 18:38 |
| *** dougsz has quit IRC | 18:39 | |
| rlljorge | mnasiadka: http://paste.openstack.org/show/788343/ | 18:45 |
| mnasiadka | rlljorge: well, you didn't say you're running it with --limit | 18:47 |
| *** tonythomas has quit IRC | 18:48 | |
| rlljorge | I try --limit for reduce output ... I will be execute without --limit | 18:48 |
| mnasiadka | rlljorge: and I don't think it's the whole output - I'm missing the task where it fails | 18:48 |
| mnasiadka | generalfuzz: pep8 failing - https://2b3ebe3b1f3307d606a3-c862a7cee9789a90e28aadd42eae2397.ssl.cf2.rackcdn.com/699312/11/check/openstack-tox-pep8/bf4f23b/job-output.txt | 18:53 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Use Docker healthchecks for core services https://review.opendev.org/676389 | 18:54 |
| rlljorge | mnasiadka: I cannot paste all content ... i dont know why ... | 18:58 |
| mnasiadka | rlljorge: so open a bug on launchpad and attach the file there | 18:59 |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Configure services to use Certificate Authority https://review.opendev.org/699312 | 19:00 |
| rlljorge | mnasiadka: 1859512 | 19:07 |
| rlljorge | mnasiadka: https://paste.ubuntu.com/p/3hsNZFpTxZ/ | 19:37 |
| *** dougsz has joined #openstack-kolla | 19:37 | |
| osmanlicilegi | rlljorge: could you verify you can ssh into compute01 | 19:50 |
| rlljorge | [root@kolla-deploy ~]# ssh compute01 | 19:52 |
| osmanlicilegi | rlljorge: there are "Failed to connect to the host via ssh" in logs | 19:52 |
| rlljorge | osmanlicilegi: I can connect with compute01 | 19:52 |
| osmanlicilegi | rlljorge: grep compute01 /etc/hosts | 19:52 |
| rlljorge | 172.16.120.136 compute01 compute01.sianet.com.br | 19:52 |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Added support for custom logging.conf https://review.opendev.org/679812 | 19:56 |
| osmanlicilegi | rlljorge: can you verify 172.16.120.136 belongs to the nic which you mentioned for network_interface | 19:56 |
| rlljorge | osmanlicilegi: yes 172.16.120.136 ( compute01 ) has the interface bond0.614 | 19:59 |
| osmanlicilegi | rlljorge: so bond0.614 should be set as network_interface. could you verify? | 20:01 |
| openstackgerrit | Doug Szumski proposed openstack/kolla master: Upgrade Kafka to 2.0.1 release https://review.opendev.org/702287 | 20:02 |
| rlljorge | osmanlicilegi: "[compute] | 20:03 |
| rlljorge | in inventory file | 20:03 |
| rlljorge | "TASK [nova-cell : Checking free port for Nova SSH]" | 20:10 |
| rlljorge | "fatal: [compute01]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ 'migration' | kolla_address }}'. Error was a <class 'kolla_ansible.exception.FilterError'>, original message: Interface 'eth0' not present on host 'compute01'"}" | 20:10 |
| *** schwicht has quit IRC | 20:10 | |
| *** schwicht has joined #openstack-kolla | 20:12 | |
| osmanlicilegi | rlljorge: grep 'network_interface:' /etc/kolla/globals.yml | 20:17 |
| rlljorge | osmanlicilegi: network_interface: "eth0" | 20:17 |
| rlljorge | all another nodes except compute01 and compute02 use eth0 as network_interface | 20:18 |
| openstackgerrit | Viktor Michalek proposed openstack/kolla-ansible master: Added support for custom logging.conf https://review.opendev.org/679812 | 20:19 |
| *** schwicht has quit IRC | 20:22 | |
| yoctozepto | mgoddard, mnasiadka, hrw, osmanlicilegi: I realized we missed a few releasenotes, time to add a linter checking if the commit has a releasenote... ;D | 20:23 |
| osmanlicilegi | rlljorge: I think your problem is interface overriding. if you override interfaces, you need to comment out them in globals.yml | 20:24 |
| rlljorge | osmanlicilegi: Ok, I will comment out in globals.yml and define host by host in inventory | 20:24 |
| osmanlicilegi | rlljorge: https://opendev.org/openstack/kolla-ansible/src/branch/master/ansible/inventory/multinode#L24 | 20:25 |
| osmanlicilegi | yoctozepto: +1 it's easy to forget release notes | 20:25 |
| *** lile has joined #openstack-kolla | 20:30 | |
| *** lile has left #openstack-kolla | 20:31 | |
| osmanlicilegi | rlljorge: I'm marking https://bugs.launchpad.net/kolla-ansible/+bug/1859512 as incomplete for now. please update it when after you retry. | 20:34 |
| openstack | Launchpad bug 1859512 in kolla-ansible "precheck fail" [Undecided,Incomplete] | 20:34 |
| *** kplant has quit IRC | 20:38 | |
| rlljorge | osmanlicilegi: prechecks executed with success after comment out *interfaces in globals.yaml and define in inventory file. | 20:46 |
| osmanlicilegi | rlljorge: great! have fun with your openstack now :] | 20:49 |
| *** TrevorV has quit IRC | 20:59 | |
| *** watersj has joined #openstack-kolla | 21:05 | |
| *** schwicht has joined #openstack-kolla | 21:10 | |
| *** damiandabrowski2 has quit IRC | 21:12 | |
| openstackgerrit | Dincer Celik proposed openstack/kolla-ansible master: Enable Glance to use iscsi backend https://review.opendev.org/699910 | 21:20 |
| *** watersj has quit IRC | 21:22 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Use distro_python_version in fernet-node-sync https://review.opendev.org/702177 | 21:30 |
| openstackgerrit | Dincer Celik proposed openstack/kolla-ansible master: Adds Docker image pruning command https://review.opendev.org/698694 | 21:36 |
| *** vesper has joined #openstack-kolla | 22:00 | |
| *** vesper11 has quit IRC | 22:01 | |
| *** rlljorge has quit IRC | 22:02 | |
| *** schwicht has quit IRC | 22:05 | |
| *** schwicht has joined #openstack-kolla | 22:06 | |
| *** vmixor has quit IRC | 22:06 | |
| *** strigazi has quit IRC | 22:12 | |
| *** strigazi has joined #openstack-kolla | 22:13 | |
| *** dougsz has quit IRC | 22:24 | |
| *** schwicht has quit IRC | 23:02 | |
| *** cah_link has quit IRC | 23:04 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Copy CA into containers. https://review.opendev.org/699888 | 23:05 |
| *** schwicht has joined #openstack-kolla | 23:35 | |
| *** schwicht has quit IRC | 23:48 | |
| *** schwicht has joined #openstack-kolla | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
