| *** kevko has quit IRC | 00:15 | |
| openstackgerrit | Merged openstack/kolla master: use freezer user for freezer_scheduler container https://review.opendev.org/c/openstack/kolla/+/765681 | 00:25 |
|---|---|---|
| openstackgerrit | Merged openstack/kolla-ansible stable/ussuri: [docs] Add python3-venv package to quickstart.rst https://review.opendev.org/c/openstack/kolla-ansible/+/767147 | 00:26 |
| openstackgerrit | Merged openstack/kolla-ansible stable/victoria: [docs] Add python3-venv package to quickstart.rst https://review.opendev.org/c/openstack/kolla-ansible/+/767146 | 00:26 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/c/openstack/kolla-ansible/+/767157 | 00:26 |
| openstackgerrit | Ghanshyam proposed openstack/kolla-ansible master: Remove retired Qinling support https://review.opendev.org/c/openstack/kolla-ansible/+/764560 | 00:35 |
| openstackgerrit | Ghanshyam proposed openstack/kolla-ansible master: Remove retired Searchlight support https://review.opendev.org/c/openstack/kolla-ansible/+/764565 | 00:37 |
| gmann | mgoddard: yoctozepto ^^ rebased these two due to merge conflict | 00:39 |
| openstackgerrit | Merged openstack/kolla stable/train: Support CentOS 8.3 https://review.opendev.org/c/openstack/kolla/+/766410 | 00:41 |
| *** maharg101 has joined #openstack-kolla | 00:51 | |
| *** maharg101 has quit IRC | 00:56 | |
| *** LinPeiWen has joined #openstack-kolla | 01:04 | |
| *** wuchunyang has joined #openstack-kolla | 01:16 | |
| *** k_mouza has joined #openstack-kolla | 01:36 | |
| *** JamesBenson has quit IRC | 01:39 | |
| *** kevinz has joined #openstack-kolla | 01:39 | |
| *** k_mouza has quit IRC | 01:41 | |
| *** dasp has quit IRC | 02:20 | |
| *** dasp has joined #openstack-kolla | 02:21 | |
| *** Chaserjim has quit IRC | 02:31 | |
| *** benj_- has joined #openstack-kolla | 02:35 | |
| *** benj_ has quit IRC | 02:35 | |
| *** benj_- is now known as benj_ | 02:35 | |
| *** LinPeiWen has quit IRC | 02:45 | |
| *** iniazi_ has quit IRC | 02:52 | |
| *** iniazi has joined #openstack-kolla | 02:53 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Pin ansible in test-requirements https://review.opendev.org/c/openstack/kolla-ansible/+/766689 | 02:58 |
| *** mgoddard has quit IRC | 02:58 | |
| *** LinPeiWen has joined #openstack-kolla | 03:05 | |
| *** mchlumsky has quit IRC | 03:31 | |
| *** dswebb has quit IRC | 03:42 | |
| *** mchlumsky has joined #openstack-kolla | 03:43 | |
| *** wuchunyang has quit IRC | 04:04 | |
| *** JamesBenson has joined #openstack-kolla | 04:24 | |
| *** JamesBenson has quit IRC | 04:28 | |
| *** JamesBenson has joined #openstack-kolla | 04:29 | |
| *** shyamb has joined #openstack-kolla | 04:37 | |
| *** k_mouza has joined #openstack-kolla | 04:48 | |
| *** k_mouza has quit IRC | 04:53 | |
| *** wuchunyang has joined #openstack-kolla | 04:54 | |
| *** wuchunyang has quit IRC | 04:59 | |
| *** vishalmanchanda has joined #openstack-kolla | 05:01 | |
| *** shyamb has quit IRC | 05:05 | |
| *** shyamb has joined #openstack-kolla | 05:05 | |
| *** JamesBenson has quit IRC | 05:29 | |
| *** JamesBenson has joined #openstack-kolla | 05:30 | |
| *** JamesBenson has quit IRC | 05:30 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-kolla | 05:33 | |
| *** JamesBenson has joined #openstack-kolla | 05:35 | |
| *** zzzeek has quit IRC | 05:39 | |
| *** JamesBenson has quit IRC | 05:39 | |
| *** zzzeek has joined #openstack-kolla | 05:39 | |
| *** shyamb has quit IRC | 05:41 | |
| *** zzzeek has quit IRC | 05:46 | |
| *** zzzeek has joined #openstack-kolla | 05:48 | |
| *** wuchunyang has joined #openstack-kolla | 05:51 | |
| *** shyamb has joined #openstack-kolla | 05:58 | |
| *** ricolin has joined #openstack-kolla | 05:59 | |
| *** zzzeek has quit IRC | 06:02 | |
| *** zzzeek has joined #openstack-kolla | 06:06 | |
| *** sorin-mihai__ has joined #openstack-kolla | 06:08 | |
| *** numans has joined #openstack-kolla | 06:10 | |
| *** shyamb has quit IRC | 06:11 | |
| *** shyamb has joined #openstack-kolla | 06:21 | |
| *** bsanjeewa has joined #openstack-kolla | 06:26 | |
| *** cah_link has joined #openstack-kolla | 06:36 | |
| *** abdysn has joined #openstack-kolla | 06:46 | |
| *** bsanjeewa has quit IRC | 07:03 | |
| *** shyamb has quit IRC | 07:03 | |
| *** bsanjeewa has joined #openstack-kolla | 07:03 | |
| *** Abdallahyas has joined #openstack-kolla | 07:04 | |
| *** shyamb has joined #openstack-kolla | 07:08 | |
| *** abdysn has quit IRC | 07:09 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add ensure-python task https://review.opendev.org/c/openstack/kayobe/+/766984 | 07:22 |
| openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add ensure-python task https://review.opendev.org/c/openstack/kayobe/+/766984 | 07:25 |
| *** kevko has joined #openstack-kolla | 07:28 | |
| *** Abdallahyas has quit IRC | 07:34 | |
| *** Abdallahyas has joined #openstack-kolla | 07:34 | |
| *** zzzeek has quit IRC | 07:40 | |
| *** zzzeek has joined #openstack-kolla | 07:41 | |
| *** zzzeek has quit IRC | 07:53 | |
| *** zzzeek has joined #openstack-kolla | 07:55 | |
| *** bsanjeewa has quit IRC | 07:55 | |
| *** shyam89 has joined #openstack-kolla | 07:56 | |
| *** shyamb has quit IRC | 07:59 | |
| *** shyam89 has quit IRC | 08:00 | |
| *** andrewbonney has joined #openstack-kolla | 08:07 | |
| *** Abdallahyas has quit IRC | 08:10 | |
| *** abdysn has joined #openstack-kolla | 08:14 | |
| *** bengates has joined #openstack-kolla | 08:15 | |
| *** gfidente|afk is now known as gfidente | 08:18 | |
| *** rpittau|afk is now known as rpittau | 08:18 | |
| ug_egon | I'm having a hard time understanding my review. Could one of you explain this newbie? https://review.opendev.org/#/c/758361/1//COMMIT_MSG | 08:21 |
| ug_egon | I already tried responding to the reviewer and emailing but so far, i received no response. | 08:21 |
| ug_egon | Why is there "Closes-Bug: 1606143" should in commit message. | 08:21 |
| openstack | bug 1606143 in kolla-ansible "SSL termination in external loadbalancer" [Wishlist,New] https://launchpad.net/bugs/1606143 | 08:21 |
| ug_egon | while my commit starts with that... | 08:22 |
| ug_egon | and what is a git title in this context? | 08:22 |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/victoria: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767300 | 08:23 |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/ussuri: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767301 | 08:24 |
| yoctozepto | gmann: thanks | 08:25 |
| yoctozepto | morning | 08:25 |
| yoctozepto | most master images have built already | 08:25 |
| yoctozepto | centos8 source still building | 08:25 |
| yoctozepto | well, built AND got published :-) | 08:26 |
| *** zzzeek has quit IRC | 08:31 | |
| *** zzzeek has joined #openstack-kolla | 08:34 | |
| *** wuchunyang has quit IRC | 08:34 | |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/train: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767302 | 08:37 |
| *** wuchunyang has joined #openstack-kolla | 08:38 | |
| *** zzzeek has quit IRC | 08:39 | |
| *** zzzeek has joined #openstack-kolla | 08:41 | |
| *** maharg101 has joined #openstack-kolla | 08:41 | |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/train: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767302 | 08:42 |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/ussuri: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767301 | 08:42 |
| openstackgerrit | Bartosz Bezak proposed openstack/kolla-ansible stable/victoria: Monasca log-metrics - Drop "notice" and "note" loglevel metrics by default https://review.opendev.org/c/openstack/kolla-ansible/+/767300 | 08:43 |
| *** dougsz has joined #openstack-kolla | 08:44 | |
| mnasiadka | yoctozepto: incredible | 08:44 |
| *** mgoddard has joined #openstack-kolla | 08:46 | |
| yoctozepto | centos8 source published too | 08:48 |
| wuchunyang | ug_egon hi please check my new reply | 08:49 |
| *** dougsz has quit IRC | 08:49 | |
| yoctozepto | and Train unblocked yesterday | 08:49 |
| yoctozepto | the next step is to do a release for T & U | 08:50 |
| yoctozepto | mnasiadka: you know how to do it, propose it and let mgoddard approve them | 08:50 |
| mnasiadka | will do | 08:51 |
| yoctozepto | oh what a spam on the k-a gate hah | 08:51 |
| *** e0ne has joined #openstack-kolla | 08:54 | |
| mnasiadka | yup, my fault :D | 08:55 |
| mnasiadka | oh, bandit doesn't like kolla_docker tmpfs change | 08:57 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Fix stop containers task in Swift rolling restart https://review.opendev.org/c/openstack/kolla-ansible/+/764087 | 08:57 |
| *** zzzeek has quit IRC | 08:59 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Fixes for CentOS 8.3 https://review.opendev.org/c/openstack/kayobe/+/766203 | 08:59 |
| *** zzzeek has joined #openstack-kolla | 09:00 | |
| *** zzzeek has quit IRC | 09:05 | |
| *** mgoddard has quit IRC | 09:06 | |
| *** zzzeek has joined #openstack-kolla | 09:06 | |
| *** mgoddard has joined #openstack-kolla | 09:07 | |
| *** zzzeek has quit IRC | 09:11 | |
| *** zzzeek has joined #openstack-kolla | 09:14 | |
| ug_egon | wuchunyang Thanks that will work! | 09:16 |
| *** adrian-a has joined #openstack-kolla | 09:17 | |
| *** shyamb has joined #openstack-kolla | 09:17 | |
| mgoddard | morning | 09:19 |
| *** zzzeek has quit IRC | 09:23 | |
| *** abdysn has quit IRC | 09:25 | |
| *** zzzeek has joined #openstack-kolla | 09:25 | |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Fixes for CentOS 8.3 https://review.opendev.org/c/openstack/kayobe/+/766203 | 09:25 |
| *** adrian-a has quit IRC | 09:29 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/victoria: Fix stop containers task in Swift rolling restart https://review.opendev.org/c/openstack/kolla-ansible/+/764085 | 09:29 |
| *** adrian-a has joined #openstack-kolla | 09:30 | |
| openstackgerrit | Egon Rijpkema proposed openstack/kolla-ansible master: keep X-Forwarded-Proto of exterbal ssl termination. https://review.opendev.org/c/openstack/kolla-ansible/+/758361 | 09:34 |
| *** shyam89 has joined #openstack-kolla | 09:36 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ussuri: Fix stop containers task in Swift rolling restart https://review.opendev.org/c/openstack/kolla-ansible/+/764086 | 09:36 |
| *** vishalmanchanda has quit IRC | 09:36 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/765685 | 09:36 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix mariadb_recovery when mariadb container is missing https://review.opendev.org/c/openstack/kolla-ansible/+/766458 | 09:37 |
| openstackgerrit | Merged openstack/kolla-ansible master: Edit ironic inspector pxe filter driver name none to noop https://review.opendev.org/c/openstack/kolla-ansible/+/765911 | 09:37 |
| openstackgerrit | Merged openstack/kolla-ansible master: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/763770 | 09:37 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix an Ansible warning https://review.opendev.org/c/openstack/kolla-ansible/+/763775 | 09:37 |
| openstackgerrit | Merged openstack/kolla-ansible master: [CI] Add trove smoke test CI https://review.opendev.org/c/openstack/kolla-ansible/+/763197 | 09:38 |
| *** shyamb has quit IRC | 09:38 | |
| openstackgerrit | Merged openstack/kolla-ansible master: CI: enable TLS in single node upgrade jobs https://review.opendev.org/c/openstack/kolla-ansible/+/754499 | 09:38 |
| *** zzzeek has quit IRC | 09:41 | |
| yoctozepto | ah how I missed this spam | 09:42 |
| *** zzzeek has joined #openstack-kolla | 09:42 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/victoria: Fix an Ansible warning https://review.opendev.org/c/openstack/kolla-ansible/+/767303 | 09:44 |
| *** bsanjeewa has joined #openstack-kolla | 09:45 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: Fix an Ansible warning https://review.opendev.org/c/openstack/kolla-ansible/+/767304 | 09:48 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/victoria: Revert "Performance: Use import_tasks in the main plays" https://review.opendev.org/c/openstack/kolla-ansible/+/767305 | 09:51 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: Revert "Performance: Use import_tasks in the main plays" https://review.opendev.org/c/openstack/kolla-ansible/+/767306 | 09:52 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Revert "Performance: Use import_tasks in the main plays" https://review.opendev.org/c/openstack/kolla-ansible/+/767307 | 09:52 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/c/openstack/kolla-ansible/+/767308 | 09:54 |
| hrw | nie to see that amount of jobs on CI again | 09:56 |
| *** zzzeek has quit IRC | 09:57 | |
| *** zzzeek has joined #openstack-kolla | 09:58 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support to kolla_docker for tmpfs mounts https://review.opendev.org/c/openstack/kolla-ansible/+/754377 | 10:02 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: Add support for using a tmpfs for Cinder image conversion https://review.opendev.org/c/openstack/kolla-ansible/+/754378 | 10:02 |
| *** bsanjeewa has quit IRC | 10:04 | |
| *** shyamb has joined #openstack-kolla | 10:06 | |
| *** vishalmanchanda has joined #openstack-kolla | 10:07 | |
| *** wuchunyang has quit IRC | 10:07 | |
| *** shyam89 has quit IRC | 10:09 | |
| *** wuchunyang has joined #openstack-kolla | 10:09 | |
| *** abdysn has joined #openstack-kolla | 10:11 | |
| *** zzzeek has quit IRC | 10:13 | |
| *** wuchunyang has quit IRC | 10:13 | |
| *** zijlboot has quit IRC | 10:14 | |
| *** zzzeek has joined #openstack-kolla | 10:14 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/c/openstack/kolla-ansible/+/767308 | 10:21 |
| hrw | kevko: are you aware of heat-base failing for debian/binary in master? | 10:28 |
| hrw | kevko: as this is binary I tend to leave it to you | 10:29 |
| hrw | kevko: and it is buster-victoria in master wallaby so maybe it is the reason | 10:30 |
| kevko | hi, | 10:31 |
| kevko | hrw: will check, i am working currently on some patchsets for kolla-ansible against master ..so i will deploy heat and will check/fix | 10:32 |
| hrw | kevko: thanks | 10:33 |
| openstackgerrit | Merged openstack/kolla master: Add daxio for nvdimm scenario https://review.opendev.org/c/openstack/kolla/+/765832 | 10:34 |
| kevko | hrw: as I currently don't have heat deployed ..do you have some log which I can check now ? | 10:34 |
| openstackgerrit | Martin Schuppert proposed openstack/kolla stable/victoria: Add daxio for nvdimm scenario https://review.opendev.org/c/openstack/kolla/+/767311 | 10:34 |
| hrw | kevko: https://1ad6f2e4d61e2bf5ff0b-20b98b64cfa6ea87451df6eaddafb782.ssl.cf5.rackcdn.com/750621/4/check/kolla-build-debian-binary/17d906e/ for example | 10:35 |
| *** zijlboot has joined #openstack-kolla | 10:38 | |
| *** JamesBenson has joined #openstack-kolla | 10:42 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/stein: Cap bandit to 1.6.3 (was Support CentOS 8.3) https://review.opendev.org/c/openstack/kolla/+/767295 | 10:43 |
| *** zzzeek has quit IRC | 10:44 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/victoria: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767312 | 10:46 |
| *** JamesBenson has quit IRC | 10:47 | |
| *** zzzeek has joined #openstack-kolla | 10:47 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767313 | 10:47 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767314 | 10:48 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767314 | 10:49 |
| hrw | +2 on all | 10:52 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/c/openstack/kolla-ansible/+/767308 | 10:54 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/victoria: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767315 | 10:54 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/ussuri: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767316 | 10:55 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767318 | 10:56 |
| *** kukacz has quit IRC | 11:04 | |
| *** zzzeek has quit IRC | 11:07 | |
| *** k_mouza has joined #openstack-kolla | 11:08 | |
| *** zzzeek has joined #openstack-kolla | 11:09 | |
| *** zijlboot_ has joined #openstack-kolla | 11:13 | |
| *** zijlboot has quit IRC | 11:15 | |
| *** JamesBenson has joined #openstack-kolla | 11:20 | |
| *** JamesBenson has quit IRC | 11:25 | |
| *** kukacz has joined #openstack-kolla | 11:26 | |
| *** zzzeek has quit IRC | 11:41 | |
| *** zzzeek has joined #openstack-kolla | 11:44 | |
| *** shyamb has quit IRC | 11:51 | |
| *** zzzeek has quit IRC | 11:55 | |
| *** zzzeek has joined #openstack-kolla | 11:57 | |
| *** JamesBenson has joined #openstack-kolla | 12:00 | |
| *** bsanjeewa has joined #openstack-kolla | 12:01 | |
| *** JamesBenson has quit IRC | 12:03 | |
| *** JamesBenson has joined #openstack-kolla | 12:04 | |
| bsanjeewa | I am having an error when running "kayobe overcloud service deploy", with external ceph configured and gnocchi enabled. gnocchi-statsd is skipped in the /venvs/kolla-ansible/share/kolla-ansible/ansible/roles/gnocchi/tasks/config.yml, but /venvs/kolla-ansible/share/kolla-ansible/ansible/roles/gnocchi/tasks/external_ceph.yml needs the directory to copy files. | 12:10 |
| bsanjeewa | http://paste.openstack.org/show/801072/ | 12:11 |
| *** zzzeek has quit IRC | 12:11 | |
| *** k_mouza has quit IRC | 12:13 | |
| *** k_mouza has joined #openstack-kolla | 12:13 | |
| *** zzzeek has joined #openstack-kolla | 12:13 | |
| *** brinzhang_ has joined #openstack-kolla | 12:17 | |
| *** k_mouza has quit IRC | 12:17 | |
| *** zzzeek has quit IRC | 12:18 | |
| *** LinPeiWen has quit IRC | 12:20 | |
| *** zzzeek has joined #openstack-kolla | 12:20 | |
| *** brinzhang has quit IRC | 12:21 | |
| *** k_mouza has joined #openstack-kolla | 12:29 | |
| *** zzzeek has quit IRC | 12:31 | |
| *** zzzeek has joined #openstack-kolla | 12:32 | |
| mgoddard | bsanjeewa: set enable_gnocchi_statsd to true in etc/kayobe/kolla/globals.yml | 12:49 |
| mgoddard | bsanjeewa: it looks like a bug - the task in external-ceph.yml should be conditional on enable_gnocchi_statsd | 12:50 |
| yoctozepto | mgoddard, mnasiadka: ok, so we are now failing mostlu on pull rate limits :D - I guess this is what we should discuss today | 12:50 |
| mnasiadka | yoctozepto: noticed that yesterday | 12:50 |
| mnasiadka | for the sake of CI we could ask docker hub what's the pull limit of this IP and fail the job when it's not sufficient ;) | 12:51 |
| mnasiadka | yoctozepto: actually, if we fail a job in pre - does it get rescheduled to a different node from nodepool? | 12:58 |
| bsanjeewa | mgoddard: thanks. A new error occured with zun, /opt/cni/ does not exist when running [Copy loopback binary from zun-cni-daemon container to host] in ansible/roles/zun/handlers/main.yml. Will https://review.opendev.org/c/openstack/kolla-ansible/+/734135/ address the issue? | 12:59 |
| mgoddard | bsanjeewa: looks relevant. Please try and comment on the review if it works. | 13:00 |
| yoctozepto | mnasiadka: it does, but note mgoddard confirmed we are using caches there, so the same cloud provider would give us the very same ip address | 13:01 |
| yoctozepto | we really need an infra registry of images | 13:02 |
| mgoddard | yoctozepto: I think there is one, but I don't know how well it would handle us | 13:02 |
| mgoddard | and if clouds are in different parts of the world and there is one registry, how will it perform? | 13:03 |
| mgoddard | but I think we need to find answers to these questions | 13:03 |
| mnasiadka | well, ideally there should be a registry that is replicated, but that means syncing gigabytes of data, which we probably don't want to do | 13:11 |
| mnasiadka | that's why maybe just checking docker hub pull limit via API and failing in pre to get a new node + disabling caching would get us somewhere | 13:12 |
| mnasiadka | last time I saw a debian kolla build job fail on fetching debian upstream container image | 13:12 |
| mnasiadka | so I don't know about caching ;) | 13:12 |
| mnasiadka | but I guess mostly we fail on k-a jobs, publishing weekly and getting it properly distributed (cache or replicated registry) would probably work best | 13:14 |
| bsanjeewa | mgoddard: where should I update the code? in src/kolla-ansible or venvs/kolla-ansible? | 13:21 |
| *** brinzhang_ has quit IRC | 13:29 | |
| *** brinzhang_ has joined #openstack-kolla | 13:29 | |
| *** k_mouza has quit IRC | 13:31 | |
| *** zzzeek has quit IRC | 13:34 | |
| *** zzzeek has joined #openstack-kolla | 13:36 | |
| *** k_mouza has joined #openstack-kolla | 13:37 | |
| *** bsanjeewa has quit IRC | 13:41 | |
| *** zzzeek has quit IRC | 13:45 | |
| *** Abdallahyas has joined #openstack-kolla | 13:46 | |
| *** zzzeek has joined #openstack-kolla | 13:49 | |
| *** abdysn has quit IRC | 13:49 | |
| *** abdysn has joined #openstack-kolla | 13:49 | |
| *** Abdallahyas has quit IRC | 13:51 | |
| *** bsanjeewa has joined #openstack-kolla | 13:54 | |
| hrw | https://review.opendev.org/c/openstack/kolla/+/765807 (tier 2 images) waits for brave reviewers | 13:58 |
| hrw | yoctozepto, mnasiadka, mgoddard, osmanlicilegi? | 13:58 |
| mgoddard | hrw: downstreaming | 13:59 |
| hrw | I keep 'WIP:' in title as I prefer not doing any changes in it before someone look and say 'ok, let go that way' | 13:59 |
| *** rafaelweingartne has joined #openstack-kolla | 14:00 | |
| mnasiadka | hrw: I'm thinking if it wouldn't be better to define a regex/names of ,,critical'' images, but maybe that's me | 14:02 |
| hrw | mnasiadka: comment on patch? | 14:03 |
| hrw | mnasiadka: I coded it as a way to tell 'those are images we care less' | 14:03 |
| hrw | mnasiadka: as we can use profiles to limit amount of 'critical images' on CI | 14:04 |
| hrw | (we use profiles on Linaro CI) | 14:05 |
| mnasiadka | commented, let's have a discussion then :D | 14:06 |
| hrw | replied | 14:08 |
| rafaelweingartne | mnasiadka: would you mind providing some more details on your requests at the patch: https://review.opendev.org/c/openstack/kolla-ansible/+/695432? | 14:10 |
| rafaelweingartne | We do not understand on what you would like us to achieve by splitting up that patch. | 14:10 |
| *** wuchunyang has joined #openstack-kolla | 14:11 | |
| mnasiadka | rafaelweingartne: less complexity, and we need more tests (CI) - that it creates what is needed. Will comment in detail tomorrow. | 14:11 |
| *** kukacz has quit IRC | 14:11 | |
| rafaelweingartne | I understand that, but we do not see what can be splited there | 14:11 |
| rafaelweingartne | For us, it does not make much sense to introduce half of those changes without it being complete | 14:12 |
| rafaelweingartne | about CI, depending on what you guys mean by CI, for instance, if it is a test to check the config files that are created, we could create those tests then, if that helps | 14:13 |
| mnasiadka | rafaelweingartne: and for me it's a big chunk of code touching various elements of k-a deployments | 14:13 |
| mnasiadka | rafaelweingartne: you can easily check if identity provider in keystone was created | 14:13 |
| mnasiadka | for example | 14:13 |
| mnasiadka | not config files, real configuration done in OpenStack services | 14:13 |
| rafaelweingartne | but this would be troublesome, we would need a CI with an IdP | 14:13 |
| rafaelweingartne | of course it is touching different files, all of them are necessary to create all of the configs | 14:14 |
| mnasiadka | well, most of features we provide are tested in a CI, so at least think about a smoke test :) | 14:14 |
| rafaelweingartne | what do you mean by smoke test? checking if the config files are what we intended to be? | 14:15 |
| *** wuchunyang has quit IRC | 14:15 | |
| mnasiadka | rafaelweingartne: idp needs to be created in keystone - check if it is created (using openstack client), at least this basic stuff | 14:20 |
| mnasiadka | and I guess there should be a test idp somewhere in the internet we could use? | 14:20 |
| rafaelweingartne | ah ok | 14:20 |
| mnasiadka | At least there's one for saml... | 14:21 |
| rafaelweingartne | so we do not need to test if it is actually working | 14:21 |
| mnasiadka | no, if that's troublesome or impossible - that can be done later or never ;) | 14:21 |
| rafaelweingartne | I do not mind us adding effort to that | 14:21 |
| rafaelweingartne | the point is that we would like some clear directions | 14:22 |
| rafaelweingartne | if that is the requirement, we will implement it, but we would like to understand if that is the requirement for us to move this on | 14:22 |
| *** abdysn has quit IRC | 14:22 | |
| rafaelweingartne | otherwise, it feels that we do the CI for OpenID Connect, and then somebody else starts to asks SAML support in the same patch, and only when that is also complete, that we can move on | 14:23 |
| rafaelweingartne | which would only add more complexity | 14:23 |
| rafaelweingartne | as you said, it is a big chunky already, and we only have that, because it is all together the basic structure we envision to add federated support + the support for OpenID Connect | 14:23 |
| openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: [doc] Fix a typo in dpdk doc https://review.opendev.org/c/openstack/kolla-ansible/+/767360 | 14:26 |
| tbarron | For a Manila document I'm working on, I'd like to check on the status of manila with kolla-ansible. | 14:29 |
| tbarron | Are there any integration challenges? Backend support issues? | 14:29 |
| tbarron | Other obstacles? | 14:30 |
| tbarron | I see https://docs.openstack.org/kolla-ansible/latest/reference/storage/manila-guide.html which mentions glusterfs in particular. | 14:30 |
| *** brinzhang_ has quit IRC | 14:31 | |
| *** brinzhang_ has joined #openstack-kolla | 14:32 | |
| *** TrevorV has joined #openstack-kolla | 14:34 | |
| *** bsanjeewa_ has joined #openstack-kolla | 14:41 | |
| *** bsanjeewa has quit IRC | 14:44 | |
| *** jbadiapa has joined #openstack-kolla | 14:48 | |
| mnasiadka | tbarron: no CI - that's the most problematic thing, we don't really know if it works :) | 14:48 |
| tbarron | mnasiadka: agree :) | 14:48 |
| *** wuchunyang has joined #openstack-kolla | 14:49 | |
| tbarron | mnasiadka: and is the obstacle there cycles and people or something else? | 14:49 |
| mnasiadka | tbarron: I think it's my cycles, at least for the Manila + CephFS part | 14:49 |
| mnasiadka | and I don't know how problematic it is to test it with glusterfs in opendev CI | 14:49 |
| mnasiadka | (haven't looked into that) | 14:49 |
| tbarron | mnasiadka: for manila + CephFS is your interest in native CephFS, CephFS gatewayed to NFS, or both? | 14:50 |
| *** bsanjeewa_ has quit IRC | 14:52 | |
| mnasiadka | native CephFS, the gatewayed NFS stuff is not really production grade, so I don't see why we should test it at all :) | 14:52 |
| mgoddard | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak | 14:52 |
| *** bsanjeewa has joined #openstack-kolla | 14:52 | |
| mgoddard | meeting in 10 ^ | 14:52 |
| tbarron | mnasiadka: ack. in tripleo we wrap the gatewayed nfs stuff with pacemaker, etc. for HA but it's a pain | 14:54 |
| *** bsanjeewa has quit IRC | 14:54 | |
| openstackgerrit | Merged openstack/kolla stable/victoria: Add daxio for nvdimm scenario https://review.opendev.org/c/openstack/kolla/+/767311 | 14:54 |
| openstackgerrit | Merged openstack/kolla-ansible stable/victoria: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767312 | 14:54 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767314 | 14:54 |
| tbarron | mnasiadka: but necessary for deployments that aren't comfortable with CephFS and tenant separation | 14:54 |
| openstackgerrit | Merged openstack/kolla-ansible stable/ussuri: docs: Add magnum guide https://review.opendev.org/c/openstack/kolla-ansible/+/767313 | 14:54 |
| *** bsanjeewa has joined #openstack-kolla | 14:55 | |
| tbarron | mnasiadka: the good news there w.r.t. native cephfs is that it should be simpler to deploy and test in CI | 14:55 |
| tbarron | mnasiadka: anyways, thanks for updating me on status! I'd like to see more use of manila with other deployment frameworks than just tripleo | 14:58 |
| tbarron | and kolla-ansible, kayobe, etc, look really nice. | 14:58 |
| mgoddard | #startmeeting kolla | 15:01 |
| openstack | Meeting started Wed Dec 16 15:01:18 2020 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| *** openstack changes topic to " (Meeting topic: kolla)" | 15:01 | |
| openstack | The meeting name has been set to 'kolla' | 15:01 |
| mgoddard | #topic rollcall | 15:01 |
| *** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:01 | |
| mgoddard | \o | 15:01 |
| yoctozepto | o/ | 15:01 |
| mnasiadka | o/ | 15:01 |
| rafaelweingartne | \o | 15:02 |
| mgoddard | #topic agenda | 15:04 |
| *** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:04 | |
| mgoddard | * Roll-call | 15:05 |
| mgoddard | * Announcements | 15:05 |
| mgoddard | * Review action items from the last meeting | 15:05 |
| mgoddard | * CI status | 15:05 |
| mgoddard | * Victoria release planning | 15:05 |
| mgoddard | * Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits | 15:05 |
| mgoddard | * CentOS 8.3 & stream https://lists.centos.org/pipermail/centos-devel/2020-December/075451.html | 15:05 |
| mgoddard | * Cinder active/active https://bugs.launchpad.net/kolla-ansible/+bug/1904062 | 15:05 |
| openstack | Launchpad bug 1904062 in kolla-ansible wallaby "external ceph cinder volume config breaks volumes on ussuri upgrade" [High,In progress] - Assigned to Michal Nasiadka (mnasiadka) | 15:05 |
| mgoddard | * Wallaby PTG actions | 15:05 |
| mgoddard | #topic announcements | 15:05 |
| *** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:05 | |
| mgoddard | I suppose we should cancel some meetings | 15:05 |
| mgoddard | the next two? | 15:05 |
| mnasiadka | for the next 3 weeks I guess, 6th Jan is designated holiday in Poland | 15:05 |
| mgoddard | oh ok | 15:06 |
| yoctozepto | ++ | 15:06 |
| *** zzzeek has quit IRC | 15:06 | |
| yoctozepto | I am running masakari's on 22nd | 15:06 |
| yoctozepto | but yeah, 23rd with kolla might be a bit late | 15:06 |
| mgoddard | #info The next 3 meetings are cancelled | 15:06 |
| mgoddard | #action mgoddard send email to openstack-discuss about meeting cancellations | 15:06 |
| wuchunyang | ok | 15:07 |
| yoctozepto | 2020-01-13 is the next kolla meeting | 15:07 |
| wuchunyang | happy new year | 15:07 |
| yoctozepto | 2021-01-13 * | 15:07 |
| yoctozepto | happy new year! :D | 15:07 |
| mgoddard | Any others? | 15:07 |
| yoctozepto | we got our CI back on track | 15:07 |
| mgoddard | #topic CI status | 15:07 |
| *** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:07 | |
| * yoctozepto sends off fireworks | 15:08 | |
| mgoddard | hooray | 15:08 |
| *** zzzeek has joined #openstack-kolla | 15:08 | |
| mgoddard | thank you to everyone involved in firefighting over the last few weeks | 15:08 |
| mgoddard | we're not done yet - kayobe is still busted | 15:09 |
| mgoddard | I had it passing yesterday, now there two new failures | 15:09 |
| yoctozepto | ah, yeah | 15:09 |
| yoctozepto | :O | 15:09 |
| yoctozepto | that went quick | 15:09 |
| yoctozepto | what kind of failures? | 15:09 |
| mgoddard | well, passing in review anyway | 15:09 |
| mgoddard | bifrost changed a default | 15:09 |
| yoctozepto | oh gosh | 15:10 |
| mgoddard | and some weird ironic locking issue | 15:10 |
| yoctozepto | :-( | 15:10 |
| * hrw at 2 meetings at time | 15:11 | |
| mgoddard | anyway let's go over the CI status | 15:11 |
| mgoddard | centos8-ceph-upgrade jobs seem to be retried 3 times only to fail in some weird way | 15:11 |
| mgoddard | still seeing it? | 15:11 |
| *** kukacz has joined #openstack-kolla | 15:11 | |
| yoctozepto | yes | 15:12 |
| yoctozepto | obviously not always | 15:12 |
| yoctozepto | https://zuul.openstack.org/builds?job_name=kolla-ansible-centos8-source-upgrade-ceph-ansible&project=openstack%2Fkolla-ansible&branch=master | 15:13 |
| yoctozepto | a SUCCESS is rare | 15:13 |
| yoctozepto | fwiw, ubuntu ain't looking any better https://zuul.openstack.org/builds?job_name=kolla-ansible-ubuntu-source-upgrade-ceph-ansible&project=openstack%2Fkolla-ansible&branch=master | 15:14 |
| mgoddard | wonderful | 15:14 |
| yoctozepto | well, I suppose DISK_FULL is kinda success | 15:14 |
| yoctozepto | I like your optimism | 15:14 |
| *** ricolin_ has joined #openstack-kolla | 15:15 | |
| kevko | guys, probably i refactor and have completed switch from haproxy to maxscale , everyone who want to check working maxscale can login here | 15:15 |
| kevko | http://185.21.197.26:8990/#/dashboard/servers | 15:15 |
| kevko | just ask for pass in private message .. | 15:16 |
| mgoddard | kevko: meeting time | 15:16 |
| yoctozepto | yup, these recent ones still failed on "Ensuring config directories exist" | 15:16 |
| mgoddard | let's move on from CI. At least we're in a better position than before, even if we have loose ends | 15:17 |
| mgoddard | #topic Victoria release planning | 15:17 |
| *** openstack changes topic to "Victoria release planning (Meeting topic: kolla)" | 15:17 | |
| yoctozepto | ubuntu does not exhibit that particular behaviour | 15:17 |
| yoctozepto | all right, Victoria not so victorious | 15:17 |
| mgoddard | end of year release not looking too likely | 15:18 |
| mgoddard | we're still basically blocked on the cinder issue | 15:18 |
| mgoddard | mnasiadka: any progress? | 15:18 |
| yoctozepto | yup, super sad this time | 15:18 |
| mnasiadka | mgoddard: have a test env, testing migration from non-cluster to cluster, should have something to push this week | 15:19 |
| mgoddard | perhaps we need to make a call about whether to release without the cinder fix | 15:19 |
| mnasiadka | backend_host probably won't take too much time, it's not much different, than we have one cinder-volume service instead of 3 or more/less | 15:19 |
| mnasiadka | mgoddard: well, we could release with an issue type reno, but it won't look very good :) | 15:20 |
| mgoddard | no, but it's no worse than what we have already | 15:20 |
| yoctozepto | ++ | 15:21 |
| mgoddard | that doesn't mean we should take the pressure off fixing the issue | 15:21 |
| yoctozepto | ++ | 15:21 |
| mgoddard | vote: release victoria without cinder active/active fix? | 15:22 |
| mnasiadka | if I won't push any updates to the change tomorrow - I'll make the releases change myself, deal? :) | 15:23 |
| yoctozepto | mnasiadka: otherwise you'll make mgoddard do them? :P | 15:23 |
| mnasiadka | unless we want to test the migration in a CI | 15:23 |
| mnasiadka | then it will take for sure more time | 15:23 |
| yoctozepto | let's release with an issue reno | 15:24 |
| yoctozepto | ussuri is already "broken" | 15:24 |
| yoctozepto | mention that | 15:24 |
| yoctozepto | it is not a *new issue* | 15:25 |
| yoctozepto | and it can be workarounded if you just control the process yourself | 15:25 |
| yoctozepto | which might be the case with an external ceph ;-) | 15:25 |
| yoctozepto | mgoddard, mnasiadka: decisions, decisions | 15:27 |
| mgoddard | ugh | 15:27 |
| mgoddard | let's give mnasiadka a day or two | 15:27 |
| yoctozepto | this week indeed | 15:27 |
| yoctozepto | I might think about extending CI testing | 15:28 |
| mgoddard | we may not even have the right people around to get a release approved before 2021 | 15:28 |
| yoctozepto | that true as well | 15:28 |
| mgoddard | #topic Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits | 15:28 |
| *** openstack changes topic to "Dockerhub pull rate limits https://etherpad.opendev.org/p/docker-pull-limits (Meeting topic: kolla)" | 15:28 | |
| yoctozepto | oh noez | 15:28 |
| yoctozepto | in nightmares it hunts me | 15:29 |
| mgoddard | you have reached your dream limit | 15:29 |
| yoctozepto | http 429 no sweet dreams for you | 15:30 |
| yoctozepto | sooo | 15:30 |
| mnasiadka | lol | 15:30 |
| yoctozepto | are we pushing towards an internal registry? | 15:30 |
| mgoddard | I've crossed through docker devil pact option 2 | 15:30 |
| yoctozepto | because I have no idea what to do about this | 15:30 |
| mnasiadka | so, mostly k-a jobs suffer? | 15:30 |
| mgoddard | which leaves switching to another devil, or using the infra registry | 15:31 |
| yoctozepto | mnasiadka: k too | 15:31 |
| mnasiadka | so, in kolla case we pull only one image from docker hub, right? | 15:31 |
| yoctozepto | maybe let's try another devil? | 15:31 |
| yoctozepto | mnasiadka: yeah 1 per distro version | 15:31 |
| mnasiadka | and we fail with that, that's some crazy stuff | 15:31 |
| yoctozepto | because we kill it with k-a jobs | 15:32 |
| mnasiadka | ok, so option 1: another devil | 15:32 |
| yoctozepto | or, well | 15:32 |
| yoctozepto | we are not using registry mirror in kolla | 15:32 |
| yoctozepto | when building those images | 15:32 |
| yoctozepto | they seem to be coming directly from the dockerhub | 15:33 |
| yoctozepto | maybe that is where our issue is | 15:33 |
| mnasiadka | I don't think centos:8 ubuntu:something and debian:devilish_number changes a lot | 15:33 |
| yoctozepto | they don't | 15:33 |
| mgoddard | if we switch to another registry, I don't know if we get the caching or the mirror | 15:33 |
| mnasiadka | so we could at least cross out those failures by using a cache | 15:33 |
| *** cah_link has quit IRC | 15:33 | |
| yoctozepto | mgoddard, mnasiadka: could you confirm that in k we are not using any mirror at the very moment | 15:33 |
| mnasiadka | do we know how many pulls credits we use per a standard k-a CI job? | 15:33 |
| *** wuchunyang has quit IRC | 15:34 | |
| yoctozepto | mnasiadka: 20-30 | 15:34 |
| *** wuchunyang has joined #openstack-kolla | 15:34 | |
| yoctozepto | *(mirror meaning caching proxy) | 15:34 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add maxscale support for database https://review.opendev.org/c/openstack/kolla-ansible/+/767370 | 15:35 |
| *** cah_link has joined #openstack-kolla | 15:35 | |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add maxscale support for database https://review.opendev.org/c/openstack/kolla-ansible/+/767370 | 15:37 |
| mnasiadka | yoctozepto: 20-30 is not bad, we have 200 pulls per 6 hours, right? | 15:37 |
| mnasiadka | or 100? | 15:37 |
| mgoddard | 100 | 15:37 |
| mgoddard | it's not enough | 15:37 |
| yoctozepto | not enuff | 15:37 |
| mgoddard | ok, key question | 15:38 |
| mnasiadka | so, if we disable using cache in k-a jobs, and add a check in pre to check if we have enough pulls for current host, is that something we could live with? | 15:38 |
| mgoddard | if I pull from github's registry using a registry mirror, will it cache the image? | 15:38 |
| mnasiadka | mgoddard: good question, I guess it should - but maybe we are both wrong in that assumption :) needs testing | 15:39 |
| mgoddard | I remember reading it doesn't | 15:39 |
| yoctozepto | you mean pull-through? | 15:40 |
| mgoddard | yes | 15:40 |
| yoctozepto | I also think it only caches the primary one | 15:41 |
| yoctozepto | might be worth gooling | 15:41 |
| yoctozepto | googling* | 15:41 |
| yoctozepto | let me see, | 15:41 |
| mnasiadka | the standard doesn't | 15:42 |
| mnasiadka | but there is stuff like https://hub.docker.com/r/tiangolo/docker-registry-proxy | 15:42 |
| mnasiadka | or probably something else we could use | 15:42 |
| mgoddard | Gotcha | 15:42 |
| mgoddard | It’s currently not possible to mirror another private registry. Only the central Hub can be mirrored. | 15:43 |
| yoctozepto | https://github.com/docker/distribution/blob/master/ROADMAP.md#proxying-to-other-registries | 15:44 |
| yoctozepto | mnasiadka: the thing is: infra has to | 15:44 |
| mnasiadka | yoctozepto: I know, that's why we need to think of something we can do now | 15:45 |
| mgoddard | yeah | 15:45 |
| yoctozepto | and they already have their hands full with the new great gerrit | 15:45 |
| yoctozepto | I think we should add cache to k | 15:45 |
| yoctozepto | as I see it is not there | 15:46 |
| yoctozepto | I wanted you to cross-validate me | 15:46 |
| mgoddard | what do you mean? | 15:46 |
| mnasiadka | and judging by the amount of storage our images need - it might be complicated to find a registry local to nodepool providers :) | 15:46 |
| yoctozepto | no hits of registry-mirrors in kolla job plays | 15:46 |
| mgoddard | oh, I see | 15:47 |
| yoctozepto | kolla-ansible yes | 15:47 |
| yoctozepto | kolla not | 15:47 |
| mgoddard | it would help for some build failures | 15:47 |
| yoctozepto | mgoddard: now you know why I thought what I thought ;-) | 15:47 |
| mgoddard | but while it might save the build jobs, the deploy jobs are likely to fail | 15:48 |
| mnasiadka | yeah, at least we can fix kolla for now - it's an easy step | 15:48 |
| yoctozepto | yes, I thought this | 15:48 |
| yoctozepto | we can move to another registry | 15:48 |
| yoctozepto | but we still need to fetch | 15:48 |
| yoctozepto | distro images | 15:48 |
| yoctozepto | so cache in kolla | 15:48 |
| yoctozepto | and that is enough | 15:48 |
| yoctozepto | for kolla | 15:48 |
| yoctozepto | then move publishing to quay | 15:49 |
| yoctozepto | or something | 15:49 |
| mnasiadka | and why quay is better? | 15:49 |
| mnasiadka | higher limits? | 15:49 |
| yoctozepto | because it has only burst limits | 15:49 |
| mgoddard | if we move to another registry we can no longer use the registry mirror, and that will increase external traffic significantly | 15:49 |
| yoctozepto | if you do too many reqs per sec | 15:49 |
| yoctozepto | but it has no mirror | 15:49 |
| yoctozepto | so is trickky | 15:49 |
| yoctozepto | indeed | 15:49 |
| mnasiadka | so, in reality the zuul pull through cache is having going over pull limits? | 15:50 |
| mnasiadka | uh, english is complicated | 15:50 |
| yoctozepto | yes, rephrase pretty please | 15:50 |
| mgoddard | yes, all pulls come from the caches | 15:50 |
| yoctozepto | in k-a jobs | 15:51 |
| mnasiadka | and the caches have one ip, so it's not a problem for them to use the quota | 15:51 |
| *** wuchunyang has quit IRC | 15:51 | |
| mnasiadka | are we sure that disabling the cache is not a better solution? :) | 15:51 |
| yoctozepto | one ip addr per cloud more or less | 15:51 |
| yoctozepto | we are pretty sure regarding the external traffic | 15:51 |
| mnasiadka | well, not better in terms of traffic, but better in terms of pull quotas | 15:52 |
| *** adrian-a has quit IRC | 15:52 | |
| mgoddard | hard to say without trying it | 15:53 |
| mgoddard | it depends on how nodepool is setup | 15:53 |
| mgoddard | if all nodes are in a single project using one router, they'll all use the same IP anyway | 15:53 |
| yoctozepto | indeed | 15:53 |
| *** k_mouza has quit IRC | 15:53 | |
| mnasiadka | well then it's even worse, if they all share one external IP via SNAT | 15:54 |
| mgoddard | yes | 15:54 |
| *** k_mouza has joined #openstack-kolla | 15:54 | |
| mnasiadka | so the only way forward is deploying a registry in each nodepool provider and try to sync them (or check for images older than X and build them) | 15:54 |
| yoctozepto | or cache quay | 15:55 |
| mnasiadka | or... check the usage of docker pull quota and decide if we need to build the images, or just pull them (if we have the same IP as the pull through cache) | 15:55 |
| mnasiadka | (docker exposes number of pull requests left in API) | 15:55 |
| yoctozepto | I wonder | 15:55 |
| yoctozepto | if we are not better of rebuilding the images now each time :P | 15:55 |
| mgoddard | it's a bit racy with multiple jobs though | 15:56 |
| mnasiadka | well, we can build on first host, push into a registry and use that registry | 15:56 |
| mnasiadka | and build only a subset that is required for CI | 15:56 |
| yoctozepto | that is what we are doing | 15:56 |
| yoctozepto | when we need to build them | 15:56 |
| yoctozepto | we are smart (TM) | 15:56 |
| yoctozepto | so what do you say | 15:57 |
| yoctozepto | we just enable building by default | 15:57 |
| yoctozepto | and see how bad that works for us | 15:57 |
| mnasiadka | well, how longer will it take? 10-20 minutes? | 15:57 |
| mgoddard | why don't we try another registry | 15:57 |
| mnasiadka | but then there's no cache? | 15:57 |
| mgoddard | all other options seem to lack a cache | 15:58 |
| yoctozepto | folks, 2 minutes | 15:58 |
| mgoddard | or require pulling in data for building | 15:58 |
| yoctozepto | enable building images each time and see how bad it goes? | 15:58 |
| yoctozepto | for the time being | 15:58 |
| yoctozepto | on master | 15:58 |
| mgoddard | could do it on master | 15:58 |
| mnasiadka | yeah, let's do it on master - and see how that goes | 15:59 |
| mnasiadka | or even only for source | 15:59 |
| yoctozepto | then basically kolla-ansible jobs work like they always do in kolla | 15:59 |
| yoctozepto | mnasiadka: no, always, otherwise we might still trigger the duckery | 15:59 |
| mgoddard | kolla-ansible pull | kolla-build | 15:59 |
| mgoddard | || | 15:59 |
| mgoddard | doesn't work for pre-upgrade though | 16:00 |
| mgoddard | also true of building | 16:00 |
| yoctozepto | no, just build on master | 16:00 |
| yoctozepto | stable have their own jobs | 16:00 |
| yoctozepto | so we would still be pulling something extra | 16:01 |
| yoctozepto | eh, dockerhub | 16:01 |
| yoctozepto | was that really necessary | 16:01 |
| yoctozepto | a'ight time is up | 16:01 |
| mnasiadka | yup | 16:02 |
| mgoddard | maybe we should just accept the Ts & Cs :D | 16:03 |
| mnasiadka | basically - ideally if infra would deploy something like Harbor in each mirror site and setup replication rules - it would be solved like that | 16:03 |
| mnasiadka | we would just publish both to docker hub and local registry | 16:03 |
| mnasiadka | wonder what is the traffic volume we create by publishing | 16:04 |
| mgoddard | ok, let's finish | 16:04 |
| mgoddard | thanks | 16:04 |
| mgoddard | #endmeeting | 16:04 |
| *** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:04 | |
| openstack | Meeting ended Wed Dec 16 16:04:30 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:04 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-12-16-15.01.html | 16:04 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-12-16-15.01.txt | 16:04 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-12-16-15.01.log.html | 16:04 |
| yoctozepto | mnasiadka: we would have to do it for infra | 16:04 |
| yoctozepto | they can't magically do it for us with 0 effort from our side | 16:05 |
| yoctozepto | thanks mgoddard | 16:05 |
| mnasiadka | yoctozepto: I'm not telling we ask them to do it, I'm happy to work on it, if they accept it as a solution :) | 16:05 |
| mnasiadka | if not, we're left with moving to Quay/Github and setting up a cache for those in opendev? | 16:06 |
| hrw | I am after internal CentOS Stream AMA meeting | 16:08 |
| hrw | information about migration from centos:8 to CentOS Stream will be published | 16:10 |
| hrw | so far we can just continue with centos:8 | 16:10 |
| sorin-mihai__ | i'm planning a production deployment and i'm comparing some things with some TripleO concepts. maybe things have changed since those documents where written, but i see that the undercloud is being deployed in just one node. all i know about kayobe so far is that it has the same role as the undercloud. i am concerned about the situation where the kayobe node would be powered off for whatever reason; how would that affect the | 16:12 |
| sorin-mihai__ | rest of the kolla-ansible deployment? i'm trying to achieve proper HA for all components of the deployment | 16:12 |
| yoctozepto | sorin-mihai__: it would not care | 16:14 |
| *** cah_link has quit IRC | 16:15 | |
| mnasiadka | hrw: I saw a mail that they are targetting for 1Q 2021 to really get Stream in shape | 16:16 |
| hrw | mnasiadka: there will be CentOS 8.4 and CentOS 8.5 in 2021 so there is a year to switch | 16:16 |
| sorin-mihai__ | yoctozepto, so the HA/redundancy is obtained just by having the 3 controllers? what if one of them goes down, how would the compute nodes be affected? i know, silly questions, but so far i didn't find documents that would discuss the "what if/worst case" | 16:18 |
| *** kukacz has quit IRC | 16:20 | |
| mnasiadka | hrw: or more | 16:29 |
| hrw | mnasiadka: depends on RDO probably | 16:30 |
| *** ricolin_ has quit IRC | 16:31 | |
| *** rafaelweingartne has quit IRC | 16:32 | |
| *** kukacz has joined #openstack-kolla | 16:43 | |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add maxscale support for database https://review.opendev.org/c/openstack/kolla-ansible/+/767370 | 16:45 |
| *** wuchunyang has joined #openstack-kolla | 16:59 | |
| *** wuchunyang has quit IRC | 17:03 | |
| *** lbotell has joined #openstack-kolla | 17:05 | |
| openstackgerrit | Martin Schuppert proposed openstack/kolla stable/ussuri: Add daxio for nvdimm scenario https://review.opendev.org/c/openstack/kolla/+/767377 | 17:05 |
| *** rpittau is now known as rpittau|afk | 17:11 | |
| *** bengates has quit IRC | 17:14 | |
| bsanjeewa | curl google.lk | 17:14 |
| bsanjeewa | sorry, wrong window | 17:15 |
| *** bengates has joined #openstack-kolla | 17:15 | |
| *** bengates has quit IRC | 17:19 | |
| *** gfidente is now known as gfidente|afk | 17:22 | |
| *** sean-k-mooney1 has joined #openstack-kolla | 17:24 | |
| *** sean-k-mooney has quit IRC | 17:25 | |
| *** k_mouza has quit IRC | 17:35 | |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add maxscale support for database https://review.opendev.org/c/openstack/kolla-ansible/+/767370 | 17:36 |
| *** bsanjeewa has left #openstack-kolla | 17:39 | |
| *** bsanjeewa has joined #openstack-kolla | 17:40 | |
| *** kukacz has quit IRC | 17:40 | |
| *** kevko has quit IRC | 17:50 | |
| *** muhaha has joined #openstack-kolla | 17:56 | |
| *** bsanjeewa has quit IRC | 17:56 | |
| *** bsanjeewa has joined #openstack-kolla | 17:57 | |
| *** cah_link has joined #openstack-kolla | 18:00 | |
| *** kukacz has joined #openstack-kolla | 18:07 | |
| *** bsanjeewa_ has joined #openstack-kolla | 18:09 | |
| *** bsanjeewa has quit IRC | 18:09 | |
| *** muhaha has quit IRC | 18:16 | |
| *** sean-k-mooney2 has joined #openstack-kolla | 18:23 | |
| *** maharg101 has quit IRC | 18:24 | |
| *** muhaha has joined #openstack-kolla | 18:26 | |
| *** sean-k-mooney1 has quit IRC | 18:26 | |
| yoctozepto | sorin-mihai__: yes, you need to have two up to have it working; one going down might cause a small disruption (as services might still be connected to the failed node and take some time to realise the fact; similarly if VIP has to move) | 18:38 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767318 | 18:39 |
| *** adrian-a has joined #openstack-kolla | 18:39 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/ussuri: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767316 | 18:39 |
| *** muhaha has quit IRC | 18:49 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Add support to kolla_docker for tmpfs mounts https://review.opendev.org/c/openstack/kolla-ansible/+/754377 | 18:51 |
| openstackgerrit | Merged openstack/kolla-ansible master: Add support for using a tmpfs for Cinder image conversion https://review.opendev.org/c/openstack/kolla-ansible/+/754378 | 18:51 |
| openstackgerrit | Merged openstack/kolla-ansible stable/victoria: Do not start swift-object-updater on stop https://review.opendev.org/c/openstack/kolla-ansible/+/767315 | 18:51 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/stein: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/c/openstack/kolla-ansible/+/767308 | 18:54 |
| *** wuchunyang has joined #openstack-kolla | 19:00 | |
| *** lbotell55 has joined #openstack-kolla | 19:04 | |
| *** wuchunyang has quit IRC | 19:05 | |
| *** jonaspaulo has joined #openstack-kolla | 19:14 | |
| *** cah_link has quit IRC | 19:16 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: CI: test elasticsearch & kibana 6.x migration (Ussuri) https://review.opendev.org/c/openstack/kolla-ansible/+/760084 | 19:18 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/ussuri: CI: test elasticsearch & kibana 6.x migration (Ussuri) https://review.opendev.org/c/openstack/kolla-ansible/+/760084 | 19:18 |
| *** kukacz has quit IRC | 19:21 | |
| adrian-a | regarding the docker limits, in another project, after contemplating to host own registry, we just bought a damn docker account, helluva cheaper than over-analyzing, deploying the own registry :) | 19:25 |
| *** andrewbonney has quit IRC | 19:26 | |
| adrian-a | + vs. the own registry maintenance cost | 19:28 |
| *** bsanjeewa_ has quit IRC | 19:34 | |
| *** zzzeek has quit IRC | 19:36 | |
| *** brtknr has quit IRC | 19:37 | |
| *** zzzeek has joined #openstack-kolla | 19:38 | |
| *** jonaspaulo has quit IRC | 19:38 | |
| *** brtknr has joined #openstack-kolla | 19:40 | |
| yoctozepto | adrian-a: I am spending enough time upstream not to spend money on it ;-) | 19:43 |
| *** kukacz has joined #openstack-kolla | 19:44 | |
| *** kukacz has quit IRC | 19:45 | |
| *** sorin-mihai__ has quit IRC | 19:49 | |
| adrian-a | definitely, I mean the Foundation. I can also sponsor a docker account, I'm spending too much time just reading about it! And a lot more wasted effort on this discussion. | 19:50 |
| adrian-a | and all the wasted time watching and rerunning CI jobs | 19:51 |
| *** e0ne has quit IRC | 19:52 | |
| yoctozepto | adrian-a: yeah, it pains me too; this was a bad year | 19:56 |
| adrian-a | anything else significant, besides the usual (Corona) and docker nuisance? | 19:59 |
| *** zzzeek has quit IRC | 20:03 | |
| yoctozepto | adrian-a: well, centos linux going away is certainly no good news if you ask me ;-) | 20:03 |
| adrian-a | yes, indeed. we'll jump into debian as probably a many folks. but we're still stuck in the rpm world with some systems. fingers crossed for rocky | 20:06 |
| *** zzzeek has joined #openstack-kolla | 20:07 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/victoria: Revert "Performance: Use import_tasks in the main plays" https://review.opendev.org/c/openstack/kolla-ansible/+/767305 | 20:07 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Revert "Performance: Use import_tasks in the main plays" https://review.opendev.org/c/openstack/kolla-ansible/+/767307 | 20:07 |
| r3ap3r | Does anyone use the Rally option when deploying with Kolla-Ansible? | 20:12 |
| *** zzzeek has quit IRC | 20:14 | |
| *** kukacz has joined #openstack-kolla | 20:17 | |
| *** zzzeek has joined #openstack-kolla | 20:17 | |
| openstackgerrit | Merged openstack/kolla-ansible master: [doc] Fix a typo in dpdk doc https://review.opendev.org/c/openstack/kolla-ansible/+/767360 | 20:20 |
| *** maharg101 has joined #openstack-kolla | 20:21 | |
| *** zzzeek has quit IRC | 20:25 | |
| *** maharg101 has quit IRC | 20:25 | |
| *** zzzeek has joined #openstack-kolla | 20:27 | |
| lbotell | Hi, I am planning to deploy one controller, one compute and three baremetal nodes | 20:50 |
| lbotell | that using kolla-ansible deploy? | 20:50 |
| *** vishalmanchanda has quit IRC | 20:56 | |
| *** sean-k-mooney has joined #openstack-kolla | 20:56 | |
| *** sean-k-mooney2 has quit IRC | 20:59 | |
| *** sorin-mihai__ has joined #openstack-kolla | 21:11 | |
| *** adrian-a has quit IRC | 21:19 | |
| *** brinzhang0 has joined #openstack-kolla | 21:42 | |
| *** TrevorV has quit IRC | 21:43 | |
| *** brinzhang_ has quit IRC | 21:45 | |
| *** kevko has joined #openstack-kolla | 21:51 | |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add maxscale support for database https://review.opendev.org/c/openstack/kolla-ansible/+/767370 | 22:00 |
| *** lbotell has quit IRC | 22:11 | |
| *** lbotell55 has quit IRC | 22:13 | |
| JamesBenson | lbotell, not sure if that is a question, but yeah, totally possible. | 22:17 |
| *** sorin-mihai__ has quit IRC | 22:42 | |
| *** sorin-mihai__ has joined #openstack-kolla | 22:42 | |
| *** sorin-mihai__ has quit IRC | 22:49 | |
| *** wuchunyang has joined #openstack-kolla | 23:02 | |
| *** kukacz has quit IRC | 23:02 | |
| *** wuchunyang has quit IRC | 23:06 | |
| *** cah_link has joined #openstack-kolla | 23:08 | |
| *** jbadiapa has quit IRC | 23:14 | |
| openstackgerrit | Kendall Nelson proposed openstack/kayobe-config master: Remove Retired Karbor Support https://review.opendev.org/c/openstack/kayobe-config/+/767435 | 23:18 |
| openstackgerrit | Kendall Nelson proposed openstack/kayobe-config-dev master: Remove Retired Karbor Support https://review.opendev.org/c/openstack/kayobe-config-dev/+/767436 | 23:20 |
| *** kukacz has joined #openstack-kolla | 23:30 | |
| *** lbotell has joined #openstack-kolla | 23:33 | |
| *** gfidente|afk has quit IRC | 23:47 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!