| *** tonythomas has quit IRC | 00:31 | |
| *** jbadiapa has quit IRC | 00:53 | |
| *** dave-mccowan has quit IRC | 00:55 | |
| *** boldo has quit IRC | 01:08 | |
| *** ivve has quit IRC | 01:27 | |
| *** xinliang has joined #openstack-kolla | 01:37 | |
| *** bold has joined #openstack-kolla | 01:41 | |
| bold | TASK [kibana : Change kibana config to set index as defaultIndex] *************************************************************fatal: [controller1]: FAILED! => {"action": "uri", "changed": false, "connection": "close", "content": "{\"error\":{\"root_cause\":[{\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse\"}],\"type\":\"mapper_parsing_exception\",\"reason\":\"failed to parse\",\"caused_by\":{\"type\":\"i_o_exception\",\"reason\":\"U | 01:41 |
|---|---|---|
| bold | nexpected character (''' (code 39)): was expecting double-quote to start field name\\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@310629c; line: 1, column: 3]\"}},\"status\":400}", "content_length": "418", "content_type": "application/json; charset=UTF-8", "elapsed": 0, "json": {"error": {"caused_by": {"reason": "Unexpected character (''' (code 39)): was expecting double-quote to start field name\n at | 01:41 |
| bold | [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper@310629c; line: 1, column: 3]", "type": "i_o_exception"}, "reason": "failed to parse", "root_cause": [{"reason": "failed to parse", "type": "mapper_parsing_exception"}], "type": "mapper_parsing_exception"}, "status": 400}, "msg": "Status code was 400 and not [200, 201]: HTTP Error 400: Bad Request", "redirected": false, "status": 400, "url": "http://192.168.10.65:920 | 01:41 |
| bold | 0/.kibana/config/*"} | 01:41 |
| bold | how to fix this error. it occurs when running kolla-ansible -i ./multinode deploy | 01:43 |
| *** jimcrowleyibm has joined #openstack-kolla | 01:53 | |
| openstackgerrit | Hongbin Lu proposed openstack/kolla master: Zun: add zun-cni-daemon image https://review.opendev.org/708273 | 02:36 |
| *** factor has quit IRC | 02:52 | |
| *** ricolin_ has joined #openstack-kolla | 03:00 | |
| *** diurnalist has joined #openstack-kolla | 03:17 | |
| *** diurnalist has quit IRC | 03:22 | |
| *** zhanglong has joined #openstack-kolla | 03:38 | |
| *** ykarel|away is now known as ykarel | 03:45 | |
| *** xinliang has quit IRC | 04:05 | |
| *** skramaja has joined #openstack-kolla | 04:15 | |
| *** negronjl has quit IRC | 04:29 | |
| *** jimcrowleyibm has quit IRC | 04:32 | |
| *** negronjl has joined #openstack-kolla | 04:32 | |
| *** jimcrowleyibm has joined #openstack-kolla | 04:37 | |
| *** jimcrowleyibm is now known as jimcrowleyibm[aw | 04:37 | |
| *** jimcrowleyibm has joined #openstack-kolla | 04:39 | |
| *** jimcrowleyibm is now known as jimcrowleyibm[aw | 04:39 | |
| *** jimcrowleyibm has joined #openstack-kolla | 04:41 | |
| *** jimcrowleyibm is now known as jimcrowleyibm[aw | 04:41 | |
| *** jimcrowleyibm[aw is now known as jimcrowleyibm | 04:41 | |
| *** jimcrowleyibm is now known as jimcrowleyibm[aw | 04:42 | |
| *** ricolin_ has quit IRC | 05:05 | |
| *** ricolin_ has joined #openstack-kolla | 05:05 | |
| *** ricolin_ has quit IRC | 05:05 | |
| *** k_mouza has joined #openstack-kolla | 05:32 | |
| *** evrardjp has quit IRC | 05:35 | |
| *** evrardjp has joined #openstack-kolla | 05:36 | |
| *** k_mouza has quit IRC | 05:36 | |
| *** suff has joined #openstack-kolla | 05:55 | |
| *** skramaja has quit IRC | 06:38 | |
| *** pbing19 has joined #openstack-kolla | 06:39 | |
| *** skramaja has joined #openstack-kolla | 06:46 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: WIP: Add mariadb_clustercheck image https://review.opendev.org/710217 | 06:57 |
| *** suff has quit IRC | 07:07 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla master: neutron: Remove networking-ovn - part 2 https://review.opendev.org/713422 | 07:07 |
| *** suff has joined #openstack-kolla | 07:08 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 07:10 |
| openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add framework to deploy user-defined containers on seed https://review.opendev.org/709689 | 07:14 |
| bold | {"msg": "The conditional check 'inventory_hostname in groups['ceph-mon'] or inventory_hostname in groups['cinder-api'] or inventory_hostname in groups['cinder-volume'] or inventory_hostname in groups['cinder-scheduler'] or inventory_hostname in groups['cinder-backup']' failed. The error was: error while evaluating conditional (inventory_hostname in groups['ceph-mon'] or inventory_hostname in groups['cinder-api'] or inventory_hostname in | 07:15 |
| bold | groups['cinder-volume'] or inventory_hostname in groups['cinder-scheduler'] or inventory_hostname in groups['cinder-backup']): Unable to look up a name or access an attribute in template string ({% if inventory_hostname in groups['ceph-mon'] or inventory_hostname in groups['cinder-api'] or inventory_hostname in groups['cinder-volume'] or inventory_hostname in groups['cinder-scheduler'] or inventory_hostname in groups['cinder-backup'] %} True {% else | 07:15 |
| bold | %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'AnsibleUndefined' is not iterable\n\nThe error appears to be in '/usr/local/share/kolla-ansible/ansible/roles/cinder/tasks/deploy.yml': line 5, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- include_tasks: config.yml\n ^ here\n"} | 07:15 |
| bold | how to fix this? | 07:16 |
| mnasiadka | first use paste.openstack.org, then paste the output WITH playbook step name :-) | 07:23 |
| *** cah_link has joined #openstack-kolla | 07:30 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Add ansible-lint to tox https://review.opendev.org/694779 | 07:32 |
| mnasiadka | INFO:kolla.common.utils.horizon: scss/src/scanner.h:17:10: fatal error: pcre.h: No such file or directory | 07:38 |
| mnasiadka | ugh | 07:38 |
| yoctozepto | woot | 07:40 |
| yoctozepto | no day w/o failure ;o | 07:40 |
| yoctozepto | looks like horizon finally switch pyscss to pyscss2 | 07:46 |
| *** bold has quit IRC | 07:46 | |
| yoctozepto | and we have no binary wheels for it | 07:46 |
| yoctozepto | and fail the compilation | 07:46 |
| yoctozepto | oh well | 07:46 |
| *** boldo has joined #openstack-kolla | 07:47 | |
| yoctozepto | "Retrying the build without the C extension now." | 07:47 |
| yoctozepto | yeah, sure, and still try to compile C | 07:47 |
| yoctozepto | well done | 07:47 |
| *** boldo has quit IRC | 07:49 | |
| *** riuzen has joined #openstack-kolla | 08:06 | |
| riuzen | when we deploy baremetal using Ironic, did kolla-ansible also provide dhcp-server for PXE or we should create first? | 08:08 |
| *** bengates has joined #openstack-kolla | 08:09 | |
| *** pbing19 has quit IRC | 08:13 | |
| *** sorin-mihai has quit IRC | 08:13 | |
| *** pbing19 has joined #openstack-kolla | 08:13 | |
| *** yoctozepto9 has joined #openstack-kolla | 08:15 | |
| *** yoctozepto has quit IRC | 08:15 | |
| *** yoctozepto9 is now known as yoctozepto | 08:15 | |
| *** sorin-mihai has joined #openstack-kolla | 08:16 | |
| *** bold has joined #openstack-kolla | 08:16 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Fix Horizon build after switch to pyscss2 which lacks binary wheel https://review.opendev.org/713589 | 08:16 |
| yoctozepto | riuzen: k-a provides it | 08:17 |
| *** abdysn has joined #openstack-kolla | 08:27 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: DNM: No binary wheels https://review.opendev.org/713591 | 08:31 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: neutron: Remove networking-ovn - part 2 https://review.opendev.org/713422 | 08:31 |
| *** abdysn has quit IRC | 08:43 | |
| *** abdysn has joined #openstack-kolla | 08:43 | |
| *** rpittau|afk is now known as rpittau | 08:45 | |
| *** yoctozepto has quit IRC | 08:49 | |
| hrw | morning | 08:52 |
| *** strigazi has joined #openstack-kolla | 08:57 | |
| *** yoctozepto has joined #openstack-kolla | 08:59 | |
| mnasiadka | yoctozepto: just thinking, is infra going to rework linters job to not use tox? :) | 09:04 |
| mnasiadka | ah, they have a step to ensure tox is installed | 09:05 |
| mnasiadka | :) | 09:05 |
| mnasiadka | hrw, yoctozepto, mgoddard: anybody planning to fix missing pcre.h in horizon, or should I? | 09:06 |
| hrw | mnasiadka: 09:16 < openstackgerrit> Radosław Piliszek proposed openstack/kolla master: Fix Horizon build after switch to pyscss2 which lacks binary wheel https://review.opendev.org/713589 | 09:08 |
| patchbot | patch 713589 - kolla - Fix Horizon build after switch to pyscss2 which la... - 1 patch set | 09:08 |
| mnasiadka | ah, ok - me blind as always | 09:08 |
| *** ykarel is now known as ykarel|away | 09:08 | |
| hrw | mnasiadka: so do not be lazy and +2+w | 09:09 |
| mnasiadka | hrw: will do, once kolla-build-centos8/debian jobs will pass in zuul :) | 09:09 |
| *** zhanglong has quit IRC | 09:09 | |
| hrw | I trust zuul more then ;d | 09:10 |
| hrw | https://review.opendev.org/#/c/710229/ anyone? | 09:10 |
| patchbot | patch 710229 - kolla - version-check: handle more situations - 1 patch set | 09:10 |
| mgoddard | bold: hi. I think I have a fix for you | 09:12 |
| mgoddard | bold: which release/branch are you using? | 09:12 |
| mgoddard | bold: https://review.opendev.org/#/c/709090/ | 09:13 |
| patchbot | patch 709090 - kolla-ansible - Fix invalid JSON body in Elasticsearch API requests - 1 patch set | 09:13 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CI: Avoid tox https://review.opendev.org/713135 | 09:20 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CI: Avoid tox https://review.opendev.org/713135 | 09:22 |
| hrw | this one removes installing tox as well | 09:22 |
| mgoddard | yoctozepto, mnasiadka, hrw: https://review.opendev.org/#/c/709090 may fix bold's issue - care to take a look? | 09:28 |
| patchbot | patch 709090 - kolla-ansible - Fix invalid JSON body in Elasticsearch API requests - 1 patch set | 09:28 |
| hrw | +W | 09:30 |
| hrw | mnasiadka: https://review.opendev.org/#/c/713272/ | 09:31 |
| patchbot | patch 713272 - kolla-ansible - CentOS 8: Use master image tag by default - 2 patch sets | 09:31 |
| mgoddard | hrw: actually wait | 09:31 |
| mgoddard | publish job failed this morning | 09:31 |
| mgoddard | so master is still centos7 | 09:31 |
| hrw | ah. | 09:32 |
| riuzen | <yoctozepto> oke found it. | 09:32 |
| mgoddard | hrw: although, the other images were published. So we are in the strange position that all images except horizon are centos8 | 09:33 |
| hrw | ;D | 09:33 |
| hrw | mgoddard: it's master. images are random state by default | 09:34 |
| mnasiadka | mgoddard: I guess it doesn't really hurt horizon is c7 :) | 09:34 |
| mgoddard | probably not | 09:34 |
| mgoddard | I'm sure we'll soon find out | 09:34 |
| *** jbadiapa has joined #openstack-kolla | 09:36 | |
| *** admin0 has quit IRC | 09:39 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: elasticsearch: use proper repo for debian/aarch64 https://review.opendev.org/713598 | 09:43 |
| hrw | https://review.opendev.org/713135 + https://review.opendev.org/713598 == aarch64 working again | 09:43 |
| patchbot | patch 713135 - kolla - CI: Avoid tox - 27 patch sets | 09:43 |
| patchbot | patch 713598 - kolla - elasticsearch: use proper repo for debian/aarch64 - 1 patch set | 09:43 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Revert "mark bifrost as unbuildable" https://review.opendev.org/713506 | 09:44 |
| yoctozepto | hrw: actually only the 2nd one as we now install tox which brings us setuptools as well | 09:44 |
| hrw | yoctozepto: anyway we can merge both | 09:45 |
| hrw | yoctozepto: INFO:kolla.common.utils.elasticsearch:Get:2 https://artifacts.elastic.co/packages/5.x/apt stable/main amd64 elasticsearch all 5.6.16 [33.8 MB] | 09:46 |
| hrw | yoctozepto: INFO:kolla.common.utils.elasticsearch:Setting up elasticsearch (5.6.16) ... | 09:46 |
| hrw | yoctozepto: INFO:kolla.common.utils.elasticsearch:Successfully tagged master1232/debian-source-elasticsearch:9.1.0 | 09:47 |
| hrw | yoctozepto: the [arch=amd64] is a trick | 09:47 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: Combined fluentd fixes https://review.opendev.org/713519 | 09:49 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 09:49 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: DNM: No binary wheels https://review.opendev.org/713591 | 09:51 |
| yoctozepto | hrw: ah, I see you rebased to remove tox, well done | 09:52 |
| openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add framework to deploy user-defined containers on seed https://review.opendev.org/709689 | 09:52 |
| hrw | yoctozepto: avoid is avoid, right? :D | 09:53 |
| hrw | 2020-03-18 09:51:44.443903 | primary | INFO:kolla.common.utils.elasticsearch:Successfully tagged kolla/debian-source-elasticsearch:9.1.0 ;D | 09:56 |
| hrw | works on CI | 09:56 |
| *** bjoernt_ has joined #openstack-kolla | 09:58 | |
| *** yoctozepto has quit IRC | 09:59 | |
| *** yoctozepto has joined #openstack-kolla | 10:03 | |
| *** bjoernt has quit IRC | 10:04 | |
| yoctozepto | hrw: right | 10:05 |
| yoctozepto | sorry, folks, it seems my network connectivity is grumpy todya | 10:05 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Revert "mark bifrost as unbuildable" https://review.opendev.org/713506 | 10:05 |
| hrw | yoctozepto: working from home? | 10:06 |
| yoctozepto | hrw: yeah, but with uni vpn | 10:08 |
| yoctozepto | home isp is fine | 10:08 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Revert "mark bifrost as unbuildable" https://review.opendev.org/713506 | 10:10 |
| *** yoctozepto67 has joined #openstack-kolla | 10:11 | |
| yoctozepto67 | let's do it webchat | 10:12 |
| *** ChanServ sets mode: +o yoctozepto | 10:13 | |
| hrw | yoctozepto: in a need of videoconf you are? | 10:13 |
| *** yoctozepto2 has joined #openstack-kolla | 10:13 | |
| *** yoctozepto has quit IRC | 10:13 | |
| *** yoctozepto2 is now known as yoctozepto | 10:13 | |
| yoctozepto67 | hrw: no, why? | 10:15 |
| *** ChanServ sets mode: +o yoctozepto67 | 10:15 | |
| *** yoctozepto67 sets mode: -b *senk*!*@* | 10:16 | |
| *** ChanServ sets mode: -o yoctozepto67 | 10:16 | |
| hrw | joking | 10:16 |
| yoctozepto67 | hrw: suche ;p | 10:17 |
| *** tonythomas has joined #openstack-kolla | 10:19 | |
| hrw | https://review.opendev.org/#/c/713492/ any brave cores? | 10:21 |
| patchbot | patch 713492 - kolla (stable/stein) - kibana, nova-libvirt: handle it properly for non-x86 - 1 patch set | 10:21 |
| hrw | and https://review.opendev.org/#/c/713491 | 10:22 |
| patchbot | patch 713491 - kolla (stable/train) - kibana, nova-libvirt: handle it properly for non-x86 - 1 patch set | 10:22 |
| yoctozepto67 | hrw: is this final? as in both merged? | 10:22 |
| hrw | final | 10:22 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CI: Avoid tox https://review.opendev.org/713135 | 10:27 |
| openstackgerrit | Mark Goddard proposed openstack/kayobe master: Remove ceph block device support https://review.opendev.org/709376 | 10:28 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: CI: Avoid tox https://review.opendev.org/713135 | 10:29 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/train: CI: Install tox https://review.opendev.org/713616 | 10:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/train: CI: install tox https://review.opendev.org/713617 | 10:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/stein: CI: Install tox https://review.opendev.org/713618 | 10:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/stein: CI: install tox https://review.opendev.org/713619 | 10:33 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/rocky: CI: Install tox https://review.opendev.org/713620 | 10:35 |
| hrw | mgoddard: so you remember https://review.opendev.org/#/c/692996 patch? | 10:37 |
| patchbot | patch 692996 - kolla (stable/stein) - Stein: support mariabackup for backups (MERGED) - 1 patch set | 10:37 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible stable/rocky: CI: install tox https://review.opendev.org/713621 | 10:38 |
| mgoddard | hrw: yes | 10:38 |
| hrw | mgoddard: I have a feeling that it was incomplete. will have to dig more in stein commits | 10:39 |
| mgoddard | hrw: have you tried to use it? | 10:40 |
| hrw | mgoddard: https://bugs.launchpad.net/kolla/+bug/1867750 | 10:40 |
| openstack | Launchpad bug 1867750 in kolla "Stein Debian MariaDB cluster failed to deploy on multi-node" [Undecided,Confirmed] - Assigned to Marcin Juszkiewicz (hrw) | 10:40 |
| hrw | mgoddard: no mariabackup on aarch64. | 10:40 |
| mgoddard | hrw: :( | 10:42 |
| mgoddard | is the mariadb package a different version? | 10:42 |
| yoctozepto | hrw: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_539/713591/2/check-arm64/kolla-build-debian-source-aarch64/539853b/kolla/build/000_FAILED_openvswitch-netcontrold.log | 10:42 |
| hrw | mgoddard: 10.3 should be on all archs | 10:42 |
| yoctozepto | looks like this one went wheel in aarch64 | 10:42 |
| yoctozepto | as it did not fail previously | 10:42 |
| hrw | yoctozepto: this image is wrong on so many levels... | 10:43 |
| yoctozepto | sure, but the point is even on aarch64 some must have come from wheels | 10:43 |
| hrw | yoctozepto: or aarch64 was failing before we added that image | 10:44 |
| hrw | building locally | 10:45 |
| hrw | INFO:kolla.common.utils.openvswitch-netcontrold:Collecting netcontrold | 10:47 |
| hrw | INFO:kolla.common.utils.openvswitch-netcontrold: Downloading https://files.pythonhosted.org/packages/60/fc/171a207c50add7eaabb81cefb6c18b5cef82836db7184fd1de42b7fa9cb8/netcontrold-1.0.15-py3-none-any.whl | 10:47 |
| hrw | INFO:kolla.common.utils.openvswitch-netcontrold:Installing collected packages: netcontrold | 10:47 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix invalid JSON body in Elasticsearch API requests https://review.opendev.org/709090 | 10:47 |
| hrw | yoctozepto: ok. I was not exact enough. | 10:48 |
| hrw | yoctozepto: on aarch64 we do binary wheels for 'arch:none' ones and build/compile all arch specific ones | 10:48 |
| stingrayza | hi all o/ | 10:49 |
| openstackgerrit | Mark Goddard proposed openstack/kolla stable/train: CentOS 8: Build bifrost image https://review.opendev.org/713509 | 10:54 |
| yoctozepto67 | hrw: ack, thanks, makes sense | 11:01 |
| *** Fl1nt has joined #openstack-kolla | 11:03 | |
| hrw | yoctozepto67: arch:none are simply python text files | 11:04 |
| *** k_mouza has joined #openstack-kolla | 11:04 | |
| yoctozepto67 | hrw: well, theoretically you could require some external preprocessor or something | 11:05 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: elasticsearch: use proper repo for debian/aarch64 https://review.opendev.org/713598 | 11:09 |
| hrw | on top of horizon | 11:09 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Custom haproxy script for monitoring galera https://review.opendev.org/710213 | 11:12 |
| mnasiadka | hrw: everything today is on top of horizon :D | 11:26 |
| hrw | yesterday was on top of bifrost | 11:27 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 11:29 |
| openstackgerrit | Mark Goddard proposed openstack/kolla master: Revert "mark bifrost as unbuildable" https://review.opendev.org/713506 | 11:34 |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 11:35 |
| openstackgerrit | Merged openstack/kolla stable/stein: kibana, nova-libvirt: handle it properly for non-x86 https://review.opendev.org/713492 | 11:37 |
| *** k_mouza has quit IRC | 11:40 | |
| *** k_mouza has joined #openstack-kolla | 11:44 | |
| *** ivve has joined #openstack-kolla | 11:47 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 11:49 |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 11:49 |
| *** diurnalist has joined #openstack-kolla | 11:49 | |
| *** diurnalist has quit IRC | 11:54 | |
| mnasiadka | yoctozepto67: https://bugs.launchpad.net/kolla-ansible/+bug/1731142 - I think we dumped clusterer in stein in favor of some better method? | 11:56 |
| openstack | Launchpad bug 1731142 in kolla-ansible rocky "The extended control node is rabbitmq error reporting" [Low,Confirmed] | 11:56 |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 11:58 |
| *** rpittau is now known as rpittau|bbl | 12:01 | |
| hrw | yoctozepto67: horizon will get binary wheels for pyscss2 soon | 12:05 |
| *** pbing19 has quit IRC | 12:05 | |
| hrw | kolla-build-debian-source-aarch64 SUCCESS in 44m 58s (non-voting) | 12:05 |
| hrw | https://review.opendev.org/#/c/713598/ ready for +W :D | 12:05 |
| patchbot | patch 713598 - kolla - elasticsearch: use proper repo for debian/aarch64 - 2 patch sets | 12:05 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 12:05 |
| *** zhanglong has joined #openstack-kolla | 12:10 | |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 12:11 |
| *** jbadiapa has quit IRC | 12:19 | |
| mnasiadka | hrw: so just +w, you have 2 +2s ;) | 12:22 |
| hrw | mnasiadka: I tend to not +W own patches ;D | 12:23 |
| mnasiadka | oh boy, +w'ed ;) | 12:23 |
| hrw | probably wrong but that's me | 12:24 |
| *** k_mouza has quit IRC | 12:32 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla stable/train: Fix Horizon build after switch to pyscss2 which lacks binary wheel https://review.opendev.org/713644 | 12:39 |
| *** yoctozepto67 has quit IRC | 12:46 | |
| *** k_mouza has joined #openstack-kolla | 12:47 | |
| openstackgerrit | Merged openstack/kolla master: Fix Horizon build after switch to pyscss2 which lacks binary wheel https://review.opendev.org/713589 | 12:50 |
| openstackgerrit | Jie Li proposed openstack/kolla-ansible stable/rocky: add region_name if enable barbican https://review.opendev.org/713646 | 12:53 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla stable/stein: mariadb: install mariabackup on Debian https://review.opendev.org/713647 | 12:54 |
| *** rpittau|bbl is now known as rpittau | 13:01 | |
| openstackgerrit | Merged openstack/kolla stable/train: kibana, nova-libvirt: handle it properly for non-x86 https://review.opendev.org/713491 | 13:02 |
| *** obre has quit IRC | 13:08 | |
| *** skramaja has quit IRC | 13:14 | |
| *** obre has joined #openstack-kolla | 13:15 | |
| *** vishalmanchanda has joined #openstack-kolla | 13:19 | |
| openstackgerrit | Jie Li proposed openstack/kolla-ansible stable/rocky: add region_name if enable barbican https://review.opendev.org/713646 | 13:21 |
| *** jbadiapa has joined #openstack-kolla | 13:24 | |
| mnasiadka | yoctozepto: do we have any haproxy logs anywhere in CI? I see it's in theory configured to send syslog messages to {{ api_interface_address }} | 13:26 |
| yoctozepto | mnasiadka: yeah, but before fluentd fix it went nowhere | 13:30 |
| yoctozepto | mnasiadka: should be there now | 13:30 |
| mnasiadka | yoctozepto: where? logs/haproxy ? | 13:30 |
| mnasiadka | ah, we don't start fluentd in mariadb CI scenario | 13:31 |
| *** zhanglong has quit IRC | 13:31 | |
| *** zhanglong has joined #openstack-kolla | 13:32 | |
| openstackgerrit | Jie Li proposed openstack/kolla-ansible master: add region_name if enable barbican https://review.opendev.org/713652 | 13:34 |
| *** TrevorV has joined #openstack-kolla | 13:36 | |
| *** umbSublime_ has joined #openstack-kolla | 13:42 | |
| openstackgerrit | Merged openstack/kolla master: neutron: Remove networking-ovn - part 2 https://review.opendev.org/713422 | 13:42 |
| openstackgerrit | Merged openstack/kolla stable/stein: CI: Install tox https://review.opendev.org/713618 | 13:42 |
| openstackgerrit | Merged openstack/kolla stable/rocky: CI: Install tox https://review.opendev.org/713620 | 13:42 |
| *** k_mouza has quit IRC | 13:43 | |
| *** spiette_ has joined #openstack-kolla | 13:43 | |
| *** spiette has quit IRC | 13:44 | |
| *** umbSublime has quit IRC | 13:44 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: OVN Support https://review.opendev.org/696841 | 13:46 |
| mnasiadka | networking-ovn begone | 13:46 |
| *** k_mouza has joined #openstack-kolla | 13:48 | |
| *** jimcrowleyibm has joined #openstack-kolla | 13:53 | |
| *** jimcrowleyibm has joined #openstack-kolla | 13:54 | |
| yoctozepto | mnasiadka: then we rather should | 13:55 |
| *** zhanglong has quit IRC | 13:57 | |
| *** zhanglong has joined #openstack-kolla | 13:58 | |
| *** diurnalist has joined #openstack-kolla | 14:02 | |
| *** zhanglong has quit IRC | 14:02 | |
| *** pbing19 has joined #openstack-kolla | 14:08 | |
| *** vedup19 has joined #openstack-kolla | 14:10 | |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 14:11 |
| *** yoctozepto68 has joined #openstack-kolla | 14:11 | |
| yoctozepto68 | arar | 14:11 |
| *** yoctozepto1 has joined #openstack-kolla | 14:12 | |
| mnasiadka | yoctozepto: yeah | 14:12 |
| *** yoctozepto has quit IRC | 14:13 | |
| *** yoctozepto1 is now known as yoctozepto | 14:13 | |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 14:13 |
| *** pbing19 has quit IRC | 14:14 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable fluentd in MariaDB jobs https://review.opendev.org/713663 | 14:16 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable fluentd in MariaDB jobs https://review.opendev.org/713663 | 14:19 |
| mnasiadka | yoctozepto68: I see it would make sense to handle per-job globals in a different way, e.g. changing tests/templates/globals-default.j2 does not trigger mariadb job in this case - maybe we should move that to Zuul vars? | 14:21 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: Custom haproxy script for monitoring galera https://review.opendev.org/710213 | 14:26 |
| openstackgerrit | Merged openstack/kolla master: elasticsearch: use proper repo for debian/aarch64 https://review.opendev.org/713598 | 14:27 |
| yoctozepto68 | mnasiadka: yeah, I consider it part of a bigger problem of testing sometimes too much, sometimes not enough | 14:27 |
| mnasiadka | yoctozepto68: let's focus on ,,not enough'' | 14:27 |
| mnasiadka | :) | 14:27 |
| yoctozepto68 | mnasiadka: yeah, we can do like devstack and interpret some pattern from zuul vars | 14:28 |
| yoctozepto68 | mnasiadka: would help in this particular case | 14:28 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Enable fluentd in MariaDB jobs https://review.opendev.org/713663 | 14:29 |
| *** sluna has quit IRC | 14:29 | |
| mnasiadka | yoctozepto68: I was rather thinking of including something like {{globals_extra}} in the template, and feed this var to zuul base job configs - I'll try to post something tomorrow | 14:29 |
| *** sluna has joined #openstack-kolla | 14:29 | |
| yoctozepto68 | mnasiadka: good, I'm open for all ideas | 14:30 |
| *** yoctozepto68 has quit IRC | 14:31 | |
| hrw | https://review.opendev.org/#/c/713616/ +1 from zuul | 14:34 |
| patchbot | patch 713616 - kolla (stable/train) - CI: Install tox - 1 patch set | 14:34 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CI: Avoid tox https://review.opendev.org/713135 | 14:35 |
| hrw | horizon rebase | 14:35 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: DNM: No binary wheels https://review.opendev.org/713591 | 14:37 |
| hrw | yoctozepto: added make | 14:37 |
| yoctozepto | hrw: ++ | 14:39 |
| hrw | yoctozepto: neutron-base needs binary wheels | 14:39 |
| hrw | textfsm is wheel only. no tarbakk | 14:39 |
| Fl1nt | Hum, I just stumble upon the following error deploying rally: standard_init_linux.go:211: exec user process caused “exec format error” | 14:41 |
| Fl1nt | I build the rally image | 14:41 |
| Fl1nt | from CentOS 7 binary package | 14:41 |
| hrw | Fl1nt: wrong arch | 14:41 |
| Fl1nt | using openstack-rally-1.1.0-2.el7 | 14:41 |
| Fl1nt | Oo | 14:41 |
| Fl1nt | can't be | 14:42 |
| Fl1nt | I build the image using x86_64 | 14:42 |
| hrw | log please | 14:42 |
| yoctozepto | hrw: lol, non-open-source? woot? | 14:42 |
| hrw | yoctozepto: https://github.com/google/textfsm | 14:44 |
| Fl1nt | @hrw, kolla-build log ? | 14:44 |
| hrw | Fl1nt: yes | 14:44 |
| Fl1nt | ok, hold on, I need to trigger a build (CICD stuff all around...). | 14:44 |
| Fl1nt | from kolla-build.conf base_arch = x86_64 | 14:45 |
| Fl1nt | all other images such as elasticsearch nova etc are fine | 14:45 |
| hrw | Fl1nt: you may skip setting base_arch. it is assumed to be native | 14:45 |
| Fl1nt | ok, cool, another default to get ride of ^^ lovin it :D | 14:46 |
| hrw | Fl1nt: you need distro/install_type and that's basically all. logs-dir is handy | 14:48 |
| hrw | Fl1nt: https://github.com/hrw/docker-utils/blob/master/kolla-build-images.sh | 14:48 |
| hrw | Fl1nt: clean_package_cache=false is good for CI builds. images are bigger but build faster | 14:48 |
| yoctozepto | hrw: I just realized we might be getting arbitrarily lower versions with wheels turned off, especially regarding transitive deps | 14:50 |
| yoctozepto | oh what a mess | 14:51 |
| hrw | yoctozepto: ? how | 14:51 |
| Fl1nt | @hrw, yep, already activated the clean package option as we have more than required room for our build and building faster is cool as for now it take us around 30mins per build. | 14:52 |
| hrw | which takes me to finishing patch for opendev ci doing same | 14:52 |
| Fl1nt | Is the OpenDev CI zuul ? | 14:54 |
| hrw | yes | 14:55 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CI: do not clean package cache if not publishing https://review.opendev.org/713670 | 14:55 |
| yoctozepto | hrw: if you can skip publishing tarball and publish only-wheel versions | 14:56 |
| yoctozepto | and then run --no-binary | 14:56 |
| hrw | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi: meeting soon | 14:56 |
| Fl1nt | At some point, I'll have to look closer at zuul, it's a topic of interest but I never had time to look at it closer. | 14:57 |
| hrw | Fl1nt: you use jenkins? | 14:57 |
| Fl1nt | yep | 14:57 |
| hrw | Fl1nt: jenkins job builder or config-in-jenkins? | 14:57 |
| Fl1nt | Jenkins pipeline multi-branch | 14:57 |
| hrw | yoctozepto: they have issue opened about lack of tarball | 14:58 |
| hrw | hm. three meetings to attend in 2 minutes | 14:58 |
| yoctozepto | hrw: sure, but any pypi project can skip tarball | 14:58 |
| yoctozepto | hrw: three? how so? | 14:58 |
| hrw | kolla, horizon and company... | 14:58 |
| yoctozepto | oh my | 14:58 |
| yoctozepto | hrw now horizon core? | 14:59 |
| hrw | yoctozepto: no. improved CI for pyscss2 today | 14:59 |
| *** priteau has joined #openstack-kolla | 15:00 | |
| mgoddard | #startmeeting kolla | 15:00 |
| openstack | Meeting started Wed Mar 18 15:00:23 2020 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: kolla)" | 15:00 | |
| openstack | The meeting name has been set to 'kolla' | 15:00 |
| mgoddard | #topic rollcall | 15:01 |
| *** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:01 | |
| mgoddard | \o | 15:01 |
| osmanlicilegi | o/ | 15:01 |
| stackedsax | \o | 15:01 |
| hrw | o/ | 15:02 |
| yoctozepto | o/ | 15:02 |
| *** vedup19 has quit IRC | 15:02 | |
| yoctozepto | sorry for breaking the chain | 15:02 |
| yoctozepto | :D | 15:02 |
| generalfuzz | o/ | 15:02 |
| mgoddard | #topic agenda | 15:02 |
| *** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:02 | |
| mgoddard | * Roll-call | 15:02 |
| mgoddard | * Announcements | 15:02 |
| mgoddard | * Review action items from last meeting | 15:03 |
| mgoddard | * CI status | 15:03 |
| mgoddard | * Kolla --SIG-- (aka Kolla Klub?) https://etherpad.openstack.org/p/kolla-sig | 15:03 |
| mgoddard | * Ussuri release planning (kayobe) | 15:03 |
| mgoddard | * Ussuri release planning (kolla & kolla ansible) | 15:03 |
| mgoddard | * Stein/Train backports | 15:03 |
| mgoddard | - https://bugs.launchpad.net/kolla/+bug/1867365 - on aarch64 we lack mariabackup in mariadb image. Train has it as we dropped Percona. Backporting to Stein == dropping Percona which makes a change on x86-64 or adding 'if !x86 then install mariadb-backup' block. | 15:03 |
| openstack | Launchpad bug 1867365 in kolla train "nova-libvirt needs fixed libvirt on AArch64 servers with ThunderX cpu" [High,Fix committed] - Assigned to Marcin Juszkiewicz (hrw) | 15:03 |
| mgoddard | #topic announcements | 15:03 |
| *** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:03 | |
| hrw | mariadb part handled by https://review.opendev.org/713647 basically | 15:03 |
| patchbot | patch 713647 - kolla (stable/stein) - mariadb: install mariabackup on Debian - 1 patch set | 15:03 |
| mgoddard | #info Kolla CentOS 8 images now tagged as master on master | 15:03 |
| yoctozepto | except horizon | 15:03 |
| mgoddard | true | 15:03 |
| yoctozepto | but it looks worky | 15:04 |
| mgoddard | Any other announcements? | 15:04 |
| mgoddard | #topic Review action items from last meeting | 15:04 |
| *** openstack changes topic to "Review action items from last meeting (Meeting topic: kolla)" | 15:04 | |
| mgoddard | mgoddard to raise a bug on Ceph job fails | 15:04 |
| mgoddard | mnasiadka to look into Ceph job fails | 15:04 |
| mgoddard | I did not do mine | 15:04 |
| yoctozepto | but mnasiadka did his | 15:05 |
| mnasiadka | Fixed | 15:05 |
| yoctozepto | at least master got nice | 15:05 |
| yoctozepto | worse for older branches still | 15:05 |
| yoctozepto | so not fully | 15:05 |
| mgoddard | yeah, I think this was about kolla ceph not ceph-ansible? | 15:05 |
| mnasiadka | Yeah, need some more time on this | 15:05 |
| yoctozepto | something went wrong regarding our ceph deploys | 15:05 |
| yoctozepto | yeah, mgoddard ++ | 15:05 |
| mnasiadka | mgoddard: but c-a also broke :) | 15:05 |
| mgoddard | indeed | 15:05 |
| yoctozepto | indeed :D | 15:05 |
| mgoddard | #action mgoddard to raise a bug on Ceph job fails | 15:06 |
| mgoddard | #action mnasiadka to look into Ceph job fails | 15:06 |
| mgoddard | #topic CI status | 15:06 |
| *** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:06 | |
| mgoddard | We had a horizon issue, has the fix merged yet? | 15:07 |
| hrw | yes | 15:07 |
| hrw | master done, train in queue | 15:07 |
| yoctozepto | train not broken | 15:07 |
| yoctozepto | as we pin releases | 15:07 |
| yoctozepto | so merge any time | 15:07 |
| hrw | horizon merges pyscss2 into train... | 15:08 |
| yoctozepto | hrw: but we have pinned releases | 15:08 |
| hrw | so once we update we break | 15:08 |
| yoctozepto | hrw: we will be blocked from updating :-) | 15:08 |
| mgoddard | Otherwise we're good | 15:08 |
| yoctozepto | seems so | 15:08 |
| mgoddard | Kayobe seems green | 15:08 |
| openstackgerrit | Grzegorz Sikorski proposed openstack/kolla-ansible stable/rocky: rabbitmq: update cluster version on reconfigure https://review.opendev.org/713628 | 15:09 |
| hrw | yoctozepto: right | 15:09 |
| yoctozepto | rocky nearing em | 15:09 |
| yoctozepto | but folks sending patches ;D | 15:09 |
| mgoddard | #topic Kolla --SIG-- (aka Kolla Klub?) https://etherpad.openstack.org/p/kolla-sig | 15:09 |
| *** openstack changes topic to "Kolla --SIG-- (aka Kolla Klub?) https://etherpad.openstack.org/p/kolla-sig (Meeting topic: kolla)" | 15:09 | |
| yoctozepto | let's kickstart this soon | 15:09 |
| mgoddard | +1 | 15:09 |
| mgoddard | I think we have around 20 people signed up now | 15:09 |
| mgoddard | #action mgoddard to spam people about kolla SIG | 15:10 |
| mgoddard | Will try to get a few more signed up | 15:10 |
| mgoddard | Then doodle poll for a slot for first meeting | 15:10 |
| Fl1nt | can I join the SIG part of the meeting? | 15:11 |
| hrw | Fl1nt: anyone can | 15:11 |
| mgoddard | Fl1nt: anyone can join in the meeting | 15:11 |
| Fl1nt | I'm in for this part so. | 15:11 |
| mgoddard | great to hear | 15:11 |
| cosmicsound | o/ | 15:12 |
| yoctozepto | Fl1nt: any part of the meeting :-) as long as on topic | 15:12 |
| Fl1nt | Ok, noticed that. | 15:12 |
| mgoddard | I don't think we have too much more to say on it at the moment - just need to get it started | 15:13 |
| mgoddard | #topic Ussuri release planning (kayobe) | 15:13 |
| *** openstack changes topic to "Ussuri release planning (kayobe) (Meeting topic: kolla)" | 15:13 | |
| *** dougsz has joined #openstack-kolla | 15:14 | |
| mgoddard | priteau, dougsz: \o | 15:15 |
| priteau | o/ | 15:15 |
| dougsz | \o/ | 15:15 |
| mgoddard | we are about 5-6 weeks from feature freeze | 15:15 |
| openstackgerrit | Merged openstack/kolla-ansible stable/stein: Combined fluentd fixes https://review.opendev.org/713519 | 15:16 |
| openstackgerrit | Merged openstack/kolla-ansible stable/rocky: Wait for MariaDB to be accessible via HAProxy https://review.opendev.org/713500 | 15:16 |
| mgoddard | so we need to be finishing things off fairly soon | 15:16 |
| mgoddard | we have quite a few patches in flight needing reviews | 15:16 |
| dougsz | sorry, are we talking kayobe or kolla here? | 15:16 |
| mgoddard | topic: Ussuri release planning (kayobe) | 15:16 |
| dougsz | will look at review list later | 15:17 |
| priteau | I saw you flagged some patches as Review-Priority, I will try to review ASAP | 15:17 |
| mgoddard | thanks | 15:17 |
| mgoddard | right, just trying to provide some guidance on priority | 15:17 |
| mgoddard | do either of you have features you would like to be included in ussuri? | 15:18 |
| mgoddard | will there be more cells work in kayobe? | 15:18 |
| priteau | The nclu patch might be nice to have | 15:18 |
| mgoddard | +1 | 15:18 |
| dougsz | not sure yet about cells, focus is on kolla-ansible patches | 15:19 |
| yoctozepto | ( dougsz: yeah, I switched order to let you have more time in case we get too chatty ) | 15:19 |
| mgoddard | good thinking | 15:20 |
| mgoddard | I think jovial was going to look at customising kayobe commands, but he's not here | 15:21 |
| *** jovial[m] has joined #openstack-kolla | 15:21 | |
| mgoddard | wow, summoned by name | 15:21 |
| mgoddard | jovial[m]: hi, welcome back | 15:21 |
| *** vedup19 has joined #openstack-kolla | 15:21 | |
| mgoddard | are you still planning to look at customising kayobe commands? | 15:22 |
| jovial[m] | hi, apologies for missing the start, got carried away writing some docs! | 15:22 |
| jovial[m] | yes, I am still keen on looking at that :) | 15:23 |
| mgoddard | ok. As I mentioned earlier we are about 5-6 weeks from feature freeze | 15:23 |
| jovial[m] | no progress to report yet though | 15:23 |
| mgoddard | which will no doubt go quite quickly | 15:23 |
| mgoddard | no problem, I know you've been busy | 15:23 |
| mgoddard | Thanks, let's move onto kolla. | 15:24 |
| mgoddard | #topic Ussuri release planning (kolla & kolla ansible) | 15:24 |
| *** openstack changes topic to "Ussuri release planning (kolla & kolla ansible) (Meeting topic: kolla)" | 15:24 | |
| mgoddard | I'll kick off with CentOS 8 | 15:24 |
| mgoddard | I added some acceptance criteria to https://etherpad.openstack.org/p/kolla-centos8 | 15:24 |
| mgoddard | So we'll have a better idea of when we're finished | 15:25 |
| mgoddard | Please read through and amend or add as necessary | 15:25 |
| mgoddard | I think kolla is quite close now - just a few last images to tidy up | 15:25 |
| mgoddard | kolla ansible also not too far off | 15:26 |
| *** abdysn has quit IRC | 15:26 | |
| mgoddard | Documentation and release notes still required | 15:26 |
| mgoddard | And some train backprots | 15:27 |
| mgoddard | That's probably all I have to say on it | 15:27 |
| mgoddard | Any thoughts? | 15:28 |
| mgoddard | Anything missing from acceptance criteria? | 15:28 |
| * yoctozepto thinking | 15:29 | |
| *** vishalmanchanda has quit IRC | 15:31 | |
| jovial[m] | looks reasonable to me at first glance | 15:32 |
| yoctozepto | ++ | 15:32 |
| yoctozepto | no more ideas, I reworded some parts and switched order | 15:33 |
| yoctozepto | we are looking good | 15:33 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: Revert "openstack-base: pin setuptools < 46 to get horizon working" https://review.opendev.org/713681 | 15:33 |
| mgoddard | cool | 15:34 |
| mgoddard | Anyone interested in picking up any remaining CentOS 8 work? | 15:34 |
| mgoddard | I thought that would make it go quiet | 15:35 |
| mgoddard | ok, let's move on | 15:35 |
| mgoddard | Any other Ussuri features we need to discuss? | 15:36 |
| generalfuzz | I would like to have a review of https://review.opendev.org/#/c/712005 - first pass at backend TLS implemented for keystone using uwsgi web server | 15:36 |
| patchbot | patch 712005 - kolla-ansible - Add support for encrypting backend HAProxy traffic - 9 patch sets | 15:36 |
| mgoddard | that's a reasonable request | 15:37 |
| mgoddard | what has been your experience of uwsgi so far generalfuzz ? | 15:38 |
| generalfuzz | mostly fine and straight forward. There are a number of peer reset exceptions always show in the log, but functionality seems to work | 15:38 |
| generalfuzz | the "peer reset" seems to be a common issue with uwsgi and tls | 15:39 |
| generalfuzz | we will need to make uwsgi available in tripleo builds | 15:40 |
| mgoddard | tripleo can provide config to remove the uwsgi package if they wish | 15:40 |
| mgoddard | what is the cause of the peer reset error? | 15:41 |
| yoctozepto | peer reset sounds bad | 15:41 |
| generalfuzz | it does, but seems like many people hit this with no functionality issues | 15:41 |
| generalfuzz | still looking into it | 15:41 |
| generalfuzz | it doesn't happen when using http - only https | 15:42 |
| yoctozepto | hmm | 15:43 |
| generalfuzz | for example - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_122/712005/9/check/kolla-ansible-ubuntu-source/12291c8/primary/logs/kolla/keystone/keystone-admin.txt | 15:43 |
| mgoddard | I was thinking we should send something to the ML about experience with uwsgi etc | 15:43 |
| generalfuzz | [uwsgi-http key: client_addr: 192.0.2.1 client_port: 42150] hr_ssl_read(): Connection reset by peer [plugins/http/https.c line 393] | 15:43 |
| yoctozepto | weird if only tls triggers it | 15:44 |
| yoctozepto | uwsgi bug? | 15:44 |
| generalfuzz | or possible configuration issue, but not an obvious one | 15:45 |
| mgoddard | do we have a suitable package for uwsgi? | 15:45 |
| yoctozepto | well, it blames peer | 15:45 |
| yoctozepto | so could be clients doing something wrong with tls | 15:46 |
| yoctozepto | https://extras.getpagespeed.com/release-el8-latest.rpm <- this looks bad bad bad | 15:46 |
| Fl1nt | or the server doing something wrong and triggering the client to reset (something like improper handle and downgrade). | 15:46 |
| stackedsax | would the tripleo build issue prevent generalfuzz' work from getting accepted? | 15:47 |
| yoctozepto | Fl1nt: indeed | 15:47 |
| generalfuzz | yoctozepto: that is for tripleo | 15:47 |
| mgoddard | where does the package come from for non-tripleo? | 15:48 |
| yoctozepto | generalfuzz: for binary in general you mean | 15:48 |
| yoctozepto | mgoddard: for source from pypi | 15:48 |
| yoctozepto | https://review.opendev.org/#/c/710879/17/docker/keystone/keystone-base/Dockerfile.j2 | 15:48 |
| patchbot | patch 710879 - kolla - Install uwsgi for Keystone - 17 patch sets | 15:48 |
| yoctozepto | all look here | 15:48 |
| generalfuzz | yoctozepto: rhel 8 doesn't have uwsgi in epel, so this is the workaround | 15:49 |
| mgoddard | we don't need to worry too much about that | 15:49 |
| generalfuzz | I will remove it then | 15:50 |
| mgoddard | we don't really test RHEL on kolla-ansible | 15:50 |
| yoctozepto | mgoddard: I think generalfuzz also meant centos8 | 15:50 |
| mgoddard | if base_distro == 'rhel' | 15:50 |
| Fl1nt | @mgoddard, working with a restricted infra, such package is going to be a nightmare for companies with high security measures and containments. it won't pass any COTS. | 15:51 |
| yoctozepto | mgoddard: ah, missed | 15:51 |
| mgoddard | Fl1nt: because it is from EPEL? | 15:51 |
| mgoddard | or do you mean https://extras.getpagespeed.com/release-el8-latest.rpm ? | 15:52 |
| mgoddard | we can drop that | 15:52 |
| Fl1nt | because the epel repo installer is from getpagespeed.com | 15:52 |
| yoctozepto | 😂 | 15:53 |
| Fl1nt | it can be override, but still, having that kind of urls triggers a lot of alerts when screening the dockerfile template in here ^^ then I need to introspect them and either validate them with the SOC team etc. | 15:54 |
| mgoddard | I mean we will drop that from the patch | 15:54 |
| Fl1nt | oh ok cool ^^ | 15:54 |
| mgoddard | it's only for RHEL, and we don't really test RHEL for kolla ansible | 15:54 |
| mgoddard | tripleo use httpd/mod_wsgi | 15:55 |
| mgoddard | which of course means we would need both in our images :( | 15:55 |
| mgoddard | any idea how big uwsgi and its dependencies are? | 15:55 |
| yoctozepto | I'm a bit confused still | 15:55 |
| Fl1nt | I personally don't use tripleO but RHEL+kolla with some customers requiring professional agreements etc. | 15:55 |
| yoctozepto | I thought we disabled epel by default | 15:55 |
| mgoddard | true, we do | 15:56 |
| yoctozepto | then how come uwsgi gets installed 😂 | 15:56 |
| mgoddard | comes from delorean-master-testing | 15:57 |
| yoctozepto | INFO:kolla.common.utils.keystone-base: uwsgi x86_642.0.18-4.el8 delorean-master-testing 385 k | 15:57 |
| mgoddard | https://3f2e3a89d291731c913a-0da77774ba1e870a646f19d85e727a87.ssl.cf2.rackcdn.com/710879/17/check/kolla-build-centos8-binary/7520d7e/kolla/build/keystone-base.log | 15:57 |
| yoctozepto | mgoddard: yeah, I pasted it for all | 15:57 |
| yoctozepto | so they do care about uwsgi | 15:57 |
| mgoddard | seems so | 15:57 |
| yoctozepto | just rhel not having it *yet* | 15:58 |
| mgoddard | wonder why it is in RDO but not RHOSP | 15:58 |
| yoctozepto | rhosp always behind | 15:58 |
| yoctozepto | still, it's delorean for ussuri | 15:58 |
| hrw | pyscss author responded ;D horizon developer and I may became comaintainers | 15:58 |
| yoctozepto | dlrn-master-rhel8-deps 1.4 MB/s | 2.4 MB 00:01 | 15:59 |
| yoctozepto | hrw: ++ but now that it moved to 2, are you going to move it back to no 2? | 15:59 |
| hrw | yoctozepto: yes | 15:59 |
| yoctozepto | looks like rhel8 also uses delorean so should be aligned... hmm | 16:00 |
| mgoddard | rhel8 builds are broken generally atm | 16:00 |
| mgoddard | Anyway, we're out of time | 16:00 |
| mgoddard | Thanks all | 16:00 |
| mgoddard | #action mgoddard to ask ML about uwsgi etc | 16:00 |
| mgoddard | #endmeeting | 16:00 |
| *** openstack changes topic to "Please feel welcome to join Kolla SIG https://etherpad.openstack.org/p/kolla-sig | Kolla IRC meetings on Wednesdays @ 15:00 UTC - see agenda @ https://goo.gl/OXB0DL | Whiteboard: https://etherpad.openstack.org/p/KollaWhiteBoard | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:00 | |
| openstack | Meeting ended Wed Mar 18 16:00:50 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-18-15.00.html | 16:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-18-15.00.txt | 16:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-03-18-15.00.log.html | 16:00 |
| hrw | thanks | 16:00 |
| generalfuzz | thanks | 16:01 |
| osmanlicilegi | thanks | 16:01 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Install uwsgi for Keystone https://review.opendev.org/710879 | 16:01 |
| yoctozepto | looking at no rhel override now to see where the problem pops up | 16:02 |
| yoctozepto | also, we probably want ubu/deb to have uwsgi too | 16:02 |
| yoctozepto | probably openstack-base should have uwsgi as it's going to be 'the server' to use | 16:02 |
| yoctozepto | mgoddard, generalfuzz | 16:02 |
| yoctozepto | if you care to continue this discussion for a moment :-) | 16:03 |
| stackedsax | I'm just a little concerned that the uwsgi work is moving the goalposts for generalfuzz's TLS work | 16:03 |
| yoctozepto | stackedsax: you mean extra delay? | 16:04 |
| yoctozepto | that's true unfortunately | 16:04 |
| stackedsax | yes. can we not take things one step at a time? | 16:04 |
| yoctozepto | stackedsax: I agree, I am also worried about that peer reset, it does not sound nice at all | 16:05 |
| stackedsax | seems like things were reasonable with the approach generalfuzz had with wsgi. | 16:05 |
| yoctozepto | stackedsax: apache is known to handle http(s) well ;p | 16:06 |
| stackedsax | introducing uwsgi at this point seems like it is destined to postpone the TLS story for a long time. | 16:06 |
| *** dking_desktop has joined #openstack-kolla | 16:07 | |
| yoctozepto | stackedsax: yeah, I'm willing to continue discussion but seems mgoddard and generalfuzz are gone | 16:07 |
| stackedsax | it seems like a more practical approach would be to handle the movement to uwsgi as a separate story | 16:08 |
| generalfuzz | still here :) | 16:08 |
| Fl1nt | @hrw, ok, so, I've just finished to rebuild and test the image, no error within the build logs, no error on push, but still the same issue. | 16:08 |
| *** vedup19 has quit IRC | 16:08 | |
| yoctozepto | I would opt even for a quick call around tls, possibly including mnasiadka to get things orderly and flowing | 16:08 |
| stackedsax | +1 that | 16:09 |
| generalfuzz | dig it | 16:09 |
| *** vedup19 has joined #openstack-kolla | 16:09 | |
| mnasiadka | orderly and flowing, sounds like a good marketing scheme | 16:13 |
| mgoddard | mnasiadka: https://bugs.launchpad.net/kolla-ansible/+bug/1867946 | 16:15 |
| openstack | Launchpad bug 1867946 in kolla-ansible train "Ceph jobs failing on Train and Stein during reconfigure" [High,Triaged] | 16:15 |
| mnasiadka | yeah, looking into it now | 16:16 |
| mnasiadka | since I need to wait for tomorrows images publish to continue with ovn | 16:16 |
| mnasiadka | :) | 16:16 |
| mgoddard | yoctozepto, generalfuzz, stackedsax: stepped away but back now | 16:17 |
| mgoddard | we can have a call. When would be a good time? | 16:17 |
| stackedsax | I mean, generalfuzz and I are usually free around the tie of this meeting most mornings. I dare say we're free now, as well :D | 16:19 |
| stackedsax | perhaps same time as the weekly meeting, but tomorrow? | 16:19 |
| mgoddard | if we're all free now that works for me | 16:20 |
| generalfuzz | +1 | 16:20 |
| mgoddard | yoctozepto, mnasiadka ? | 16:20 |
| yoctozepto | tomorrow 3 PM UTC ? | 16:21 |
| mgoddard | ok | 16:21 |
| mnasiadka | yeah, why not | 16:21 |
| mgoddard | I'll setup a google meet | 16:21 |
| yoctozepto | ++ | 16:22 |
| Fl1nt | ok, I even introspect the pulled image, it's a AMD64 one, nothing wrong within the image, don't get why docker doesn't want to run it... | 16:22 |
| stackedsax | thanks, mgoddard | 16:22 |
| hrw | re | 16:23 |
| hrw | https://review.opendev.org/#/c/713135/ waits for reviews. zuul fine except c8-k-a | 16:23 |
| patchbot | patch 713135 - kolla - CI: Avoid tox - 30 patch sets | 16:23 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible stable/train: DNM: Debug Ceph permission weirdness on reconfigure https://review.opendev.org/713691 | 16:24 |
| hrw | https://review.opendev.org/#/c/713647/ needs second core | 16:24 |
| patchbot | patch 713647 - kolla (stable/stein) - mariadb: install mariabackup on Debian - 1 patch set | 16:24 |
| yoctozepto | hrw: let me review that coauthored patch which grew from one line to a few more :-) | 16:24 |
| hrw | so is https://review.opendev.org/#/c/713616/ | 16:24 |
| patchbot | patch 713616 - kolla (stable/train) - CI: Install tox - 1 patch set | 16:24 |
| hrw | yoctozepto: it was also rewrite of run.yml scripts ;D | 16:25 |
| *** k_mouza has quit IRC | 16:27 | |
| yoctozepto | hrw: yeah, that's your part :-) | 16:28 |
| *** diurnalist has quit IRC | 16:29 | |
| *** k_mouza has joined #openstack-kolla | 16:30 | |
| *** diurnalist has joined #openstack-kolla | 16:35 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Avoid tox (and clean up gate setup) https://review.opendev.org/713134 | 16:36 |
| mnasiadka | yoctozepto: any idea why fluentd fails so miserably on add fluentd to mariadb CI change? https://zuul.opendev.org/t/openstack/build/ba032fb7155c4cdca354a078d67a0a0a/log/primary/logs/docker_logs/fluentd.txt ;-) | 16:38 |
| yoctozepto | mnasiadka: you reading my mind, but no idea so far :-) | 16:42 |
| yoctozepto | https://67bcfa89baec475c7d11-bb81120481bfc1a9d88d226642da11d8.ssl.cf5.rackcdn.com/713663/3/check/kolla-ansible-centos8-source-mariadb/ba032fb/primary/logs/kolla_configs/fluentd/input/00-global.conf | 16:42 |
| yoctozepto | hmm | 16:42 |
| mgoddard | maybe because lots of things are disabled? | 16:43 |
| yoctozepto | path {% for service, enabled in services if enabled | bool %}/var/log/kolla/{{ service }}/*.log{% if not loop.last %},{% endif %}{% endfor %} | 16:43 |
| yoctozepto | yeah, all openstack services | 16:43 |
| yoctozepto | so this ends up being empty | 16:43 |
| mnasiadka | so I found a bug... | 16:44 |
| yoctozepto | mnasiadka: I'll handle this, we can treat it as bug | 16:44 |
| mnasiadka | yoctozepto: thanks :) | 16:44 |
| *** bengates has quit IRC | 16:44 | |
| * hrw off | 16:49 | |
| openstackgerrit | Merged openstack/kolla stable/train: Fix Horizon build after switch to pyscss2 which lacks binary wheel https://review.opendev.org/713644 | 17:05 |
| *** k_mouza has quit IRC | 17:23 | |
| Fl1nt | \o/ VICTORY | 17:26 |
| Fl1nt | all right | 17:26 |
| Fl1nt | finally found out what's going on with my images | 17:27 |
| Fl1nt | so | 17:27 |
| Fl1nt | when the base image (or any image) isn't able to curl dumb-init and end up with a 404 the process continue without marking the image as erroneous and so you end up with all your images being stale as the dumb-init called by the entrypoint isn't present | 17:29 |
| yoctozepto | Fl1nt: woop woop! what was it? | 17:29 |
| yoctozepto | :O | 17:29 |
| yoctozepto | that's a bug, needs reporting | 17:29 |
| Fl1nt | Additionnaly, why does the dockerfile.j2 template do a RUN CURL for RPM based where RDO get the packages and pip repositories too. | 17:32 |
| *** evrardjp has quit IRC | 17:36 | |
| *** evrardjp has joined #openstack-kolla | 17:36 | |
| Fl1nt | ? | 17:36 |
| Fl1nt | sorry, was on a call while I typed the question ^^ | 17:36 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Fix deployment of fluentd without any enabled OpenStack services https://review.opendev.org/713704 | 17:37 |
| Fl1nt | so, all in all, within the dockerfile.j2 once dumb_init is used it try to curl for the yelp dumb-init package, I think a more robust solution would be to rely on pip, does anyone have thoughts on this? | 17:38 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: CI: Enable fluentd in MariaDB jobs https://review.opendev.org/713663 | 17:41 |
| *** priteau has quit IRC | 17:42 | |
| yoctozepto | Fl1nt: which branch is this? | 17:43 |
| *** k_mouza has joined #openstack-kolla | 17:43 | |
| Fl1nt | all, from 7.1.0 to master | 17:43 |
| Fl1nt | Here are for the tags: https://opendev.org/openstack/kolla/src/tag/7.1.0/docker/base/Dockerfile.j2 https://opendev.org/openstack/kolla/src/tag/8.0.2/docker/base/Dockerfile.j2 https://opendev.org/openstack/kolla/src/tag/9.0.1/docker/base/Dockerfile.j2 | 17:46 |
| Fl1nt | same for all stable/x branch | 17:48 |
| Fl1nt | same for all stable/x branches | 17:48 |
| mgoddard | curl -f will fail properly. Should always be used in scripts | 17:51 |
| yoctozepto | I thought we moved to pip's, maybe it was in ussuri... | 17:53 |
| Fl1nt | it may be related to CentOS not getting libc.so properly used with ld. | 17:54 |
| Fl1nt | @yoctozepto, where are the ussuri sources? | 17:54 |
| yoctozepto | Fl1nt: master plus what is proposed in gerrit | 17:55 |
| yoctozepto | but no, that did not happen | 17:55 |
| yoctozepto | I remember we must have at least discussed that at some point | 17:55 |
| Fl1nt | yeah, checked master earlier. | 17:55 |
| yoctozepto | for now makes sense to raise a bug and fix curl calls | 17:56 |
| Fl1nt | ok, I'll note that on my todo. | 17:58 |
| Fl1nt | are bugs filled with storyboard now? | 17:58 |
| Fl1nt | or directly within gitea? | 17:58 |
| yoctozepto | Fl1nt: nah, we are on launchpad | 17:58 |
| yoctozepto | https://bugs.launchpad.net/kolla | 17:59 |
| Fl1nt | >_< ouch | 17:59 |
| Fl1nt | ok ^^ | 17:59 |
| Fl1nt | I'll create all my required account this weekend, but gosh, the contributing process of OS is... challenging ^^ | 17:59 |
| yoctozepto | Fl1nt: it's actually one account to rule them all | 18:00 |
| yoctozepto | Fl1nt: just not github | 18:00 |
| yoctozepto | but ubuntu's | 18:01 |
| Fl1nt | yep, that's yet another account ^^ | 18:10 |
| Fl1nt | but anyway, I'll do necessary | 18:10 |
| Fl1nt | BTW | 18:10 |
| Fl1nt | the problem is bigger than just curl | 18:10 |
| Fl1nt | when openstack-base fail due to the 404 Curl | 18:11 |
| Fl1nt | it's marked as successfully build | 18:11 |
| Fl1nt | BUT | 18:11 |
| Fl1nt | some images detect that base parent is stale and so are maked as ERROR with the status: parent_error. | 18:12 |
| Fl1nt | for now on my side I just override the dumb_init block. | 18:12 |
| Fl1nt | All right dear people of this chan, that's it for me for today, going AFK. | 18:20 |
| Fl1nt | see you | 18:20 |
| *** suff has quit IRC | 18:20 | |
| *** k_mouza has quit IRC | 18:30 | |
| openstackgerrit | Merged openstack/kayobe master: Remove ceph block device support https://review.opendev.org/709376 | 18:35 |
| *** sri__ has joined #openstack-kolla | 18:39 | |
| *** k_mouza has joined #openstack-kolla | 18:44 | |
| *** k_mouza has quit IRC | 18:48 | |
| *** k_mouza has joined #openstack-kolla | 18:49 | |
| *** k_mouza has quit IRC | 18:54 | |
| *** k_mouza has joined #openstack-kolla | 19:21 | |
| *** dougsz has quit IRC | 19:25 | |
| *** k_mouza has quit IRC | 19:26 | |
| *** diurnalist has quit IRC | 19:30 | |
| *** dougsz has joined #openstack-kolla | 19:40 | |
| *** rpittau is now known as rpittau|afk | 19:40 | |
| *** diurnalist has joined #openstack-kolla | 19:41 | |
| *** gmann is now known as gmann_afk | 19:57 | |
| *** gfidente is now known as gfidente|afk | 20:08 | |
| *** k_mouza has joined #openstack-kolla | 20:14 | |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Fix deployment of fluentd without any enabled OpenStack services https://review.opendev.org/713704 | 20:18 |
| *** k_mouza has quit IRC | 20:19 | |
| *** vedup19 has quit IRC | 20:19 | |
| *** gmann_afk is now known as gmann | 20:33 | |
| *** cah_link has quit IRC | 20:41 | |
| *** sri__ has quit IRC | 20:46 | |
| *** jbadiapa has quit IRC | 21:13 | |
| *** openstackstatus has joined #openstack-kolla | 21:16 | |
| *** ChanServ sets mode: +v openstackstatus | 21:16 | |
| *** Fl1nt has quit IRC | 21:45 | |
| *** dougsz has quit IRC | 21:50 | |
| *** suff has joined #openstack-kolla | 22:00 | |
| *** suff has quit IRC | 22:09 | |
| *** bersace has quit IRC | 22:12 | |
| *** k_mouza has joined #openstack-kolla | 22:15 | |
| *** k_mouza has quit IRC | 22:19 | |
| *** suff has joined #openstack-kolla | 22:31 | |
| *** suff has quit IRC | 22:38 | |
| *** TrevorV has quit IRC | 22:41 | |
| *** ivve has quit IRC | 22:50 | |
| *** threestrands has joined #openstack-kolla | 22:58 | |
| *** sean-k-mooney1 has joined #openstack-kolla | 23:02 | |
| *** sean-k-mooney has quit IRC | 23:04 | |
| openstackgerrit | Merged openstack/kolla stable/stein: mariadb: install mariabackup on Debian https://review.opendev.org/713647 | 23:19 |
| *** jonaspaulo has quit IRC | 23:32 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!