| *** zzzeek has joined #tripleo | 00:02 | |
| *** sanjayu__ has quit IRC | 00:19 | |
| *** rcernin has quit IRC | 00:28 | |
| *** rcernin has joined #tripleo | 00:31 | |
| *** ayoung has quit IRC | 00:49 | |
| *** ayoung has joined #tripleo | 00:50 | |
| *** cfontain has joined #tripleo | 00:57 | |
| *** cfontain has quit IRC | 01:03 | |
| *** ayoung has quit IRC | 01:10 | |
| *** ayoung has joined #tripleo | 01:11 | |
| *** lbragstad_ has joined #tripleo | 01:33 | |
| *** lbragstad has quit IRC | 01:36 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 01:47 |
|---|---|---|
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common stable/ussuri: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750278 | 01:47 |
| *** ayoung has quit IRC | 01:54 | |
| *** ayoung has joined #tripleo | 01:55 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Fetch non tripleo containers in provider job and consume it https://review.opendev.org/749645 | 01:59 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Fetch non tripleo containers in provider job and consume it https://review.opendev.org/749645 | 01:59 |
| *** ekultails has quit IRC | 02:04 | |
| *** artom has quit IRC | 02:12 | |
| openstackgerrit | likui proposed openstack/diskimage-builder master: Stop install unnecessary packages when run tox -edocs https://review.opendev.org/751150 | 02:15 |
| *** lbragstad_ has quit IRC | 02:17 | |
| *** zzzeek has quit IRC | 02:27 | |
| *** zzzeek has joined #tripleo | 02:33 | |
| *** ayoung has quit IRC | 02:47 | |
| *** ayoung has joined #tripleo | 03:00 | |
| *** udesale has joined #tripleo | 03:12 | |
| *** jmasud has quit IRC | 03:21 | |
| *** jmasud has joined #tripleo | 03:23 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Fetch non tripleo containers in provider job and consume it https://review.opendev.org/749645 | 03:28 |
| *** skramaja has joined #tripleo | 03:42 | |
| *** rcernin has quit IRC | 03:53 | |
| *** ykarel|away has joined #tripleo | 03:54 | |
| *** udesale has quit IRC | 04:13 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Move get network configs action to utils https://review.opendev.org/737847 | 04:15 |
| *** rcernin has joined #tripleo | 04:15 | |
| *** ratailor has joined #tripleo | 04:18 | |
| *** ysandeep|away is now known as ysandeep | 04:24 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Move get network configs action to utils https://review.opendev.org/737847 | 04:27 |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #tripleo | 04:33 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Move get network configs action to utils https://review.opendev.org/737847 | 04:37 |
| *** ykarel|away is now known as ykarel | 04:38 | |
| *** skramaja has quit IRC | 04:38 | |
| *** skramaja has joined #tripleo | 04:39 | |
| *** cloudnull has quit IRC | 04:51 | |
| *** cloudnull has joined #tripleo | 04:52 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Move get network configs action to utils https://review.opendev.org/737847 | 04:56 |
| *** bandini has joined #tripleo | 05:04 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Make sure we noop the Mysql_ providers https://review.opendev.org/750678 | 05:05 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: Make sure we noop the Rabbitmq_ providers https://review.opendev.org/750763 | 05:05 |
| *** ratailor has quit IRC | 05:06 | |
| *** ratailor has joined #tripleo | 05:07 | |
| *** marios has joined #tripleo | 05:08 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common master: Move get network configs action to utils https://review.opendev.org/737847 | 05:08 |
| *** ratailor has quit IRC | 05:11 | |
| jpalanis_ | chandankumar, hi | 05:20 |
| Tengu | EmilienM: matter of broken CI, and CI not failing on the patches that were moving installation paths for two packages. promotion is the only place where all those patches were together, apparently. | 05:31 |
| chandankumar | jpalanis_: Hello | 05:32 |
| *** xek has joined #tripleo | 05:35 | |
| Tengu | EmilienM: (broken due to docker.io - that made us lose about 2 weeks of rechecks and all) | 05:36 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/python-tripleoclient stable/ussuri: Allow to pass a static inventory to the validator CLI https://review.opendev.org/750331 | 05:38 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations stable/ussuri: New validation: ensure we have access to the right package version https://review.opendev.org/750626 | 05:44 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: WIP pass build_repo_dir + fix tempest cloud name for standalone upgrade https://review.opendev.org/750443 | 05:47 |
| jpalanis_ | chandankumar, https://review.opendev.org/#/c/737847/ failing cross repo dependency merge conflict issue but i think rebased correctly. | 05:49 |
| chandankumar | jpalanis_: I am also hitting against my review not sure why | 05:50 |
| jpalanis_ | chandankumar, ok | 05:50 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 05:51 |
| *** lmiccini has joined #tripleo | 05:51 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 05:52 |
| *** slaweq has joined #tripleo | 06:02 | |
| openstackgerrit | Bhagyashri Shewale proposed openstack/openstack-tempest-skiplist master: Add octavia test to skip list https://review.opendev.org/751162 | 06:04 |
| Tengu | is the CI running on Windows? "2020-09-10 09:17:06.162987 | secondary | ERROR: All items complete" sounds like the nice "ERROR: no error" to me -.- | 06:13 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations master: Correct how the image_serve validation works https://review.opendev.org/743956 | 06:14 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/python-tripleoclient master: Do not relabel /etc/yum.repos.d nor /etc/pki/rpm-gpg https://review.opendev.org/749051 | 06:16 |
| *** slaweq has quit IRC | 06:18 | |
| *** zzzeek has quit IRC | 06:23 | |
| *** zzzeek has joined #tripleo | 06:24 | |
| *** udesale has joined #tripleo | 06:24 | |
| *** xek has quit IRC | 06:27 | |
| *** rcernin has quit IRC | 06:28 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible master: Adds filter for relative file paths https://review.opendev.org/751165 | 06:33 |
| openstackgerrit | yatin proposed openstack/tripleo-puppet-elements stable/ussuri: Remove overcloud-opstools element https://review.opendev.org/751166 | 06:33 |
| *** rcernin has joined #tripleo | 06:46 | |
| *** zzzeek has quit IRC | 06:51 | |
| *** apetrich has joined #tripleo | 06:53 | |
| *** zzzeek has joined #tripleo | 06:53 | |
| *** jcapitao has joined #tripleo | 06:55 | |
| *** apetrich has quit IRC | 06:55 | |
| *** apetrich has joined #tripleo | 06:56 | |
| *** pcaruana has joined #tripleo | 07:00 | |
| *** jtomasek has joined #tripleo | 07:02 | |
| *** slaweq has joined #tripleo | 07:03 | |
| *** jcapitao has quit IRC | 07:04 | |
| *** jcapitao has joined #tripleo | 07:06 | |
| *** amoralej|off is now known as amoralej | 07:07 | |
| *** jcapitao has quit IRC | 07:21 | |
| *** saneax has joined #tripleo | 07:24 | |
| *** jcapitao has joined #tripleo | 07:26 | |
| *** bogdando has joined #tripleo | 07:28 | |
| *** ysandeep is now known as ysandeep|lunch | 07:34 | |
| *** saneax has quit IRC | 07:44 | |
| *** jpena|off is now known as jpena | 07:44 | |
| bandini | anyone seen '2020-09-11T06:17:57.203400262+00:00 stderr F <13>Sep 11 06:17:57 puppet-user: Error: Evaluation Error: Error while evaluating a Resource Statement, Duplicate declaration: Nova_config[DEFAULT/max_concurrent_builds] is already declared at (file: /etc/puppet/modules/nova/manifests/compute.pp, line: 370); cannot redeclare (file: /etc/puppet/modules/nova/manifests/compute/ironic.pp, line: | 07:45 |
| bandini | 32) (file: /etc/puppet/modules/nova/manifests/compute/ironic.pp, line: 32, column: 5) on node undercloud.ooo.test' already? (rdo https://logserver.rdoproject.org/78/750678/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/fd5bb97/logs/undercloud/var/log/containers/stdouts/container-puppet-nova.log.txt.gz) | 07:45 |
| *** jcapitao has quit IRC | 07:45 | |
| Tengu | hmmm nope. | 07:45 |
| Tengu | not yet | 07:45 |
| bandini | seems I73bbca9caf733c8b11335c988b8dbcc33c354150 broke it | 07:46 |
| ramishra | bandini: https://review.opendev.org/#/c/750967/ will fix it I guess | 07:47 |
| *** shyamb has joined #tripleo | 07:47 | |
| Tengu | if tempest stops being nasty | 07:48 |
| bandini | ramishra: aha thanks Rabi! | 07:49 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations stable/train: New validation: Ensure sVirt is active https://review.opendev.org/751180 | 08:01 |
| *** jpich has joined #tripleo | 08:03 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates master: python3-novaclient installation via THT https://review.opendev.org/746745 | 08:04 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: Revert "Make sure python3-novaclient is installed before creating fence_compute" https://review.opendev.org/751182 | 08:04 |
| *** gfidente has joined #tripleo | 08:05 | |
| *** hoonetorg has joined #tripleo | 08:11 | |
| *** xaban has joined #tripleo | 08:13 | |
| xaban | Hi, via which template file or environments can we configure additional static routes for the overcloud nodes? Any documentation regarding it? | 08:14 |
| *** pcaruana has quit IRC | 08:19 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: haproxy bundle https://review.opendev.org/750455 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: mysql https://review.opendev.org/750456 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: manila-share https://review.opendev.org/750457 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: cinder backup/volume https://review.opendev.org/750458 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: ovn dbs https://review.opendev.org/750459 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: redis bundle https://review.opendev.org/750460 | 08:22 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: rabbitmq https://review.opendev.org/750461 | 08:22 |
| ykarel | xaban, see https://opendev.org/openstack/tripleo-heat-templates/commit/f5f655379725ed72cbc00005be3417fde82a1953 | 08:25 |
| ykarel | as per it u can set using <NetworkName>InterfaceRoutes environment for the network u want to add | 08:25 |
| xaban | ykarel: may I show you an example that I am using, which is not working? | 08:26 |
| ykarel | yes share | 08:27 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible master: Add filter for absolute path of file/dir https://review.opendev.org/751165 | 08:28 |
| *** shyamb has quit IRC | 08:28 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/ussuri: pcs commands on host: rabbitmq https://review.opendev.org/750461 | 08:28 |
| xaban | ykarel: https://nopaste.xyz/?bfecc32e65c9e41c#6YHdek3e7q71pmGMvidkBU9ykGjhhNrFJEFiV2dU8Tpc | 08:29 |
| xaban | shouldn't this add the static route: 10.26.15.0/24 via 10.26.15.254 ?? | 08:29 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/train: Bind mount /var/lib/container-config-scripts inside the restart bundles https://review.opendev.org/751184 | 08:29 |
| *** jcapitao has joined #tripleo | 08:30 | |
| *** pcaruana has joined #tripleo | 08:31 | |
| ykarel | xaban, can u check os-net-config on controller node to see if that config landed there | 08:34 |
| ykarel | yes if config is right, routes should have been set | 08:34 |
| ykarel | at /etc/os-net-config/config.json | 08:35 |
| *** ysandeep|lunch is now known as ysandeep | 08:36 | |
| xaban | Unfortunately, answer is negative. | 08:36 |
| xaban | Change: 2018-03-20, hmm, the file is not changed for quite some time | 08:37 |
| ykarel | ohh so you are doing update not a fresh deploy? | 08:38 |
| *** ssbarnea has joined #tripleo | 08:38 | |
| xaban | ykarel: indeed | 08:38 |
| ykarel | i think for network config, update needs to be enabled by flag, ramishra hjensas__ may have more idea on it | 08:39 |
| ramishra | xaban: use NetworkDeploymentActions: ['CREATE', 'UPDATE'] in one of your env files | 08:39 |
| ykarel | yes ^ flag, thanks ramishra | 08:39 |
| xaban | Hmm | 08:39 |
| ramishra | you can use {{role.name}}NetworkDeploymentActions if you want to limit it to a role | 08:41 |
| ramishra | ex. ControllerNetworkDeploymentActions: ['CREATE', 'UPDATE'] for controllers | 08:42 |
| *** derekh has joined #tripleo | 08:42 | |
| xaban | ramishra and ykarel: thanks! if it correct that an update - by default - doesn't update the networking config on the nodes? | 08:43 |
| *** ccamacho has joined #tripleo | 08:44 | |
| ramishra | xaban: yes | 08:45 |
| xaban | Good to know, thanks! | 08:45 |
| *** tosky has joined #tripleo | 08:47 | |
| *** ccamacho has quit IRC | 08:48 | |
| hjensas__ | xaban: it's not updated by default because it's not always safe. Some services especially on controllers (pacemaker, galera) does'nt always handle a network interface being brought down and up again. | 08:51 |
| *** hjensas__ is now known as hjensas | 08:51 | |
| *** xaban has quit IRC | 08:51 | |
| hjensas | xaban: os-net-config did get better since it was put in place, it will for many cases now update running config without bringing down/up interfaces. When it's possible ... It's a good idea to use the role specific parameter to enable the update, if you know you only need to update Compute's for example. | 08:53 |
| *** xek has joined #tripleo | 08:53 | |
| *** shyamb has joined #tripleo | 08:55 | |
| *** pojadhav is now known as pojadhav|afk | 08:55 | |
| *** ccamacho has joined #tripleo | 09:02 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/tripleo-upgrade stable/ussuri: Add neutron-ovs inclusion before other user environments https://review.opendev.org/751190 | 09:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/tripleo-upgrade stable/train: Add neutron-ovs inclusion before other user environments https://review.opendev.org/751191 | 09:06 |
| *** ssbarnea has quit IRC | 09:06 | |
| *** pcaruana has quit IRC | 09:07 | |
| *** pcaruana has joined #tripleo | 09:08 | |
| *** rcernin has quit IRC | 09:10 | |
| *** shyam89 has joined #tripleo | 09:11 | |
| openstackgerrit | Harald Jensås proposed openstack/python-tripleoclient master: Convert to 'raw' images when uploading https://review.opendev.org/749466 | 09:12 |
| *** shyamb has quit IRC | 09:15 | |
| *** ratailor has joined #tripleo | 09:16 | |
| *** xek has quit IRC | 09:17 | |
| *** cyberpear has quit IRC | 09:17 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Add support for Glance disk_formats configuration https://review.opendev.org/751200 | 09:17 |
| *** chem has joined #tripleo | 09:18 | |
| *** jtomasek has quit IRC | 09:28 | |
| shyam89 | Hi owalsh: Let me know once you get online | 09:33 |
| shyam89 | We need your advice on what approach we should use to add trilio mysql calls in puppet-tripleo module | 09:35 |
| *** saneax has joined #tripleo | 09:35 | |
| shyam89 | https://github.com/openstack/puppet-tripleo/blob/stable/train/manifests/profile/base/database/mysql.pp | 09:35 |
| *** ykarel is now known as ykarel|lunch | 09:36 | |
| bhagyashri|rover | Tengu, Hi | 09:36 |
| bhagyashri|rover | Tengu, master jobs are still failing :( | 09:37 |
| openstackgerrit | Rajesh Tailor proposed openstack/tripleo-heat-templates master: Fix typos https://review.opendev.org/751212 | 09:38 |
| *** sanjayu_ has joined #tripleo | 09:41 | |
| *** saneax has quit IRC | 09:42 | |
| Tengu | bhagyashri|rover: yeah - we're talking about it on #rdo and #oooq - the correction patch is blocked due to validations-common not being up-to-date in CI. we might get a way out. | 09:45 |
| openstackgerrit | Rajesh Tailor proposed openstack/tripleo-heat-templates master: Complete missing description https://review.opendev.org/751222 | 09:46 |
| Tengu | bhagyashri|rover: sorry for the whole sh*t - this kind of things are hard to catch (and correct, seeing how things are pushed to promotion) | 09:46 |
| bhagyashri|rover | Tengu, ok np looking at discussion. | 09:46 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 09:47 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 09:53 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-ansible master: OvS DPDK and SRIOV derive parameters https://review.opendev.org/737482 | 09:59 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Consume ceph, prom, grafana containers from quay.io https://review.opendev.org/750119 | 10:05 |
| *** shyam89 has quit IRC | 10:08 | |
| *** ratailor_ has joined #tripleo | 10:09 | |
| *** ratailor has quit IRC | 10:11 | |
| *** d34dh0r53 has quit IRC | 10:11 | |
| *** mnaser has quit IRC | 10:11 | |
| *** donnyd has quit IRC | 10:11 | |
| *** d34dh0r53 has joined #tripleo | 10:11 | |
| *** donnyd has joined #tripleo | 10:11 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Fetch non tripleo containers in provider job and consume it https://review.opendev.org/749645 | 10:14 |
| *** ykarel|lunch is now known as ykarel | 10:16 | |
| *** mnaser has joined #tripleo | 10:16 | |
| *** mnaser has quit IRC | 10:17 | |
| *** mnaser has joined #tripleo | 10:17 | |
| *** jmasud has quit IRC | 10:20 | |
| *** tkajinam has quit IRC | 10:22 | |
| *** jmasud has joined #tripleo | 10:22 | |
| *** shyamb has joined #tripleo | 10:30 | |
| openstackgerrit | likui proposed openstack/diskimage-builder master: replace imp module https://review.opendev.org/751236 | 10:31 |
| *** shyamb has quit IRC | 10:33 | |
| *** jpich has quit IRC | 10:39 | |
| *** jpich has joined #tripleo | 10:40 | |
| *** jcapitao is now known as jcapitao_lunch | 10:47 | |
| *** chem has quit IRC | 10:50 | |
| openstackgerrit | Merged openstack/tripleo-ansible stable/ussuri: Use bool filter when checking use_default_templates var https://review.opendev.org/751088 | 10:51 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Don't hide logs in molecule https://review.opendev.org/751050 | 10:51 |
| *** shyamb has joined #tripleo | 10:52 | |
| *** chem has joined #tripleo | 10:53 | |
| openstackgerrit | Sandeep Yadav proposed openstack/tripleo-quickstart-extras master: Use osp_release flag to set selinux mode https://review.opendev.org/751243 | 10:56 |
| *** artom has joined #tripleo | 11:01 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-validations master: Revert "Change path for validation Ansible files" https://review.opendev.org/751244 | 11:03 |
| *** sanjayu__ has joined #tripleo | 11:03 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-validations master: Revert "Change path for validation Ansible files" https://review.opendev.org/751244 | 11:04 |
| *** sanjayu_ has quit IRC | 11:05 | |
| *** rcernin has joined #tripleo | 11:06 | |
| *** cyberpear has joined #tripleo | 11:08 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Add support for Glance disk_formats configuration https://review.opendev.org/751200 | 11:10 |
| *** shyam89 has joined #tripleo | 11:11 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations master: Revert "Change path for validation Ansible files" https://review.opendev.org/751244 | 11:12 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Add support for Glance disk_formats configuration https://review.opendev.org/751200 | 11:12 |
| *** shyamb has quit IRC | 11:15 | |
| *** rcernin has quit IRC | 11:20 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-ansible master: Add tripleo-lvmfilter role to restrict visible block devices for LVM2 https://review.opendev.org/697990 | 11:24 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations master: Ensure we get a prefix for validation roles https://review.opendev.org/751247 | 11:25 |
| Tengu | matbu: -^ | 11:25 |
| *** sanjayu__ has quit IRC | 11:28 | |
| *** jtomasek has joined #tripleo | 11:28 | |
| *** chem has quit IRC | 11:29 | |
| *** shyam89 has quit IRC | 11:30 | |
| *** shyam89 has joined #tripleo | 11:30 | |
| *** chem has joined #tripleo | 11:31 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-ansible master: Add tripleo-lvmfilter role to restrict visible block devices for LVM2 https://review.opendev.org/697990 | 11:32 |
| shyam89 | owalsh: Hi | 11:38 |
| owalsh | shyam89: hi | 11:40 |
| *** jpena is now known as jpena|lunch | 11:42 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Run tripleo_lvmfilter role to restrict block devices visible to LVM2 https://review.opendev.org/697997 | 11:42 |
| shyam89 | owalsh: Trilio service needs to create it's database and keystone resources. For that we need to add it's call in https://github.com/openstack/puppet-tripleo/blob/stable/train/manifests/profile/base/database/mysql.pp | 11:43 |
| shyam89 | and https://github.com/openstack/puppet-tripleo/blob/stable/train/manifests/profile/base/keystone.pp | 11:43 |
| shyam89 | We are planning to merge the changes in upstream. But for this release, can we do changes in puppet-tripleo on undercloud and upload that to overcloud? | 11:44 |
| shyam89 | Using upload-puppet-module tool | 11:44 |
| shyam89 | owalsh: Let me know it's pros and cons | 11:44 |
| shyam89 | and if it will work | 11:44 |
| *** xek has joined #tripleo | 11:45 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Run tripleo_lvmfilter role to restrict block devices visible to LVM2 https://review.opendev.org/697997 | 11:47 |
| *** raildo has joined #tripleo | 11:48 | |
| owalsh | shyam89: keystone/db are not really my area... I'd ask EmilienM or mwhahaha for advice on this | 11:51 |
| owalsh | shyam89: given the time zones maybe better to discuss on openstack-discuss mailing list (start the subject with [tripleo]) | 11:52 |
| shyam89 | owalsh: okay | 11:53 |
| shyam89 | thanks | 11:53 |
| shyam89 | EmilienM: mwhahaha: Let me know your thoughts on my query ^^ | 11:54 |
| ykarel | shyam89, btw did the addition of db and keystone calls in tht worked for you? | 11:55 |
| ykarel | just for testing | 11:55 |
| shyam89 | ykarel: No, I did not seen their execution in any log file. Also no effect on the keystone and db resources | 11:56 |
| ykarel | shyam89, at what place u added the calls? | 11:57 |
| ykarel | execution should be there if added at correct place | 11:57 |
| *** jcapitao_lunch is now known as jcapitao | 11:59 | |
| *** rfolco|ruck has joined #tripleo | 12:01 | |
| *** rlandy has joined #tripleo | 12:01 | |
| shyam89 | ykarel: I added it here for db: https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/database/mysql-pacemaker-puppet.yaml#L178 | 12:02 |
| shyam89 | ykarel: like this: http://paste.openstack.org/show/797770/ | 12:03 |
| shyam89 | I observed that puppet module got loaded, but not executed | 12:03 |
| shyam89 | maybe puppet tags issue? | 12:03 |
| shyam89 | I observed that this call uses 'file' puppet tag. https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/database/mysql-pacemaker-puppet.yaml#L172 | 12:04 |
| ykarel | shyam89, where u obsolerved puppet module got loaded? | 12:05 |
| shyam89 | in container-puppet-mysql.log file | 12:06 |
| shyam89 | ykarel: ^^ | 12:06 |
| ykarel | shyam89, check logs for mysql_init_bundle | 12:09 |
| ykarel | uhhk | 12:10 |
| ykarel | shyam89, so u need to add call at https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/database/mysql-pacemaker-puppet.yaml#L276 | 12:10 |
| ykarel | instead of the earlier place | 12:10 |
| ykarel | as there is puppet tags file, and only file resources will be applied | 12:10 |
| shyam89 | okay | 12:11 |
| shyam89 | I see | 12:12 |
| shyam89 | ykarel: I will give a try | 12:12 |
| shyam89 | But before that, I will test upload puppet module for tripleo puppet module | 12:12 |
| *** xek has quit IRC | 12:12 | |
| shyam89 | ykarel: thanks. This is helpful. | 12:13 |
| *** Goneri has joined #tripleo | 12:16 | |
| rlandy | EmilienM: hi - how do we know where we pull the ubi8 image from in container builds? which registry? | 12:16 |
| *** amoralej is now known as amoralej|lunch | 12:17 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Make sure we noop the Rabbitmq_ providers https://review.opendev.org/750763 | 12:18 |
| openstackgerrit | Merged openstack/tripleo-common master: Run tripleo-build-containers-ubi-8 on BuildahBuilder code https://review.opendev.org/750800 | 12:18 |
| EmilienM | rlandy: good question | 12:18 |
| EmilienM | let me find out | 12:18 |
| EmilienM | but ubi8 comes from redhat registry | 12:18 |
| ykarel | registry.redhat | 12:18 |
| EmilienM | yeah ^ that probably | 12:18 |
| ykarel | to be specific registry.redhat.io | 12:19 |
| EmilienM | registry.access.redhat.com/ubi8 | 12:19 |
| EmilienM | or that ^ | 12:19 |
| EmilienM | on my computer, it pulls it from registry.access.redhat.com/ubi8 | 12:19 |
| rlandy | EmilienM: talking with Shreshtha | 12:19 |
| rlandy | we're trying to sync up where we take ubi8 vs the ondeck image brew uses | 12:20 |
| ykarel | shyam89, for keystone i think u need to use ansible way because of https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/keystone/keystone-container-puppet.yaml#L382 | 12:20 |
| ykarel | https://github.com/openstack/puppet-tripleo/blob/stable/train/manifests/profile/base/keystone.pp#L203-L213 | 12:20 |
| shyam89 | ykarel: yes, I used ansible way. Working fine on 16.1. But for RHOSP13 support, we need to use old way. | 12:21 |
| ykarel | shyam89, ok for 13 it will work | 12:22 |
| EmilienM | rlandy, ykarel : i think it really depends on the /etc/containers/registries.conf | 12:22 |
| EmilienM | and the "unqualified-search-registries" | 12:22 |
| EmilienM | :q! | 12:22 |
| EmilienM | i finally left VI ! | 12:23 |
| ykarel | EmilienM, yes it will look in order in registeries defined in ^ | 12:23 |
| ykarel | and ubi8 will come from redhat registeris | 12:23 |
| ykarel | shyam89, for queens u can add at https://github.com/openstack/tripleo-heat-templates/blob/stable/queens/puppet/services/keystone.yaml#L511 | 12:25 |
| shyam89 | ykarel: okay, let me check | 12:27 |
| shyam89 | ykarel: good one, thanks | 12:28 |
| *** shyam89 has quit IRC | 12:29 | |
| *** pojadhav|afk is now known as pojadhav | 12:34 | |
| *** raildo_ has joined #tripleo | 12:35 | |
| *** raildo has quit IRC | 12:35 | |
| *** ratailor_ has quit IRC | 12:42 | |
| *** skramaja has quit IRC | 12:43 | |
| *** jpena|lunch is now known as jpena | 12:44 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/ussuri: Add neutron-ovs inclusion before other user environments https://review.opendev.org/751190 | 12:45 |
| *** lbragstad_ has joined #tripleo | 12:49 | |
| *** zzzeek has quit IRC | 12:50 | |
| *** lbragstad_ is now known as lbragstad | 12:51 | |
| *** zzzeek has joined #tripleo | 12:51 | |
| *** raildo has joined #tripleo | 12:53 | |
| *** raildo_ has quit IRC | 12:53 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/train: Add tripleo_states to default callback whitelist https://review.opendev.org/741015 | 12:56 |
| *** ekultails has joined #tripleo | 13:00 | |
| *** zzzeek has quit IRC | 13:04 | |
| *** zzzeek has joined #tripleo | 13:07 | |
| *** matbu has quit IRC | 13:09 | |
| *** matbu has joined #tripleo | 13:10 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Add a note about verifying container images after hotfix https://review.opendev.org/750807 | 13:11 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-ansible master: OvS DPDK and SRIOV derive parameters https://review.opendev.org/737482 | 13:14 |
| openstackgerrit | wes hayutin proposed openstack/ansible-role-collect-logs master: use sova to check for selinux denials https://review.opendev.org/751273 | 13:19 |
| weshay|ruck | Tengu, https://review.opendev.org/#/c/751273/ | 13:19 |
| *** amoralej|lunch is now known as amoralej | 13:19 | |
| weshay|ruck | Tengu, this patch caused a selinux denial btw https://review.opendev.org/#/c/739481/ | 13:20 |
| Tengu | weshay|ruck: linked to podman version upstream I guess. | 13:20 |
| Tengu | iirc ykarel worked in order to get 1.6.4-15 in there. | 13:20 |
| weshay|ruck | aye | 13:20 |
| Tengu | but nice thing adding selinux in the parsing. | 13:21 |
| Tengu | what's sova? | 13:21 |
| openstackgerrit | John Fulton proposed openstack/python-tripleoclient master: WIP/DNM: stub for ceph-export https://review.opendev.org/750812 | 13:21 |
| Tengu | weshay|ruck: since you're here - care to add some ptl ack on https://review.opendev.org/#/c/750970/ and https://review.opendev.org/#/c/750964/ ? That will help with the blocked promotion.. | 13:22 |
| Tengu | weshay|ruck: (I've update the cix card with the plan, and some details about the path taken to get in this situation..) | 13:23 |
| weshay|ruck | sova is a little tool to parse logs for common errors.. you see it drop a log file in each job.. in logs.. https://logserver.rdoproject.org/81/739481/8/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-vexxhost/9788565/logs/ | 13:24 |
| weshay|ruck | No_failure_reason_found.log | 13:24 |
| weshay|ruck | don't ask about the names there | 13:24 |
| weshay|ruck | not my cup of tea | 13:24 |
| weshay|ruck | https://1dc19a8f26b6d8fa6b0f-1ea15234ebc61f9ad6f390769a79bf22.ssl.cf2.rackcdn.com/750071/1/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/06adabe/logs/ | 13:24 |
| weshay|ruck | in that log you'll find.. _Undercloud_install_failed.log | 13:24 |
| weshay|ruck | and we capture that info in our monitoring http://dashboard-ci.tripleo.org/d/wb8HBhrWk/cockpit?orgId=1 | 13:25 |
| weshay|ruck | Failure cause | 13:25 |
| * weshay|ruck looks at reviews | 13:25 | |
| *** tmazur has joined #tripleo | 13:27 | |
| *** TrevorV has joined #tripleo | 13:27 | |
| ykarel | weshay|ruck, Tengu hmm i didn't removed DNM https://review.opendev.org/#/c/746111/ | 13:27 |
| ykarel | tested train at https://review.rdoproject.org/r/#/c/28958/ | 13:27 |
| Tengu | weshay|ruck: ah, that's the tool providing those files. thanks! | 13:28 |
| Tengu | ykarel: ah. well. guess we can drop the DNM right? | 13:28 |
| ykarel | btw where u seen denials? | 13:28 |
| Tengu | probably when tempest starts a VM. | 13:29 |
| weshay|ruck | ykarel, https://logserver.rdoproject.org/81/739481/8/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-vexxhost/9788565/logs/overcloud-novacompute-0/var/log/audit/audit.log.txt.gz | 13:29 |
| ykarel | are we running with enforcing? | 13:29 |
| Tengu | hmmm | 13:29 |
| weshay|ruck | ykarel, enforcing is not allowed upstream or 3rd party | 13:29 |
| Tengu | weshay|ruck: https://logserver.rdoproject.org/81/739481/8/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-vexxhost/9788565/logs/overcloud-novacompute-0/var/log/extra/denials.txt.gz only one denial. | 13:29 |
| weshay|ruck | it was caught on our osp runs | 13:30 |
| Tengu | not linked to the patch | 13:30 |
| ykarel | weshay|ruck, hmm looking for that only | 13:30 |
| weshay|ruck | yup.. that's all it takes | 13:30 |
| ykarel | how it got caught | 13:30 |
| weshay|ruck | same job runs on osp w/ enforcing | 13:30 |
| ykarel | ok good | 13:30 |
| ykarel | but upstream patch dnm will not help the osp jobs | 13:31 |
| Tengu | the chronyd denial isn't linked to anything with nova, and it's a "temporary" one actually, until chronyd is restarted or something | 13:31 |
| Tengu | osp should have 1.6.4-15 already. | 13:31 |
| Tengu | unless the wrong container-tools stream is activated? | 13:31 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Add option to pull a custom base dockerfile - ubi8 https://review.opendev.org/751277 | 13:36 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Decrease Swift proxy timeouts for GET/HEAD requests https://review.opendev.org/751279 | 13:36 |
| *** udesale_ has joined #tripleo | 13:37 | |
| Tengu | folks, can we get +A on https://review.opendev.org/#/c/751244/ so that we unlock the promotion pipeline? | 13:38 |
| Tengu | thanks - and sorry for that failure -.-' | 13:38 |
| Tengu | thanks weshay|ruck ! | 13:38 |
| Tengu | weshay|ruck: once that one is merged, any ETA for a promotion on master? | 13:38 |
| Tengu | provided, of course, nothing else breaks :) | 13:39 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-validations master: Remove objects migrated to validations-common https://review.opendev.org/713204 | 13:39 |
| *** udesale has quit IRC | 13:39 | |
| weshay|ruck | Tengu, we've been dark for 2 days or so.. all jobs failing | 13:39 |
| weshay|ruck | Tengu, so.. not sure.. let me check the components | 13:39 |
| * Tengu ducks in a really dark, dark corner | 13:39 | |
| weshay|ruck | looks pretty good http://dashboard-ci.tripleo.org/d/FE8Hf29Wz/component-pipeline?orgId=1 | 13:40 |
| weshay|ruck | everything there is promoting quickly | 13:40 |
| Tengu | so that's a good news then | 13:40 |
| weshay|ruck | ooof .. except for tempest | 13:40 |
| Tengu | yeah, was about to talk about it | 13:40 |
| *** saneax has joined #tripleo | 13:40 | |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Fix pathing for container image prepare https://review.opendev.org/751100 | 13:41 |
| weshay|ruck | it's only two days old | 13:41 |
| Tengu | finger crossed everything will be OK - meaning we can finish the checklist Monday. | 13:44 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Fix pathing for container image prepare https://review.opendev.org/751100 | 13:48 |
| *** errantekarmico has joined #tripleo | 13:50 | |
| *** matbu has quit IRC | 13:51 | |
| *** matbu has joined #tripleo | 13:53 | |
| *** gmann is now known as gmann_afk | 14:05 | |
| fultonj | Tengu: what's the lastest version of ansible you're using with tripleo-lab ? | 14:15 |
| fultonj | e.g. are you using it with ansible-2.9.13-1 ? | 14:16 |
| Tengu | fultonj: ansible 2.9.13 | 14:17 |
| fultonj | ack, thanks | 14:18 |
| Tengu | ansible-2.9.13-1.fc32.noarch yeè | 14:18 |
| Tengu | fultonj: are you hitting any issue? | 14:20 |
| fultonj | permissions issue during undercloud install | 14:20 |
| fultonj | but i'm running an old version of tripleo lab from jul1 | 14:20 |
| fultonj | with latest main from tripleo | 14:21 |
| fultonj | so i'm going to upgrade tripleo-lab first | 14:21 |
| fultonj | i had upgrded ansible right before though so i wanted to rule that out | 14:21 |
| fultonj | with all the permissions from from 2.9.12, 2.9.11 had to ask | 14:21 |
| Tengu | oh, ok | 14:24 |
| jpalanis_ | fultonj, hi | 14:34 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-ansible master: OvS DPDK and SRIOV derive parameters https://review.opendev.org/737482 | 14:34 |
| fultonj | hi jpalanis_ | 14:34 |
| jpalanis_ | fultonj, planning to do the separate patch for molecule and documentation changes | 14:34 |
| jpalanis_ | fultonj, we can work and merge this two patches to remove the mistral dependencies | 14:35 |
| fultonj | jpalanis_: is it working in your env? | 14:35 |
| jpalanis_ | fultonj, yes | 14:35 |
| fultonj | in a real deploy? | 14:35 |
| fultonj | 'openstack overcloud deploy -p ...' sets the dpdk params right? | 14:36 |
| jpalanis_ | fultonj, it was working with my code ovs dpdk and sriov. but not tested yet with latest code | 14:36 |
| jpalanis_ | fultonj, mean using hci merged code | 14:37 |
| jpalanis_ | fultonj, should work. will test and let you know | 14:37 |
| jpalanis_ | fultonj, already added modules and filters test cases, only molecule NFV mock and document parts pending | 14:38 |
| fultonj | jpalanis_: that's great | 14:38 |
| fultonj | jpalanis_: is there mock data i could try the ansible module with? | 14:39 |
| fultonj | that's not essential but would be a nice to have | 14:39 |
| jpalanis_ | fultonj, for ansible modules, already used mock data in test cases. | 14:40 |
| jpalanis_ | fultonj, just need to run the test cases | 14:40 |
| *** errantekarmico has quit IRC | 14:40 | |
| jpalanis_ | fultonj, each test cases, different mock data needed | 14:40 |
| fultonj | jpalanis_: so you think it could merge once CI is green? | 14:41 |
| fultonj | jpalanis_: https://review.opendev.org/#/c/578947/ | 14:42 |
| jpalanis_ | fultonj, ok. will do the additional patches for enhancements and incases any issues | 14:42 |
| fultonj | should ^ be abandoned? | 14:42 |
| jpalanis_ | fultonj, yes, will do it. need to rework | 14:43 |
| fultonj | https://review.opendev.org/#/c/737847/ looks like it's getting close | 14:43 |
| fultonj | you know why it failed tripleo-ci-centos-8-containers-multinode ? | 14:43 |
| fultonj | https://zuul.opendev.org/t/openstack/build/e291a6ca67bb400d9e5ea33861376194 | 14:44 |
| fultonj | i kind of doubt it's related but it should be looked into and if it's not related rechecked | 14:44 |
| jpalanis_ | fultonj, https://zuul.opendev.org/t/openstack/build/e291a6ca67bb400d9e5ea33861376194 | 14:45 |
| jpalanis_ | fultonj, i think its not related to my changes. | 14:45 |
| mwhahaha | fultonj: it oom'd | 14:46 |
| mwhahaha | EmilienM, cloudnull -^ containers-multinode in master oom'd too https://review.opendev.org/#/c/737847/ | 14:46 |
| fultonj | ack | 14:46 |
| fultonj | ok | 14:46 |
| fultonj | just wanted to be sure before rechdcking | 14:46 |
| jpalanis_ | fultonj, ok | 14:46 |
| mwhahaha | somethings changed recently where we're oom'ing | 14:47 |
| EmilienM | let me look | 14:47 |
| EmilienM | do we have a logstash query? | 14:48 |
| mwhahaha | probably not | 14:48 |
| mwhahaha | logstash is always days behind anyway | 14:49 |
| *** jtomasek has quit IRC | 14:49 | |
| EmilienM | i used to have a magic gerrit query to find what landed in tripleo very recently | 14:49 |
| weshay|ruck | heh.. this one https://review.opendev.org/#/q/status:merged+tripleo+age:3week | 14:50 |
| mwhahaha | it's ooming in train+plus so i wonder if we have some base os changes that are now consuming more ram | 14:51 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Remove objects migrated to validations-common https://review.opendev.org/713204 | 14:52 |
| EmilienM | yeah let's rpm diff | 14:53 |
| EmilienM | my favorite friday morning game | 14:53 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Remove objects migrated to validations-common https://review.opendev.org/713204 | 14:53 |
| fultonj | Tengu: still hitting permissions issue | 14:54 |
| fultonj | http://paste.openstack.org/show/797780/ | 14:54 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Run tripleo_lvmfilter role to restrict block devices visible to LVM2 https://review.opendev.org/697997 | 14:54 |
| Tengu | fultonj: did you update the tripleo-operator-ansible ? | 14:55 |
| Tengu | fultonj: you can run "ansible-playbook config-host.yaml" | 14:55 |
| Tengu | that will update and deploy the operator. | 14:56 |
| fultonj | ansible-playbook -i inventory.yaml config-host.yaml | 14:56 |
| fultonj | i ran before running | 14:56 |
| Tengu | weird... | 14:56 |
| * fultonj just re-ran and chaged=0 | 14:56 | |
| Tengu | thought.... I don't think this is due to tripleo-lab. Are you re-using an existing undercloud VM, or is it a brand new one? | 14:57 |
| fultonj | i had tripleo lab destroy and rebuild eveything | 14:58 |
| Tengu | meh...... | 14:58 |
| fultonj | i'll get to the bottom of it eventually | 14:58 |
| Tengu | what are the permission on that ansible.log ? | 14:58 |
| EmilienM | mwhahaha: first of, new ansible (2.9.13 vs 2.9.11) | 14:58 |
| EmilienM | https://www.diffchecker.com/0H5Tfsb5 | 14:58 |
| fultonj | hmmmm | 14:59 |
| fultonj | lol | 14:59 |
| fultonj | -rw-r--r--. 1 root root 5710 Sep 11 14:36 ansible.log | 14:59 |
| EmilienM | mwhahaha: i took a failing job from today on master and a working job on master 3 days ago | 14:59 |
| mwhahaha | why dowe have lower ansible-runner now? | 14:59 |
| mwhahaha | 1.4.5 vs 1.4.6 | 15:00 |
| Tengu | fultonj: wondering if it's not coming from the prior container prepare command.... | 15:00 |
| EmilienM | mwhahaha: no, it's newer now | 15:00 |
| mwhahaha | oh inverse | 15:00 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Run tripleo_lvmfilter role to restrict block devices visible to LVM2 https://review.opendev.org/697997 | 15:00 |
| mwhahaha | so either ansible or ansible runner mabe | 15:00 |
| EmilienM | i mean there are a few python deps which could be the reason too but not sure | 15:00 |
| bandini | mmh ussuri jobs are failing with containers 404 but only on the overcloud lesigh | 15:05 |
| ykarel | both ansible and ansible-runner were updated recently on master | 15:05 |
| *** xek has joined #tripleo | 15:05 | |
| ykarel | is seen issue master only? | 15:05 |
| EmilienM | I'm looking at ansible release notes | 15:06 |
| mwhahaha | ykarel: no there's a bug around train too | 15:07 |
| EmilienM | did we also bump ansible in train? | 15:07 |
| ykarel | mwhahaha, in train we didn't updated so may be something else | 15:07 |
| ykarel | recently not updated in Train | 15:08 |
| mwhahaha | we haven't touched this code in a while, so i was wondering if it was the recent switch to tcib containers | 15:08 |
| EmilienM | no pushups today please | 15:09 |
| weshay|ruck | PUSH UPS | 15:10 |
| weshay|ruck | one for each container | 15:10 |
| EmilienM | mwhahaha: https://www.diffchecker.com/msqVQgZv | 15:12 |
| EmilienM | profile_tasks_limit ? | 15:13 |
| mwhahaha | no | 15:13 |
| mwhahaha | i don't think at least | 15:14 |
| *** bandini has quit IRC | 15:14 | |
| EmilienM | and it would have failed on master before | 15:15 |
| *** ysandeep is now known as ysandeep|away | 15:15 | |
| ykarel | mwhahaha, i see ur comment wrt swap size | 15:15 |
| ykarel | https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/prepare-node/tasks/main.yaml#L3 | 15:15 |
| mwhahaha | yea i'd recommend 4g | 15:15 |
| ykarel | role var configure_swap_size | 15:15 |
| ykarel | currently 1024 is used | 15:15 |
| mwhahaha | i run 8+4 in my dev env | 15:16 |
| mwhahaha | haven't oom'd in a very very very long time | 15:16 |
| mwhahaha | it's likely the stock undercloud is using all 8 and when we run the prepare process for teh overcloud, it's blowing up | 15:16 |
| mwhahaha | container image prepare uses a decent amount of ram | 15:17 |
| ykarel | can try swap bump, but would be good to know why suddenly it starting oom | 15:18 |
| *** xek has quit IRC | 15:19 | |
| *** lmiccini has quit IRC | 15:19 | |
| mwhahaha | EmilienM: might also compare dstat files | 15:19 |
| EmilienM | brb | 15:21 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add release version package https://review.opendev.org/751317 | 15:21 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add release version package https://review.opendev.org/751317 | 15:25 |
| *** gfidente has quit IRC | 15:26 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add release version package https://review.opendev.org/751317 | 15:30 |
| *** zzzeek has quit IRC | 15:32 | |
| *** zzzeek has joined #tripleo | 15:34 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Convert to 'raw' images when uploading https://review.opendev.org/749466 | 15:39 |
| openstackgerrit | Paul Leimer proposed openstack/tripleo-heat-templates master: Gather more extra stats with the collectd virt plugin https://review.opendev.org/751330 | 15:42 |
| *** ykarel is now known as ykarel|away | 15:42 | |
| mwhahaha | i wonder if it's crashing during the updating | 15:44 |
| *** udesale_ has quit IRC | 15:45 | |
| *** jcapitao has quit IRC | 15:46 | |
| *** marios is now known as marios|out | 15:50 | |
| *** zzzeek has quit IRC | 15:55 | |
| *** ekultails has left #tripleo | 15:56 | |
| *** ekultails has joined #tripleo | 15:56 | |
| *** zzzeek has joined #tripleo | 15:56 | |
| *** bogdando has quit IRC | 15:58 | |
| *** marios|out has quit IRC | 16:00 | |
| *** zzzeek has quit IRC | 16:03 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-quickstart master: Fix containers config file on fs037 https://review.opendev.org/751340 | 16:05 |
| *** zzzeek has joined #tripleo | 16:05 | |
| openstackgerrit | Merged openstack/tripleo-validations master: Revert "Change path for validation Ansible files" https://review.opendev.org/751244 | 16:06 |
| *** ykarel|away has quit IRC | 16:06 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-operator-ansible master: Operator does not need to become for image prepare https://review.opendev.org/751344 | 16:14 |
| *** derekh has quit IRC | 16:23 | |
| *** zzzeek has quit IRC | 16:26 | |
| *** zzzeek has joined #tripleo | 16:26 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-common master: WIP: Add variable to override default labels on base dockerfile https://review.opendev.org/751348 | 16:29 |
| *** saneax has quit IRC | 16:31 | |
| *** jpich has quit IRC | 16:34 | |
| *** artom has quit IRC | 16:35 | |
| *** artom has joined #tripleo | 16:35 | |
| *** rfolco|ruck is now known as rfolco|ruck|brb | 16:39 | |
| *** raildo_ has joined #tripleo | 16:46 | |
| *** raildo has quit IRC | 16:49 | |
| cloudnull | mwhahaha EmilienM re: oom - wasn't there changes ekultails and dpeacock were working on, adjusting the memlimits for mariadb? did those land? | 17:00 |
| cloudnull | could they be pusing us over the edge | 17:00 |
| EmilienM | https://review.opendev.org/#/c/747992/ | 17:01 |
| EmilienM | it wasn't backported | 17:01 |
| EmilienM | ekultails: ^ btw https://review.opendev.org/#/c/747992/8/deployment/database/mysql-base.yaml@65 | 17:01 |
| EmilienM | ekultails: please send a patch to address this comment | 17:01 |
| cloudnull | ok. | 17:01 |
| cloudnull | maybe we're just landing on new nodes w/ less ram? | 17:01 |
| cloudnull | like a new provide just came online ? | 17:02 |
| cloudnull | or the swapiness is set to 0? | 17:02 |
| chandankumar | jpalanis_: patch in merge conflict issue is fixed now by infra side | 17:06 |
| ekultails | EmilienM, aschultz: Thanks for the catch. I'll take care of it. | 17:09 |
| ekultails | cloudnull: We're looking into making the MariaDB tuning somewhat dynamic and potentially expanding that to other services. | 17:10 |
| openstackgerrit | Luke Short proposed openstack/tripleo-heat-templates master: Use consistent naming for MysqlInnodbBufferPoolSize https://review.opendev.org/751361 | 17:14 |
| *** jpena is now known as jpena|off | 17:34 | |
| *** gmann_afk is now known as gmann | 17:42 | |
| *** apetrich has quit IRC | 17:44 | |
| *** ekultails has quit IRC | 17:45 | |
| *** rfolco|ruck|brb is now known as rfolco|ruck | 17:46 | |
| *** ccamacho has quit IRC | 17:50 | |
| mwhahaha | cloudnull: i don't think we've changed anything really. i also don't think we do any swapiness/oom tuning | 17:57 |
| mwhahaha | cloudnull: EmilienM: I was wondering if we should think about exposing OOMScoreAdjust in the podman systemd service units | 17:57 |
| EmilienM | ehhhhh not sure i want to go into that business | 17:58 |
| mwhahaha | we probably should have been in that business but yea | 17:58 |
| EmilienM | what value would you put? | 17:59 |
| EmilienM | -500 ? | 17:59 |
| mwhahaha | anything? just to make it lower priority | 18:02 |
| mwhahaha | -100 | 18:02 |
| EmilienM | worth giving a try | 18:03 |
| mwhahaha | it's not going to solve this problem | 18:04 |
| mwhahaha | but might prevent other shit | 18:04 |
| matbu | weshay|ruck: can you merged that https://review.opendev.org/#/c/713204/ to see if it can unblock the promotion | 18:10 |
| weshay|ruck | looking | 18:11 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-quickstart master: Fix containers config file on fs037 https://review.opendev.org/751340 | 18:11 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Make sure we noop the Mysql_ providers https://review.opendev.org/750678 | 18:26 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add release version package https://review.opendev.org/751317 | 18:30 |
| *** rh-jelabarre has quit IRC | 18:36 | |
| *** chem has quit IRC | 18:36 | |
| *** rh-jelabarre has joined #tripleo | 18:36 | |
| *** chem has joined #tripleo | 18:38 | |
| rlandy | cloudnull: https://review.opendev.org/#/c/751277 Add option to pull a custom base dockerfile - ubi8 | 18:38 |
| openstackgerrit | Lance Bragstad proposed openstack/tripleo-heat-templates master: Update AdminToken description https://review.opendev.org/751380 | 18:40 |
| EmilienM | rlandy: +2 | 18:40 |
| lbragstad | qq - is there a way to explicitly override a default set by tht on older releases? | 18:40 |
| rlandy | thanks | 18:40 |
| mwhahaha | rlandy, cloudnull: not exactly a good variable name | 18:41 |
| rlandy | mwhahaha: base_image? | 18:42 |
| mwhahaha | probably better | 18:42 |
| mwhahaha | containers_base_image | 18:43 |
| mwhahaha | something like that | 18:43 |
| rlandy | ok - updating | 18:43 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Add option to pull a custom base dockerfile - ubi8 https://review.opendev.org/751277 | 18:43 |
| cloudnull | anyone want to give https://review.opendev.org/#/c/751074/ a nudge | 18:46 |
| *** ekultails has joined #tripleo | 18:58 | |
| weshay|ruck | folks.. all the validations folks are away | 19:10 |
| weshay|ruck | do we still need tripleo-validations in master.. if we have validations-common and validations-lib ? | 19:11 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Add --temp-dir to overcloud image build https://review.opendev.org/751388 | 19:11 |
| mwhahaha | i was wondering the same thing, if we don't then validations-common could obsolete the tripleo version but i think the tripleo version is still required | 19:12 |
| weshay|ruck | k.. I sent them an email | 19:18 |
| weshay|ruck | we'll find out | 19:18 |
| weshay|ruck | mwhahaha, at the very least we should move tripleo-validations to the validations component | 19:18 |
| * mwhahaha shrugs | 19:19 | |
| *** social has joined #tripleo | 19:24 | |
| *** xek has joined #tripleo | 19:47 | |
| *** pcaruana has quit IRC | 19:55 | |
| openstackgerrit | Merged openstack/tripleo-operator-ansible master: Add comment specifying tripleo_overcloud_node_delete_nodes as a list https://review.opendev.org/750737 | 19:56 |
| lbragstad | it looks like the AdminToken is set by default on train deployments, even though it was removed in later releases https://review.opendev.org/#/c/717421/ | 20:00 |
| lbragstad | i have a patch to update the docs https://review.opendev.org/#/c/751380/ but is there a way to explicitly unset that variable at deploy time? | 20:01 |
| mwhahaha | admintoken is reused for somethign isn't it? | 20:04 |
| mwhahaha | lbragstad: AdminToken is still used as the keystone db password, so you wouldn't want to unset it | 20:05 |
| lbragstad | ahhh | 20:12 |
| lbragstad | i'm missing the part where tht hands that off to puppet-keystone then | 20:12 |
| mwhahaha | keystone::db::mysql::password: {get_param: AdminToken} | 20:12 |
| lbragstad | yeah - that part makes sense | 20:12 |
| lbragstad | wait - so there isn't a way to disable AdminToken without disabling database authentication on train | 20:13 |
| mwhahaha | you can just not set it for the admin token part | 20:13 |
| * mwhahaha looks | 20:14 | |
| mwhahaha | keystone::admin_token: {get_param: AdminToken} | 20:15 |
| mwhahaha | removing that | 20:15 |
| mwhahaha | or blanking that as necessary | 20:15 |
| lbragstad | ControllerExtraConfig | 20:15 |
| lbragstad | i wonder if that would work? | 20:15 |
| lbragstad | otherwise - i think i'd need to back port https://review.opendev.org/#/c/717421/ | 20:15 |
| mwhahaha | if you set keystone::admin_token that way, it should override the THT one | 20:15 |
| mwhahaha | though backporting that is probably more correct | 20:15 |
| mwhahaha | i don't know when we stopped relying on the admin token everywhere | 20:16 |
| lbragstad | you think? i wasn't sure if that would be accepted since it would break things | 20:16 |
| mwhahaha | what would it break? | 20:16 |
| lbragstad | i have a stable/train env up and you can technically use the admin token still | 20:16 |
| mwhahaha | it's more of what does the puppet-keystone bits require | 20:16 |
| lbragstad | because it's set - even though `keystone-manage bootstrap` is used | 20:16 |
| mwhahaha | i want to say we stopped using that with containers tho | 20:17 |
| mwhahaha | we had to backport some fixes around that stuff | 20:17 |
| mwhahaha | but it hink we did that in train | 20:17 |
| mwhahaha | https://review.opendev.org/#/c/699017/1/deployment/keystone/keystone-container-puppet.yaml | 20:18 |
| mwhahaha | so i think you can just cherry pick that change back | 20:18 |
| openstackgerrit | Lance Bragstad proposed openstack/tripleo-heat-templates stable/train: Remove deprecated puppet-keystone modules related to admin auth https://review.opendev.org/751423 | 20:20 |
| lbragstad | tht parameters support nil, right? keystone::admin_token: nil for example | 20:23 |
| openstackgerrit | Merged openstack/tripleo-validations master: Remove objects migrated to validations-common https://review.opendev.org/713204 | 20:24 |
| mwhahaha | no | 20:26 |
| mwhahaha | removing it will cause it to fall back to the default from the puppet module which in theory it should drop from the configs (if the default value is $::os_service_default) | 20:29 |
| lbragstad | you mean removing it here - https://review.opendev.org/#/c/751423/1/deployment/keystone/keystone-container-puppet.yaml@394 ? | 20:31 |
| mwhahaha | yes | 20:32 |
| lbragstad | yeah - ok, that makes sense | 20:32 |
| lbragstad | in the mean time, setting it to empty string using ControllerExtraConfig should work around the issue (i'm testing this now)? | 20:32 |
| mwhahaha | maybe? | 20:33 |
| mwhahaha | i'd have to look what happens in the puppet module | 20:33 |
| mwhahaha | set it to <SERVICE DEFAULT> | 20:35 |
| mwhahaha | hax | 20:35 |
| mwhahaha | keystone::admin_token: "<SERVICE DEFAULT>" in an extraconfig | 20:36 |
| mwhahaha | should reproduce unsetting it | 20:36 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Set install of ipa packages to false - use local forwarder https://review.opendev.org/741035 | 20:36 |
| lbragstad | mwhahaha ok - trying that now | 20:37 |
| *** TrevorV has quit IRC | 20:37 | |
| lbragstad | side note; is anyone else noticing issues deploying stable/train rdo packages, specifically with the overcloud container starting in step 3? | 21:03 |
| * lbragstad is able to recreate on ovb and deployed-server environments | 21:04 | |
| lbragstad | overcloud containers* | 21:04 |
| *** raildo_ has quit IRC | 21:06 | |
| *** rlandy is now known as rlandy|afk | 21:09 | |
| *** rfolco|ruck has quit IRC | 21:20 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: Add guard to ensure a mapping has a valid key https://review.opendev.org/751074 | 21:33 |
| openstackgerrit | John Fulton proposed openstack/python-tripleoclient master: Add openstack overcloud export ceph https://review.opendev.org/750812 | 21:35 |
| *** fultonj has quit IRC | 21:38 | |
| *** xek has quit IRC | 21:49 | |
| *** rlandy|afk is now known as rlandy | 21:53 | |
| *** cfontain has joined #tripleo | 22:02 | |
| *** rcernin has joined #tripleo | 22:02 | |
| *** rcernin has quit IRC | 22:03 | |
| *** rcernin has joined #tripleo | 22:03 | |
| *** tmazur has quit IRC | 22:53 | |
| *** tosky has quit IRC | 23:20 | |
| *** chem has quit IRC | 23:39 | |
| *** chem has joined #tripleo | 23:41 | |
| *** ekultails has quit IRC | 23:42 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!