| *** langdon has joined #tripleo | 00:00 | |
| *** zzzeek has quit IRC | 00:12 | |
| *** zzzeek has joined #tripleo | 00:16 | |
| *** tosky has quit IRC | 00:20 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/train: Make sure qemu CA has correct permissions https://review.opendev.org/c/openstack/puppet-tripleo/+/764193 | 00:43 |
|---|---|---|
| *** jamesdenton has quit IRC | 00:48 | |
| *** jamesden_ has joined #tripleo | 00:49 | |
| openstackgerrit | Amy Marrich proposed openstack/tripleo-docs master: Added Victoria release https://review.opendev.org/c/openstack/tripleo-docs/+/765692 | 00:55 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Reimplement os_tripleo_baremetal_node_introspection https://review.opendev.org/c/openstack/tripleo-ansible/+/764115 | 01:30 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates stable/victoria: Ensure cloud-init has finished before puppet run https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765736 | 03:00 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates stable/ussuri: Ensure cloud-init has finished before puppet run https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765737 | 03:00 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates stable/train: Ensure cloud-init has finished before puppet run https://review.opendev.org/c/openstack/tripleo-heat-templates/+/764933 | 03:01 |
| *** ramishra has joined #tripleo | 03:01 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible stable/ussuri: Stop NetworkManager from updating resolv.conf https://review.opendev.org/c/openstack/tripleo-ansible/+/765738 | 03:07 |
| *** ramishra has quit IRC | 03:08 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible stable/train: Stop NetworkManager from updating resolv.conf https://review.opendev.org/c/openstack/tripleo-ansible/+/765718 | 03:09 |
| *** ramishra has joined #tripleo | 03:11 | |
| *** d34dh0r53 has joined #tripleo | 03:59 | |
| *** skramaja has joined #tripleo | 04:14 | |
| *** ratailor has joined #tripleo | 04:24 | |
| *** jamesden_ has quit IRC | 04:34 | |
| *** jamesdenton has joined #tripleo | 04:35 | |
| *** hakhande has joined #tripleo | 04:57 | |
| *** hakhande has joined #tripleo | 04:58 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-ansible stable/train: Update kernel args grub file entry inline with upgrade changes https://review.opendev.org/c/openstack/tripleo-ansible/+/753207 | 05:04 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/ussuri: Adding Ceph Dashboard to the Edge roles https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765292 | 05:15 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Map build host repos to distro.repos.d instead of yum.repos.d https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/761632 | 05:15 |
| *** raukadah is now known as chandankumar | 05:17 | |
| *** ramishra has quit IRC | 05:31 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #tripleo | 05:33 | |
| *** udesale has joined #tripleo | 05:43 | |
| *** ramishra has joined #tripleo | 05:57 | |
| *** chandankumar is now known as chkumar|ruck | 06:09 | |
| *** chkumar|ruck is now known as chkumar|rover | 06:14 | |
| *** chkumar|rover is now known as chkumar|ruck | 06:35 | |
| *** saneax has joined #tripleo | 06:37 | |
| openstackgerrit | chandan kumar proposed openstack/tripleo-quickstart-extras master: Fallback to rdo registry if quay gets down https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/765731 | 06:48 |
| *** lmiccini has joined #tripleo | 06:53 | |
| openstackgerrit | chandan kumar proposed openstack/tripleo-quickstart-extras master: Fallback to rdo registry if quay gets down https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/765731 | 07:16 |
| *** waleedm has joined #tripleo | 07:32 | |
| *** rcernin has quit IRC | 07:55 | |
| *** bandini has joined #tripleo | 08:03 | |
| *** jcapitao has joined #tripleo | 08:06 | |
| *** hakhande has quit IRC | 08:07 | |
| *** shyamb has joined #tripleo | 08:14 | |
| *** pcaruana has joined #tripleo | 08:15 | |
| *** frenzy_friday has joined #tripleo | 08:23 | |
| *** rcernin has joined #tripleo | 08:25 | |
| *** rcernin has quit IRC | 08:30 | |
| *** bandini has quit IRC | 08:31 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Update role-addition.yml playbook to manage molecule.yaml file again https://review.opendev.org/c/openstack/tripleo-validations/+/765323 | 08:33 |
| *** rpittau|afk is now known as rpittau | 08:46 | |
| *** jpich has joined #tripleo | 08:48 | |
| *** lucasagomes has joined #tripleo | 08:54 | |
| *** shyamb has quit IRC | 09:01 | |
| *** derekh has joined #tripleo | 09:04 | |
| *** cylopez has joined #tripleo | 09:05 | |
| *** shyamb has joined #tripleo | 09:05 | |
| *** xek_ has joined #tripleo | 09:17 | |
| *** zzzeek has quit IRC | 09:19 | |
| *** zzzeek has joined #tripleo | 09:20 | |
| *** chem has joined #tripleo | 09:23 | |
| *** rcernin has joined #tripleo | 09:25 | |
| Tengu | hjensas: heya! apparently we CAN run all the validation subset happening during an OC deploy with pre-deployed nodes (at least in a novaless environment). Meaning we can stop enforcing "--disable-validations" param when we use pre-deployed server. I have an OC deploy running right now with a tweaked tripleoclient that just disables the check on the param. | 09:27 |
| Tengu | let's see if it crashes later. | 09:28 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/ussuri: [Container build] raise Exception post logging error https://review.opendev.org/c/openstack/python-tripleoclient/+/764180 | 09:29 |
| *** rcernin has quit IRC | 09:30 | |
| *** pojadhav|afk is now known as pojadhav | 09:33 | |
| *** hakhande has joined #tripleo | 09:41 | |
| Tengu | err... meh. my automated thing did add the --disable-validations, and I didn't see it. #automationAtItsFinest | 09:44 |
| Tengu | ah bah. works as well so far. | 09:54 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Run Swift replicators in single replica mode https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765775 | 09:58 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-common master: Cleanup and add test coverage for inventory without overcloud https://review.opendev.org/c/openstack/tripleo-common/+/765524 | 10:05 |
| *** dtantsur|afk is now known as dtantsur | 10:07 | |
| *** rcernin has joined #tripleo | 10:21 | |
| *** rcernin has quit IRC | 10:21 | |
| openstackgerrit | Merged openstack/tripleo-common stable/victoria: Fix handling of default_tag https://review.opendev.org/c/openstack/tripleo-common/+/764679 | 10:28 |
| *** cmorey has joined #tripleo | 10:33 | |
| openstackgerrit | mbu proposed openstack/tripleo-validations master: Generate inventory without any overcloud https://review.opendev.org/c/openstack/tripleo-validations/+/764955 | 10:34 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Modify overcloud-deploy to support multiple stacks https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/763786 | 10:35 |
| *** matbu has quit IRC | 10:35 | |
| *** matbu has joined #tripleo | 10:38 | |
| *** jcapitao has quit IRC | 10:39 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Do not relabel Swift files on every container start https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765778 | 10:40 |
| *** jcapitao has joined #tripleo | 10:45 | |
| hjensas | Tengu: oh, that's cool. Let's se how the overcloud deploy run goes. | 11:17 |
| *** tosky has joined #tripleo | 11:22 | |
| *** hakhande has quit IRC | 11:28 | |
| *** ratailor has quit IRC | 11:32 | |
| *** ratailor has joined #tripleo | 11:33 | |
| *** cmorey has quit IRC | 11:36 | |
| openstackgerrit | Itzik Brown proposed openstack/tripleo-upgrade stable/train: [DO NOT MERGE]Revert "Add the -n flag in the run parameters conversion script" https://review.opendev.org/c/openstack/tripleo-upgrade/+/765789 | 11:39 |
| openstackgerrit | Itzik Brown proposed openstack/tripleo-upgrade stable/train: [DO NOT MERGE]Revert "Add the -n flag in the run parameters conversion script" https://review.opendev.org/c/openstack/tripleo-upgrade/+/765790 | 11:40 |
| *** cmorey has joined #tripleo | 11:42 | |
| *** bogdando has joined #tripleo | 11:45 | |
| *** bogdando has left #tripleo | 11:45 | |
| *** hakhande has joined #tripleo | 11:45 | |
| *** shyamb has quit IRC | 11:46 | |
| *** moguimar has joined #tripleo | 11:48 | |
| *** shyamb has joined #tripleo | 11:48 | |
| *** raildo has joined #tripleo | 11:48 | |
| cmorey | Is it possible to add vxlan support to an existing deployment? (because i don't have geneve support in the overcloud kernel due to being on centos7), | 11:49 |
| *** hakhande has quit IRC | 11:53 | |
| *** rfolco has joined #tripleo | 11:54 | |
| *** udesale_ has joined #tripleo | 11:57 | |
| *** udesale has quit IRC | 11:59 | |
| Tengu | hjensas: still doing some tests, it failed but I doubt it's due to validations | 12:01 |
| *** zzzeek has quit IRC | 12:03 | |
| *** zzzeek has joined #tripleo | 12:05 | |
| *** jcapitao is now known as jcapitao_lunch | 12:09 | |
| *** dprince has joined #tripleo | 12:33 | |
| *** chem has quit IRC | 12:35 | |
| *** chem has joined #tripleo | 12:36 | |
| *** zzzeek has quit IRC | 12:38 | |
| *** jpich has quit IRC | 12:38 | |
| *** jpich has joined #tripleo | 12:38 | |
| openstackgerrit | Shnaidman Sagi proposed openstack/puppet-tripleo stable/ussuri: [ussuri] Migrate to content provider jobs/template https://review.opendev.org/c/openstack/puppet-tripleo/+/757829 | 12:39 |
| *** zzzeek has joined #tripleo | 12:39 | |
| *** shyamb has quit IRC | 12:47 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-puppet-elements master: [WIP] - Include cephadm in the overcloud image https://review.opendev.org/c/openstack/tripleo-puppet-elements/+/765512 | 12:49 |
| *** shyamb has joined #tripleo | 12:57 | |
| *** alexmcleod_ has quit IRC | 12:59 | |
| *** alexmcleod_ has joined #tripleo | 12:59 | |
| *** efoley has joined #tripleo | 12:59 | |
| *** rlandy has joined #tripleo | 13:00 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/python-tripleoclient master: Stop configuring install_command in tox. https://review.opendev.org/c/openstack/python-tripleoclient/+/693502 | 13:01 |
| Tengu | hjensas: soooo.... currently deploy a 3ctrl + 1compute OC with metalsmith provisionning, and without the "--disable-validations". seems to work. Step 4. | 13:09 |
| Tengu | first attempt failed on mysql deploy for some reason, but I would say "not related to validations". | 13:09 |
| *** dpeacock has quit IRC | 13:10 | |
| *** dpeacock has joined #tripleo | 13:11 | |
| *** saneax has quit IRC | 13:14 | |
| openstackgerrit | Jesse Pretorius proposed openstack/tripleo-heat-templates stable/victoria: Enable tripleo_free strategy for upgrade https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765744 | 13:16 |
| *** jcapitao_lunch is now known as jcapitao | 13:17 | |
| openstackgerrit | Jesse Pretorius proposed openstack/tripleo-heat-templates stable/ussuri: Enable tripleo_free strategy for upgrade https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765745 | 13:19 |
| *** dpeacock has quit IRC | 13:23 | |
| Tengu | hjensas: "Overcloud Deployed without error" with the following command: http://paste.openstack.org/show/800788/ | 13:24 |
| Tengu | hjensas: guess we can actually drop the mandatory "--disable-validations" when --deployed-server is in use. | 13:25 |
| Tengu | brb | 13:25 |
| *** dpeacock has joined #tripleo | 13:29 | |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: WIP Use heat server side env merging https://review.opendev.org/c/openstack/python-tripleoclient/+/765808 | 13:30 |
| cmorey | according to https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/network_isolation.html i can add VXLAN capability after the fact, so I set NeutronNetworkType: 'vxlan,vlan' and NeutronTunnelTypes: 'vxlan' but i'm getting a StackValidationFailed error, "Parameter 'NeutronNetworkType' is invalid: Invalid default vxlan,vlan (["vxlan", "vlan"] is not an allowed value ["geneve", "vlan", "flat"])" | 13:34 |
| *** shyamb has quit IRC | 13:34 | |
| cmorey | (it's also complaining about an unused parameter, NeutronTunnelTypes), | 13:35 |
| *** shyamb has joined #tripleo | 13:35 | |
| cloudnull | mornings | 13:36 |
| cmorey | has support for vxlan been dropped in train? | 13:38 |
| supamatt | morning | 13:39 |
| *** ysandeep is now known as ysandeep|afk | 13:42 | |
| cmorey | looks like the error is coming from tht/deployment/neutron/neutron-plugin-ml2-ovn.yaml | 13:43 |
| beagles | cmorey: it depends on the version you are using | 13:44 |
| beagles | cmorey: vxlan was not intially supported with ovn | 13:44 |
| cmorey | beagles, running train on centos7 | 13:44 |
| cmorey | my kernel is too old for Geneve, and it looks like OVN doesn't support multi-chassis VLAN tenant networks, | 13:45 |
| cmorey | https://opendev.org/openstack/tripleo-heat-templates/commit/ae4aac8cb57a69d82c35e90090d52f5b30b335a6 | 13:46 |
| beagles | cmorey: the tripleo heat templates were modified to allow vxland only recently https://review.opendev.org/c/openstack/neutron/+/734889 | 13:46 |
| *** Goneri has joined #tripleo | 13:48 | |
| cmorey | beagles, ah | 13:48 |
| cmorey | can i update to newer tht? | 13:48 |
| supamatt | well my thought was he'd be a really good replacement for dvd in the ESS team, he could sit in support but still contribute to engineering work and then eventually make that transition when he's ready | 13:50 |
| supamatt | err soops wrong channel | 13:50 |
| cmorey | supamatt, :) | 13:50 |
| beagles | cmorey: not without also using the corresponding version of other tripleo pieces | 13:50 |
| beagles | cmorey: i.e. you can't just use master tht without new tripleo-ansible, puppet-tripleo, etc etc etc etc | 13:51 |
| cmorey | beagles, I'm guessing that means going (or trying to) go from train -> ussuri? | 13:51 |
| cmorey | Victoria even | 13:51 |
| beagles | cmorey: afaict that patch hasn't been backported to any stable branches - the required neutron patch hasn't been either https://review.opendev.org/c/openstack/neutron/+/734889 | 13:53 |
| beagles | cmorey: if you need vxlan you pretty much need to use ml2/ovs | 13:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/train: Update container build to better reign in threads https://review.opendev.org/c/openstack/tripleo-common/+/752324 | 13:55 |
| hjensas | Tengu: sorry for the slow replies, I have child with stomack-flu at home. Thanks for doing the testing, I'll do some myself as well and remove the "--disable-validations" requirement. | 13:56 |
| cmorey | beagles, i don't specifically need vxlan, i just need floating IPs to work for me (it seems the tunnels aren't being created | 13:57 |
| cmorey | beagles, if i switch to ml/ovs will that solve things? | 13:57 |
| Tengu | hjensas: np - family first! do you want to jump on my current env to check it? | 13:58 |
| cloudnull | weshay|ruck on https://bugs.launchpad.net/tripleo/+bug/1907006 - I saw then error when doing deployment with my old network templates using master. I had to first convert my templates and turn the ansible integration off | 13:58 |
| openstack | Launchpad bug 1907006 in tripleo "standalone deployment victoria failing on AttributeError: 'NoneType' object has no attribute 'items'" [Critical,Triaged] | 13:58 |
| beagles | cmorey: it might be worthwhile trying to troubleshoot why OVN isn't working for you | 13:58 |
| cloudnull | so this very well may be an issue only in upgrades? | 13:58 |
| beagles | cmorey: have you asked about it on openstack-neutron? | 13:59 |
| cloudnull | -cc Tengu | 13:59 |
| cloudnull | hjensas ^ | 13:59 |
| cloudnull | ramishra ^ | 14:00 |
| *** ratailor has quit IRC | 14:00 | |
| Tengu | cloudnull: probably a promotion issue as ramishra told in another place. | 14:00 |
| Tengu | would be good ensuring packages are up-to-date with the new network templates | 14:00 |
| cloudnull | ++ this lp came up on the cix call | 14:00 |
| cmorey | beagles, I haven't, because i think i'm hitting https://bugzilla.redhat.com/show_bug.cgi?id=1704596 | 14:00 |
| openstack | bugzilla.redhat.com bug 1704596 in python-networking-ovn "FIP traffix does not work on OVN-DVR setup when using VLAN tenant network type" [High,Assigned] - Assigned to jlibosva | 14:00 |
| Tengu | cloudnull: I know :) | 14:00 |
| * cloudnull is just blasting it out so everyone knows :P | 14:01 | |
| beagles | cmorey: have you tried deploying with dvr disabled? | 14:02 |
| cmorey | beagles, i haven't tried disabling DVR though, trying it now actually, if that doesn't work, i'll try openstack-neutron | 14:03 |
| beagles | cmorey: k | 14:03 |
| cmorey | beagles, i may have tried it in the past, i've tried that many things over the last month or so, but at least now the deployment is passing cleanly, | 14:03 |
| cmorey | can i set that without deleting and re-deploying? (Ie can i just deploy with that set to false)? | 14:04 |
| beagles | that's a good question | 14:04 |
| *** waleedm has quit IRC | 14:04 | |
| beagles | cmorey: it might work | 14:04 |
| cmorey | beagles, i'll probably find out soon enough :) | 14:04 |
| beagles | cmorey: you might run into issues if you have existing networking resources created in the overcloud - especially routers | 14:05 |
| hjensas | cloudnull: we put in some changes https://review.opendev.org/763296 and https://review.opendev.org/q/topic:%22bug%252F1904809%22 to fix some similar issues. | 14:06 |
| cloudnull | ++ | 14:06 |
| beagles | cmorey: at best those routers won't work but at worst neutron might choke on the ports etc because they have values assigned to them that are not supported by the component configuration | 14:06 |
| cloudnull | looking | 14:06 |
| cmorey | at this rate i might be better off investing the time to patch the LSI SAS2008 back into the overcloud | 14:06 |
| cmorey | beagles, ok. i'll quickly nuke the only router in the system | 14:07 |
| cloudnull | weshay|ruck, hjensas - so maybe it is like Tengu (via ramishra) said, LP 1907006 may just be a promotion issue at this point? | 14:09 |
| openstack | Launchpad bug 1907006 in tripleo "standalone deployment victoria failing on AttributeError: 'NoneType' object has no attribute 'items'" [Critical,Triaged] https://launchpad.net/bugs/1907006 | 14:09 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/train: [Container build] raise Exception post logging error https://review.opendev.org/c/openstack/python-tripleoclient/+/764160 | 14:10 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add a debug tasks on pull_success output https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/762400 | 14:10 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/train: Removed ubi-8 based jobs https://review.opendev.org/c/openstack/python-tripleoclient/+/761862 | 14:10 |
| hjensas | cloudnull: it may be promotion issue, or there is some other bug. I find these ansible j2 issues quite hard to see, for example the error that bubbles up without this fix https://review.opendev.org/c/openstack/tripleo-ansible/+/763427/5/tripleo_ansible/roles/tripleo_network_config/templates/multiple_nics/multiple_nics.j2 was similar. | 14:10 |
| openstackgerrit | Rajesh Tailor proposed openstack/python-tripleoclient stable/victoria: Add config-download-dir flag to cell export command https://review.opendev.org/c/openstack/python-tripleoclient/+/765746 | 14:11 |
| hjensas | cloudnull: I mean, it even took ramishra a while to see that. :) | 14:11 |
| openstackgerrit | Rajesh Tailor proposed openstack/python-tripleoclient stable/ussuri: Add config-download-dir flag to cell export command https://review.opendev.org/c/openstack/python-tripleoclient/+/765747 | 14:11 |
| openstackgerrit | Rajesh Tailor proposed openstack/python-tripleoclient stable/ussuri: Add config-download-dir flag to cell export command https://review.opendev.org/c/openstack/python-tripleoclient/+/765747 | 14:13 |
| hjensas | cloudnull: might be we need to merge https://review.opendev.org/c/openstack/tripleo-heat-templates/+/764964 to Victoria? | 14:14 |
| cloudnull | ++ I can keep an eye on that review thoughout the day to try and will it through the gate. | 14:17 |
| cloudnull | looks like you just rechecked it so I'll keep the status page up and watch the paint dry :D | 14:17 |
| *** shyamb has quit IRC | 14:18 | |
| *** TrevorV has joined #tripleo | 14:19 | |
| *** owalsh has quit IRC | 14:19 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Add qemu metadata to compute node when tls for live migration https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765251 | 14:26 |
| ramishra | hjensas, cloudnull commented in LP https://bugs.launchpad.net/tripleo/+bug/1907006/comments/2 | 14:28 |
| openstack | Launchpad bug 1907006 in tripleo "standalone deployment victoria failing on AttributeError: 'NoneType' object has no attribute 'items'" [Critical,Triaged] | 14:28 |
| cloudnull | thank you ramishra | 14:29 |
| *** elGamal has quit IRC | 14:29 | |
| *** tmazur has joined #tripleo | 14:32 | |
| *** elGamal has joined #tripleo | 14:33 | |
| *** ysandeep|afk is now known as ysandeep | 14:33 | |
| openstackgerrit | Martin Schuppert proposed openstack/python-tripleoclient stable/train: [train/stein] Handle UseTLSTransportForNbd for previous created envs https://review.opendev.org/c/openstack/python-tripleoclient/+/760789 | 14:35 |
| *** artom has joined #tripleo | 14:37 | |
| mwhahaha | weshay|ruck: where are we at with getting the stable/ussuri tripleo-common jobs un-broken | 14:40 |
| weshay|ruck | sec | 14:42 |
| *** owalsh has joined #tripleo | 14:42 | |
| cloudnull | anything I can help with on that front? | 14:43 |
| mwhahaha | the switch to providers patch still hasn't landed | 14:43 |
| mwhahaha | it needs to get merged asap | 14:43 |
| mwhahaha | https://review.opendev.org/c/openstack/tripleo-common/+/757821 | 14:43 |
| *** fultonj has joined #tripleo | 14:43 | |
| weshay|ruck | mwhahaha, ya.. there are 3 patches | 14:45 |
| mwhahaha | also we're getting ftbs errors in rdo, looks like podman 1.6.4 is being excluded | 14:46 |
| weshay|ruck | mwhahaha, chkumar|ruck and marios are ruck/rovers this week.. and we've synced and the focus will be on upstream jobs.. and pass rates | 14:47 |
| *** owalsh has quit IRC | 14:47 | |
| weshay|ruck | mwhahaha, the biggest contributor to the suck atm.. is the content-providers pass rate | 14:47 |
| mwhahaha | looks like i should go back to bed. already started out this morning wonderfully by sleeping through my alarm | 14:48 |
| weshay|ruck | it's in the 70 - 80 percent pass.. it's not high enough | 14:48 |
| weshay|ruck | this is also impacting upstream quite a bit https://bugs.launchpad.net/tripleo/+bug/1907006 | 14:50 |
| openstack | Launchpad bug 1907006 in tripleo "standalone deployment victoria failing on AttributeError: 'NoneType' object has no attribute 'items'" [Critical,Triaged] | 14:50 |
| weshay|ruck | which kevin is looking at afaik | 14:51 |
| mwhahaha | fun | 14:52 |
| cloudnull | weshay|ruck looks like that LP is +/- wrapped up with - https://review.opendev.org/q/topic:%22bug%252F1904809%22 | 14:54 |
| weshay|ruck | cloudnull, hjensas do we need to clear queues to land this stuff? | 14:55 |
| cloudnull | also https://bugs.launchpad.net/tripleo/+bug/1907006/comments/2 | 14:55 |
| openstack | Launchpad bug 1907006 in tripleo "standalone deployment victoria failing on AttributeError: 'NoneType' object has no attribute 'items'" [Critical,Triaged] | 14:55 |
| weshay|ruck | ya | 14:55 |
| cloudnull | there may be a promotion issue which is keeping us in purgatory | 14:56 |
| weshay|ruck | cloudnull, naw | 14:56 |
| weshay|ruck | not now that mwhahaha correctly pointed out.. current was fucked | 14:56 |
| weshay|ruck | cloudnull, something merges in a tripleo project.. it should be available upstream immediately | 14:57 |
| cloudnull | ok | 14:57 |
| weshay|ruck | if there is something that is NOT tripleo.. it would need a promotion | 14:57 |
| weshay|ruck | everything in https://review.opendev.org/q/topic:%22bug%252F1904809%22 is tripleo | 14:58 |
| weshay|ruck | this needs a wf https://review.opendev.org/c/openstack/tripleo-heat-templates/+/763379 | 14:58 |
| openstackgerrit | Sergey Galas' proposed openstack/os-net-config master: basic support for Linux tap devices https://review.opendev.org/c/openstack/os-net-config/+/765816 | 14:58 |
| *** owalsh has joined #tripleo | 14:58 | |
| cloudnull | yup that one is the only code impact the other open change is docs | 14:59 |
| cloudnull | also https://review.opendev.org/c/openstack/tripleo-heat-templates/+/764964 - but that is going back for victoria | 15:00 |
| *** xek__ has joined #tripleo | 15:01 | |
| *** xek_ has quit IRC | 15:04 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-specs master: Introduce tripleo-ceph-ganesha spec https://review.opendev.org/c/openstack/tripleo-specs/+/759395 | 15:07 |
| *** ekultails has joined #tripleo | 15:08 | |
| fultonj | mwhahaha: following up on our conversation at the PTG. cephadm will stand up ganesha standalone. tripleo won't need to fork any ceph-ansible code. spec updated to reflect this https://review.opendev.org/c/openstack/tripleo-specs/+/759395 | 15:14 |
| mwhahaha | k | 15:15 |
| openstackgerrit | wes hayutin proposed openstack/python-tripleoclient master: [UX] Align to left the table for the validations run results https://review.opendev.org/c/openstack/python-tripleoclient/+/764642 | 15:15 |
| fultonj | tbarron: gouthamr: want to review the udpated spec ^ ? | 15:16 |
| weshay|ruck | cloudnull, fyi https://6d3496c40c56d07d1060-baa753fa9aa73e2e1b575c4411692489.ssl.cf1.rackcdn.com/765404/1/gate/openstack-tox-py36/4eead1d/job-output.txt in gate | 15:19 |
| weshay|ruck | 36, 38 and linters blew up | 15:19 |
| fultonj | weshay|ruck: do you recommend we not +w jobs until RDO issue is addressed? | 15:22 |
| fultonj | http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019249.html | 15:22 |
| cloudnull | weird ... | 15:22 |
| cloudnull | I wonder if setuptools / pip has decided to make some new breaking changes? | 15:22 |
| * cloudnull spins up reproducer | 15:22 | |
| mwhahaha | yes | 15:22 |
| mwhahaha | that was last weeks thing | 15:23 |
| mwhahaha | 20.4 or whatever | 15:23 |
| cloudnull | \o/ | 15:23 |
| fultonj | noone said not to re-run check-rdo | 15:24 |
| tbarron | fultonj: yessir :) | 15:30 |
| fultonj | thanks tbarron | 15:30 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Only install kolla if working with OS ver<=7 https://review.opendev.org/c/openstack/tripleo-ci/+/764761 | 15:35 |
| *** hberaud has quit IRC | 15:36 | |
| cloudnull | so mwhahaha was there a set of reviews that we landed to address those setuptools / pip changes I can crib from? | 15:39 |
| cloudnull | I guess the same needs to be applied to tripleo-ansible | 15:40 |
| mwhahaha | i don't think there was a change | 15:40 |
| cloudnull | ok | 15:40 |
| mwhahaha | unless it was a global constraint or something | 15:40 |
| mwhahaha | i'm not aware of a tripleo change | 15:40 |
| mwhahaha | though i may just not have seen it | 15:40 |
| cloudnull | -cc weshay|ruck | 15:40 |
| mwhahaha | [qa][all] pip 20.3 breaks jobs | 15:41 |
| mwhahaha | was the upstream ml post about it | 15:41 |
| cloudnull | k | 15:41 |
| cloudnull | I have an 1804 node up and will be trying to reproduce it | 15:41 |
| mwhahaha | they pinned in dev stack | 15:41 |
| cloudnull | see if i can work out a resolution | 15:41 |
| cloudnull | that pin seems like it should go in the python3-tox zuul job definition? | 15:42 |
| *** ysandeep is now known as ysandeep|away | 15:42 | |
| * cloudnull will go read the ML thread on it | 15:42 | |
| gmann | mwhahaha: cloudnull this one - http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019149.html | 15:42 |
| *** hberaud has joined #tripleo | 15:42 | |
| gmann | no actual solution yet but workaround of pinning is merged in devstack. | 15:42 |
| cloudnull | gmann ++ | 15:43 |
| itailev | anyone else encountered deployment errors as in https://bugzilla.redhat.com/show_bug.cgi?id=1904220? | 15:43 |
| openstack | bugzilla.redhat.com bug 1904220 in openstack-tripleo-common "openstack overcloud deploy fails during config-download phase" [High,New] - Assigned to apetrich | 15:43 |
| weshay|ruck | ramishra, http://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a72/765736/1/check/tripleo-ci-centos-8-undercloud-upgrade-victoria/a72b125/logs/undercloud/var/log/extra/package-list-installed.txt | 15:50 |
| weshay|ruck | ramishra, python3-tripleoclient.noarch 13.4.2-0.20201206191839.98fdc7d.el8 @delorean-current | 15:50 |
| ramishra | weshay|ruck: yes I've seen that, how would you explain https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a72/765736/1/check/tripleo-ci-centos-8-undercloud-upgrade-victoria/a72b125/logs/undercloud/home/zuul/undercloud_install.log with that? | 15:51 |
| weshay|ruck | ramishra, just noting.. delorean-current is not a promoted repo | 15:52 |
| ramishra | that's like pretty old code | 15:52 |
| mwhahaha | ramishra: that would be the invocation for stable/ussuri | 15:52 |
| weshay|ruck | ramishra, it's the undercloud install.. it's installling ussuri first | 15:52 |
| ramishra | ussuri that file would be there | 15:53 |
| ramishra | so it's using victoria tht with ussuri tripleoclient? | 15:53 |
| weshay|ruck | that's what an upgrade does... first step | 15:53 |
| mwhahaha | no | 15:53 |
| mwhahaha | not for install | 15:53 |
| mwhahaha | it should be ussuri for all | 15:53 |
| ramishra | yep | 15:54 |
| weshay|ruck | yep | 15:54 |
| * mwhahaha checks package versions | 15:54 | |
| weshay|ruck | http://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a72/765736/1/check/tripleo-ci-centos-8-undercloud-upgrade-victoria/a72b125/logs/undercloud/etc/yum.repos.d/delorean.repo | 15:54 |
| mwhahaha | i know what it is | 15:55 |
| openstackgerrit | Sergey Galas' proposed openstack/os-net-config master: basic support for Linux tap devices https://review.opendev.org/c/openstack/os-net-config/+/765816 | 15:55 |
| mwhahaha | getting repo | 15:55 |
| ramishra | I thought this upgrade jobs do the same release upgrade.. remember hearing that from someone | 15:55 |
| mwhahaha | http://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a72/765736/1/check/tripleo-ci-centos-8-undercloud-upgrade-victoria/a72b125/logs/undercloud/etc/yum.repos.d/gating.repo | 15:55 |
| mwhahaha | so it's fetching THT from the change at teh wrong place | 15:55 |
| mwhahaha | 12.4.x is THT for ussuri | 15:56 |
| mwhahaha | but 13.x is installed | 15:56 |
| mwhahaha | from the gating repo | 15:56 |
| * mwhahaha sighs | 15:56 | |
| weshay|ruck | openstack-tripleo-heat-templates.noarch 13.0.1-0.20201207031758.4d96ae3.el8 @gating-repo | 15:57 |
| * weshay|ruck looks | 15:57 | |
| mwhahaha | so for upgrade jobs, gating repo needs to be the ussuri changes first and then you need a victoria gating repo for the upgrade job | 15:57 |
| mwhahaha | we've not noticed because we're generally backwards compatible for >1 release | 15:57 |
| mwhahaha | though this would account for random failure crap | 15:57 |
| weshay|ruck | agree.. that is how it should work | 15:58 |
| mwhahaha | victoria should still have disable-nova though? | 15:58 |
| ramishra | mwhahaha: no it won't, we dropped that in victoria | 15:58 |
| *** jamesdenton has quit IRC | 15:58 | |
| mwhahaha | i thought it was still available in victoira? | 15:59 |
| *** skramaja has quit IRC | 15:59 | |
| * mwhahaha shrugs | 15:59 | |
| *** jamesdenton has joined #tripleo | 15:59 | |
| ramishra | https://review.opendev.org/c/openstack/tripleo-heat-templates/+/750271 | 15:59 |
| mwhahaha | ehhhhh | 15:59 |
| mwhahaha | sep 16th | 16:00 |
| mwhahaha | heh | 16:00 |
| *** marios|rover has joined #tripleo | 16:01 | |
| *** dhill has quit IRC | 16:14 | |
| cmorey | beagles, the re-deploy seems to have mostly worked, existing instances have been able to re-connect to their volumes, but i appear to have broken new volume creation (which may or may not be related) | 16:18 |
| cmorey | (but i can at least attach a floating ip now, which i couldn't before, so thanks ) | 16:20 |
| *** lmiccini has quit IRC | 16:20 | |
| beagles | cmorey: cool | 16:20 |
| *** holser has joined #tripleo | 16:21 | |
| *** jpich has quit IRC | 16:22 | |
| *** jpich has joined #tripleo | 16:22 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Include default forwarder address in all cases for IPA https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/765342 | 16:38 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Make standalone-ipa jobs voting again - minus train https://review.opendev.org/c/openstack/tripleo-ci/+/762766 | 16:39 |
| openstackgerrit | Martin Magr proposed openstack/tripleo-heat-templates stable/train: Add possibility to set logging source for Horizon https://review.opendev.org/c/openstack/tripleo-heat-templates/+/762217 | 16:39 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-ci master: Make standalone-ipa jobs voting again - minus train https://review.opendev.org/c/openstack/tripleo-ci/+/762766 | 16:39 |
| cmorey | So, new (blank) volumes appear to work, but i can't create a volume from an image | 16:39 |
| cmorey | ah ha,.. i think it didn't like the fact the source image was on the old (pre-ceph) store | 16:42 |
| weshay|ruck | ramishra, mwhahaha I think this spells out the issue https://bugs.launchpad.net/tripleo/+bug/1907122 | 16:44 |
| openstack | Launchpad bug 1907122 in tripleo "gating repo improperly enabled across releases, enabling the wrong patches to be integrated into build" [Critical,Triaged] | 16:44 |
| mwhahaha | yea | 16:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Move enable ksm on compute node to deploy step 1 https://review.opendev.org/c/openstack/tripleo-heat-templates/+/763741 | 16:53 |
| *** rpittau is now known as rpittau|afk | 16:58 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: pytest-html report for tcib container builds https://review.opendev.org/c/openstack/tripleo-ci/+/755124 | 17:01 |
| *** slaweq has quit IRC | 17:05 | |
| *** marios|rover is now known as marios|out | 17:05 | |
| *** slaweq has joined #tripleo | 17:07 | |
| *** marios|out has quit IRC | 17:09 | |
| *** jpich has quit IRC | 17:26 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-common master: Add daxio package for nvdimm scenario https://review.opendev.org/c/openstack/tripleo-common/+/765833 | 17:26 |
| *** dhill has joined #tripleo | 17:30 | |
| *** lucasagomes has quit IRC | 17:33 | |
| *** udesale_ has quit IRC | 17:34 | |
| openstackgerrit | mbu proposed openstack/tripleo-validations master: Add tripleo-validations role for component testing https://review.opendev.org/c/openstack/tripleo-validations/+/755390 | 17:38 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 17:44 |
| openstackgerrit | mbu proposed openstack/tripleo-quickstart-extras master: add validations playbook for VF component testing https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/755382 | 17:44 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-quickstart/+/765835 | 17:47 |
| *** jcapitao has quit IRC | 17:49 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 17:50 |
| *** dsneddon has joined #tripleo | 17:50 | |
| *** cylopez has quit IRC | 17:51 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: tq/release: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-quickstart/+/765835 | 17:51 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: pytest-html report for tcib container builds https://review.opendev.org/c/openstack/tripleo-ci/+/755124 | 17:54 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Network ports module https://review.opendev.org/c/openstack/tripleo-ansible/+/761908 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Introduce role/instance 'networks' key https://review.opendev.org/c/openstack/tripleo-ansible/+/762160 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Populate network ports env module https://review.opendev.org/c/openstack/tripleo-ansible/+/764638 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Add role_net_map to expand roles output https://review.opendev.org/c/openstack/tripleo-ansible/+/764639 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Provison/Unprovision instance network ports https://review.opendev.org/c/openstack/tripleo-ansible/+/764640 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Provision workflow managed/unmanaged node support https://review.opendev.org/c/openstack/tripleo-ansible/+/762162 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Support for unmanaged servers in provision playbook https://review.opendev.org/c/openstack/tripleo-ansible/+/765545 | 17:56 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-ansible master: Expand roles honstname to role mapping https://review.opendev.org/c/openstack/tripleo-ansible/+/765837 | 17:56 |
| *** derekh has quit IRC | 17:58 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: pytest-html report for tcib container builds https://review.opendev.org/c/openstack/tripleo-ci/+/755124 | 18:06 |
| *** dtantsur is now known as dtantsur|afk | 18:28 | |
| *** dprince has quit IRC | 18:37 | |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: pytest-html report for tcib container builds https://review.opendev.org/c/openstack/tripleo-ci/+/755124 | 18:50 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 18:53 |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: Stop NetworkManager from updating resolv.conf https://review.opendev.org/c/openstack/tripleo-ansible/+/765718 | 19:09 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: tq/release: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-quickstart/+/765835 | 19:14 |
| openstackgerrit | Luke Short proposed openstack/python-tripleoclient stable/train: Expose --ansible-forks https://review.opendev.org/c/openstack/python-tripleoclient/+/765855 | 19:24 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 19:25 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates stable/victoria: Ensure cloud-init has finished before puppet run https://review.opendev.org/c/openstack/tripleo-heat-templates/+/765736 | 19:27 |
| *** waleedm has joined #tripleo | 19:32 | |
| *** waleedm has quit IRC | 19:39 | |
| mwhahaha | does anyone remember if zuul merges vars: on a job definition or is it a straight replacement? | 19:51 |
| mwhahaha | i think it's a merge but i don't remember exactly | 19:51 |
| openstackgerrit | Amy Marrich proposed openstack/tripleo-docs master: Added Victoria release https://review.opendev.org/c/openstack/tripleo-docs/+/765692 | 19:53 |
| openstackgerrit | David Vallee Delisle proposed openstack/tripleo-ansible master: tripleo_deploy_artifacts has wrong tar syntax https://review.opendev.org/c/openstack/tripleo-ansible/+/765861 | 19:53 |
| valleedelisle | mwhahaha, regarding this commit ^^ I just realize that it's conflicting with https://review.opendev.org/c/openstack/tripleo-ansible/+/763957 | 19:57 |
| valleedelisle | mwhahaha, should I abandon mine? I made a unittest as well. | 19:57 |
| mwhahaha | there were already tests so i'd abandon yorus | 19:57 |
| mwhahaha | needs someone to approve the other | 19:57 |
| * mwhahaha throws things at cloudnull | 19:57 | |
| valleedelisle | mwhahaha, the test is a bit incomplete imho | 19:58 |
| valleedelisle | but I don't mind abandoning | 19:58 |
| mwhahaha | feel free to rebase yours if you want to improve that | 19:58 |
| valleedelisle | ack will do | 19:58 |
| mwhahaha | imho we shouldn't be testing teh functionality of tar but whatever | 19:58 |
| valleedelisle | ok I'll just abandon I guess | 19:59 |
| *** dciabrin_ has joined #tripleo | 20:02 | |
| valleedelisle | mwhahaha, I was wondering why we redirect stderr? It would have been easier to figure out if we didn't have this pipe redirection. | 20:04 |
| mwhahaha | to capture it in the result | 20:04 |
| mwhahaha | is what it's supposed to do | 20:04 |
| valleedelisle | hmm I didn't see it, maybe I missed it | 20:05 |
| *** dciabrin has quit IRC | 20:05 | |
| mwhahaha | we dont' appear to use it in tyhose | 20:05 |
| mwhahaha | but we shouldn't be spitting stdout/stderr in the output if we can avoid it | 20:05 |
| valleedelisle | ack | 20:06 |
| spotz | Hey all looks like we have a tox dependency issue calling the six module, opendev said it's repo related not system wide | 20:06 |
| mwhahaha | feel free to propose some fixes for that but this isn't really supported outside of a dev/poc concept | 20:06 |
| mwhahaha | spotz: log? | 20:06 |
| spotz | mwhahaha: https://review.opendev.org/c/openstack/tripleo-docs/+/765692 is where I saw it, asked in opendev channel | 20:07 |
| mwhahaha | Could not import extension sphinxcontrib.mermaid (exception: No module named 'six') | 20:07 |
| mwhahaha | likely need to bump sphinx or fix a dep | 20:07 |
| *** matbu has quit IRC | 20:08 | |
| mwhahaha | i doubt it's tripleo specific | 20:08 |
| spotz | clarkb seemed to think so, I thought it might be farther reaching. I'll look for something in another project to patch:) | 20:09 |
| *** matbu has joined #tripleo | 20:09 | |
| mwhahaha | we could fix it in tripleo-docs but it's not something we should likely be specifying | 20:09 |
| spotz | clarkb - looking quickly at that package it doesn't seem to explicitly depend on six but does import it | 20:10 |
| spotz | That's all I got:*( | 20:11 |
| mwhahaha | we can add six to docs/requirements.txt | 20:11 |
| mwhahaha | but that seems totally a hack | 20:11 |
| spotz | mwhahaha: I'm updating my OSA repo let me see what's there | 20:12 |
| mwhahaha | meh it's bogdando's fault | 20:12 |
| mwhahaha | probably for pdf support | 20:12 |
| mwhahaha | https://review.opendev.org/c/openstack/tripleo-docs/+/682362 | 20:12 |
| spotz | Bleh! | 20:13 |
| spotz | So I guess we do need to add it | 20:13 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-docs master: Add six dep for docs https://review.opendev.org/c/openstack/tripleo-docs/+/765866 | 20:13 |
| mwhahaha | added -^ | 20:14 |
| mwhahaha | we'll see if that blows up | 20:14 |
| spotz | heheh | 20:14 |
| *** jamesdenton has quit IRC | 20:15 | |
| spotz | I +1'd but yeah it might be one of those I can't pass cause I need this thing situations:( | 20:15 |
| *** jamesdenton has joined #tripleo | 20:15 | |
| mwhahaha | there should be a six requirement in sphinxcontrib-mermaid | 20:16 |
| mwhahaha | similar to https://github.com/pypa/virtualenv/blob/main/setup.cfg#L46 | 20:16 |
| mwhahaha | so it's a bug against them | 20:16 |
| * mwhahaha files bug | 20:17 | |
| * spotz finds bugs | 20:18 | |
| mwhahaha | https://github.com/mgaitan/sphinxcontrib-mermaid/issues/57 | 20:19 |
| cloudnull | weshay|ruck did we ever get that RPM made for the log collection role ? | 20:19 |
| weshay|ruck | ya | 20:19 |
| weshay|ruck | where you looking for it, rdo or osp? | 20:19 |
| weshay|ruck | https://trunk.rdoproject.org/centos8-master/component/common/09/a8/09a886105a69bc77fe25d4288e3a057faa0a672d_6c7d0507/rpmbuild.log | 20:20 |
| weshay|ruck | from https://trunk.rdoproject.org/centos8-master/status_report.html | 20:20 |
| cloudnull | I have delorean master on my undercloud and was looking for the package | 20:21 |
| cloudnull | to make changes to the sos report bits | 20:21 |
| weshay|ruck | k.. if it's a recent build.. should be there | 20:22 |
| weshay|ruck | like PREGO | 20:22 |
| cloudnull | ah its there, ansible-role-collect-logs I was just searching the wrong name | 20:23 |
| cloudnull | so where do we want to make that a dep? | 20:23 |
| weshay|ruck | good question | 20:24 |
| mwhahaha | we don't | 20:24 |
| * mwhahaha runs | 20:24 | |
| cloudnull | maybe tripleoclient ? | 20:24 |
| weshay|ruck | mwhahaha, go revert some patches would ya :) | 20:24 |
| * cloudnull runs | 20:24 | |
| mwhahaha | no | 20:24 |
| mwhahaha | imho we shouldn't be using that in sos | 20:24 |
| * weshay|ruck fetches my bike | 20:24 | |
| mwhahaha | ANYWAY the question would be where do you need/want it | 20:25 |
| mwhahaha | and i'll tell you where the dep goes | 20:25 |
| mwhahaha | and then i'll -2 it | 20:25 |
| * mwhahaha runs | 20:25 | |
| weshay|ruck | lolz | 20:25 |
| weshay|ruck | just needs to work w/ sos.. so cloudnull the openstack sos plugin rpm I guess | 20:25 |
| cloudnull | I think it would be useful to have a playbook to collect logs across the cluster, maybe a command to facilitate it? | 20:25 |
| cloudnull | I can do that | 20:26 |
| mwhahaha | we already had a command to do that | 20:26 |
| cloudnull | revisit the playbook idea later | 20:26 |
| mwhahaha | fwiw | 20:26 |
| mwhahaha | but it invokes sos | 20:26 |
| mwhahaha | i debate the correctness of using ansible via sos but i've basically just ignored that for now | 20:27 |
| mwhahaha | or ansible to invoke sos to invoke ansible | 20:27 |
| cloudnull | yea, something similar , but not 50GiB. | 20:27 |
| cloudnull | I think the issue now is that sos is basically missing everything useful for tripleo deployments | 20:27 |
| mwhahaha | which i get, but i don't think that role really solves that | 20:28 |
| weshay|ruck | let's write a new role | 20:28 |
| * weshay|ruck runs | 20:28 | |
| cloudnull | IDK if it solves it either, but i feel like it makes it better? | 20:28 |
| mwhahaha | let's not write a role | 20:28 |
| mwhahaha | questionable but whatevers | 20:29 |
| weshay|ruck | the parts that are nice imho.. is the container scraping | 20:29 |
| weshay|ruck | re: logs, rpms.. etc | 20:29 |
| mwhahaha | that doesn't need a role | 20:29 |
| mwhahaha | that would be good outside of openstack | 20:29 |
| spotz | pre-introspection failure questions - here or ironic?:) | 20:29 |
| mwhahaha | ironic | 20:29 |
| weshay|ruck | mwhahaha, so.. upstream and internal ci are using the role | 20:30 |
| spotz | thanks | 20:30 |
| weshay|ruck | why not sos | 20:30 |
| mwhahaha | weshay|ruck: because customers aren't CI | 20:30 |
| mwhahaha | but carry on | 20:30 |
| mwhahaha | the logs required out of customers are not 1:1 CI | 20:30 |
| weshay|ruck | ya.. ok | 20:30 |
| weshay|ruck | so what | 20:31 |
| mwhahaha | that role is a bazooka | 20:31 |
| weshay|ruck | yes... and when customers run sos... that's what you want no/ | 20:31 |
| mwhahaha | we don't need too grab 100G of logs out of our customers | 20:31 |
| weshay|ruck | ? | 20:31 |
| weshay|ruck | it's not that much | 20:31 |
| mwhahaha | that role is a solution for requirements we have not generated | 20:31 |
| mwhahaha | we know it works for CI, but we don't requirements around what we're missing from sos and the priority of things we should collect that way | 20:32 |
| mwhahaha | i also don't necessarily agree we should be going into our customer containers and running stuff | 20:33 |
| weshay|ruck | it can be an option no? | 20:33 |
| mwhahaha | i don't know, because we don't have requirements? | 20:33 |
| weshay|ruck | and we never will | 20:33 |
| mwhahaha | sooo should we be solving for something we don't have requirements for? | 20:33 |
| weshay|ruck | if it makes our lives better, imho yes | 20:33 |
| mwhahaha | kinda think we've done that in the past and it hasn't necessarily worked out for the best | 20:33 |
| mwhahaha | so i think it's skipping the fact that we have people who's job is to pinpoint this stuff before it gets to the point where we get involved | 20:34 |
| weshay|ruck | cloudnull, can we make it an option, default-collect vs.. extensive-collect | 20:34 |
| mwhahaha | and because we're not actually working with those folks, we're working around that issue | 20:34 |
| weshay|ruck | mwhahaha, fwiw.. morazi asked for it | 20:34 |
| mwhahaha | ok, what was his requirements/reasoning? | 20:35 |
| *** JayF has joined #tripleo | 20:35 | |
| weshay|ruck | meh.. no clue | 20:35 |
| *** JayF has left #tripleo | 20:35 | |
| mwhahaha | right | 20:35 |
| mwhahaha | so that's likely the larger issue | 20:35 |
| weshay|ruck | other than getting the rpms off the installed env | 20:35 |
| weshay|ruck | and containers | 20:35 |
| mwhahaha | so i think container reporting would be beneficial but i don't think that role is the solution | 20:35 |
| weshay|ruck | he wants to diff a straight.. sub-man what's avail vs. what's actually installed | 20:36 |
| mwhahaha | don't think we should be doing that | 20:36 |
| cloudnull | weshay|ruck sure | 20:36 |
| weshay|ruck | mwhahaha, if you have the default avail, what's the diff? | 20:36 |
| mwhahaha | default avail ? | 20:37 |
| spotz | mwhahaha: Got sent back here:) | 20:37 |
| weshay|ruck | default behavior of sos, and the openstack plugin | 20:37 |
| weshay|ruck | lolz | 20:37 |
| mwhahaha | spotz: what's the error? | 20:37 |
| spotz | On tripleo validator run pre-introspection should I be concerned about failures on undercloud-neutron-sanity-check and ctlplane-ip-range. I did verify the ctlplane-ip-range is 20 IPs and the nodes themselves did introspect once I corrected a MAC issue | 20:37 |
| spotz | I had to bump the ram and cpu higher to get that to pass so have a note to update the docs accordingly if needed | 20:38 |
| mwhahaha | weshay|ruck: i'd like to have further discussion around this log/sos integration before we keep going. i question motivation and wonder if it's covering for something else | 20:38 |
| mwhahaha | spotz: ¯\_(ツ)_/¯ | 20:39 |
| mwhahaha | i have no idea what those validations are | 20:39 |
| spotz | The weird thing is the docs say minumum of 20 IPs doe the ctlplace IP range and technically I have 21. It's just weird | 20:40 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 20:40 |
| spotz | I'll keep going and see if it deploys | 20:40 |
| mwhahaha | if you have validation problems, Tengu matbu dpeacock might be your people. though check again tomorrow morning | 20:41 |
| * mwhahaha drives the bus over validations and makes sure to throw it in reverse a few times | 20:41 | |
| mwhahaha | cloudnull: where's the rfe around the sos/log collect thing | 20:43 |
| spotz | hehe | 20:44 |
| cloudnull | there is one , one sec | 20:44 |
| cloudnull | https://bugzilla.redhat.com/show_bug.cgi?id=1475040 | 20:45 |
| openstack | bugzilla.redhat.com bug 1475040 in rhosp-director "[RFE] Director plug-in to collect logs from all Openstack nodes to ease log collection" [Medium,New] - Assigned to aschultz | 20:45 |
| mwhahaha | right the thing that's already been implemented | 20:45 |
| mwhahaha | so where's teh ask to do the additional log collection | 20:45 |
| cloudnull | per-phil's comment and the convo w/ morazi / weshay|ruck it seems like there was a desire for more indepth collection | 20:47 |
| mwhahaha | right so that should be a new rfe with details | 20:48 |
| cloudnull | but this is the only RFE along these lines that i'm aware of | 20:48 |
| mwhahaha | because that one I implemented back in 2017 | 20:48 |
| mwhahaha | and the only reason it was still open because we don't have testing | 20:48 |
| cloudnull | ok | 20:48 |
| mwhahaha | https://review.opendev.org/q/topic:%2522bp/capture-environment-status-and-logs%2522 | 20:48 |
| mwhahaha | and this ask is a giant pet peeve of mine | 20:48 |
| mwhahaha | so please include me on any discussions around it | 20:49 |
| mwhahaha | since i implemented it 3 years ago and no one wants to own it | 20:49 |
| cloudnull | sure thing 👍 | 20:49 |
| spotz | oh weird no nova endpoint:( | 20:50 |
| mwhahaha | spotz: nova is no longer deployed as of victoria | 20:50 |
| mwhahaha | spotz: what version are you using? | 20:50 |
| spotz | mwhahaha: Victoria:) I'm working on a blog on virtual tripleo deployment | 20:51 |
| mwhahaha | you need to use the new baremetal deployment method | 20:51 |
| mwhahaha | so openstack overcloud node provision | 20:51 |
| mwhahaha | etc | 20:51 |
| spotz | Which is why I'll have a bunch of doc patches potentially | 20:51 |
| spotz | Ok I'll go back to the docs:) | 20:52 |
| mwhahaha | it's in the docs | 20:52 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Build container images in ci centos gate job https://review.opendev.org/c/openstack/tripleo-quickstart/+/763548 | 20:52 |
| mwhahaha | just likely not well formatted | 20:52 |
| mwhahaha | https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/provisioning/baremetal_provision.html | 20:52 |
| *** haleyb has quit IRC | 20:53 | |
| spotz | Yeah I was still using MAC and switched to ports for my instack.yaml. So lots of changes between train and victoria | 20:54 |
| *** haleyb has joined #tripleo | 20:54 | |
| weshay|ruck | mwhahaha, cloudnull k.. going back to mang.. and asking for more details | 20:54 |
| mwhahaha | weshay|ruck: i agree our logging is likely insufficient, but we need the right people to say what's missing and not just collect the world | 20:55 |
| weshay|ruck | meh.. I disagree.. by asking your going to get 10 people that say 10 different things and 8 of them will be worthless | 20:56 |
| mwhahaha | no | 20:57 |
| mwhahaha | because in this case we want the 10 different things | 20:57 |
| mwhahaha | to make sure that our implementation captures the 10 diferent things | 20:57 |
| weshay|ruck | mwhahaha, not mention that only collects what you ask it to | 20:57 |
| mwhahaha | so it's useless because we don't know what we want | 20:58 |
| mwhahaha | got it | 20:58 |
| weshay|ruck | ultimately it comes back to this group if support can't figure it out.. w/ what's collected now via sos, it seems worthless to me | 20:59 |
| mwhahaha | my push back has been that we don't have people who are driving this and owning it | 20:59 |
| weshay|ruck | if you don't want 100mb of logs.. then just configure it to collect less | 20:59 |
| mwhahaha | (going on years now) | 20:59 |
| mwhahaha | so just collecting the world is not addressing teh issue | 20:59 |
| weshay|ruck | we don't need to collect the world | 21:00 |
| mwhahaha | we are not solving the actual problem | 21:00 |
| mwhahaha | we're ignoring it and working around it without making sure we're fixing what we actually need to fix | 21:00 |
| weshay|ruck | I would think gss would be the authority here | 21:01 |
| mwhahaha | which should be providing requirements | 21:01 |
| weshay|ruck | k.. | 21:01 |
| mwhahaha | the reality is that like openstackclient and things customers have to use, we don't have people who actually work on improving them and making sure they solve things for them | 21:02 |
| mwhahaha | log collection and what people actually need to solve problems needs a driver and just collecting all the things is incorrect | 21:02 |
| openstackgerrit | Rafael Folco proposed openstack/tripleo-ci master: pytest-html report for tcib container builds https://review.opendev.org/c/openstack/tripleo-ci/+/755124 | 21:03 |
| mwhahaha | our sos plugins have been ignored for years and we're missing logs because they were never updated | 21:03 |
| weshay|ruck | ya | 21:06 |
| *** raildo has quit IRC | 21:07 | |
| mwhahaha | weshay|ruck: so we can ship that role independently | 21:07 |
| mwhahaha | i got no problems with that | 21:07 |
| mwhahaha | but i don't think it should be used to improve log collection for the openstack sos stuff | 21:07 |
| weshay|ruck | fine | 21:07 |
| mwhahaha | having support be able to provide a playbook to fetch stuff would be greatr | 21:08 |
| mwhahaha | but sosreport should be fixed to actually have stuff we're missing | 21:08 |
| weshay|ruck | ya | 21:08 |
| mwhahaha | these are the things i'm pointing out in that they solve different problems | 21:08 |
| mwhahaha | both may be needed | 21:08 |
| mwhahaha | but we need to understand what the issues are | 21:08 |
| mwhahaha | requiring that role via tripleoclient would satisfy that shipping by default for the undercloud | 21:10 |
| *** efoley has quit IRC | 21:15 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fallback to rdo registry if quay gets down https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/765731 | 21:16 |
| *** jamesdenton has quit IRC | 21:18 | |
| *** jamesdenton has joined #tripleo | 21:18 | |
| *** sshnaidm has quit IRC | 21:49 | |
| openstackgerrit | mbu proposed openstack/tripleo-quickstart-extras master: add validations playbook for VF component testing https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/755382 | 21:57 |
| *** slaweq has quit IRC | 21:58 | |
| openstackgerrit | mbu proposed openstack/tripleo-validations master: Add tripleo-validations role for component testing https://review.opendev.org/c/openstack/tripleo-validations/+/755390 | 21:58 |
| *** rcernin has joined #tripleo | 22:00 | |
| *** hberaud has quit IRC | 22:02 | |
| *** odyssey4me has quit IRC | 22:02 | |
| *** johnsom has quit IRC | 22:02 | |
| *** LaurentDumont has quit IRC | 22:02 | |
| *** sshnaidm has joined #tripleo | 22:08 | |
| *** pcaruana has quit IRC | 22:21 | |
| openstackgerrit | mbu proposed openstack/tripleo-validations master: Add tripleo-validations role for component testing https://review.opendev.org/c/openstack/tripleo-validations/+/755390 | 22:26 |
| openstackgerrit | mbu proposed openstack/tripleo-quickstart-extras master: add validations playbook for VF component testing https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/755382 | 22:27 |
| *** mburned has quit IRC | 22:28 | |
| *** xek__ has quit IRC | 22:28 | |
| *** fuzzball81 has quit IRC | 22:29 | |
| *** LaurentDumont has joined #tripleo | 22:30 | |
| *** hberaud has joined #tripleo | 22:30 | |
| *** odyssey4me has joined #tripleo | 22:30 | |
| *** johnsom has joined #tripleo | 22:30 | |
| *** rfolco has quit IRC | 22:34 | |
| morazi | weshay|ruck, fwiw I said it seemed like it would be useful for providing support and detecting drift in what is installed vs what we would expect to be installed but I didn't ask for an rfe to go into director I don't think | 22:40 |
| morazi | weshay|ruck, I was originally thinking it would be an external thing though | 22:41 |
| mwhahaha | we don't have a way to do that | 22:43 |
| mwhahaha | other than rpm -v | 22:43 |
| *** rfolco has joined #tripleo | 22:43 | |
| mwhahaha | and the log collection isn't related to that necessarily | 22:43 |
| mwhahaha | there are other things like container versions but that's a different thing as well | 22:43 |
| morazi | indeed. that is probably a telling thing re: when I originally brought the idea up. ;-) | 22:44 |
| morazi | mwhahaha, but yea, originally I had suggested that it might be handy to be able to operate on a sos-report but it wasn't really within the context of ooo directly when I brought it up | 22:46 |
| mwhahaha | we can solve for somethings but we need to understand the common issues | 22:47 |
| weshay|ruck | the openstack stack plugin is ooo? | 22:47 |
| weshay|ruck | news to me | 22:47 |
| *** rfolco has quit IRC | 22:48 | |
| mwhahaha | it's not | 22:48 |
| mwhahaha | it's in sos | 22:48 |
| mwhahaha | there are sos plugins for each service | 22:48 |
| mwhahaha | they all need updating | 22:48 |
| mwhahaha | the configs don't work since queens with containerization | 22:48 |
| weshay|ruck | seems like something a pm should figure out | 22:50 |
| mwhahaha | a novel concept | 22:50 |
| morazi | at the time, we were seeing a lot of issues with clearly out of date deployments that were not being flagged as such. I suspect that is likely still the case, but I'd imagine now it is probably more useful to quickly see the relationship between deployed containers and registry | 22:50 |
| * mwhahaha has been asking for years now | 22:50 | |
| weshay|ruck | mwhahaha, have you tried flipping a table? | 22:50 |
| mwhahaha | many | 22:50 |
| mwhahaha | i gave up and let kevin take over | 22:50 |
| * weshay|ruck knows what I'm getting mwhahaha for xmas | 22:50 | |
| * mwhahaha runs away | 22:50 | |
| morazi | weshay|ruck, getting him more tables? | 22:51 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 22:52 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: tq/release: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-quickstart/+/765835 | 23:02 |
| cloudnull | anyone around mind giving https://review.opendev.org/c/openstack/tripleo-ansible/+/757648 a look ? | 23:07 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Remove /etc/ceph dependency on puppet services https://review.opendev.org/c/openstack/puppet-tripleo/+/763545 | 23:20 |
| *** hoonetorg has quit IRC | 23:20 | |
| *** mvalsecc has joined #tripleo | 23:22 | |
| *** cmorey has quit IRC | 23:30 | |
| openstackgerrit | Shnaidman Sagi proposed openstack/tripleo-ci master: toci- ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-ci/+/765834 | 23:33 |
| sshnaidm | weshay|ruck, ^^ | 23:33 |
| *** hoonetorg has joined #tripleo | 23:34 | |
| openstackgerrit | Shnaidman Sagi proposed openstack/tripleo-quickstart master: tq/release: ensure the gating repo is release aware https://review.opendev.org/c/openstack/tripleo-quickstart/+/765835 | 23:37 |
| *** stevebaker has quit IRC | 23:38 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!