| *** dviroel|afk is now known as dviroel | 00:41 | |
| *** dviroel is now known as dviroel|out | 00:50 | |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add standalone ovn-controller role https://review.opendev.org/c/openstack/tripleo-ansible/+/845653 | 01:04 |
|---|---|---|
| *** rlandy is now known as rlandy|out | 01:58 | |
| opendevreview | gaobin proposed openstack/tripleo-heat-templates master: fixed word case https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857249 | 02:06 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-common master: Drop unnecessary usage of staticmethod https://review.opendev.org/c/openstack/tripleo-common/+/857250 | 02:08 |
| lecris[m] | jm1: jpodivin Thanks for the support yesterday, the issue was indeed a red-herring. The actual cause was a TLS issue which makes much more sense for when it was occurring | 02:47 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 03:12 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-heat-templates master: Move host_prep_tasks to standalone tripleo_neutron https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857253 | 03:16 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 03:22 |
| opendevreview | Manojkatari proposed openstack/puppet-tripleo master: Add missing parameters for NFS cinder backends. https://review.opendev.org/c/openstack/puppet-tripleo/+/854315 | 03:24 |
| opendevreview | Manojkatari proposed openstack/puppet-tripleo master: Add missing parameters for RBD cinder backends. https://review.opendev.org/c/openstack/puppet-tripleo/+/857077 | 03:24 |
| lecris[m] | Anyone know where the TLS configuration for sqlalchemy/oslo/keystone (internal connection to sql part) are? | 03:25 |
| lecris[m] | I have looked for something like `oslo_config`, but that is already deprecated and removed | 03:26 |
| opendevreview | Takashi Kajinami proposed openstack/puppet-pacemaker master: TripleO: Switch to CentOS Stream 9 job https://review.opendev.org/c/openstack/puppet-pacemaker/+/840970 | 03:32 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 03:35 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 03:54 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 03:58 |
| opendevreview | David Sedgmen proposed openstack/puppet-tripleo stable/train: Resues fix from "Always update the local certmonger ca cert" https://review.opendev.org/c/openstack/tripleo-heat-templates/+/785020 [Train ONLY] in wallby this was moved to tripleo-ansible and is not straight forward to refactor into the current implementation in tripleo-ansible https://review.opendev.org/c/openstack/puppet-tripleo/+/855310 | 04:21 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 04:21 |
| *** ysandeep|out is now known as ysandeep | 04:42 | |
| tkajinam | bhagyashris|ruck, hi. I've enabled promotion blocker tag on https://bugs.launchpad.net/tripleo/+bug/1974047 because it has been blocking puppet-pacemaker CI for a while. I was trying to find out the case but could not find any problem so far and need some help/eyes with it | 04:51 |
| tkajinam | it's quite strange the job is passing in the other repos but fails only in puppet-pacemaker repo | 04:51 |
| bhagyashris|ruck | tkajinam, for cix you will need to add the milestone on bug | 04:53 |
| bhagyashris|ruck | will check | 04:53 |
| tkajinam | bhagyashris|ruck, ah, thanks for reminder. I added the milestone | 04:55 |
| ykarel | tkajinam, it's failing because it's using ubi8 as base image, should be c9 stream image | 04:56 |
| ykarel | the reason it's passing other places might be other places use content provider jobs | 04:57 |
| ykarel | setting containers_base_image: "quay.io/centos/centos:stream9" should work for that job | 04:59 |
| ykarel | infact containers_base_image: registry.access.redhat.com/ubi9:latest should be used as used in other jobs | 05:01 |
| tkajinam | ykarel, hmm ok. yeah that is the problem. though I don't know why we do not get it from the centos 9 sc004 job template | 05:02 |
| *** ysandeep is now known as ysandeep|brb | 05:02 | |
| opendevreview | Takashi Kajinami proposed openstack/puppet-pacemaker master: TripleO: Switch to CentOS Stream 9 job https://review.opendev.org/c/openstack/puppet-pacemaker/+/840970 | 05:02 |
| ykarel | tkajinam, since only one tripleo job is running there, running content provider additionally doesn't make much sense as it will require one additional node for full job run | 05:04 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-ci master: Force usage of ubi9 image for all CentOS 9 standalone jobs https://review.opendev.org/c/openstack/tripleo-ci/+/857261 | 05:04 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-ci master: Force usage of ubi9 image for all CentOS 9 standalone jobs https://review.opendev.org/c/openstack/tripleo-ci/+/857261 | 05:06 |
| tkajinam | ykarel, I'll workaround by adding override in puppet-pacemaker but ideally we need something like this ^^^ | 05:06 |
| tkajinam | ykarel, by the way thanks a ton for quickly pointing that ! | 05:06 |
| * lecris[m] sent a code block: https://matrix.org/_matrix/media/r0/download/matrix.org/JzTXVtOYEkbFgqQWBAlCHSVp | 05:07 | |
| ykarel | tkajinam, yes agree default needs to be fixed, but not sure if we need that at job level | 05:10 |
| ykarel | will check that letter | 05:10 |
| tkajinam | it should be good as long as we can unblock the gate. | 05:16 |
| lecris[m] | Is this configuration with `/usr/share/openstack-tripleo-heat-templates/environments/services/ironic-overcloud.yaml` being tested in the CI? | 05:27 |
| ykarel | i don't think ^ being tested in CI | 05:31 |
| lecris[m] | Ok. There's an issue with the deployment currently: https://bugs.launchpad.net/tripleo/+bug/1989405 | 05:33 |
| opendevreview | Merged openstack/tripleo-heat-templates master: Undercloud install: Create clouds.yaml with mode 600 https://review.opendev.org/c/openstack/tripleo-heat-templates/+/855881 | 05:33 |
| lecris[m] | But isn't nova-ironic being migrated to an ansible-role? | 05:34 |
| lecris[m] | This one https://review.opendev.org/c/openstack/tripleo-ansible/+/855873? | 05:35 |
| tkajinam | lecris[m], that one does not touch nova-ironic | 05:35 |
| *** ysandeep|brb is now known as ysandeep | 05:35 | |
| tkajinam | just only ironic-api | 05:35 |
| tkajinam | and ironic_pxe_* | 05:35 |
| lecris[m] | Kay, nothing on nova-ironic currently then? | 05:36 |
| tkajinam | there is nothing on-going now | 05:37 |
| opendevreview | Luis Tomas Bolivar proposed openstack/tripleo-heat-templates stable/wallaby: Ensure ovn_bgp_agent parameters are properly handled https://review.opendev.org/c/openstack/tripleo-heat-templates/+/854575 | 05:40 |
| opendevreview | chandan kumar proposed openstack/tripleo-common master: [DNM] Build tripleo-ansible-ee container https://review.opendev.org/c/openstack/tripleo-common/+/850458 | 05:40 |
| lecris[m] | I am comparing `nova-conductor-container-puppet.yaml` with `nova-ironic-container-puppet` that call the same `nova_statedir_owner`, but I don't see any apparent difference | 05:42 |
| opendevreview | Luis Tomas Bolivar proposed openstack/tripleo-heat-templates stable/wallaby: Ensure ovn_bgp_agent parameters are properly handled https://review.opendev.org/c/openstack/tripleo-heat-templates/+/854575 | 05:43 |
| opendevreview | Luis Tomas Bolivar proposed openstack/tripleo-heat-templates stable/wallaby: Ensure ovn_bgp_agent parameters are properly handled https://review.opendev.org/c/openstack/tripleo-heat-templates/+/854575 | 05:44 |
| lecris[m] | Oh found it: https://opendev.org/openstack/tripleo-heat-templates/commit/cb889805334a7cd7325b2a9a1efe2bd00bd48c31 | 05:45 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-heat-templates master: nova-ironic: Fix selinux denial when relabeling /var/lib/nova https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857263 | 05:45 |
| opendevreview | Merged openstack/tripleo-ansible master: Improve and correct nftables role https://review.opendev.org/c/openstack/tripleo-ansible/+/856487 | 05:45 |
| tkajinam | lecris[m], try this ^^^ | 05:45 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-heat-templates master: nova-ironic: Fix selinux denial when relabeling /var/lib/nova https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857263 | 05:46 |
| lecris[m] | Yeah, I'll check it, but it should be just that | 05:47 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 05:47 |
| lecris[m] | Maybe the hole commit will need to be reverted, I'll keep you posted on the review there | 05:48 |
| lecris[m] | s/hole/whole/ | 05:48 |
| tkajinam | lecris[m], are you using centos 8 or centos 9 ? | 05:49 |
| lecris[m] | 9 | 05:49 |
| tkajinam | ok | 05:49 |
| tkajinam | seems we are running CI with selinux permissive :-( | 05:50 |
| lecris[m] | Oh, there's ya problem :D | 05:51 |
| lecris[m] | Also about my various TLS issues, https://bugs.launchpad.net/tripleo/+bug/1989395: It seems that overriding `tripleo_internal_tls_ca_file` with the file in `InternalTLSCAFile` solves it. I have orverriden on the whole `tripleo_container_standalone` for testing, but probably it needs to be patched on all service heat templates | 05:53 |
| lecris[m] | It's a niche deployment scenario so I don't think it is/can be covered by CI | 05:58 |
| opendevreview | yatin proposed openstack/openstack-tempest-skiplist master: Revert "Skip neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON" https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/857067 | 06:18 |
| opendevreview | Luca Miccini proposed openstack/tripleo-heat-templates master: Ensure /run/frr is present after (re)boot https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857265 | 06:19 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 06:24 |
| *** tkajinam is now known as Guest162 | 06:33 | |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 06:41 |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 06:46 |
| Tengu | hello there! folks, care to have a look at https://review.opendev.org/c/openstack/tripleo-ansible/+/853481 and, maybe, nudget it to the gate? :) | 06:53 |
| Tengu | tkajinam: we're at one +W on https://review.opendev.org/q/topic:standalone%252Fapache to be able to get rid of puppetlabs-apache :) | 06:54 |
| Tengu | «just saying» ;) | 06:54 |
| opendevreview | Marios Andreou proposed openstack/tripleo-upgrade stable/queens: Remove linter testing as pip dependencies won't install anymore. https://review.opendev.org/c/openstack/tripleo-upgrade/+/856723 | 07:20 |
| opendevreview | Marios Andreou proposed openstack/tripleo-upgrade stable/queens: Remove linter testing as pip dependencies won't install anymore. https://review.opendev.org/c/openstack/tripleo-upgrade/+/856723 | 07:25 |
| marios | chem: fyi left you a comment - you have duplicate zuul layout thats why it was running :D | 07:27 |
| marios | chem: ^^ | 07:27 |
| chem | marios: thanks :) | 07:27 |
| Tengu | chandankumar: thanks for your comment on the tripleo_httpd_* change proposal. | 07:30 |
| *** jm1|ruck is now known as jm1|rover | 07:33 | |
| *** jpena|off is now known as jpena | 07:37 | |
| opendevreview | Sofer Athlan-Guyot proposed openstack/tripleo-upgrade stable/queens: Remove linter testing as pip dependencies won't install anymore. https://review.opendev.org/c/openstack/tripleo-upgrade/+/856723 | 07:44 |
| opendevreview | Sofer Athlan-Guyot proposed openstack/tripleo-upgrade stable/queens: Ensure fencing is correctly handled during update https://review.opendev.org/c/openstack/tripleo-upgrade/+/856126 | 07:46 |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Removing superfluous imports and variable https://review.opendev.org/c/openstack/validations-libs/+/857037 | 07:48 |
| chandankumar | Tengu: thank you, :-) please have a look https://review.opendev.org/c/openstack/tripleo-ansible/+/855358 when free! | 07:48 |
| Tengu | an easy one :) | 07:50 |
| opendevreview | Sofer Athlan-Guyot proposed openstack/tripleo-heat-templates master: Ensure container's image get updated if their name stay the same. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/850933 | 07:51 |
| opendevreview | Manojkatari proposed openstack/puppet-tripleo master: Add missing parameters for RBD cinder backends. https://review.opendev.org/c/openstack/puppet-tripleo/+/857077 | 08:02 |
| chandankumar | tkajinam: please have a look at this one https://review.opendev.org/c/openstack/tripleo-ci/+/850736 when around, thank you :-) | 08:03 |
| tkajinam | Tengu, I've left a comment in the first tripleo-ansible patch. the 2nd one looks good. the tht patch also looks good but I'm still trying to understand how we set option for a specific vhost | 08:04 |
| opendevreview | mbu proposed openstack/python-tripleoclient master: Add Ansible execution environment support https://review.opendev.org/c/openstack/python-tripleoclient/+/776564 | 08:05 |
| Tengu | tkajinam: ah, the per-vhost/service options: it's done using t-h-t and the custom ansible role, such as tripleo_ironic. | 08:06 |
| Tengu | tkajinam: t-h-t can use a "custom namespace" in the group_var, like I generate in the new apache-baremetal-ansible file, using the SERVICE_NAME as a distinct """namespace""" - it's then translated in the service ansible role, such as tripleo_ansible | 08:07 |
| tkajinam | that basically requires re-defining the same vars in each role, right ? | 08:10 |
| Tengu | tkajinam: yeah, more or less: https://review.opendev.org/c/openstack/tripleo-ansible/+/855873/14/tripleo_ansible/roles/tripleo_ironic/tasks/api_httpd.yaml#19 | 08:11 |
| Tengu | if this answers your question... | 08:12 |
| Tengu | BUT note: those "namespaced keys" are needed only for the global things we're setting in the apache-baremetal-ansible | 08:12 |
| Tengu | other keys are just set in the t-h-t service file, such as https://review.opendev.org/c/openstack/tripleo-heat-templates/+/854568/18/deployment/ironic/ironic-api-container-puppet.yaml#377 | 08:13 |
| tkajinam | I'm saying this from my curiosity but I'm wondering whether we can pass a single dict to override each options | 08:13 |
| Tengu | so mostly... we're talking "only" about some keys like TLS configuration, and module loading | 08:13 |
| tkajinam | it might be overkilling as we intentionally selected what we believe useful only and expose these | 08:13 |
| Tengu | I thought about that, but I think the way I implemented it is easier to understand and to maintain | 08:13 |
| Tengu | especially since the apache-baremetal-ansible is centralizing some actions around TLS certificates. | 08:14 |
| opendevreview | Manojkatari proposed openstack/puppet-tripleo master: Add missing parameters for NFS cinder backends. https://review.opendev.org/c/openstack/puppet-tripleo/+/854315 | 08:16 |
| tkajinam | Tengu, we don | 08:19 |
| tkajinam | sorry. pushed Enter too early | 08:19 |
| tkajinam | Tengu, we don't need to expose every parameter because some parameters might be for internal use. My concern is that in case we found one good vhost option which we want to make configurable then we might end up adding the same parameter in multiple roles. | 08:20 |
| tkajinam | as we have done in puppet for long. After adding a single parameter in puppet-openstacklib, I always need to update wsgi::apache in 15+ modules to allow usage of that parameter | 08:21 |
| Tengu | tkajinam: sorry, on a call | 08:23 |
| tkajinam | Tengu, no problem | 08:23 |
| tkajinam | chandankumar, done | 08:23 |
| opendevreview | Manojkatari proposed openstack/puppet-tripleo master: Add missing parameters for RBD cinder backends. https://review.opendev.org/c/openstack/puppet-tripleo/+/857077 | 08:23 |
| chandankumar | tkajinam: thanks! | 08:24 |
| chandankumar | Tengu: https://review.opendev.org/c/openstack/tripleo-ci/+/850736 another one good to go! | 08:25 |
| opendevreview | Takashi Kajinami proposed openstack/tripleo-heat-templates master: nova-ironic: Fix selinux denial when relabeling /var/lib/nova https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857263 | 08:36 |
| *** ysandeep is now known as ysandeep|lunch | 09:06 | |
| opendevreview | Sofer Athlan-Guyot proposed openstack/tripleo-heat-templates master: Ensure container's image get updated if their name stay the same. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/850933 | 09:12 |
| opendevreview | wangjiaqi proposed openstack/os-net-config master: Use py3 as the default runtime for tox https://review.opendev.org/c/openstack/os-net-config/+/857302 | 09:13 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-ansible master: Ensure the openvswitch service is enabled and deps are installed https://review.opendev.org/c/openstack/tripleo-ansible/+/855975 | 09:26 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-ansible master: Ensure the openvswitch service is enabled and deps are installed https://review.opendev.org/c/openstack/tripleo-ansible/+/855975 | 09:26 |
| opendevreview | Cristian Le proposed openstack/tripleo-heat-templates master: Add internal TLS CA file to group vars https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857336 | 09:39 |
| lecris[m] | ^ I think there should be a better place to put this instead of the keystone, but at least it does resolve the discrepancy issue | 09:41 |
| jm1 | Tengu: o/ trying again today XD who can fix this cap issue? https://bugs.launchpad.net/tripleo/+bug/1989247 | 09:41 |
| lecris[m] | jm1: It is a red herring, the issue seems to be something else | 09:42 |
| lecris[m] | It would be great to remove that error so that it doesn't confuse others though | 09:43 |
| jm1 | lecris[m]: exactly ;) | 09:43 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-ansible master: Ensure the openvswitch service is enabled and deps are installed https://review.opendev.org/c/openstack/tripleo-ansible/+/855975 | 09:45 |
| lecris[m] | Weird ipa behaviour. After the deployment finished, I cannot dig to the top-level ipa dns zone. I thought it was a hiccup, but it happened twice now | 09:48 |
| opendevreview | Merged openstack/tripleo-quickstart-extras master: Fix resolution of tripleo_ceph_deploy_container_namespace https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/857201 | 09:50 |
| Tengu | tkajinam: back - but away for lunch again, sorry! took some more time. | 09:55 |
| Tengu | jm1: well, sorry - was on a call - now lunch... I'll read the LP and come back. | 09:56 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-heat-templates master: Use python to template cell urls https://review.opendev.org/c/openstack/tripleo-heat-templates/+/851917 | 09:56 |
| Tengu | jm1, lecris[m] you can `git grep cap_add' in tripleo-heat-templates | 09:57 |
| Tengu | this is how you can add capabilities to containers. | 09:57 |
| Tengu | should solve your thing. | 09:57 |
| Tengu | #lunch | 09:57 |
| lecris[m] | How can I debug 503 issue for horizon? | 10:01 |
| jm1 | Tengu: can u send a patch for that cap_add thingy? | 10:04 |
| jm1 | Tengu: pls :) | 10:04 |
| lecris[m] | I forgot for which service this one occured | 10:05 |
| lecris[m] | Looks pretty straightforward though | 10:06 |
| lecris[m] | Oh, but it has to be handled on tripleo-ansible | 10:08 |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Logging facility for VF https://review.opendev.org/c/openstack/validations-libs/+/857198 | 10:15 |
| opendevreview | Cristian Le proposed openstack/tripleo-ansible master: Add audit write cap https://review.opendev.org/c/openstack/tripleo-ansible/+/857338 | 10:15 |
| lecris[m] | jm1: Do you remember other services that could have this issue? | 10:15 |
| lecris[m] | Though I don't know if adding this like such is appropriate so open for discussion | 10:16 |
| opendevreview | chandan kumar proposed openstack/tripleo-quickstart-extras master: Initial playbook to compute node using standalone ansible playbook https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843835 | 10:18 |
| jm1 | lecris[m]: no idea. which is why i am asking Tengu for help here ^^ | 10:20 |
| lecris[m] | 👍️ Good strategy :D | 10:20 |
| lecris[m] | `environments/services/ironic-overcloud.yaml` has not been tested in quite a while:... (full message at <https://matrix.org/_matrix/media/r0/download/matrix.org/gLyDMdWmOxnSAGDniYYMKfuj>) | 10:23 |
| opendevreview | chandan kumar proposed openstack/tripleo-ci master: Add tripleo-external-compute-deployment-pipeline job template https://review.opendev.org/c/openstack/tripleo-ci/+/843836 | 10:25 |
| *** rlandy|out is now known as rlandy | 10:29 | |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Logging facility for VF https://review.opendev.org/c/openstack/validations-libs/+/857198 | 10:33 |
| *** ysandeep|lunch is now known as ysandeep | 10:43 | |
| opendevreview | Merged openstack/tripleo-heat-templates master: Fix Update ovn_controller. from external-update playbook https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857199 | 11:00 |
| lecris[m] | Anyone know what could cause this?... (full message at <https://matrix.org/_matrix/media/r0/download/matrix.org/OvwshVilFwPcKTMCAUlfEama>) | 11:03 |
| lecris[m] | Are there reported errors of neutron+TLS? The deployment works, but `neutron-api` service does not work because of connection to `ovs` presumably | 11:13 |
| Tengu | jm1: seeing Dan answer on the github, I'd rather not add the CAP_AUDIT_WRITE. | 11:13 |
| Tengu | probably something to correct in the container service so that it doesn't rely on audit. | 11:13 |
| jm1 | Tengu: whatever it takes to silence that msg ;) | 11:16 |
| lecris[m] | Would adding a line to /etc/sudoers do that? | 11:16 |
| Tengu | I don't think so, no | 11:17 |
| opendevreview | Merged openstack/tripleo-ansible master: Add openssh test_deps as container manager needs it https://review.opendev.org/c/openstack/tripleo-ansible/+/855358 | 11:17 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-ansible master: New roles for apache/httpd management https://review.opendev.org/c/openstack/tripleo-ansible/+/853481 | 11:17 |
| Tengu | tkajinam: -^^ updated according to your comments | 11:18 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-ansible master: New tripleo_ironic role https://review.opendev.org/c/openstack/tripleo-ansible/+/855873 | 11:19 |
| lecris[m] | Are these new roles basically identical to the heat template ones or are there some fixes in them as well? | 11:21 |
| Tengu | lecris[m]: what new roles? | 11:27 |
| Tengu | the ones I pushed for httpd management? | 11:27 |
| lecris[m] | The latter, `tripleo_ironic` | 11:28 |
| Tengu | ah, that. it only covers the httpd config | 11:28 |
| opendevreview | Vijayalakshmi proposed openstack/tripleo-ansible stable/wallaby: TripleO os_net_config playbooks should allow re-run https://review.opendev.org/c/openstack/tripleo-ansible/+/857346 | 11:28 |
| lecris[m] | But there are also `neutron_api` and so on | 11:28 |
| Tengu | yes. there will be a tripleo_neutron soon-ish | 11:28 |
| Tengu | I sent a mail to the "discuss" ML about that, earlier today. | 11:29 |
| Tengu | "[tripleo] puppetlabs-apache: soon a just a bad memory?" | 11:29 |
| lecris[m] | What do these migrations aim to solve? | 11:30 |
| Tengu | it's all written in the mail. | 11:30 |
| Tengu | basically: remove a 3rd party dependency that usually breaks us on new releases, get a smaller footprint... | 11:31 |
| Tengu | and some other considerations. | 11:31 |
| lecris[m] | Thnx for the tldr | 11:31 |
| lecris[m] | Btw a quick question about undercloud, what's the purpose of the different `admin`, `public` and local ip addresses/names? | 11:33 |
| *** dviroel|out is now known as dviroel | 11:36 | |
| opendevreview | Douglas Viroel proposed openstack/tripleo-heat-templates master: Test custom cpu_model on nested virt job https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857226 | 11:45 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-heat-templates master: Switch default firewall engine https://review.opendev.org/c/openstack/tripleo-heat-templates/+/852808 | 11:47 |
| Tengu | ysandeep: -^^ added depends-on your 2 oooq+oooq-extras patches | 11:47 |
| Tengu | that should make zuul green on that patch. | 11:47 |
| Tengu | hopefully. | 11:47 |
| Tengu | ysandeep: meaning: testproject doesn't need to depends-on them now. Sorry if it crashed an on-going run though :/. | 11:48 |
| ysandeep | Tengu, ack, rdo ci is still in shape due to mirror issue | 11:48 |
| Tengu | just want to validate things withing that patch. | 11:48 |
| Tengu | ysandeep: ok - so I didn't hose things :). which is good. | 11:48 |
| ysandeep | Tengu, yeah even better | 11:48 |
| opendevreview | Jaganathan Palanisamy proposed openstack/tripleo-common master: Derive parameters clean up https://review.opendev.org/c/openstack/tripleo-common/+/851194 | 11:56 |
| Tengu | lecris[m], jm1 so, maybe we can edit the sudo.conf (not sudoers) in order to avoid any discussion with the audit thingy. According to the manpage, starting with 1.9.1, there's a "sudoers_audit" plugin now - maybe we can either exchange it with another thing, or mute its "audit log" capacities. | 11:58 |
| Tengu | it needs some more readong. | 11:58 |
| Tengu | *reading | 11:58 |
| Tengu | but that may be a path to follow. | 11:58 |
| Tengu | but from what I read, the plugin doesn't offer much related to audit log. | 11:58 |
| Tengu | `man sudo.conf' offers some more explanation, we may get to something nice... but it will require testing, of course. | 11:59 |
| opendevreview | Merged openstack/tripleo-heat-templates stable/wallaby: Remove unneeded parameter tripleo_frr_ovn_bgp_agent_bridge_mappings https://review.opendev.org/c/openstack/tripleo-heat-templates/+/855330 | 12:08 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-ansible master: Ensure the openvswitch service is enabled and deps are installed https://review.opendev.org/c/openstack/tripleo-ansible/+/855975 | 12:14 |
| opendevreview | Oliver Walsh proposed openstack/tripleo-ansible master: Ensure the openvswitch service is enabled and deps are installed https://review.opendev.org/c/openstack/tripleo-ansible/+/855975 | 12:20 |
| jm1 | Tengu: thanks man! i guess you are also pretty booked but maybe you could paste your discoveries in the bug and give us some direction to whom (which dfg) we could assign that bug? | 12:29 |
| Tengu | jm1: gimme 5 minutes - my laptop had a crash -.-. | 12:30 |
| Tengu | of course. | 12:30 |
| jm1 | Tengu: thank you :D (fyi ruck and rovers are mostly fighting intermittent and transient failures nowadays, we barely have time to dig into one of the bugs ourselfs :/ ) | 12:31 |
| Tengu | lecris[m]: had to -1 your patch adding the capability (cf discussion here - I'll push data to the LP) | 12:32 |
| *** frenzyfriday is now known as frenzyfriday|lunch | 12:32 | |
| Tengu | ade_lee: heya! would you have some time to check on https://bugs.launchpad.net/tripleo/+bug/1989247 and, especially, comment #12 ? | 12:35 |
| Tengu | ade_lee: apparently, under some circumstances, "sudo" wants to write in the audit log, and this is prevented from containers (https://github.com/containers/podman/issues/15626#issuecomment-1236306369) - I'd rather avoid re-adding the capability, and think it would be better to properly configure sudo. | 12:35 |
| ade_lee | Tengu, ack - will look | 12:36 |
| Tengu | thanks! | 12:37 |
| Tengu | jm1: things are under control. More or less. | 12:37 |
| lecris[m] | <Tengu> "lecris: had to -1 your patch..." <- No worries | 12:38 |
| Tengu | lecris[m]: added some more context/comment | 12:39 |
| chandankumar | bogdando: slagle please have a look at this issue https://review.opendev.org/c/openstack/tripleo-ansible/+/842437/16#message-b674e65d15e6935100268c09e210503e6e74c7cc | 12:39 |
| Tengu | brb | 12:39 |
| chandankumar | it is blocking testing of standalone roles | 12:39 |
| opendevreview | Jakob Meng proposed openstack/openstack-tempest-skiplist master: Skip neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON on c9 master network comp. https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/857422 | 12:41 |
| bogdando | chandankumar, abishop: ^^ the real error is Error: statfs /var/lib/config-data/ansible-generated/iscsid: no such file or directory | 12:41 |
| bogdando | unlikely /lib/modules mount | 12:42 |
| bogdando | and that is related to new ansible config provider | 12:42 |
| chandankumar | bogdando: yes, as other standalone jobs are passing | 12:48 |
| chandankumar | Let me open a bug to track that | 12:48 |
| Tengu | ade_lee: if needed, we can collaborate on that sudo.conf thingy. | 12:57 |
| Tengu | imho there are potential improvements in the containers sudo things. Maybe we should even avoid calling "sudo" from within the container and rely on the "--user root" for specific subset of commands ? | 12:58 |
| Tengu | lecris[m]: incidentally.... I see the db_sync is supposed to run as root at this point. I'm a bit surprised to see "sudo" being involved.. (cc ade_lee ) | 12:59 |
| lecris[m] | It is `sudo -u keystone` | 13:00 |
| Tengu | ah. meh. | 13:01 |
| Tengu | didn't check deeper. | 13:01 |
| Tengu | maybe we can try to log only sudo denials. | 13:02 |
| Tengu | that should do it. more or less. | 13:02 |
| lecris[m] | Tengu: But it is running as root so would there be sudo denials? | 13:04 |
| lecris[m] | I was thinking that if a specific command is written in sudoers, then it would not check audit | 13:05 |
| lecris[m] | But I don't know sudo configs | 13:05 |
| opendevreview | Merged openstack/openstack-tempest-skiplist master: Skip neutron_tempest_plugin.api.test_port_forwardings.PortForwardingTestJSON on c9 master network comp. https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/857422 | 13:05 |
| Tengu | lecris[m]: so by default sudo logs all - successes and denials. | 13:15 |
| Tengu | lecris[m]: so if we configure it to log only denials, we shouldn't hit any issue, since things are allowed. | 13:16 |
| Tengu | we basically don't really care about the sudo log when things are running as planed, do we? | 13:16 |
| Tengu | so imho, we can configure sudo to not log successes, and be off with it. | 13:16 |
| lecris[m] | 👍 sounds reasonable | 13:17 |
| lecris[m] | Is it ok not to log unexpected sudo commands | 13:17 |
| lecris[m] | Realistically wouldn't occur, but 🤷♂️ | 13:18 |
| *** dasm|off is now known as dasm | 13:21 | |
| *** pdeore is now known as pdeore|afk | 13:24 | |
| *** frenzyfriday|lunch is now known as frenzyfriday | 13:26 | |
| opendevreview | Mikolaj Ciecierski proposed openstack/tripleo-heat-templates stable/wallaby: Fix Update ovn_controller. from external-update playbook https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857433 | 13:27 |
| Tengu | that's for Security ;) | 13:43 |
| opendevreview | yatin proposed openstack/tripleo-heat-templates master: Allow Undercloud to be deployed with Ml2 OVN https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857436 | 13:48 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-heat-templates master: Avoid a pipe and grep call https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857437 | 13:55 |
| opendevreview | yatin proposed openstack/tripleo-quickstart master: Switch fs001 to Deploy Undercloud with OVN https://review.opendev.org/c/openstack/tripleo-quickstart/+/853503 | 14:12 |
| opendevreview | Manojkatari proposed openstack/tripleo-heat-templates master: support tripleo_etcd ansible role https://review.opendev.org/c/openstack/tripleo-heat-templates/+/849844 | 14:20 |
| Tengu | chandankumar: heya! is there anything in addition to https://review.opendev.org/c/openstack/tripleo-quickstart/+/856603 to do? apparently, zuul is still failing with the OC deploy (OC node unreachable). | 14:27 |
| Tengu | chandankumar: I first thought it was due to the nftables switch, but the failure happens far, far before we actually set anything in the firewall. | 14:27 |
| Tengu | chandankumar: for instance https://review.opendev.org/c/openstack/tripleo-heat-templates/+/852808 - and https://f0b14fac337d4721647f-43001c5c460bd4e8e3c2be96e8c7a69c.ssl.cf2.rackcdn.com/852808/12/check/tripleo-ci-centos-9-containers-multinode/005472b/logs/undercloud/home/zuul/overcloud_deploy.log | 14:27 |
| Tengu | Wait for connection to become available | 192.168.24.3 | error={"changed": false, "elapsed": 2407, "msg": "timed out waiting for ping module test: Data could not be sent to remote host \"192.168.24.3\". Make sure this host can be reached over ssh: ssh: connect to host 192.168.24.3 port 22: No route to host\r\n"} | 14:28 |
| chandankumar | Tengu: weired | 14:35 |
| chandankumar | https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-containers-multinode seems to be healthy | 14:36 |
| chandankumar | may be something got merged or got updated caused that? | 14:36 |
| chandankumar | if dependent patches is not causing it | 14:36 |
| Tengu | hmm. | 14:41 |
| Tengu | uho. ok. I see far too many dropped packets. | 14:42 |
| Tengu | ysandeep: apparently the vxlan opening needs to be in some other jobs :) | 14:42 |
| Tengu | ysandeep: check that out: https://f0b14fac337d4721647f-43001c5c460bd4e8e3c2be96e8c7a69c.ssl.cf2.rackcdn.com/852808/12/check/tripleo-ci-centos-9-containers-multinode/005472b/logs/undercloud/var/log/extra/dropped-packets.txt | 14:42 |
| Tengu | ysandeep: so you want to expand a bit the scope of your patch imho. we probably should open the vxlan related ports to all of the jobs involving overcloud nodes. | 14:43 |
| ysandeep | Tengu, https://f0b14fac337d4721647f-43001c5c460bd4e8e3c2be96e8c7a69c.ssl.cf2.rackcdn.com/852808/12/check/tripleo-ci-centos-9-containers-multinode/005472b/logs/undercloud/home/zuul/undercloud-parameter-defaults.yaml ^^ undercloud already have those rules | 14:43 |
| ysandeep | checking dropped-packet file | 14:44 |
| chandankumar | Tengu: what is the issue there? | 14:44 |
| Tengu | o_O | 14:44 |
| Tengu | chandankumar: UC can't talk to OC node. apparently infra is using vxlan, and while it should be OK, it doesn't seem to be THAT ok. | 14:45 |
| ysandeep | Tengu, we shouldn't have drops for 4789 its here already: https://f0b14fac337d4721647f-43001c5c460bd4e8e3c2be96e8c7a69c.ssl.cf2.rackcdn.com/852808/12/check/tripleo-ci-centos-9-containers-multinode/005472b/logs/undercloud/home/zuul/undercloud-parameter-defaults.yaml | 14:45 |
| chandankumar | ah ok | 14:45 |
| Tengu | SRC=158.69.64.138 DST=158.69.65.240 vs ip saddr 158.69.65.240 | 14:45 |
| Tengu | ysandeep: we probably want to either allow the range (/24 or so), or take the right IP for the SRC. | 14:45 |
| ysandeep | Tengu, I am in a mtg but lets talk tomorrow morning o/ | 14:46 |
| Tengu | here, vxlan packets come from SRC=158.69.64.138 while we're allowing saddr 158.69.65.240 | 14:46 |
| Tengu | ysandeep: sure thing! | 14:46 |
| Tengu | I'll be in a meeting in ~15 minutes as well. | 14:46 |
| Tengu | maybe "ip daddr 158.69.65.240 udp dport 4789 counter packets 0 bytes 0 accept comment "020 Allow VXLan from CI infra network" | 14:47 |
| Tengu | would be better. | 14:47 |
| Tengu | though it's a bit too wide imho. | 14:47 |
| Tengu | (anything hitting that IP (which is public) on that port will be allowed - not sure this is that safe) | 14:47 |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Logging facility for VF https://review.opendev.org/c/openstack/validations-libs/+/857198 | 14:49 |
| opendevreview | Jiri Podivin proposed openstack/validations-libs master: Logging facility for VF https://review.opendev.org/c/openstack/validations-libs/+/857198 | 14:52 |
| Tengu | corresponds to a /29 netmask. | 14:52 |
| *** ykarel is now known as ykarel|afk | 14:54 | |
| Tengu | ok. that .138 is known within the deploy, according to https://f0b14fac337d4721647f-43001c5c460bd4e8e3c2be96e8c7a69c.ssl.cf2.rackcdn.com/852808/12/check/tripleo-ci-centos-9-containers-multinode/005472b/logs/quickstart_files/ssh.config.ansible | 14:56 |
| Tengu | looks like a bastion or something. | 14:56 |
| Tengu | so we may get that IP in some way in the deploy, and allow it. | 14:56 |
| Tengu | aw dang. no. it's not a /29, I didn't realize it's 138 vs 240 :). | 14:57 |
| *** eliadcohen__ is now known as eliadcohen | 15:04 | |
| *** ykarel|afk is now known as ykarel | 15:10 | |
| opendevreview | James Slagle proposed openstack/tripleo-ansible master: Add initial standalone playbooks and inventory for a compute node https://review.opendev.org/c/openstack/tripleo-ansible/+/840509 | 15:15 |
| opendevreview | Merged openstack/tripleo-heat-templates stable/wallaby: Add 'ResellerAdmin' role to ceilometer user https://review.opendev.org/c/openstack/tripleo-heat-templates/+/856491 | 15:19 |
| slagle | bogdando: you have +2's from me on the nova roles | 15:23 |
| slagle | i'd be happy to get these merged and iteratively improve from there | 15:23 |
| opendevreview | chandan kumar proposed openstack/tripleo-heat-templates master: [WIP] Include tripleo_iscsid/tasks/configure.yml https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857464 | 15:26 |
| chandankumar | bogdando: slagle for this https://bugs.launchpad.net/tripleo/+bug/1989482 trying with above fix | 15:26 |
| chandankumar | ^^ | 15:26 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-ansible master: New roles for apache/httpd management https://review.opendev.org/c/openstack/tripleo-ansible/+/853481 | 15:28 |
| opendevreview | chandan kumar proposed openstack/tripleo-ci master: Add tripleo-external-compute-deployment-pipeline job template https://review.opendev.org/c/openstack/tripleo-ci/+/843836 | 15:28 |
| opendevreview | Cedric Jeanneret proposed openstack/tripleo-ansible master: New tripleo_ironic role https://review.opendev.org/c/openstack/tripleo-ansible/+/855873 | 15:28 |
| slagle | chandankumar: looking | 15:35 |
| bogdando | slagle: thanks! | 15:38 |
| *** marios is now known as marios|out | 15:49 | |
| *** dviroel is now known as dviroel|lunch | 15:51 | |
| sdanni | Hi! We are working on deploying overcloud with SSL. We use letsencrypt certificates which will expire in 90 days. I wonder what is the common way to renew ssl certificates in openstack environment without redeploying overcloud? | 15:52 |
| *** ysandeep is now known as ysandeep|out | 15:57 | |
| opendevreview | Daniel Bengtsson proposed openstack/tripleo-heat-templates master: Remove the python3-openclient and add a new task. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857469 | 16:01 |
| lecris[m] | sdanni: Does certmonger work with letsencrypt? | 16:02 |
| lecris[m] | At the very least you can make a certbot service and restart haproxy and horizon | 16:05 |
| lecris[m] | Reload rather than restart | 16:06 |
| sdanni | lecris[m]: i just googled it. I think there are some tools for certmonger to communicate with letsencrypt CA but none of them are officially supported | 16:07 |
| sdanni | lecris[m]: do you mean I can manually replace the certs and keys on overcloud nodes and then reload haproxy? | 16:08 |
| lecris[m] | Yeah. I don't know if openstack terminates or passthrough's the tls protocol. If it's the latter that should be the feasible way | 16:10 |
| lecris[m] | If the former you could add acme to haproxy iirc | 16:11 |
| sdanni | lecris[m]: if certmonger works with letsencrypt, could the process be simpler? | 16:13 |
| lecris[m] | Yes, because certmonger support is built-in | 16:13 |
| sdanni | lecris[m]: gotcha! thanks | 16:19 |
| *** jpena is now known as jpena|off | 16:35 | |
| opendevreview | Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/wallaby: WIP: FFWD3: enample templated haproxy/heat images https://review.opendev.org/c/openstack/tripleo-heat-templates/+/852898 | 16:35 |
| *** dviroel|lunch is now known as dviroel | 16:57 | |
| opendevreview | Merged openstack/tripleo-heat-templates master: Correct label for /run/libvirt https://review.opendev.org/c/openstack/tripleo-heat-templates/+/856535 | 18:01 |
| opendevreview | Merged openstack/tripleo-ansible master: Run molecule tests locally without zuul https://review.opendev.org/c/openstack/tripleo-ansible/+/856339 | 18:39 |
| opendevreview | Douglas Viroel proposed openstack/tripleo-heat-templates master: Test custom cpu_model on nested virt job https://review.opendev.org/c/openstack/tripleo-heat-templates/+/857226 | 18:50 |
| *** dviroel is now known as dviroel|brb | 20:10 | |
| *** rlandy is now known as rlandy|bbl | 21:27 | |
| *** dasm is now known as dasm|off | 22:28 | |
| *** dasm|off is now known as Guest305 | 23:03 | |
| *** rlandy|bbl is now known as rlandy | 23:27 | |
| opendevreview | Brendan Shephard proposed openstack/tripleo-ansible master: Add Neutron API standalone role https://review.opendev.org/c/openstack/tripleo-ansible/+/857252 | 23:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!