| *** rlandy is now known as rlandy|ruck | 00:11 | |
| *** pojadhav|afk is now known as pojadhav | 04:56 | |
| *** ysandeep|out is now known as ysandeep | 05:24 | |
| *** amoralej|off is now known as amoralej | 07:00 | |
| marios | o/ needs reviews please https://review.opendev.org/c/openstack/tripleo-quickstart/+/816313 thank you | 07:38 |
|---|---|---|
| ysandeep | revoted on latest patchset ^^ | 07:50 |
| marios | akahat|rover: soniya29|ruck: o/ please see tripleo (gate blocker? multinode job) and also fyi https://review.opendev.org/c/openstack/tripleo-ci/+/815108/6#message-df817d0433c86aa60dfd5eac6edb09350fe5d738 | 07:54 |
| marios | thanks ysandeep | 07:55 |
| *** ykarel__ is now known as ykarel | 08:19 | |
| soniya29|ruck | marios, ack | 08:21 |
| *** soniya29|ruck is now known as soniya29|ruck|lunch | 08:21 | |
| *** pojadhav is now known as pojadhav|lunch | 09:21 | |
| *** soniya29|ruck|lunch is now known as soniya29|ruck | 09:35 | |
| *** ykarel is now known as ykarel|lunch | 09:42 | |
| *** ysandeep is now known as ysandeep|afk | 09:44 | |
| *** pojadhav|lunch is now known as pojadhav | 10:04 | |
| akahat|rover | slaweq, o/ | 10:09 |
| akahat|rover | slaweq, we have one promotion blocker: https://bugs.launchpad.net/tripleo/+bug/1950916. | 10:10 |
| akahat|rover | slaweq, i was digging in the logs.. i found that one network got created. but looks like on neutron side it has some issue. | 10:11 |
| slaweq | akahat|rover: hi, sorry, I was in the meeting | 10:39 |
| slaweq | let me check that | 10:39 |
| akahat|rover | slaweq, looks like router is not there:https://bugs.launchpad.net/tripleo/+bug/1950916/comments/5 | 10:40 |
| *** ykarel|lunch is now known as ykarel | 10:43 | |
| slaweq | akahat|rover: router 93922895-a67b-4e2a-abee-ddff06583470 seems to be created properly on l3 agent's side in logs which You were investigating | 10:45 |
| slaweq | FIP 192.168.24.172 too | 10:45 |
| slaweq | akahat|rover: is it failing like that every time? | 10:45 |
| slaweq | or sometimes? | 10:45 |
| akahat|rover | slaweq, it is failing every time. | 10:46 |
| slaweq | akahat|rover: ok, can You maybe run that job again and put node on hold so I can ssh to it and try to investigate why it's failing? | 10:46 |
| soniya29|ruck | akahat|rover, marios: https://bugs.launchpad.net/tripleo/+bug/1950964 | 10:47 |
| akahat|rover | slaweq, yes. i've already started running jobs. i'll hold it. | 10:47 |
| marios | soniya29|ruck: thanks | 10:49 |
| marios | soniya29|ruck: strange doesn't seem to be something recent there https://bugs.launchpad.net/tripleo/+bug/1950964/comments/1 (param is only in that file according to grep) | 10:56 |
| *** ysandeep|afk is now known as ysandeep | 10:57 | |
| slaweq | akahat|rover: thx | 10:57 |
| akahat|rover | ykarel, o/ | 10:58 |
| *** jpena|off is now known as jpena | 10:59 | |
| akahat|rover | ykarel, could you please hold sc001 job node: https://review.rdoproject.org/zuul/status#36756 | 10:59 |
| ykarel | akahat|rover, added to autohld | 11:00 |
| akahat|rover | ykarel, thank you. | 11:00 |
| soniya29|ruck | marios, okay | 11:02 |
| marios | soniya29|ruck: worth digging there or tripleo-ansible see if there are any recent commits that might be resposible and reach out to DF | 11:06 |
| marios | akahat|rover: ^^^ re https://bugs.launchpad.net/tripleo/+bug/1950964 | 11:06 |
| akahat|rover | marios, already on it. | 11:07 |
| marios | ack thanks akahat|rover soniya29|ruck | 11:07 |
| *** rlandy is now known as rlandy|ruck | 11:19 | |
| soniya29|ruck | marios, sure..checking | 11:19 |
| rlandy|ruck | soniya29|ruck: akahat|rover: hey - let's sync https://meet.google.com/vyh-awmk-wst?pli=1&authuser=0 | 11:21 |
| akahat|rover | rlandy|ruck, joining | 11:22 |
| rlandy|ruck | slaweq: hi - we are have a second CIX blocking issue - on wallaby that may be neutron or may be compute. https://bugs.launchpad.net/tripleo/+bug/1950815 | 11:36 |
| rlandy|ruck | can you take a look at ^^ and let us know if that lands with your team or DF or compute | 11:37 |
| slaweq | rlandy|ruck: this failed test neutron_tempest_plugin.api.test_ports.PortsTestJSON | 11:38 |
| slaweq | test_create_update_port_with_dns_name seems like Neutron related | 11:38 |
| slaweq | so it would be for DFG:Networking | 11:38 |
| slaweq | but I don't have time now to check it | 11:38 |
| slaweq | I can take a look tomorrow probably | 11:38 |
| rlandy|ruck | slaweq: thanks - is there someone else we should ping with this - otherwise we will leave until tomorrow | 11:39 |
| slaweq | rlandy|ruck: I will check it tomorrow | 11:54 |
| rlandy|ruck | thanks | 11:54 |
| rlandy|ruck | marios: hi - can we get your w+ on https://review.opendev.org/c/openstack/tripleo-common/+/817889 | 12:02 |
| marios | rlandy|ruck: looking | 12:04 |
| rlandy|ruck | akahat|rover: who is supposed to provide the permanent fix for https://bugs.launchpad.net/tripleo/+bug/1950148? | 12:12 |
| akahat|rover | rlandy|ruck, oh. patch is got merged. i'll close it. | 12:14 |
| rlandy|ruck | akahat|rover: bug says the workaround was fixed | 12:14 |
| rlandy|ruck | we still need a proper fix | 12:14 |
| rlandy|ruck | chandan wanted it open | 12:14 |
| rlandy|ruck | ysandeep: thanks for your note to steve | 12:15 |
| rlandy|ruck | I left him a note last night as well | 12:15 |
| rlandy|ruck | I will try catch him in my afternoon | 12:15 |
| akahat|rover | rlandy|ruck, okay.. let me check once with raukadah, and i | 12:15 |
| akahat|rover | i'll update it. | 12:16 |
| rlandy|ruck | there is a CIX open | 12:16 |
| rlandy|ruck | will bring it to the bug scrub as well | 12:16 |
| akahat|rover | hey slaweq, please share your key, so i can add it to the node. | 12:16 |
| ysandeep | rlandy|ruck, :) you are welcome | 12:18 |
| ysandeep | rlandy|ruck: imho... if resource is not a constraint(we need to check with Alan).. we should move default centos-ci check job from fs052 to minimal , that will add more value. | 12:21 |
| rlandy|ruck | ysandeep: ack | 12:21 |
| rlandy|ruck | then we will catch the failures | 12:21 |
| rlandy|ruck | let's bring up at scrum | 12:21 |
| ysandeep | rlandy|ruck: ack | 12:22 |
| rlandy|ruck | checking in on 16.2 container build failure | 12:22 |
| ysandeep | still an issue ^^ | 12:23 |
| rlandy|ruck | ysandeep: ack - CIXed and pinged delivery again | 12:23 |
| *** amoralej is now known as amoralej|lunch | 12:27 | |
| slaweq | akahat|rover: I'm afk now. I will send ot to you in about 20 minutes when I will come back | 12:47 |
| rlandy|ruck | akahat|rover: can you check the sc010 and ovn provider results in check on other branches | 12:47 |
| rlandy|ruck | so we can know if we can make them all voting? | 12:48 |
| akahat|rover | slaweq, okay | 12:49 |
| akahat|rover | rlandy|ruck, yes. i've already checked. | 12:49 |
| rlandy|ruck | arxcruz: hey - any progress on https://review.opendev.org/c/openstack/tripleo-ci/+/816991? | 12:55 |
| arxcruz | rlandy|ruck: sorry, not yet, the testproject isn't working :( | 12:56 |
| arxcruz | still investigating | 12:56 |
| rlandy|ruck | akahat|rover: scrum time | 13:02 |
| slaweq | akahat|rover: my ssh key http://pastebin.test.redhat.com/1008244 | 13:09 |
| rlandy|ruck | Review list: | 13:10 |
| marios | rlandy|ruck: workflow & ping ruck|rovers @ https://review.opendev.org/c/openstack/tripleo-ci/+/815108/6#message-df817d0433c86aa60dfd5eac6edb09350fe5d738 * watching https://zuul.openstack.org/status#815108 | 13:10 |
| akahat|rover | slaweq, zuul@38.102.83.147 | 13:10 |
| marios | & standby with https://review.rdoproject.org/r/c/rdo-jobs/+/36720 | 13:10 |
| marios | rlandy|ruck: whole_disk_images * selling https://review.opendev.org/c/openstack/tripleo-quickstart/+/816313 | 13:11 |
| marios | rlandy|ruck: https://review.rdoproject.org/r/c/testproject/+/36267 | 13:12 |
| marios | rlandy|ruck: https://review.opendev.org/c/openstack/tripleo-operator-ansible/+/817928 | 13:13 |
| slaweq | akahat|rover: ok, let me check it | 13:13 |
| pojadhav | rlandy|ruck, https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/817924 | 13:17 |
| akahat|rover | rlandy|ruck, https://review.rdoproject.org/r/c/testproject/+/36756 | 13:29 |
| arxcruz | need to bring my daughter from school, brb in 20ish minutes | 13:35 |
| slaweq | akahat|rover: I checked env on zuul@38.102.83.147 | 13:35 |
| slaweq | First of all it's different env than You were debugging in https://bugs.launchpad.net/tripleo/+bug/1950916/comments/5 | 13:36 |
| slaweq | this one is ML2/OVN and the other one is ML2/OVS | 13:36 |
| slaweq | but I don't think that this is an issue really | 13:36 |
| slaweq | it seems for me that vms created there aren't started | 13:36 |
| slaweq | if You check console log of any of them VMs there, it is always empty | 13:37 |
| slaweq | I booted one VM manually and its console log is empty also | 13:37 |
| slaweq | I think that someone from the compute team should take a look into that first | 13:37 |
| slaweq | for now I don't think it's really networking issue | 13:38 |
| *** amoralej|lunch is now known as amoralej | 13:38 | |
| akahat|rover | slaweq, yes. i also noted that console logs are empty in tests. | 13:39 |
| akahat|rover | slaweq, okay. i'll ping someone in compute team. | 13:40 |
| slaweq | ++ | 13:40 |
| akahat|rover | slaweq, so that router missing is not the error? | 13:40 |
| slaweq | there wasn't missing router | 13:40 |
| slaweq | where You saw that? | 13:40 |
| akahat|rover | here: https://bugs.launchpad.net/tripleo/+bug/1950916/comments/5 | 13:41 |
| akahat|rover | not found in namespace qrouter | 13:41 |
| akahat|rover | Error: [Errno 2] No such file or directory: '/var/lib/neutron/ha_confs/93922895-a67b-4e2a-abee-ddff06583470.pid.keepalived' | 13:41 |
| slaweq | no, it's not error | 13:42 |
| slaweq | it's like that because it was new router so there wasn't that file created yet | 13:42 |
| slaweq | nothing wrong there | 13:42 |
| slaweq | akahat|rover: it's just DEBUG message | 13:42 |
| akahat|rover | slaweq, okay. Thank you for information. :) | 13:43 |
| slaweq | yw | 13:43 |
| rlandy|ruck | akahat|rover: so we're passing this to the compute team? | 13:58 |
| akahat|rover | rlandy|ruck, yes. it's not from the networking side. | 13:59 |
| akahat|rover | so we need to pass it to the compute team. | 13:59 |
| rlandy|ruck | akahat|rover: moving to #rhos-dev | 14:00 |
| akahat|rover | akc | 14:00 |
| *** jbadiapa_ is now known as jbadiapa | 14:31 | |
| arxcruz | marios: rlandy|ruck https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/36760 | 14:33 |
| arxcruz | the dashboard | 14:33 |
| marios | arxcruz: ack adding to reivews | 14:33 |
| rlandy|ruck | arxcruz: you are core on python tempestconf? | 14:38 |
| arxcruz | rlandy|ruck: yes | 14:38 |
| rlandy|ruck | can you join https://meet.google.com/ugp-erxb-psj?authuser=0? | 14:39 |
| rlandy|ruck | re: scenario010 and python tempestconf | 14:39 |
| rlandy|ruck | arxcruz: https://review.opendev.org/c/openinfra/python-tempestconf/+/817348 | 14:42 |
| rlandy|ruck | arxcruz: https://review.opendev.org/c/openinfra/python-tempestconf/+/814373 | 14:44 |
| *** soniya29|ruck is now known as soniya29|ruck|dinner | 15:00 | |
| rlandy|ruck | http://tripleo-cockpit.usersys.redhat.com/d/9DmvErfZz/cockpit?orgId=1 is down - no our fault :( | 15:02 |
| *** ykarel is now known as ykarel|away | 15:03 | |
| rlandy|ruck | marios: want to sync? | 15:15 |
| marios | yes rlandy|ruck | 15:15 |
| rlandy|ruck | marios: https://meet.google.com/pcb-apfv-yzr?pli=1&authuser=0 | 15:16 |
| *** soniya29|ruck|dinner is now known as soniya29|ruck | 15:34 | |
| ysandeep | rlandy|ruck, once you get free, want to chat about 16.2 situation? | 15:38 |
| rlandy|ruck | ysandeep: sure - will ping | 15:39 |
| ysandeep | rlandy|ruck, ack, afk for dinner for few mins | 15:40 |
| rlandy|ruck | ysandeep: ok - ping when you are back from dinner | 15:40 |
| pojadhav | rlandy|ruck, multinode-ipa job for C9 is failing at "configure FreeIPA" with error "IPv6 stack is enabled in the kernel but there is no interface that has ::1 address assigned. Add ::1 address resolution to 'lo' interface. You might need to enable IPv6 on the interface 'lo' in sysctl.conf." | 15:58 |
| pojadhav | rlandy|ruck, after some findings I get to know that we need to modify /etc/sysctl.conf file and add entry "net.ipv6.conf.lo.disable_ipv6 = 0". | 16:00 |
| pojadhav | may be this is useful to troubleshoot issue : https://www.devopszones.com/2019/10/freeipa-installation-gives-following.html | 16:00 |
| pojadhav | but unable to get, where exactly need this change. | 16:00 |
| pojadhav | https://logserver.rdoproject.org/19/36719/1/check/periodic-tripleo-ci-centos-9-standalone-on-multinode-ipa-master/61201d6/job-output.txt | 16:03 |
| ysandeep | rlandy|ruck, /me back if you want to chat | 16:03 |
| ysandeep | pojadhav, I can check about IPV6 issue tomorrow | 16:04 |
| pojadhav | ysandeep, sure will sync up on that. thank you! | 16:05 |
| rlandy|ruck | ysandeep: https://meet.google.com/fky-oqkd-yzt?pli=1&authuser=0 | 16:07 |
| *** jpena is now known as jpena|off | 16:08 | |
| rlandy|ruck | ysandeep: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-build-containers-ubi-8-internal-rhel-8-build-push-upload-rhos-16.2/070e6e4/logs/container-builds/4ae817df-2c09-4321-a713-3ddd7e63a73a/base/rsyslog/rsyslog-build.log | 16:13 |
| rlandy|ruck | http://download.eng.bos.redhat.com/brewroot/repos/rhos-16.2-rhel-8-trunk-build/latest/x86_64/pkglist has toplink/vol/rhel-8/packages/rsyslog/8.1911.0/7.el8_4.2/x86_64/rsyslog-elasticsearch-8.1911.0-7.el8_4.2.x86_64.rpm and toplink/vol/rhel-8/packages/rsyslog/8.1911.0/7.el8_4.2/noarch/rsyslog-doc-8.1911.0-7.el8_4.2.noarch.rpm | 16:13 |
| rlandy|ruck | rsyslog-8.2102.0-5.el8.x86_64 | 16:13 |
| ysandeep | rlandy|ruck, https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-jobs.git;a=blob;f=zuul.d/tripleo-build-containers.yaml#l136 | 16:17 |
| rlandy|ruck | ysandeep: http://git.app.eng.bos.redhat.com/git/rhos-release.git/tree/rhos-release | 16:20 |
| rlandy|ruck | https://catalog.redhat.com/software/containers/ubi8/ubi/5c359854d70cc534b3a3784e?tag=8.4-213&push_date=1635845149000>i-tabs=registry-tokens | 16:22 |
| rlandy|ruck | podman pull registry.access.redhat.com/ubi8/ubi:8.4-213 | 16:23 |
| *** amoralej is now known as amoralej|off | 16:24 | |
| ysandeep | rlandy|ruck, http://pastebin.test.redhat.com/1008315 | 16:28 |
| ysandeep | rlandy++ | 16:33 |
| *** ysandeep is now known as ysandeep|out | 16:34 | |
| rlandy|ruck | pojadhav: sorry - been working with compute team on blocker | 17:04 |
| rlandy|ruck | if ysandeep|out hasn;t sorted this by tomorrow, pls ping me - we can look | 17:04 |
| rlandy|ruck | sshnaidm: hi - you around today? | 17:30 |
| sshnaidm | rlandy|ruck, I'm off today | 17:30 |
| rlandy|ruck | sshnaidm: tomorrow? | 17:31 |
| sshnaidm | rlandy|ruck, working tomorrow | 17:31 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!