| *** rlandy is now known as rlandy|out | 00:10 | |
| *** yadnesh|away is now known as yadnesh | 04:00 | |
| pojadhav | arxcruz|ruck : Wish you a very Happy Birthday :) | 05:39 |
|---|---|---|
| akahat | o/ | 05:42 |
| soniya29|rover | arxcruz|ruck, Happy birthday :) | 05:56 |
| arxcruz|ruck | pojadhav: soniya29|rover thanks! :) | 06:25 |
| arxcruz|ruck | Tengu: hey, this https://opendev.org/openstack/tripleo-quickstart-extras/commit/5c22039cd0134d7b5a9c8b82a88c29c9f2c2b1a7 breaks periodic-tripleo-ci-centos-9-standalone-external-compute-target-host-zed | 06:34 |
| marios | o/ | 06:45 |
| soniya29|rover | marios, arxcruz|ruck, 17 rhel9 jobs are getting skippped, line hasn't fully kicked | 07:26 |
| soniya29|rover | do we need to rekick the line again? | 07:26 |
| chandankumar | Happy Birthday my dear friend arxcruz|ruck :-) | 07:26 |
| marios | soniya29|rover: o/ looking | 07:29 |
| marios | arxcruz|ruck: happy birthday :) | 07:29 |
| marios | soniya29|rover: did you check the image build failure? | 07:29 |
| marios | soniya29|rover: please check to see if it is a legit new issue | 07:30 |
| soniya29|rover | marios, ack | 07:30 |
| marios | soniya29|rover: arxcruz|ruck: reminder meeting in 1 hour for sync please (see calendar, the invite i sent on friday was recurring) | 07:30 |
| soniya29|rover | marios, okay | 07:31 |
| *** soniya29|rover is now known as soniya29|rover|lunch | 07:38 | |
| Tengu | arxcruz|ruck: heya! hmmmm do you have some more data? | 07:53 |
| Tengu | and I don't think I've seen that job in the list of the ones kicked in by the change.... | 07:54 |
| Tengu | ah, "periodic". «great»....... | 07:54 |
| marios | need vote here please when you have time https://review.opendev.org/c/openstack/tripleo-heat-templates/+/867026 | 07:54 |
| Tengu | arxcruz|ruck: and happy birthday since it seems to be The D-Day :) | 07:55 |
| Tengu | marios: nudged. | 07:55 |
| marios | thx Tengu o/ | 07:56 |
| Tengu | arxcruz|ruck: hm, I don't seem to be able to find that job here? https://zuul.opendev.org/t/opendev/builds?job_name=periodic-tripleo-ci-centos-9-standalone-external-compute-target-host-zed&skip=0 | 07:56 |
| Tengu | unless I'm searching in the wrong plac. | 07:57 |
| Tengu | *place | 07:57 |
| chandankumar | Tengu: you are looking at wrong place | 07:57 |
| chandankumar | Tengu: https://review.rdoproject.org/zuul/status | 07:57 |
| Tengu | ah, rdo. | 07:58 |
| Tengu | of course. | 07:58 |
| chandankumar | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-standalone-external-compute-target-host-zed&skip=0 | 07:58 |
| chandankumar | it is broken now | 07:58 |
| chandankumar | interesting | 07:58 |
| Tengu | yep, on it. I'll check the log and try to find the reason. | 07:58 |
| Tengu | master was fine. | 07:58 |
| Tengu | ok, the "validate" script. | 07:58 |
| Tengu | "fun". the remote node isn't listed in nova. | 07:59 |
| Tengu | https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-external-compute-target-host-zed/ea1f97a/logs/undercloud/home/zuul/validate_external_compute.log.txt.gz | 07:59 |
| Tengu | HYPER=node-0003464364.novalocal we don't see that node in the hypervisor listing bellow. | 08:00 |
| Tengu | ok, node isn't deployed for some reason. | 08:00 |
| Tengu | *again*. | 08:00 |
| Tengu | https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-standalone-external-compute-target-host-zed/ea1f97a/logs/undercloud/home/zuul/tripleo-deploy/edpm-inventory/host_vars/subnode-1.gz | 08:01 |
| Tengu | ok, it's missing a patch | 08:01 |
| Tengu | chandankumar: remember the one removing the ansible_connection from the host_vars? | 08:01 |
| Tengu | it's missing in Zed I guess. | 08:01 |
| chandankumar | yes | 08:01 |
| chandankumar | we need to backport that | 08:02 |
| Tengu | on it | 08:02 |
| Tengu | https://review.opendev.org/c/openstack/tripleo-ansible/+/870492 | 08:02 |
| Tengu | please vote :) | 08:02 |
| Tengu | no conflict. | 08:02 |
| Tengu | arxcruz|ruck: -^^^ will fix the zed jobs | 08:02 |
| arxcruz|ruck | chandankumar: Thanks! :* | 08:02 |
| arxcruz|ruck | marios: ack | 08:03 |
| arxcruz|ruck | Tengu: thanks! :) | 08:03 |
| Tengu | arxcruz|ruck: that's my b-day gift: fix the job. ;) | 08:03 |
| arxcruz|ruck | Tengu: best gift ever :) | 08:03 |
| Tengu | thought so ;) | 08:03 |
| arxcruz|ruck | marios: if you may vote ^ it's a zed promotion blocker | 08:04 |
| arxcruz|ruck | marios: is it okay if i ran the testproject with a depends-on and get a promotion? | 08:04 |
| Tengu | arxcruz|ruck: is there an LP already? if yes, I can link it on the commit message. | 08:07 |
| arxcruz|ruck | Tengu: there isn't, i just notice this morning, was investigating before open one | 08:08 |
| Tengu | arxcruz|ruck: ok! well, #done then :) | 08:08 |
| Tengu | just a matter of getting it merged. | 08:08 |
| Tengu | that was a fast one ;) | 08:08 |
| Tengu | (especially since I know *where* to look for *what* - I hit that one in master before ;)) | 08:08 |
| arxcruz|ruck | :) | 08:09 |
| marios | arxcruz|ruck: soniya29|rover|lunch: o/ joining | 08:30 |
| *** soniya29|rover|lunch is now known as soniya29|rover | 08:30 | |
| soniya29|rover | yeah | 08:30 |
| soniya29|rover | i am in | 08:30 |
| *** jpena|off is now known as jpena | 08:42 | |
| * soniya29|rover needs a break | 10:21 | |
| *** soniya29|rover is now known as soniya29|rover|afk | 10:21 | |
| *** ysandeep is now known as ysandeep|afk | 10:21 | |
| marios | biab | 10:37 |
| *** rlandy|out is now known as rlandy | 11:08 | |
| *** soniya29|rover|afk is now known as soniya29|rover | 11:09 | |
| rlandy | arxcruz|ruck: happy birthday!! | 11:11 |
| rlandy | arxcruz|ruck: soniya29|rover: marios: need anything rr-wise? | 11:11 |
| arxcruz|ruck | rlandy: thanks! | 11:11 |
| arxcruz|ruck | rlandy: all stable upstream, just zed, but i'm monitoring a required patch to get the promotion | 11:12 |
| soniya29|rover | rlandy, nope | 11:12 |
| rlandy | got the jenkins jobs passing over the weekend - so hopefully that sorted downstream | 11:12 |
| marios | rlandy: had our usual sync earlier arxcruz|ruck and soniya29|rover are going over stuff | 11:12 |
| marios | rlandy: lets talk in few mins | 11:12 |
| marios | soniya29|rover: did you file the new 17 bug? | 11:12 |
| rlandy | arxcruz|ruck: marios: k - we can talk about it after review | 11:12 |
| rlandy | all set for CIX? | 11:12 |
| marios | soniya29|rover: i dont see it on hackmd or cix bord | 11:13 |
| marios | board | 11:13 |
| marios | yeah should be rlandy we updated those in the call | 11:13 |
| soniya29|rover | marios, let me rekick those jobs again | 11:14 |
| marios | soniya29|rover: i thought you already did that earlier ... | 11:14 |
| * marios reviews call | 11:15 | |
| dviroel | o/ | 11:37 |
| chandankumar | dviroel: welcome back sir o/ | 11:42 |
| dviroel | chandankumar: o/ tks | 11:42 |
| marios | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/1d57b2aa86ee43869f7e78bbb731b83b | 11:42 |
| * soniya29|rover brb | 11:45 | |
| *** soniya29|rover is now known as soniya29|rover|brb | 11:45 | |
| chandankumar | Tengu: you might be interested in this pr https://github.com/openstack-k8s-operators/openstack-ansibleee-operator/pull/53 | 11:46 |
| rlandy | soniya29|rover|brb: pls ping when you are back | 11:49 |
| rlandy | we need a bug ASAP on the image build on 17 | 11:50 |
| rlandy | I need to work with you to get that logged and debug info added before CIX | 11:50 |
| *** soniya29|rover|brb is now known as soniya29|rover | 12:01 | |
| rlandy | soniya29|rover: hello | 12:02 |
| rlandy | let's talk | 12:02 |
| rlandy | https://meet.google.com/kwp-irwy-auq?pli=1&authuser=0 | 12:02 |
| rlandy | soniya29|rover: ^^ | 12:02 |
| * pojadhav afk | 12:05 | |
| Tengu | checking | 12:11 |
| rlandy | Tengu: hey - can I borrow your debug skills for 5? I have a log diff I want to run by you | 12:46 |
| rlandy | marios: soniya29|rover: https://bugzilla.redhat.com/show_bug.cgi?id=2161262 updated with diff info | 12:56 |
| rlandy | you can see the install diff bet passing and failing there | 12:57 |
| rlandy | dviroel: hey - you back today? | 12:58 |
| marios | jgilaber: frenzy_friday: joining | 12:59 |
| marios | rlandy: thank you | 12:59 |
| soniya29|rover | rlandy, https://code.engineering.redhat.com/gerrit/c/testproject/+/438702 | 13:01 |
| rlandy | soniya29|rover: ok - hold one node there | 13:02 |
| * rlandy meeting now | 13:02 | |
| Tengu | rlandy: hmm, am I missing a meeting? | 13:12 |
| rlandy | soniya29|rover: pls see 17.1 | 13:38 |
| rlandy | you have build image failsure there | 13:38 |
| rlandy | pls check the bug if it needs to be updated now | 13:38 |
| rlandy | Tengu: no | 13:38 |
| soniya29|rover | rlandy, ack | 13:38 |
| rlandy | soniya29|rover: pls update adding 17.1 logs if relevant | 13:38 |
| soniya29|rover | rlandy, yes | 13:38 |
| *** amoralej is now known as amoralej|lunch | 13:39 | |
| soniya29|rover | rlandy, trying to hold the node for the bug | 13:41 |
| Tengu | rlandy: ok - if I miss anything, please let me know, I'm not yet used to the new amount of meeting ^^ | 13:42 |
| soniya29|rover | rlandy, updated the bug | 13:50 |
| rlandy | dviroel: hey - alan wants to talk with you re: hive/zuul requirements | 13:53 |
| soniya29|rover | rlandy, i have hold the node for https://bugzilla.redhat.com/show_bug.cgi?id=2161262 | 13:55 |
| rlandy | soniya29|rover: can you ssh to the node? | 13:55 |
| soniya29|rover | rlandy, need to try | 13:56 |
| soniya29|rover | give me a moment | 13:56 |
| dviroel | rlandy: ok, I am looking on Jira comments now | 13:59 |
| pojadhav | rlandy, dasm|off : scrum | 14:01 |
| pojadhav | dviroel, ^^ | 14:01 |
| *** dasm|off is now known as dasm | 14:03 | |
| *** ysandeep|afk is now known as ysandeep | 14:14 | |
| *** amoralej|lunch is now known as amoralej | 14:17 | |
| Tengu | jm1: heya! fyi: https://zuul.opendev.org/t/openstack/build/7216b6c3001142f792715c81a054ec77/log/job-output.txt#24019 this means the galaxy proxy is at last working as it should! | 14:28 |
| Tengu | jm1: will try to get it merged this week. Meaning we should be able to use the proxy-cache for galaxy. | 14:28 |
| jm1 | Tengu++ Aaaaaaawesome 🥳 | 14:30 |
| * pojadhav brb | 14:46 | |
| rlandy | Tengu: chandankumar: will send you guys to baremetal info after todays meetings | 15:01 |
| rlandy | so you can access the boxes | 15:01 |
| * dviroel lunch | 15:02 | |
| *** dviroel is now known as dviroel|lunch | 15:03 | |
| Tengu | rlandy: yay, hardware for play-time :) | 15:03 |
| *** ysandeep is now known as ysandeep|afk | 15:12 | |
| kkula | Hello! could you take a look on RDO dashboard? http://dashboard-ci.tripleo.org/d/wplmDxdVz/puppet-promotion-pipelines?orgId=1 once again, it has issue with being empty. Last time it was solved (somehow) by running script pulling git changes. frenzy_friday? | 15:18 |
| frenzy_friday | kkula, huh that is strange. I'll check this week | 15:18 |
| kkula | thanks! | 15:19 |
| rlandy | marios: did we get your trigger patch sorted? | 15:23 |
| rlandy | arxcruz|ruck: you can pass a longer hold time if needed | 15:24 |
| arxcruz|ruck | rlandy: oh, i did not know that :) | 15:24 |
| arxcruz|ruck | i thought 24 hours was the max | 15:25 |
| * rlandy gets | 15:26 | |
| rlandy | arxcruz|ruck: pls see pvt | 15:26 |
| *** dviroel|lunch is now known as dviroel | 15:28 | |
| chandankumar | see ya people! | 15:59 |
| marios | rlandy: yes merged that | 16:00 |
| rlandy | marios: k - cool | 16:00 |
| marios | https://review.rdoproject.org/r/c/config/+/46584 fyi | 16:01 |
| ysandeep|afk | chandankumar, o/ gd nyt, let's catch up tomorrow's morning to discuss more about keeping common code in ci directory in install_yamls | 16:03 |
| soniya29|rover | rlandy, arxcruz|ruck, leaving for the day | 16:04 |
| soniya29|rover | marios, ^^ | 16:04 |
| *** ysandeep|afk is now known as ysandeep|out | 16:04 | |
| rlandy | soniya29|rover: pls pick up what is still unsolved of the 17/17.1 bz tomorrow | 16:04 |
| rlandy | I will comment with what I have | 16:05 |
| soniya29|rover | rlandy, ack | 16:05 |
| rlandy | soniya29|rover: but pls read the BZ when you come on line | 16:05 |
| rlandy | and take action to follow up | 16:05 |
| soniya29|rover | rlandy, okay | 16:05 |
| rlandy | soniya29|rover: pls also keep an eye on downstream components | 16:05 |
| arxcruz|ruck | i need to step out for one hour | 16:05 |
| arxcruz|ruck | but it's almost eod here as well | 16:06 |
| rlandy | arxcruz|ruck: ack - pls ping if anything to watch | 16:06 |
| soniya29|rover | rlandy, ack | 16:06 |
| * rlandy only looking at soniya29|rover's one bz atm | 16:06 | |
| *** soniya29|rover is now known as soniya29|rover|out | 16:07 | |
| * marios off in few | 16:29 | |
| Tengu | ok Team - off as well :). See you tomorrow! | 16:48 |
| rlandy | have a good night | 16:49 |
| *** amoralej is now known as amoralej|off | 17:20 | |
| *** jpena is now known as jpena|off | 17:46 | |
| rlandy | dviroel: around? want to sync for a few? | 17:53 |
| dviroel | rlandy: sure | 17:55 |
| rlandy | dviroel: https://meet.google.com/xcg-rwac-pbw?pli=1&authuser=0 | 17:57 |
| rlandy | dviroel: dasm: so ... maybe you know this ... | 20:11 |
| rlandy | how would it be possible for 'openstack overcloud --help' to work from command line but not script? | 20:11 |
| rlandy | it seems like the client install is recognized from command line but not a script run | 20:12 |
| rlandy | ++ openstack overcloud --help | 20:12 |
| rlandy | Could not load 'overcloud_admin_authorize': module 'dns.rdtypes' has no attribute 'ANY' | 20:12 |
| rlandy | ^^ script run | 20:12 |
| dasm | rlandy: sourced credentials? | 20:18 |
| dasm | rlandy: what kind of script are you talking about? | 20:19 |
| rlandy | dasm: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17.1-rhel9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-9-buildimage-overcloud-full-rhos-17.1/f1f9738/logs/build_images.sh.txt.gz | 20:19 |
| dasm | checking | 20:20 |
| rlandy | openstack --debug overcloud image build will work from command line | 20:20 |
| dasm | is it possible that the script runs under different username? without proper credentials? | 20:20 |
| dviroel | maybe they are using different python libraries, from different environments? one is using from a specific virtual env? | 20:22 |
| rlandy | this is downstream so the install should all be rpm | 20:23 |
| dasm | https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17.1-rhel9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-9-buildimage-overcloud-full-rhos-17.1/f1f9738/logs/build.log.txt.gz | 20:24 |
| dasm | sounds like mismatch in versions | 20:24 |
| dasm | which is strange, because over here it starts without issues: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17.1-rhel9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-9-buildimage-overcloud-full-rhos-17.1/f1f9738/logs/build-err.log.txt.gz | 20:25 |
| dasm | just later it fails | 20:25 |
| rlandy | dviroel: so I think you are right ... | 20:27 |
| rlandy | if I get rid of the source /home/zuul/workspace/venv/bin/activate | 20:27 |
| rlandy | it does start | 20:28 |
| rlandy | it fails | 20:28 |
| rlandy | but starts | 20:28 |
| dviroel | you can compare both with pip freeze output | 20:30 |
| dviroel | and check rdtypes | 20:30 |
| rlandy | just trying to get a clean run | 20:35 |
| rlandy | sec | 20:35 |
| rlandy | dviroel: dasm: so here's the thing ... | 20:40 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17.1-rhel9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-9-buildimage-overcloud-full-rhos-17.1/f1f9738/logs/build_images.sh.txt.gz | 20:40 |
| rlandy | "/home/zuul/workspace/venv" exists | 20:40 |
| rlandy | so it sources it | 20:40 |
| rlandy | even though for downstream it's an rpm install | 20:41 |
| rlandy | if I comment that out, it gets further | 20:41 |
| * rlandy tries a patch | 20:41 | |
| * rlandy wasted a ton of time on this debug | 20:43 | |
| rlandy | adding notes and moving on | 20:43 |
| dasm | rlandy: ack | 20:46 |
| dasm | any recent change? | 20:46 |
| rlandy | idk - need to check with rel del | 20:47 |
| dasm | ack | 20:47 |
| rlandy | dviroel: hmmm ... https://bugzilla.redhat.com/show_bug.cgi?id=2161262 - last comment | 20:55 |
| rlandy | think maybe getting this | 20:55 |
| rlandy | ansible-collections-openstack.cloud==1.8.0 | 20:55 |
| rlandy | changed collections install? | 20:56 |
| rlandy | jm1[m]: you still around? | 20:57 |
| rlandy | dviroel++ thanks for that hint | 21:12 |
| dviroel | \o/ | 21:18 |
| * dasm => offline | 22:36 | |
| dasm | o/ | 22:36 |
| *** dasm is now known as dasm|off | 22:37 | |
| * rlandy out see you all tomorrow | 22:59 | |
| *** rlandy is now known as rlandy|out | 22:59 | |
| *** dviroel is now known as dviroel|out | 23:25 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!