Thursday, 2023-01-05

akahat|rovero/04:25
*** pojadhav is now known as pojadhav|ruck04:36
ysandeepgood morning o/05:44
ysandeepakahat|rover, some jobs failed in your testrun because of missing containers: https://review.rdoproject.org/zuul/status#46525 05:56
ysandeepWould be good to also include container build in testproject itself.05:56
ysandeepakahat|rover, do you want to meet for few to sort out the next steps?05:59
ysandeepakahat|rover, regarding your testrun which is running component jobs, it will fail because https://review.opendev.org/c/openstack/tripleo-quickstart/+/869099 check release files don't have systemd in exclude06:01
ysandeepakahat|rover, I am updating your patch to just exclude systemd for a test06:04
marioso/06:05
ysandeepmarios, good morning o/06:05
mariosmorning06:07
ysandeeppojadhav|ruck, hey o/ could you please swap chandankumar next ruck/rover swift with me(starting tomorrow). He covered me last week.06:15
pojadhav|ruckysandeep, yep06:15
ysandeeppojadhav|ruck, thanks!06:15
pojadhav|ruckysandeep, done06:17
pojadhav|ruckbhagyashris, soniya29 : swapped your names too as requested06:18
ysandeeppojadhav++ 06:18
soniya29pojadhav|ruck, thank you06:21
chandankumarysandeep: thank you :-)06:22
chandankumarGood morning #oooq06:22
ysandeepchandankumar, good morning o/ and thanks to you!06:24
ysandeepakahat|rover, updated testprojects, lets see after few hours.06:25
akahat|roverysandeep, o/ thanks.. i've added different packages in the releases.06:30
akahat|roverso we can narrow down the package list06:31
akahat|roverysandeep, also build-containers job is failing because of https://bugs.launchpad.net/tripleo/+bug/200162606:31
ysandeepakahat|rover, ^^ Fixed now, i have included container build in testproject itself06:33
ysandeepat this point we have tried different packages, lets try systemd alone in exclude - testproject running right now06:34
akahat|roverysandeep, zed is not yet merged: https://review.opendev.org/c/openstack/tripleo-common/+/86920706:34
ysandeepahh okay06:35
mariosbogdan need to remove his -1 there06:37
mariosfor the zed patch 06:37
mariosbut he is not on irc yet 06:37
akahat|roveryes.. we need to wait for hi.m06:37
marios(it isn't going to gate i think because of bogdan -1)06:37
akahat|roveris review.opendev.org is down? not able to post the comment.06:47
mariosah i see Tengu did go through with making it non voting on master patch (comment is there akahat|rover )06:48
akahat|rovermarios, yes.. he fixed master.. this patch wil fix zed: https://review.opendev.org/c/openstack/tripleo-common/+/869207/2#message-18bab14c2c57f4b4260a29f7877275b0191f920806:49
mariosakahat|rover: ysandeep: chandankumar: please vote there when you have sec we need that back in 06:49
marioshttps://review.opendev.org/c/openstack/tripleo-common/+/869242/206:50
mariosok cool so master is unblocked at least 06:50
marios(gate i mean)06:50
akahat|roverwe don't have nay hack to remove w -1 from bogdan? I think admins can do it.06:51
mariosakahat|rover: looks like only owner can do it06:56
mariosshould be around soon still 0756 in central eu06:56
akahat|roverok. 06:57
akahat|rovermarios, chandankumar ysandeep please take a look when you free: https://review.opendev.org/c/openstack/tripleo-common/+/86920807:02
mariosack akahat|rover but for same repo better to rebase instead of depends-on 07:04
akahat|rovermarios, tired with rebase.. was not working.. so updated with depends-on.07:05
mariosakahat|rover: what didnt work 07:05
chandankumaryes depends on does not make sense here, rebase would be better07:05
akahat|roverrebase from gerrit ui didn't worked.07:06
akahat|roverlet me try again07:06
mariosakahat|rover: its ok doesn't matter now just for future ref07:09
mariosakahat|rover: doesnt matter because other patch merged already 07:10
akahat|roveryeah.. it got merged.. rebased and removed depends-on.07:10
mariosack revoted07:11
mariosakahat|rover: pojadhav|ruck: lets do quick sync at usual time please ? ( invite sent) should be brief today 07:28
akahat|rovermarios, yeah.. sure. accepted.07:29
* akahat|rover lunch07:30
*** soniya29 is now known as soniya29|lunch07:30
pojadhav|ruckmarios, ack07:31
pojadhav|ruckmarios, ysandeep : i think we should have RR handover today before planning meeting, otherwise it will too late for us. what say ?07:36
ysandeeppojadhav|ruck, works for me before planning mtg 07:37
pojadhav|ruckysandeep, thanks, will send invite for the same07:38
mariospojadhav|ruck: sure07:40
mariosthx pojadhav|ruck 07:41
mariospojadhav|ruck: add a google meet to the event 07:42
pojadhav|ruckalready added07:42
pojadhav|rucki can see google meet in the invite07:42
mariospojadhav|ruck: yeah my bad thanks07:43
ysandeepakahat|rover, all the 3 jobs passed node provisioning with systemd exclude: https://review.rdoproject.org/zuul/status#4521707:47
mariosysandeep: great lets try rerun it few times today please?07:48
ysandeepyeah, that's the plan, few more jobs are running here already: https://review.rdoproject.org/zuul/status#46525 07:49
* pojadhav|ruck afk for half hour07:52
Tengumarios: heya!07:59
marios\o07:59
Tengumaster is merged, so we should be unlocked. zed and re-enabling master upgrade are in gate.07:59
mariosyeah thanks07:59
Tenguwe're on track now.07:59
mariosi saw and we workflowed the partial revert07:59
mariosthanks Tengu 07:59
Tengunp07:59
Tenguhappy to help :)07:59
mariosbut we need bogdan to remove his -107:59
mariosfrom zed08:00
Tenguah, right.08:00
mariosshould be in soon hopefully 08:00
Tenguthat's why -w is annoying.08:00
Tengugrumpf.... that edpm thing is driving me mad.08:01
Tenguchandankumar: heya! seems it's still not deploying on the right node..... how can I get the subnode-1 IP in order to create the appropriate entry in the inventory?08:02
chandankumarTengu: /etc/nodepool/sub_nodes_private contains the ip of second node08:04
chandankumaryou can find this file on undercloud08:04
chandankumarjust do a cat 08:04
akahat|roverysandeep, \o/ that's good news. :) 08:04
chandankumarsomething like this https://review.rdoproject.org/r/c/testproject/+/46393/20/playbook/run.yaml#14908:04
Tenguhmm ok.08:05
Tengucalling slurp will be more "ansible", but ok.08:05
Tengulemme do some tweaking.08:05
*** soniya29|lunch is now known as soniya2908:14
*** ysandeep is now known as ysandeep|lunch08:16
mariosakahat|rover: o/ 08:31
* pojadhav|ruck lunch08:46
*** ysandeep|lunch is now known as ysandeep09:34
ysandeepakahat|rover, two periodic line job also passed node provisionin, I am rerunning some jobs 09:37
akahat|roverysandeep, this also shows good results: https://review.rdoproject.org/r/c/testproject/+/4521709:38
akahat|roveri'll re-run all the fs01 jobs.. it will give us good results09:38
ysandeepakahat|rover, I have updated https://review.rdoproject.org/r/c/testproject/+/45217 , lets see09:39
akahat|roveroh.. you also update it :|09:39
ysandeepakahat|rover, if all passes this time, lets remove the DNM from the main patch 09:39
akahat|roverysandeep, +109:40
ysandeepakahat|rover, https://review.rdoproject.org/r/c/testproject/+/45217/8/.zuul.yaml soo many ovb together might cause resource crunch09:40
ysandeepakahat|rover, could you please comment few for now09:41
akahat|roverysandeep, ah.. 2 2 nodes.. 09:41
akahat|roveryes.. let's go with master first.09:41
arxcruzhey guys, i need to go to the doctor, brb in one or two hours09:41
ysandeeparxcruz, o/ sure09:45
Tenguchandankumar: wow..... it's fun: apparently I have *exactly* the same behavior on my local lab.10:07
Tenguso there's something fishy with either the playbook, or the inventory. Need to investigate a bit.10:08
Tenguat least, it's faster to debug locally :).10:08
Tenguespecially since I can interrupt pretty fast.10:08
Tengufun... fun, fun, fun.10:14
Tengu[WARNING]: Could not match supplied host pattern, ignoring: []    <- I think this is the issue.10:15
Tenguand this one comes from... gimme a sec...10:15
Tenguhosts: "{{ deploy_source_host | default([]) }}:{{ deploy_target_host | default('overcloud') }}"10:15
Tenguand I think here we get a tiny issue.10:15
Tenguo____O10:25
Tenguchandankumar: it's..... eewww.. guess I'll need to grab some more coffee. Like, some casks actually.10:26
chandankumarTengu: glad you found the issue. :-)10:26
chandankumarsorry little busy with docs writting.10:26
Tenguchandankumar: well, I still don't understand what's going on10:27
Tengubut there's something really, really fishy in the way ansible takes and understand the inventory....10:27
Tenguchandankumar: https://paste.openstack.org/show/bODLyx65Z1lneE0UPmL4/  so, the ansible_host is correct; the ansible_fqdn and ansible_hostname are NOT.10:29
Tengui.e. it's taking the IP of the compute, and does something wrong in order to map it to the current host (which is, well, named "standalone", for that matter.10:30
Tenguoh. wrong. ok. still "undercloud". only the prompt is correct.10:30
Tenguanyway. it's running against the undercloud/standalone instead of the remote, while it DOES have the correct IP.10:30
akahat|roverysandeep, i've updated your patch.. if we have enough passing jobs we can push it to gate. https://review.opendev.org/c/openstack/tripleo-quickstart/+/86860510:41
akahat|rovermarios, ^^10:42
ysandeepakahat|rover, could you please update the comments also10:42
ysandeepfrom Pin kernel to pin systemd10:42
mariosthx 10:44
akahat|roverdone10:51
ysandeepakahat|rover, thanks, need some more small changes: https://review.opendev.org/c/openstack/tripleo-quickstart/+/86860510:53
ysandeephttps://review.opendev.org/c/openstack/tripleo-quickstart/+/868605/15#message-e5dc90dcb7f47ff8aa030d90db83fa0b4f8a7ce610:53
akahat|roverysandeep, ok10:57
ysandeepakahat|rover, ++ thank you, looks good now11:03
ysandeepThe only issue I see is, we are currently excluding affected 252* version but as soon as newer systemd releases (and if our bug is not solved by then) we will again hit this issue.11:06
* ysandeep not sure of cadence of systemd release11:07
Tenguchandankumar: ok. apparently something in the edpm playbooks make it works only for local connection.... still need to dig a bit more, but that's one of the issue.11:08
ysandeepakahat|rover, 4/4 jobs passed node provision again: https://review.rdoproject.org/zuul/status#45217 , looks like we finally have the culprit11:08
Tengui.e. it remaps things to the current localhost. "yay".11:08
TenguI'll check with James as well, maybe it's intended. Would be bad imho, but heh.11:08
*** rlandy|out is now known as rlandy11:09
Tenguwondering if the way the inventory is built/displayed isn't the main issue.......11:10
chandankumarTengu: when I was testing, I was running the playbook from second node to deploy compute on the second node itself11:16
chandankumarif I remove this task https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/external-data-plane-management/tasks/compute_services_on_target_host.yml#L2911:17
chandankumarit does not make any difference11:17
chandankumarit will run using localhost11:17
chandankumarthe way you are trying to execute from the same node, I have not tried that11:17
chandankumars/same/undercloud node11:18
chandankumarlet's check with james what he says about that11:18
arxcruzback11:22
rlandymarios: akahat|rover: pojadhav|ruck: should we sync for a few?11:35
pojadhav|ruckrlandy, yep11:36
mariosrlandy: ok 11:38
ysandeepdo you mind me listening in the background as I am the next ruck/rover, We will probably need a shorter handover sync if I join now.11:38
mariosyes good idea ysandeep 11:38
mariosimo we can even skip that 11:39
rlandymarios: ysandeep: okp you are on next week11:39
mariosyes11:40
rlandymarios: ysandeep: so up to you if you want to sync now or after scrum11:40
rlandyI think now is better11:40
ysandeepmarios, +1 to just meet now and skip later 11:40
mariosakahat|rover: pojadhav|ruck: around?11:40
pojadhav|ruckyes11:40
rlandyafter scrum is late for ysandeep and pojadhav|ruck 11:40
mariosrlandy: we have it scheduled *before* scrum but still now is good if akahat|rover is around11:40
mariosk lets try in 20 mins top of the hour rlandy ok for you then or have calls11:41
mariosgive akahat|rover a chance if he is afk/break 11:41
rlandymarios: ok - can you move the meeting there then?11:42
mariossure pojadhav|ruck has to do it it is hers can you please pojadhav|ruck ?11:42
mariospojadhav|ruck: move the rr hanforr in 20 mins 11:42
pojadhav|ruckmarios, ack 11:42
rlandythanks11:42
mariosakahat|rover: ysandeep: ^^^ 20 mins 11:42
rlandymarios; and sorry - misread the BEFORE part11:43
mariosack 11:43
rlandysoniya29: hey - you around?11:43
ysandeepmarios, ack works for me11:43
*** dviroel|ourt is now known as dviroel11:45
pojadhav|ruckfolks, I have created new agenda for 73 sprint planning, kindly add agenda if any.. thank you !! https://hackmd.io/0ruKGb6eTwGvdvwWE4nk8g#2023-01-05-Unified-Sprint-73-Planning11:46
akahat|rovermarios, pojadhav|ruck rlandy i'm here.11:47
pojadhav|ruck*add new topics/tasks that needs to be done this sprint.11:47
soniya29rlandy, yes11:48
pojadhav|ruckakahat|rover, i have set RR handover in around 15 mins.11:48
* soniya29 stepping out for an hour11:51
mariosakahat|rover: call in 9 mins11:51
pojadhav|ruckmarios, rlandy : RR handover12:04
Tenguchandankumar: host_vars/lab2-oc0-compute-1:ansible_connection: local12:05
Tenguof course.......12:05
Tengu#facepalm12:05
Tenguchandankumar: so this explain the issue in CI as well :).12:07
Tengueasy enough to correct.12:07
akahat|rovermarios, rlandy https://review.opendev.org/c/openstack/tripleo-quickstart/+/86860512:11
Tenguimho this host_vars/localhost shouldn't list that ansible_connection parameter, it should be in the inventory 02-computes12:15
Tengudang. that one was a hard nut to crash.12:15
chandankumarTengu: awesome :-)12:24
chandankumaryou can send a review to fix tripleo-ansible inventory12:25
chandankumarhttps://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/inventory12:25
chandankumarwe ship this file in the tripleo-ansiblee container12:25
Tengusure - lemme do some more testing, but apparently it should really, really work better now.12:28
Tengudang.12:28
*** ysandeep is now known as ysandeep|afk12:32
* pojadhav|ruck afk for ~1 hr12:34
Tenguchandankumar: https://review.opendev.org/c/openstack/tripleo-ansible/+/86935712:34
Tengualso took the opportunity to make the inventory files format more consistent.12:34
Tengupfff.... how many days did I lose because of that? ;_;12:35
* akahat|rover back in hour.. 12:41
chandankumarTengu: great. 13:01
chandankumarmore than 2 weeks13:01
chandankumarbut it was a great improvement to the roles and inventory :-)13:01
Tenguheh... yeah. still.13:02
soniya29rlandy, hello13:49
* soniya29 i am back13:49
rlandysoniya29: hi - let's meet for 513:49
soniya29okay13:50
rlandysoniya29: https://meet.google.com/jmg-ytnf-hax?pli=1&authuser=013:50
*** dasm|off is now known as dasm13:50
dasmo/13:51
rlandydasm: gm14:00
dasmrlandy: \o14:02
TenguFolks, we may have an issue in tripleo-ansible repository: https://zuul.opendev.org/t/openstack/builds?job_name=openstack-tox-pep8&project=openstack%2Ftripleo-ansible&skip=0   not sure yet about the cause, probably something in the tox.ini, I think Rabi corrected something not that long ago somewhere. So just FYI.14:07
Tengulast successful: https://review.opendev.org/c/openstack/tripleo-ansible/+/869020/ (in Gate) - then all red.14:08
Tengumaybe related (that's the patch I thought about, from Rabi)14:08
dasmakahat|rover: pojadhav|ruck ^ are you aware of that?14:09
akahat|roverdasm, yes.. its known14:10
dasmTengu: ^14:10
dasmakahat|rover: ack14:10
Tengu'k - is there already some patch around?14:11
Tenguoh, wait - patch is still active.14:12
Tenguguess we'll have to wait a bit.14:13
akahat|roverq: playbooks from rdo picking master here: PRE-RUN END RESULT_NORMAL: [trusted : review.rdoproject.org/config/playbooks/tripleo-rdo-base/container-login.yaml@master]14:20
akahat|roverand it removed from master here: https://opendev.org/openstack/tripleo-ansible/commit/69bac5fdc5306c022cc17d0ecc6a9db8c5adab7914:20
akahat|roverregarding this: https://logserver.rdoproject.org/openstack-periodic-integration-zed-centos9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-zed/8a282a5/job-output.txt14:21
akahat|roverlp: https://bugs.launchpad.net/tripleo/+bug/200191614:21
akahat|roverjob broken since long time.. https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-9-scenario010-kvm-standalone-wallaby&skip=014:22
pojadhav|ruckfolks, sprint planning meeting in 4 mins : arxcruz, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat|rover , chandankumar, frenzy_friday, dviroel, rlandy, dasm, Tengu, jgilaber 14:26
pojadhav|ruckhttps://hackmd.io/0ruKGb6eTwGvdvwWE4nk8g#2023-01-05-Unified-Sprint-73-Planning14:26
pojadhav|ruckhttps://meet.google.com/avd-ijsb-ejt?authuser=014:26
*** ysandeep|afk is now known as ysandeep14:26
Tenguwow.... my external compute is deployed.14:29
pojadhav|ruckdasm, chandankumar, dviroel : planning meeting14:30
* pojadhav|ruck dinner15:44
chandankumarsee ya people tomorrow!15:45
rlandyakahat|rover: marios: ysandeep: ok to merge this after gate clears failing tests? https://review.opendev.org/c/openstack/tripleo-quickstart/+/86860515:48
rlandyI voted15:48
ysandeeprlandy, yes15:48
ysandeep10/10 job passed with systemd in exclude15:49
rlandygreat15:50
rlandybhagyashris; hi - can you come to review meeting on friday and we can merge up your patches?15:51
ysandeeprlandy, for tomorrow's podified friday call.. doug/me would like us to spend sometime dividing our work in jira tasks.15:51
ysandeepdviroel, ^^ 15:51
rlandyysandeep++ 15:52
rlandysounds good15:52
* ysandeep out, see everyone tomorrow15:59
*** ysandeep is now known as ysandeep|out16:00
*** dviroel is now known as dviroel|lunch16:01
*** soniya29 is now known as soniya29|out16:08
*** pojadhav|ruck is now known as pojadhav|out16:13
bhagyashrisrlandy, sure16:32
Tenguchandankumar: fyi, EDPM is now deploying fine on tripleo-lab, had to tweak some stuff. I have a better understanding about the CI job itself, and I'm confident we should get something fine shortly now.16:32
* marios off in few have a good one oooq o/ 16:32
rlandyrechecking https://review.opendev.org/c/openstack/tripleo-ansible/+/869020 16:33
rlandymarios: have a good night16:33
Tengurlandy: is the Zed patch merged? the one about the missing files?16:33
mariosthanks rlandy 16:33
Tenguit is16:33
Tengu\o/16:33
rlandy<opendevreview> Merged openstack/tripleo-common stable/zed: Fix missing 50-edk2-ovmf-*.json files handling  https://review.opendev.org/c/openstack/tripleo-common/+/86920716:33
mariosyes https://review.opendev.org/c/openstack/tripleo-common/+/86920716:34
mariosfinally16:34
rlandyTengu: and the upgrades voting16:34
Tenguyep - and I saw the last patch of that serie, re-enabling the upgrades jobs16:34
Tenguyeah16:34
Tenguthat was a nice run.16:34
rlandyrechecking https://review.opendev.org/c/openstack/tripleo-ansible/+/869020 and 16:34
rlandythe growvols wo16:34
*** marios is now known as marios|out16:35
rlandy https://review.opendev.org/c/openstack/tripleo-quickstart/+/86860516:35
rlandylunch - brb16:35
Tenguok folks - getting offline. See you tomorrow16:46
dasmTengu: o/17:02
*** dviroel|lunch is now known as dviroel17:10
dasmdviroel: o/ can you +2 this one? it lost reviews due to rebasing: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4566917:35
reviewbotDo you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks.17:35
dasmdviroel: rlandy would be able to +W outstanding patches17:35
* dviroel check17:36
akahat|roverrlandy, https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605 is good to go, voted.17:55
rlandyakahat|rover: thanks17:56
* akahat|rover will be out in few mins.. see you tomorrow!17:58
rlandydviroel: ok - one left of dasm''s patches to look at pls: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45699 18:05
rlandyif you agree, I'll w that18:05
dviroelrlandy: done, lgtm18:36
dasm thanks18:38
rlandydasm: ok - that should be all of your patches18:51
rlandydasm: any idea what's going on here with tox test failures: https://review.opendev.org/c/openstack/tripleo-repos/+/866316?18:53
rlandydropping review time ... 19:02
rlandyjust need one more core vote on: 19:02
rlandy    https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605/ Exclude latest systemd*-252-* packages19:02
dasmjust joined for review. no one's here. dropping and going over the list on my own19:05
*** dviroel is now known as dviroel|afk21:19
*** rlandy is now known as rlandy|out22:13
dasmo/22:33
*** dasm is now known as dasm|off22:33

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!