Wednesday, 2021-09-01

« Tuesday, 2021-08-31 Index Thursday, 2021-09-02 »
*** ysandeep|away is now known as ysandeep00:59
slaweqchandankumar: hi, how are You?05:55
chandankumarslaweq: Hello, I am good05:55
slaweqchandankumar: can You help me undestand why ci in https://review.rdoproject.org/r/c/testproject/+/35120 is failing?05:55
chandankumarslaweq: thanks for asking, How are you?05:55
chandankumarslaweq: let me check05:55
slaweqI wanted to test if one of the previously skipped tests will now be fine but I can't :/05:56
slaweqor maybe You have link to that job which is running perodically tests from skiplist? I can maybe check there if that test is passing05:57
ykarelslaweq, may be you looking for jobs similar to https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-scenario001-skipped-tempest-master06:02
ykarelfor other release can change "master"06:02
chandankumarslaweq: above job failed at undercloud install https://logserver.rdoproject.org/20/35120/1/check/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-ussuri/5dde5b8/logs/undercloud/home/zuul/undercloud_install.log.txt.gz while uploading the image is givjng 40406:02
ykareli just see your last two messages so missed context06:02
chandankumarslaweq: let me take a look at ussuri skipped job06:02
slaweqchandankumar: ykarel thx06:05
chandankumarhttps://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-scenario007-skipped-tempest-ussuri&project=openstack/tripleo-ci06:06
chandankumarmost of the ussuri jobs are broken06:06
chandankumarmost of the skipped ussuri jobs are failing before tempest run06:06
*** ykarel is now known as ykarel|away06:07
ykarel|awayme out, not feeling weel06:07
chandankumarykarel|away: take care :-)06:07
chandankumarslaweq: https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable3 so in last periodic run container build job was failing, https://review.rdoproject.org/r/c/testproject/+/35120 uses force_periodic to true, so the job will pull container with tripleo-ci-testing hash (which was suppose to be container build job which got failed in this run) are also failing06:11
chandankumardue to missing containers on registry06:11
chandankumarslaweq: https://review.opendev.org/c/openstack/tripleo-ci/+/806645 once this merges, will fix the container build then we can retrigger the job06:12
slaweqchandankumar++ ykarel++ thx a lot for help06:14
slaweqchandankumar: I can run my test job without force_periodic: true - maybe that will work right now06:15
chandankumarslaweq: yes, but we need this https://review.rdoproject.org/r/c/rdoinfo/+/35105/1/tags/ussuri.yml fix06:16
chandankumarlet me send a patch in tq release file to install it on undercloud06:16
chandankumarslaweq: neutron-tempest-plugin1.2.0 is available in current-tripleo06:19
chandankumarhttps://trunk.rdoproject.org/centos8-ussuri/component/network/7d/59/7d5988b29b6eb27b2dc954ea287c85bfa2ec67c4_a3633030/06:19
chandankumarso without force_periodic it will work and we donot need to make changes in tq release file06:19
slaweqchandankumar: so we should be good to go with https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/805899 if that test job will pass, right?06:23
chandankumarslaweq: yes06:24
slaweqthx06:24
*** jpena|off is now known as jpena07:03
*** amoralej|off is now known as amoralej07:05
bhagyashriszbr Hi around?07:11
zbrhi! in 30min07:11
bhagyashrisok np07:12
zbri am back08:05
bhagyashriszbr, https://meet.google.com/xnf-tvdh-pmk?authuser=008:08
*** arxcruz is now known as arxcruz|training08:16
ysandeepmarios: welcome back! Hope you enjoyed your PTOs08:23
*** ysandeep is now known as ysandeep|lunch08:26
marioso/ ysandeep|lunch thanks 08:29
soniya29chandankumar, kopecmartin, arxcruz|training, ysandeep|lunch, please add/edit today's agenda for the tempest meeting08:56
*** ysandeep|lunch is now known as ysandeep09:00
*** sshnaidm|afk is now known as sshnaidm09:08
bhagyashriszbr, fyi https://review.opendev.org/c/openstack/tripleo-repos/+/80687909:38
zbrbhagyashris: no1 prio is to make sanity voting.09:41
bhagyashrisyes doing 09:41
bhagyashriszbr, Done  https://review.opendev.org/c/openstack/tripleo-repos/+/806880 09:44
bhagyashristhanks :)09:44
zbrin fact you only need to remove the voting, default is true.09:45
bhagyashriszbr, ok i wll do that09:47
zbri will be out for ~1.5h today, need to take blood test10:25
*** pojadhav is now known as pojadhav|brb10:46
marioszbr: chandankumar: o/ hey i am working there (checking in case you were doing something) https://review.opendev.org/c/openstack/tripleo-quickstart/+/791486/47#message-f19da2ac2f0c3aa53aee30a9330390367278e47d 10:46
chandankumarmarios: please go ahead10:48
zbrmarios: chandankumar am I wrong o patch #46 did pass on both upstream/rdo. If that is true lets merge in that form and look to other changes another time.10:50
zbrthat patch is already 4mo old...10:51
zbrmarios: please +2 https://review.opendev.org/c/openstack/tripleo-repos/+/805400 asap10:53
zbri just realised that the fixes to sanity did were not merged.10:53
marioszbr: ack adding to reviews 10:54
zbri really need to ensure sanity passes as otherwise people will fail to read its output. this morning i had a talk with bhagyashris around that.10:54
*** jpena is now known as jpena|lunch11:30
*** rlandy is now known as rlandy|rover11:38
rlandy|roverdviroel|ruck: hey - how are things today?11:39
dviroel|ruckrlandy|rover: hi, the containers build fix merged, but not in time for the last train pipeline run11:42
rlandy|roverdviroel|ruck: we can rekick that line11:42
rlandy|roversec11:42
dviroel|ruckyep, would be  a good idea, next run is 18:00 utc11:42
*** pojadhav|brb is now known as pojadhav11:43
rlandy|roverdviroel|ruck: does ussuri also need a rekick?11:45
dviroel|ruckrlandy|rover: it should start in 15 min (12:00 utc)11:46
rlandy|rovergreat11:46
rlandy|roverdviroel|ruck: only the 16.2 tripleo component is old11:46
rlandy|rovertripleo in 17 promoted11:47
dviroel|ruckwhat is this refresh fail?11:48
rlandy|roverdviroel|ruck: where is a refresh fail?11:50
dviroel|rucktripleo component 16.2 and upstream11:51
dviroel|ruckhttp://tripleo-cockpit.usersys.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=111:51
dviroel|rucklike this one here https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-tripleo-train/3e4fc57/logs/]11:52
rlandy|roveroh failure log11:53
rlandy|rover FATAL | Wait for containers to start for step 3 using paunch | standalone-0 | error={"ansible_job_id": "984259434115.114221", "attempts": 1200, "changed": false, "finished": 0, "started": 1}11:54
rlandy|roverhttps://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-tripleo-train/3e4fc57/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz11:54
rlandy|roverthat is a sova record11:54
rlandy|roverfor the error11:55
rlandy|rovermarios: hey - welcome back - how was your vacation?11:55
rlandy|roverperiodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-tripleo-train11:58
rlandy|roverdviroel|ruck: ^^ I don;t think that job ever passed11:58
rlandy|roverit was an experiment in train11:58
rlandy|rovervoting afterwards11:58
mariosrlandy|rover: o/ hi was nice thanks :)11:58
soniya29chandankumar, rlandy|rover, weshay|ruck, arxcruz|training , tempest meeting?12:02
chandankumarrlandy|rover: https://review.rdoproject.org/r/c/config/+/35230 please have a look when free12:02
rlandy|roverhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-tripleo-rhos-16.2&pipeline=openstack-component-tripleo12:03
rlandy|rover^^ that's the issue for tripleo 16.212:03
soniya29rlandy|rover, weshay|ruck, arxcruz|training, ^^12:04
rlandy|roversoniya29: I have a clash with the progran call12:04
ysandeepkopecmartin: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/806574 12:04
rlandy|rover2021-08-31 20:26:27.331827 | primary | PLAY [Prepare the SSL Configuration for the overcloud deployment] **************12:06
kopecmartinysandeep: interesting, thanks12:06
rlandy|roverdviroel|ruck: ovb on tripleo component12:07
rlandy|roverysandeep: ^^ know about this error? or worth rerun on 16.2 https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-tripleo-rhos-16.2/583e952/job-output.txt12:08
ysandeeprlandy|rover: yes already rerunning it here: https://code.engineering.redhat.com/gerrit/c/testproject/+/211494 12:08
rlandy|roverysandeep: yep - I see thanks12:09
rlandy|rovergoing to rerun periodic-tripleo-ci-rhel-8-multinode-1ctlr-featureset010-tripleo-rhos-16.212:09
rlandy|roveralso failed12:09
ysandeeprlandy|rover, we have atleast 1 green run for ^^ above job for current hash in testing . http://pastebin.test.redhat.com/991032 12:11
rlandy|roveryeo12:12
rlandy|roveryep12:12
rlandy|roverysandeep: and the good news is that it's the only component that got called out12:13
*** amoralej is now known as amoralej|lunch12:14
ysandeeprlandy|rover: :( dang, we got new content on 27th(friday), That tempest allow bug was blocking fs001 till yesterday12:17
rlandy|roverysandeep: it's ok - we'll get that component cleared shortly12:18
ysandeeprlandy|rover, fyi.. reported this one for 17 https://bugzilla.redhat.com/show_bug.cgi?id=2000070 12:20
soniya29rlandy|rover, okay, no problem12:20
pojadhavarxcruz, zbr, sshnaidm, rlandy|rover , marios, ysandeep, bhagyashris, soniya29 , akahat, weshay|ruck , chandankumar, frenzy_friday, dviroel|ruck  : RETRO in next 5 mins12:25
pojadhavhttps://meet.google.com/kkp-bejs-vvo?authuser=012:25
pojadhavhttps://miro.com/app/board/o9J_lz-Gr0g=/12:25
*** jpena|lunch is now known as jpena12:30
bhagyashrisakahat, chandankumar dviroel|ruck soniya29 sshnaidm weshay|ruck retro time12:31
soniya29bhagyashris, i am in :)12:31
bhagyashrischandankumar, akahat ^12:32
*** amoralej|lunch is now known as amoralej13:08
chandankumarThank you pojadhav nice retro :-)13:14
bhagyashrisThanks pojadhav :)13:15
dviroel|ruckrlandy|rover: fyi https://review.rdoproject.org/r/c/testproject/+/34983/10#message-4479ab3f7f1dd2d83fd0b768a019f9e2f64684d913:15
pojadhavchandankumar, bhagyashris  :-)13:15
bhagyashrisdviroel|ruck, hey have a sec? just want to update few things 13:16
chandankumarsshnaidm: please have a look https://review.rdoproject.org/r/c/config/+/35230 thanks!13:17
sshnaidmchandankumar, I'll +w, but need to keep eye on it, because you know - no testing13:24
sshnaidmchandankumar, if something is wrong, all container jobs will fail13:24
dviroel|ruckbhagyashris: hey, yes :)13:25
bhagyashrisgive me 5 mins13:42
bhagyashrisdviroel|ruck, https://meet.google.com/vue-xagq-pjn?authuser=013:43
rlandy|roverdviroel|ruck: very nice on scenario00113:57
mariospojadhav: o/ hey when you have time please check https://review.rdoproject.org/r/c/rdo-jobs/+/34927/6#message-8d65ee6fc3377bab74841fef1db5682f35d0da10 -1 is for the removal of the wallaby jobs i think that may be accidental? 14:08
zbri am back14:31
zbrrlandy|rover: dviroel|ruck a review on https://review.opendev.org/c/openstack/tripleo-repos/+/805400 would be apprecatiated.14:32
rlandy|roverzbr: voted - will W+ once dviroel|ruck votes14:33
pojadhavmarios, thanks for review.. yeah it was accidental.. i will fix the review comments. :)14:35
dviroel|ruckzbr: rlandy|rover review done14:36
rlandy|roverw+'ed14:36
chandankumarsshnaidm: rlandy|rover https://review.rdoproject.org/r/c/config/+/35155 merge merge please14:37
chandankumarsshnaidm: found this https://review.opendev.org/c/openstack/tripleo-ansible/+/806428 in our debugging14:37
chandankumarlessons donot use latest version of podman14:38
rlandy|roverdone14:39
chandankumarrlandy|rover: jpena sshnaidm thanks :-)14:41
* dviroel|ruck lunch14:48
frenzy_fridayHey zbr arxcruz|training do you have some time, I need a little help with the ER bot15:12
*** chem is now known as Guest598915:16
rlandy|roverperiodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-tripleo-rhos-16.2 (3rd attempt) p assed yay15:47
rlandy|roverysandeep++15:48
*** jpena is now known as jpena|off16:01
ysandeeprlandy|rover: ah, nice.. tripleo component should promote soon16:02
ysandeeprlandy|rover: we are getting close to fixing 17 bm job, https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/806724 and https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/26889316:03
rlandy|roververy nice16:03
*** marios is now known as marios|out16:04
ysandeepI hope i am rightly assuming environment_type == 'baremetal' is just set for baremetal deployments, https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/806724/2/roles/overcloud-prep-config/tasks/main.yml#34 16:05
dviroel|ruckrlandy|rover: this one is also passing https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/80589916:08
ysandeeprlandy|rover: pojadhav and I have decided to have joint call thrice a week(just me and her) for like half an hour for downstream ruck/rover sync, so that we can divide and conquer the job failures.. 16:09
ysandeepIt took longer than ~2 hours for us today, but hopefully things will be smooth soon16:10
ysandeep17 components are all green16:10
*** ysandeep is now known as ysandeep|out16:15
* ysandeep|out out for the day, see you guys tomorrow o/16:15
*** amoralej is now known as amoralej|off16:15
rlandy|roverdviroel|ruck: ok - so we can merge those reverts16:25
rlandy|roverdviroel|ruck: w+'ed https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/805899\16:25
dviroel|ruckno 100% sure that they solved the issue, but we may want to revert this one too https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/806794 and keep watching 16:28
dviroel|rucknot all succeeded again here https://review.rdoproject.org/r/c/testproject/+/34983/10#message-3994708b13e12ce0ba86f7649f5cc401f65c516616:29
dviroel|ruckthose tests failed in this run https://logserver.rdoproject.org/83/34983/10/check/periodic-tripleo-ci-centos-8-scenario001-standalone-glance-master/4e0d4fa/logs/16:30
dviroel|ruckmaybe disabling cephadm in step3 mitigates the issue, but not sure if will complete solve the problem16:30
rlandy|roverk - let me look at  that one more in a few 16:33
rlandy|roverjust want to get the openvswitch review in16:33
rlandy|roverlunch brb16:49
dviroel|ruckchasing train here https://review.rdoproject.org/r/c/testproject/+/3513217:03
rlandy|roveroh dear ovb17:11
rlandy|roverthanks17:11
rlandy|roverdeployment issue17:12
rlandy|rover021-09-01 16:06:52 | sed: couldn't open temporary file /home/heat-admin/.ssh/sedkDjOHs: Permission denied17:12
rlandy|rover2021-09-01 16:06:52 | Could not import keys to one of ['192.168.24.8', '192.168.24.19', '192.168.24.15', '192.168.24.29']. Original error message: Command '['ssh', '-o', 'ConnectionAttempts=6', '-o', 'ConnectTimeout=30', '-o', 'StrictHostKeyChecking=no', '-o', 'PasswordAuthentication=no', '-o', 'UserKnownHostsFile=/dev/null', '-i', '/home/zuul/.ssh/id_rsa', '-l', 'heat-admin', '192.168.24.8', "sed -i -e '/TripleO split 17:12
rlandy|roverstack short term key/d' $HOME/.ssh/authorized_keys"]' returned non-zero exit status 4.17:12
rlandy|rover^^ real issue?17:12
rlandy|rover2021-09-01 15:49:41 | sed: couldn't open temporary file /home/heat-admin/.ssh/sedmapNtx: P17:13
rlandy|roverdviroel|ruck: ^^ we have  areal problem17:14
rlandy|roverwill likely reproduce17:15
rlandy|roverchecking component line17:15
dviroel|ruckyeah, all ovb failed17:17
rlandy|roverhttps://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/b80c17c/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz17:17
rlandy|roversame problem17:17
rlandy|roverdviroel|ruck: creating bug17:17
rlandy|roverso we can track debug/log17:17
dviroel|ruckack17:19
rlandy|roverdviroel|ruck: https://bugs.launchpad.net/tripleo/+bug/194235617:23
dviroel|ruck++17:23
rlandy|roverdviroel|ruck; going to ask on tripleo17:43
rlandy|rover#tripleo17:43
rlandy|roverno idea here :)17:43
dviroel|ruckyeah, was digging here, no idea yet17:44
rlandy|roveralex will know in 2 minutes17:45
rlandy|roverdviroel|ruck: easiest is to ssh to the one of the nodes where you have a running job right now18:01
rlandy|roverhold the node18:01
rlandy|roverand dig around 18:01
rlandy|roverthe /ssh folder permissions18:01
rlandy|roverI need to prep for an interview candidate18:02
rlandy|roverwill look back here in a bit18:02
dviroel|ruckyes18:05
dviroel|ruckrlandy|rover: where should i look for the cloud-init changes?18:05
dviroel|ruckmaybe i can find something18:05
rlandy|rover cloud_config18:09
rlandy|roveror at least os_cloud_config is where the trouble often was18:09
rlandy|rovercloud-init is available on the os18:10
rlandy|roverbut could be installed from elsewhere18:10
rlandy|roverdviroel|ruck: on the node 18:12
rlandy|roverhttp://pastebin.test.redhat.com/99116118:12
dviroel|ruckand /home/heat-admin/.ssh permissions?18:13
rlandy|roverdir not created yet18:14
rlandy|roverstill in undercloud setup18:14
rlandy|roverif it's only ovb18:14
rlandy|rovercould be an images thing18:14
rlandy|roverhttp://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/centos/8-stream/AppStream/x86_64/os/Packages/18:15
rlandy|roverdviroel|ruck: ^^ cloud-init was updated on 08/2518:15
rlandy|roverhttps://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train18:17
rlandy|rover08/27 was last success there18:17
rlandy|roverfailed tempest 18:17
rlandy|roverso it passed deploy18:17
rlandy|roverdviroel|ruck: k - really need to go prep18:19
rlandy|roverback to look again in a bit18:19
rlandy|rovercan you get onto the node?18:19
rlandy|roverssh zuul@38.102.83.13418:19
dviroel|rucklet me see18:19
rlandy|roveryour keys should be on there18:19
dviroel|ruckhum, asking password18:20
dviroel|ruckmay not be there18:20
rlandy|roverlet me check your keys18:21
rlandy|roverhmmm18:21
rlandy|roverseems not18:21
* rlandy|rover gets18:21
dviroel|ruckhttps://github.com/viroel.keys18:21
rlandy|rovertry now18:22
dviroel|ruckrlandy|rover: working now18:22
dviroel|rucktks18:22
rlandy|roverdviroel|ruck: we nede to check into why your keys are not there18:23
rlandy|roverdviroel|ruck: this is why we need to lock stream18:24
dviroel|ruckyeah =/18:24
rlandy|roveralso it's only train18:26
rlandy|rovercloud-init should have hit everywhere18:26
rlandy|roverlet's take this debg as far as we can - we can pull in the DF engineers more 18:26
rlandy|roverdebug18:26
rlandy|roverhttps://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/9d5ff817ea8d75d837d26cf34d1dd6b949d7b4e0/periodic-tripleo-centos-8-buildimage-overcloud-full-train/3cd93cf/build.log18:33
rlandy|rovermaybe from image builds18:33
rlandy|rover2021-09-01 12:14:35.652 |  cloud-init                               noarch  21.1-6.el8                   appstream                1.0 M18:33
dviroel|ruckis this node on hold? 38.102.83.13418:49
rlandy|rovernot at the moment18:50
* rlandy|rover holds18:50
rlandy|roverone sec18:50
rlandy|roverdviroel|ruck: should be held now18:52
dviroel|ruckgreat, thanks18:52
rlandy|rovercomparing cloud init in previous image build logs18:52
dviroel|rucki did compare periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train, passing and failing18:54
dviroel|rucklast succeeded:18:54
dviroel|ruckcloud-init.noarch                             21.1-3.el8                                @appstream                         18:54
dviroel|ruckfirst failure:18:54
dviroel|ruckcloud-init.noarch                             21.1-6.el8                                @appstream18:54
rlandy|roveryep18:54
rlandy|roverdviroel|ruck: so on the held node18:55
rlandy|roveronce it fails18:55
rlandy|roverdowngrade cloud-init and run deploy again18:55
rlandy|roverlet's see if we can get it to work with downgraded rpm18:55
rlandy|roverin fact downgrade now if you can18:55
rlandy|roverare both available for install18:55
rlandy|roverdviroel|ruck: ^^ was that compare on the undercloud or overcloud?18:58
dviroel|ruckovercloud18:58
dviroel|ruckhttps://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/cbc1d80/logs/overcloud-controller-1/var/log/extra/package-list-installed.txt.gz18:58
rlandy|roverok - so image builds18:58
rlandy|roverhmmm ... checking current-tripleo19:02
dviroel|ruckrlandy|rover: http://pastebin.test.redhat.com/99117419:05
rlandy|roverpls put that in the bug for tracking19:06
dviroel|ruckack19:07
dviroel|rucksame thing for other overcloud note19:08
dviroel|ruckdeploy just failed19:08
rlandy|roverhttps://github.com/openstack/os-net-config/commit/69699cda8ff1deda7c121be7660145a79325876619:12
rlandy|rovermaybe19:12
rlandy|roverwe may need to use old overcloud images19:19
rlandy|roverand deploy with those19:19
dviroel|ruckcloud-init.log19:24
dviroel|ruckhttps://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/cbc1d80/logs/overcloud-controller-0/var/log/cloud-init.log.txt.gz19:24
dviroel|ruck..19:24
dviroel|ruck2021-08-28 21:57:54,216 - util.py[DEBUG]: Changing the ownership of /home/heat-admin/.ssh to 0:019:24
dviroel|ruckwhile in passing jobs is:19:24
dviroel|ruck2021-08-27 21:52:29,295 - util.py[DEBUG]: Changing the ownership of /home/heat-admin/.ssh to 1000:100119:24
dviroel|ruckneed to check overcloud cloud-init scripts19:25
rlandy|rovergoing to chase ussuri19:32
rlandy|rover#- periodic-tripleo-ci-centos-8-ovb-1ctlr_1comp-featureset002-ussuri19:34
rlandy|rover #- periodic-tripleo-ci-centos-8-scenario010-standalone-ussuri19:34
rlandy|roveractually should promote19:34
rlandy|rover2021-09-01 19:04:04,654 3791530 INFO     promoter Running: env ANSIBLE_LOG_PATH=/home/promoter/web/promoter_logs/container-push/20210901-190351.log ANSIBLE_DEBUG=False ansible-playbook -v -e @/tmp/tmpjasfarz9.yaml /home/promoter/ci-config/ci-scripts/container-push/container-push.yml19:35
rlandy|roveryes it is19:35
rlandy|roverdviroel|ruck: pls paste all your debug into  the LP s we can hand it over to frenzy_friday in the start of her morning19:36
rlandy|roverand see if we should lock cloud-init19:36
dviroel|ruckrlandy|rover: ok,  will do19:40
* dviroel|ruck need to take a walk, brb in a couple of min19:51
rlandy|roversure20:01
*** rlandy|rover is now known as rlandy|rover|mtg20:29
dviroel|ruck job from master https://logserver.rdoproject.org/26/806926/2/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/77510da/logs/overcloud-controller-0/var/log/extra/package-list-installed.txt.gz20:39
dviroel|ruckcloud-init.noarch                             21.1-6.el8                                @appstream                     20:39
dviroel|ruck2021-09-01 18:00:30,325 - util.py[DEBUG]: Changing the ownership of /home/heat-admin/.ssh to 0:0 20:40
dviroel|ruckbut don't fail20:40
*** rlandy|rover|mtg is now known as rlandy|rover21:19
rlandy|roverdviroel|ruck: hey21:19
rlandy|roverdviroel|ruck: no worries - you can just leave all your notes on the LP21:20
rlandy|roverAdding a ping now for frenzy_friday and chandankumar to take a look in their morning21:20
rlandy|roveralso updating the ussuri standalone upgrades card21:20
dviroel|ruckrlandy|rover: doing some tests here too21:21
dviroel|ruckin the node21:21
rlandy|roversure21:24
rlandy|roverupdated the standalone-ussuri-upgrade card21:28
rlandy|roverthe solution is right but in the wrong place - will have to check with marios about the best place to put it.21:28
rlandy|roverdviroel|ruck: ^^21:28
dviroel|ruckrlandy|rover: i have a workaround for this bug, without locking cloud-init version21:43
dviroel|ruckin tripleoclient21:43
rlandy|roverdviroel|ruck++ very nice21:56
dviroel|ruckrlandy|rover: https://review.opendev.org/c/openstack/python-tripleoclient/+/80699322:14
dviroel|ruckonly affects stable/train22:14
* rlandy|rover looks22:15
dviroel|ruckanother possible fix is to make cloud-init set the expected permissions on .ssh dir22:15
rlandy|roverdviroel|ruck: this seems reasonable to me22:16
rlandy|roverare you still testing it?22:17
rlandy|roverwhen ready pls add kevin and alex on the review22:17
rlandy|rovermaybe cedric22:17
rlandy|roverso they can vote on it22:17
dviroel|rucktested in the node hold22:18
rlandy|roverdviroel|ruck: also - you can add Closes-Bug: #194235622:18
rlandy|roverok22:18
rlandy|roverso you can remove the WIP then and put it in for review22:18
dviroel|ruckyes, set as wip22:18
rlandy|roverdviroel|ruck: thank you for following this all up!!22:19
rlandy|roververy nicely done22:19
dviroel|ruckwill this work in testproject with depends-on?22:19
rlandy|roveryes it should22:20
rlandy|roverbecause the change is not in the image build22:20
rlandy|roverbut the cloud actions22:20
rlandy|roverwe should get build-test packages on that22:20
rlandy|roverdviroel|ruck: stepping away for a few hours (going to volunteer job) - will check back in on the test run later22:23
*** rlandy|rover is now known as rlandy|rover|bbl22:23
dviroel|ruckok22:24
*** dviroel|ruck is now known as dviroel|out22:43
dviroel|outhttps://review.rdoproject.org/r/c/testproject/+/3523522:43
« Tuesday, 2021-08-31 Index Thursday, 2021-09-02 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!