| *** ykarel|away is now known as ykarel | 04:32 | |
| *** amoralej|off is now known as amoralej | 06:10 | |
| bhagyashris|ruck | arxcruz, hi | 06:29 |
|---|---|---|
| ykarel | bhagyashris|ruck, chkumar|rover are you aware about ci jobs config errors in https://review.rdoproject.org/zuul/config-errors? | 06:57 |
| ykarel | mainly the promotion-staging one or jobs defined in ci-config | 06:58 |
| ykarel | i didn't checked if those jobs are running or being monitored by CI team | 06:58 |
| chkumar|rover | ykarel: we donot monitor those, but good to check those | 07:02 |
| ykarel | chkumar|rover, yes please | 07:02 |
| ykarel | we are checking non promoter ones, u can check those | 07:02 |
| marios | bhagyashris|ruck: maybe we need to shift the scrum by half hour today there is a clash with all-hands fyi | 08:36 |
| bhagyashris|ruck | marios, ack | 08:39 |
| *** chem is now known as Guest874 | 09:16 | |
| *** ykarel is now known as ykarel|lunch | 09:17 | |
| arxcruz | bhagyashris|ruck: hey | 09:53 |
| bhagyashris|ruck | arxcruz, i have one question- here on the dashboard http://dashboard-ci.tripleo.org/d/3pUqDadGk/tempest-skipped-tests?orgId=1 i see some of the test are not present in the skiplist https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml | 10:18 |
| arxcruz | bhagyashris|ruck: because it's being skipped the whole namespace | 10:19 |
| arxcruz | for example TestNetworkBasicOps | 10:19 |
| bhagyashris|ruck | for example this test tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details is shown on the dashboard but not in ths skiplist | 10:19 |
| arxcruz | tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_subnet_details might not be in the skiplist, because tempest.scenario.test_network_basic_ops.TestNetworkBasicOps is being skipped | 10:20 |
| bhagyashris|ruck | i try to search tempest.scenario.test_network_basic_ops.TestNetworkBasicOps but that is also not in the skiplist | 10:21 |
| bhagyashris|ruck | and on the dashboard that test is skipped for all the branches as well | 10:25 |
| bhagyashris|ruck | looks like the dashboard is not showing the correct result | 10:25 |
| *** ykarel|lunch is now known as ykarel | 11:00 | |
| *** dviroel|out is now known as dviroel | 11:02 | |
| arxcruz | bhagyashris|ruck: https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml#L1131 | 11:47 |
| arxcruz | TestNetworkBasicOps is skipped | 11:47 |
| arxcruz | bhagyashris|ruck: nevermind, it's newton lol | 11:47 |
| arxcruz | bhagyashris|ruck: https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml#L1755 test_subnet_details is also being skipped | 11:48 |
| bhagyashris|ruck | arxcruz, yes some are there some are not | 11:50 |
| *** amoralej is now known as amoralej|lunch | 11:50 | |
| arxcruz | bhagyashris|ruck: i'll take a look | 11:50 |
| bhagyashris|ruck | arxcruz, thanks | 11:50 |
| *** amoralej|lunch is now known as amoralej|away | 11:50 | |
| bhagyashris|ruck | arxcruz, plz review https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800886 when you have time | 11:54 |
| arxcruz | bhagyashris|ruck: was doing that just right now :) | 11:55 |
| bhagyashris|ruck | arxcruz, ok thanks :) | 11:55 |
| * frenzy_friday lunch | 11:55 | |
| bhagyashris|ruck | arxcruz, weshay|ruck chkumar|rover please have a look when you have time https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800913 | 12:26 |
| chkumar|rover | done | 12:27 |
| arxcruz | bhagyashris|ruck: did you check all the branches right ? | 12:29 |
| bhagyashris|ruck | arxcruz, yes | 12:30 |
| weshay|ruck | arxcruz++ bhagyashris|ruck++ chkumar|rover++ | 12:44 |
| zbr | i need some clarifications regarding where the get_hash config is supporsed to be loaded from. | 12:57 |
| zbr | at this moment load_config has some kind of weird lookup logic, apparently looking for current directry config.yaml or usr/local/etc/tripleo_get_hash/config.yaml --- as you can guess, that is not reliable. | 12:59 |
| dviroel | zbr: marios will be the person to answer you about that | 13:07 |
| zbr | i have one test that confuses me, now i am trying to see if there are also others. | 13:08 |
| marios | zbr: lets talk on scrum i was going to ask for a sync call with you and dviroel re get hash and tripleo-repos setup.cfg moving forward | 13:16 |
| marios | zbr: i can explain the 'weird lookup logic too' (You're weird!) | 13:16 |
| marios | ;) | 13:16 |
| dviroel | chkumar|rover: lol, i was about to ask in bhagyashris|ruck's patch, why we keep these skip tests for old releases like liberty, mitaka, newton | 13:17 |
| dviroel | chkumar|rover: now i saw your patch removing them :) | 13:17 |
| zbr | probaly it would be better just after scrum to avoid boredom for others. | 13:18 |
| marios | zbr: yes i was going to suggest tomorrow morning ususally we don't have many scheduled calls... ah dviroel timezone... ok his morning then like early afternoon for us. lets setup on scrum anyway | 13:23 |
| bhagyashris|ruck | akahat, arxcruz chkumar|rover frenzy_friday marios pojadhav rlandy sshnaidm zbr weshay|ruck dviroel scrum meeting in 1 min | 13:29 |
| bhagyashris|ruck | Review list | 13:33 |
| bhagyashris|ruck | https://review.opendev.org/q/topic:%22scenario010-branchful%22+(status:open%20OR%20status:merged) | 13:33 |
| marios | https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/793331/14#message-8e221153fe33586675eb5e36f6f0b54fc639908f | 13:36 |
| bhagyashris|ruck | https://review.opendev.org/c/openstack/tripleo-quickstart/+/791486 | 13:38 |
| rlandy | https://review.opendev.org/q/owner:+pojadhav%2540redhat.com+topic:+wallaby-upgrade-jobs | 13:41 |
| rlandy | https://review.opendev.org/c/openstack/ansible-role-tripleo-modify-image/+/793234 | 13:45 |
| bhagyashris|ruck | https://review.rdoproject.org/r/q/topic:%22refactor_job_names%22+(status:open%20OR%20status:merged) | 13:45 |
| frenzy_friday | weshay|ruck, I have a new pastebin if you want to try https://paste.opendev.org/show/807504/ But it may not work (also the patch has been updated, you have to pull again) | 13:52 |
| marios | great scrum... blockers and even time for some chat... 22 mins. ++ | 13:52 |
| weshay|ruck | frenzy_friday, rock... trying | 13:55 |
| marios | zbr: dviroel: sent invite for tomorrow 'usual' scrum time | 14:00 |
| marios | zbr: let's sync our morning tomorrow - ping me when you get in for the 'quick sync' call to unblock you? | 14:00 |
| marios | zbr: then we can meet with douglas for the main chat as scheduled | 14:00 |
| weshay|ruck | frenzy_friday, re: INFO:paramiko:Trying ssh connection to weshayutin@review.opendev.org:29418 | 14:00 |
| zbr | ok | 14:01 |
| weshay|ruck | frenzy_friday, /me assumes.. this is an error right? | 14:01 |
| weshay|ruck | DEBUG:paramiko.transport:Dropping user packet because connection is dead. | 14:01 |
| weshay|ruck | or is that normal for make build | 14:01 |
| frenzy_friday | no, thats the end of make buuild | 14:01 |
| frenzy_friday | You can try make up now | 14:01 |
| marios | zbr: meantime can you please add this to your reviews list https://review.opendev.org/c/openstack/tripleo-quickstart/+/791486 & depends-on https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/793331 as mentioned in scrum just now | 14:01 |
| weshay|ruck | ah cool | 14:01 |
| weshay|ruck | frenzy_friday, thanks | 14:01 |
| frenzy_friday | weshay|ruck, It should run with a lot of debug outputs till you get er-cron | crond: wakeup dt=60 | 14:02 |
| weshay|ruck | looking better | 14:02 |
| weshay|ruck | er-web exited with code 1 | 14:02 |
| weshay|ruck | Aborting on container exit... | 14:02 |
| weshay|ruck | Stopping er-bot ... done | 14:02 |
| weshay|ruck | Stopping er-cron ... done | 14:02 |
| weshay|ruck | make: *** [Makefile:59: up] Error 1 | 14:02 |
| marios | rlandy: weshay|ruck: please add to your reviews for when you next have time thanks https://review.opendev.org/c/openstack/tripleo-quickstart/+/791486 & depends-on https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/793331 | 14:02 |
| zbr | marios: imho, i would prefer if we directly install the module as part of a collection and avoid the hacky installation from these reviews | 14:03 |
| zbr | to explain it: we would no longer need to sudo, just galaxy build/install inside the same folder. | 14:04 |
| *** chkumar|rover is now known as chandankumar | 14:05 | |
| weshay|ruck | frenzy_friday, make up should start the containers? | 14:07 |
| frenzy_friday | weshay|ruck, yes. I forgot to add one more step in the paste | 14:07 |
| rlandy | frenzy_friday: is the VM updated? should I try make up again? | 14:10 |
| rlandy | build worked | 14:10 |
| rlandy | marios: ack - pls bug me again tomorrow if I dont get to it | 14:11 |
| frenzy_friday | rlandy, not yet. Right now in the original health vm it looks like the containers are running but nginx shows 404 | 14:11 |
| marios | rlandy: thanks | 14:11 |
| rlandy | chandankumar: bhagyashris|ruck: have you see any ovb failures with "Property error: resources.NovaCompute.properties.image: Error validating value 'overcloud-full': No images matching {'name': 'overcloud-full'}." in deploy? | 15:33 |
| rlandy | showing up in downstream | 15:33 |
| rlandy | just check if you have seen the same in rdo | 15:33 |
| rlandy | weshay|ruck: do you know why we switched baremetal in the integration lines to fs035 and leave the component lines on fs001? | 15:35 |
| weshay|ruck | rlandy, better coverage | 15:35 |
| chandankumar | rlandy: yes | 15:35 |
| rlandy | barmetal and tripleo on 16.2 are way behind | 15:35 |
| chandankumar | rlandy: https://bugs.launchpad.net/tripleo/+bug/1936287 | 15:36 |
| rlandy | chandankumar: thank yoou | 15:36 |
| rlandy | you | 15:36 |
| rlandy | https://github.com/openstack/tripleo-quickstart-extras/commit/c9676e9115f1aa097b528ca70119465991416767 | 15:42 |
| rlandy | wonder if its not this | 15:42 |
| rlandy | sshnaidm: https://github.com/openstack/tripleo-quickstart-extras/commit/9c6667d9156f7228bc91871db820823485ba94ed - should train be included in the legacy here? | 15:44 |
| rlandy | noting https://bugs.launchpad.net/tripleo/+bug/1936287 | 15:44 |
| rlandy | train jobs not finding images | 15:45 |
| * rlandy tried | 15:45 | |
| rlandy | tries | 15:45 |
| sshnaidm | hmm.. seems like tripleo operator worked: tripleo.operator.tripleo_overcloud_image_upload https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-train/ab8385f/job-output.txt | 15:47 |
| sshnaidm | rlandy, https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-train/ab8385f/logs/undercloud/home/zuul/overcloud_image_upload.log.txt.gz | 15:47 |
| sshnaidm | rlandy, if with train it works, let's use legacy, I can look later what is the difference | 15:49 |
| rlandy | sshnaidm: going to try the adding train back to legacy | 15:49 |
| rlandy | it's taking out the train and 16.2 lines atm | 15:49 |
| sshnaidm | ack | 15:49 |
| rlandy | thanks | 15:49 |
| *** marios is now known as marios|out | 16:25 | |
| *** ykarel is now known as ykarel|away | 16:32 | |
| *** sshnaidm is now known as sshnaidm|afk | 16:35 | |
| rlandy | lunch - biab | 17:10 |
| weshay|ruck | rlandy, https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800975 not sure if I have that right | 17:43 |
| weshay|ruck | rlandy, can you add something to the Examples: https://opendev.org/openstack/openstack-tempest-skiplist | 17:44 |
| * dviroel will get his first dose of vaccine on saturday :) | 17:50 | |
| * rlandy looks | 17:54 | |
| rlandy | dviroel: :) ... don't make plans on sunday | 17:54 |
| rlandy | https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/doc/source/listyaml/listyaml.rst | 17:56 |
| dviroel | rlandy: yeah =| | 17:56 |
| rlandy | guess that's all there is on installer | 17:56 |
| rlandy | https://opendev.org/openstack/openstack-tempest-skiplist/src/commit/85a392c55f55bd83a370c677c3174d567f288b1e/roles/list_skipped/library/list_skipped.py#L80 | 17:57 |
| rlandy | looking at tox failure | 17:58 |
| rlandy | weshay|ruck: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800975 - installer looks right but the test line needs to be spaced in | 18:00 |
| rlandy | weshay|ruck: container build job ... https://logserver.rdoproject.org/53/18953/91/check/tripleo-build-containers-stream9-development/2c98310/logs/containers-built.log | 18:01 |
| rlandy | running | 18:01 |
| rlandy | fixing the build failure | 18:02 |
| rlandy | stream 9 | 18:02 |
| rlandy | https://review.rdoproject.org/zuul/stream/46d090c2487a4e8db2bd06f7580a69d1?logfile=console.log - fix for images on train works | 18:03 |
| rlandy | weshay|ruck: ignore that spacing column | 18:12 |
| rlandy | it's the viewer I am using | 18:13 |
| rlandy | got that review - looking at it | 18:13 |
| rlandy | review updated | 18:15 |
| rlandy | https://logserver.rdoproject.org/95/24995/58/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/46d090c/job-output.txt | 19:04 |
| rlandy | ok - that get past image download - now introspect fails on train | 19:04 |
| weshay|ruck | ya.. saw it was busted | 19:05 |
| rlandy | weshay|ruck: both | 19:05 |
| rlandy | same issue on train and 16.2 | 19:06 |
| weshay|ruck | rlandy, rabi has a comment on your patch | 19:09 |
| rlandy | looking | 19:09 |
| rlandy | k - editing that patch in a bit | 19:10 |
| rlandy | https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800980 - updated | 19:31 |
| rlandy | weshay|ruck: ^^ | 19:32 |
| weshay|ruck | rock on.. thank you! | 19:35 |
| rlandy | baremetal is still running - so waiting until that is done | 19:50 |
| rlandy | to update train patch | 19:50 |
| *** dviroel is now known as dviroel|brb | 19:53 | |
| *** dviroel|brb is now known as dviroel | 20:27 | |
| rlandy | "success": true - yay finally one bm pass | 20:43 |
| rlandy | https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/800980 ok to merge? | 21:08 |
| weshay|ruck | rlandy, aye | 21:14 |
| rlandy | thanks | 21:14 |
| *** dviroel is now known as dviroel|out | 21:24 | |
| rlandy | 2021-07-15 16:24:03.168466 | primary | | 21:46 |
| rlandy | 2021-07-15 16:24:03.168535 | primary | TASK [os_tempest : Execute tempest tests] ************************************** | 21:46 |
| rlandy | 2021-07-15 16:24:03.168544 | primary | Thursday 15 July 2021 16:24:03 -0400 (0:00:00.036) 0:04:10.110 ********* | 21:46 |
| rlandy | 2021-07-15 21:39:16.770334 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master] | 21:46 |
| rlandy | 2021-07-15 21:39:16.791288 | POST-RUN START: [trusted : code.engineering.redhat.com/openstack/tripleo-ci-internal-config/playbooks/dlrn/post-dlrn.yaml@master] | 21:46 |
| rlandy | weshay|ruck: BM on tripleo component keeps timing out ^^ | 21:46 |
| rlandy | it's blocking promotion on the 16.2 component line | 21:46 |
| weshay|ruck | in the post? | 21:46 |
| rlandy | can we reduce tempest there? | 21:47 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/7e1d7db9d705403dbf943f6f09cfb83f?logfile=console.log | 21:47 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-bm_envC-3ctlr_1comp-featureset001-tripleo-rhos-16.2 | 21:47 |
| rlandy | on good days it times out | 21:47 |
| rlandy | or we download images | 21:47 |
| rlandy | we are at the longest job time | 21:48 |
| weshay|ruck | rlandy, more concerned with why bm is failing | 21:49 |
| weshay|ruck | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-bm_envC-3ctlr_1comp-featureset001-tripleo-rhos-16.2&pipeline=openstack-component-tripleo | 21:49 |
| rlandy | sec - just kicking off job to test marios patch | 21:50 |
| rlandy | weshay|ruck: numerous failures | 21:51 |
| rlandy | restarted some hardware | 21:51 |
| rlandy | then there was rabi's patch with networking | 21:51 |
| rlandy | then sandeep's patch with operators | 21:52 |
| rlandy | been chasing this job for days | 21:52 |
| rlandy | that's why tripleo has not promoted on 16.2 | 21:52 |
| rlandy | job is criteria | 21:52 |
| rlandy | the BM job from the baremetal line just passed | 21:53 |
| rlandy | with patch to get train off operators | 21:53 |
| rlandy | weshay|ruck: maybe worth skipping bm in tripleo criteria to get component to promote | 21:58 |
| rlandy | at least now | 21:58 |
| *** rlandy is now known as rlandy|bbl | 22:08 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!