| *** brault has quit IRC | 00:16 | |
| *** brault has joined #oooq | 00:18 | |
| *** hamzy has joined #oooq | 01:01 | |
| *** apetrich has quit IRC | 02:09 | |
| *** dtrainor has quit IRC | 02:12 | |
| *** dtrainor has joined #oooq | 02:15 | |
| *** ykarel has joined #oooq | 02:22 | |
| *** ykarel has quit IRC | 03:39 | |
| *** ykarel has joined #oooq | 03:57 | |
| *** dsneddon has quit IRC | 03:57 | |
| *** epoojad1 has joined #oooq | 04:08 | |
| *** holser has joined #oooq | 04:13 | |
| *** epoojad1 has quit IRC | 04:15 | |
| *** epoojad1 has joined #oooq | 04:15 | |
| *** brault has quit IRC | 04:19 | |
| *** dsneddon has joined #oooq | 04:26 | |
| *** ykarel has quit IRC | 04:31 | |
| *** epoojad1 has quit IRC | 04:34 | |
| *** soniya29 has joined #oooq | 04:41 | |
| *** raukadah is now known as chandankumar | 04:47 | |
| *** surpatil has joined #oooq | 04:49 | |
| *** ratailor has joined #oooq | 05:05 | |
| *** ykarel has joined #oooq | 05:06 | |
| *** epoojad1 has joined #oooq | 05:14 | |
| *** surpatil has quit IRC | 05:14 | |
| *** surpatil has joined #oooq | 05:14 | |
| *** dsneddon has quit IRC | 05:15 | |
| *** dsneddon has joined #oooq | 05:20 | |
| *** dsneddon has quit IRC | 05:24 | |
| *** holser has quit IRC | 05:27 | |
| *** marios has joined #oooq | 05:27 | |
| *** udesale has joined #oooq | 05:33 | |
| *** ykarel_ has joined #oooq | 05:34 | |
| *** marios is now known as marios|rover | 05:35 | |
| *** ykarel has quit IRC | 05:36 | |
| *** udesale has quit IRC | 05:38 | |
| *** udesale has joined #oooq | 05:39 | |
| *** jfrancoa has joined #oooq | 05:42 | |
| *** dsneddon has joined #oooq | 05:54 | |
| *** ccamacho has joined #oooq | 05:59 | |
| *** sshnaidm_ has joined #oooq | 05:59 | |
| *** sshnaidm|pto has quit IRC | 06:00 | |
| *** sshnaidm has joined #oooq | 06:03 | |
| *** ldumont has quit IRC | 06:04 | |
| *** sshnaidm_ has quit IRC | 06:04 | |
| *** ldumont has joined #oooq | 06:05 | |
| *** sshnaidm_ has joined #oooq | 06:07 | |
| *** sshnaidm has quit IRC | 06:10 | |
| *** ldumont has quit IRC | 06:10 | |
| *** ldumont has joined #oooq | 06:10 | |
| *** ykarel__ has joined #oooq | 06:14 | |
| *** holser has joined #oooq | 06:15 | |
| *** ykarel_ has quit IRC | 06:17 | |
| *** yolanda__ has quit IRC | 06:20 | |
| *** yolanda has joined #oooq | 06:20 | |
| *** ykarel__ is now known as ykarel | 06:28 | |
| *** ratailor_ has joined #oooq | 06:38 | |
| *** ratailor has quit IRC | 06:40 | |
| *** saneax has joined #oooq | 06:42 | |
| *** dsneddon has quit IRC | 06:45 | |
| *** matbu has joined #oooq | 06:57 | |
| *** jpena|off is now known as jpena | 07:00 | |
| *** holser has quit IRC | 07:04 | |
| *** tesseract has joined #oooq | 07:12 | |
| *** udesale has quit IRC | 07:13 | |
| *** udesale has joined #oooq | 07:13 | |
| *** tosky has joined #oooq | 07:19 | |
| *** dsneddon has joined #oooq | 07:19 | |
| chandankumar | sshnaidm_: Welcome back :-) | 07:22 |
|---|---|---|
| *** apetrich has joined #oooq | 07:26 | |
| *** ykarel is now known as ykarel|lunch | 07:35 | |
| *** kopecmartin|off is now known as kopecmartin | 07:38 | |
| *** dtantsur|afk is now known as dtantsur | 07:53 | |
| *** holser has joined #oooq | 07:55 | |
| *** holser has quit IRC | 08:10 | |
| *** holser has joined #oooq | 08:10 | |
| *** dsneddon has quit IRC | 08:12 | |
| *** dsneddon has joined #oooq | 08:13 | |
| *** dsneddon has quit IRC | 08:17 | |
| *** ykarel|lunch is now known as ykarel | 08:29 | |
| *** amoralej is now known as amoralej|mtg | 08:32 | |
| *** akahat has joined #oooq | 08:48 | |
| *** dsneddon has joined #oooq | 08:49 | |
| *** chem|eod is now known as chem | 08:50 | |
| marios|rover | ykarel: so i think we might have to scale back the tempest tests if nothing else, at least temprarily unless we have some better way before EOD today (EU time at least) cos we are red on master | 09:16 |
| marios|rover | ykarel: thanks very much ... fs20 yeah rfolco|ruck filed a new bug for that last night was switching to that next | 09:16 |
| *** amoralej|mtg is now known as amoralej | 09:26 | |
| ykarel | marios|rover, yes agree need to have a proper plan | 09:26 |
| ykarel | and good to have all info in one bug if they are related | 09:26 |
| *** sshnaidm_ is now known as sshnaidm|pto | 09:27 | |
| chandankumar | marios|rover: ykarel my reproducer failed at other place | 09:27 |
| sshnaidm|pto | chandankumar, nope :) | 09:27 |
| chandankumar | sshnaidm|pto: let me know when you are back! | 09:27 |
| ykarel | chandankumar, what issue? | 09:27 |
| ykarel | standalone full tempest? | 09:27 |
| *** dsneddon has quit IRC | 09:29 | |
| chandankumar | ykarel: yes, | 09:29 |
| ykarel | chandankumar, and what's the issue? | 09:31 |
| ykarel | at what step it failed | 09:31 |
| chandankumar | ykarel: one min | 09:31 |
| chandankumar | kopecmartin: fixed https://bugzilla.redhat.com/show_bug.cgi?id=1743569, please test it | 09:32 |
| openstack | bugzilla.redhat.com bug 1743569 in python-stestr "missing stestr only stestr-3 available" [Medium,Modified] - Assigned to chkumar | 09:32 |
| ykarel | chandankumar, ack | 09:33 |
| kopecmartin | chandankumar: ok | 09:35 |
| chandankumar | ykarel: the reproducer got failed at run-test : shell | 09:38 |
| ykarel | chandankumar, login to node and see quickstart_install.log at /home/zuul/workspace | 09:39 |
| ykarel | or check logs, reproducer pushes logs at :8000 | 09:39 |
| *** derekh has joined #oooq | 09:43 | |
| *** dsneddon has joined #oooq | 09:59 | |
| *** ratailor_ has quit IRC | 10:12 | |
| chandankumar | ykarel: marios|rover http://logs.rdoproject.org/71/20171/60/check/periodic-tripleo-ci-centos-7-standalone-full-tempest-train/dea9b9a/logs/undercloud/var/log/extra/errors.txt.txt.gz | 10:12 |
| chandankumar | ykarel: marios|rover ERROR neutron.plugins.ml2.managers [req-65806fe3-9e5b-4b5c-827b-a2488b38d1ae efd0b97a36ab41f68de5f66bf0080492 2f954310ba2e4a3d89313d7499ae5fe5 - default default] Failed to bind port f1a0b5cc-6b44-432f-b7c7-7ac96870169f on host badhost2 for vnic_type normal using segments [{'network_id': 'bb97b07a-5b31-4b59-93e6-a3b83a0cd142', 'segmentation_id': 45, 'physical_network': None, 'id': | 10:13 |
| chandankumar | 'a566b2cf-b507-4cba-a9da-3574a89db0d4', 'network_type': u'geneve'}] | 10:13 |
| chandankumar | ykarel: just grep for ids you can find that | 10:13 |
| chandankumar | http://logs.rdoproject.org/71/20171/60/check/periodic-tripleo-ci-centos-7-standalone-full-tempest-train/dea9b9a/logs/undercloud/home/zuul/tempest/tempest.log.txt.gz | 10:13 |
| *** ratailor has joined #oooq | 10:13 | |
| ykarel | chandankumar, sorry context? u mean ^^ is causing timeout issue? | 10:15 |
| chandankumar | ykarel: I think so | 10:15 |
| ykarel | chandankumar, ack good to see other runs too if there is same error and also look timing | 10:15 |
| ykarel | it doesn't look that error is eating up much time though | 10:15 |
| ykarel | all those errors around 07:28:53 | 10:16 |
| ykarel | and from that error it looks it can be a negative test | 10:16 |
| ykarel | but need to check code to confirm, saying as per:- badhost2 | 10:17 |
| chandankumar | ykarel: sorry it is found in passing job | 10:17 |
| chandankumar | wait | 10:17 |
| ykarel | need to check passing one which taking less time | 10:18 |
| ykarel | < 2 hours | 10:18 |
| marios|rover | chandankumar: ykarel yeah it does looks like bona fide timeout... might be just few mins... e.g. see fs20 http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/3fbae7e/logs/undercloud/home/zuul/tempest.log.txt.gz | 10:23 |
| marios|rover | first line - last line 2019-10-14 04:09:22 --> 2019-10-14 06:51:12 | 10:23 |
| marios|rover | ykarel: and if you see the last line its actually in the process of reporting the results... compare to good file at http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/47fc7f9/logs/undercloud/home/zuul/tempest.log.txt.gz | 10:23 |
| marios|rover | but still... even the 'good' job takes like 2019-10-07 16:25:39 --> 2019-10-07 18:52:57 2.5 + hours | 10:24 |
| marios|rover | that seems insane | 10:24 |
| marios|rover | chandankumar: ^^ | 10:24 |
| marios|rover | rfolco|ruck: chandankumar: ykarel i updated the bug to include fs20 https://bugs.launchpad.net/tripleo/+bug/1847585 | 10:25 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,New] | 10:25 |
| ykarel | chandankumar, yes and now it's not finishing in 2.5 hours also, it still need to run few hundreds test | 10:29 |
| ykarel | sorry marios|rover ^^ | 10:29 |
| marios|rover | ykarel: ack ... not sure if we want to bump timeout or do something about the tests. maybe both ... timeout first cos red and then revisit what we're running there in the meantime and lower timeout again asap | 10:31 |
| ykarel | marios|rover, yes ^^ can be a backup plan if nothing constructive found soon | 10:31 |
| ykarel | but for fs020 already timeout for job is 5 hours | 10:31 |
| ykarel | increasing more seems not that good | 10:32 |
| marios|rover | ouch | 10:32 |
| marios|rover | ack ykarel | 10:32 |
| *** ratailor_ has joined #oooq | 10:38 | |
| ykarel | marios|rover, chandankumar btw i am looking at https://review.opendev.org/#/q/topic:token-cache+(status:open+OR+status:merged), at first glance it looks it can affect the timing | 10:39 |
| ykarel | but let's see if can found more info, before trying revert | 10:39 |
| marios|rover | ykarel: ack thanks | 10:39 |
| ykarel | let me see if i have environment, i can try disabling that locally | 10:40 |
| marios|rover | ykarel: great | 10:40 |
| *** ratailor has quit IRC | 10:40 | |
| *** dsneddon has quit IRC | 10:46 | |
| *** soniya29 has quit IRC | 10:49 | |
| ykarel | marios|rover, chandankumar see http://paste.openstack.org/show/783897/ | 10:52 |
| ykarel | i can observe good difference with cache enabled=false vs enable=true | 10:52 |
| chandankumar | ykarel: can you revert and try that | 10:53 |
| ykarel | chandankumar, ok can try, seems both needs revert | 10:54 |
| ykarel | or we can just try enabling cache, that should also work, wdys? | 10:55 |
| chandankumar | ykarel: just try with enable=true and see | 10:56 |
| *** dsneddon has joined #oooq | 10:56 | |
| ykarel | chandankumar, ack /me prepares patch | 10:56 |
| marios|rover | nice ykarel lets try it | 10:57 |
| marios|rover | ykarel: chandankumar but still though we need to talk about those tests... i think more than 2 hours is already too much | 10:58 |
| chandankumar | marios|rover: Yes, I am figuring out a way to stdout the test name time when that executes | 10:59 |
| ykarel | marios|rover, yes that's a different story though, i remember when we fixed a performance regression last year, it was around 1.5 hour for full tempest | 10:59 |
| chandankumar | so that we can compare with runs which tests has taken to much time and then find the id and dig it | 10:59 |
| ykarel | but now it's more than 2 hours | 10:59 |
| *** recheck has quit IRC | 11:00 | |
| *** recheck has joined #oooq | 11:00 | |
| *** dsneddon has quit IRC | 11:00 | |
| *** soniya29 has joined #oooq | 11:02 | |
| *** recheck has quit IRC | 11:03 | |
| *** dsneddon has joined #oooq | 11:03 | |
| *** recheck has joined #oooq | 11:04 | |
| *** recheck has quit IRC | 11:05 | |
| *** recheck has joined #oooq | 11:05 | |
| *** recheck has quit IRC | 11:06 | |
| *** recheck has joined #oooq | 11:06 | |
| *** recheck has quit IRC | 11:06 | |
| *** recheck has joined #oooq | 11:07 | |
| *** recheck has quit IRC | 11:08 | |
| *** recheck has joined #oooq | 11:08 | |
| *** dsneddon has quit IRC | 11:08 | |
| *** recheck has quit IRC | 11:11 | |
| *** recheck has joined #oooq | 11:11 | |
| panda|off | chandankumar: question | 11:12 |
| chandankumar | panda|off: anwser | 11:14 |
| chandankumar | *answer | 11:14 |
| chandankumar | panda|off: yes please ask | 11:14 |
| panda|off | chandankumar: surprise statement. acknowledgements. grettings. | 11:14 |
| panda|off | chandankumar: w/r/t podman gating. Did we ever start a discussion with people in pdoman github org to push the app and .zuul.yaml files ? | 11:14 |
| marios|rover | chandankumar: so fs20/fs21 are running full tempest? | 11:14 |
| chandankumar | panda|off: fs020 -> full tempest on overcloud | 11:15 |
| chandankumar | fs021 -> skip list on overcloud | 11:15 |
| marios|rover | chandankumar: ie. like standalone-full-tempest-master? | 11:15 |
| marios|rover | chandankumar: ah ok but f20 like ^^ ? full? | 11:15 |
| marios|rover | 14:15 < chandankumar> panda|off: fs020 -> full tempest on overcloud | 11:15 |
| marios|rover | chandankumar: thanks ^ | 11:16 |
| *** udesale has quit IRC | 11:16 | |
| marios|rover | chandankumar: gonna propose the timeout bump +1 hour as fall back but that can only be temporary measure... not even sure we want to do it its crazy 5 hours already | 11:16 |
| chandankumar | panda|off: EmilienM told when we start working, he will point us and start the conversation to the right person in podman team | 11:17 |
| chandankumar | need to wait for EmilienM to come online | 11:17 |
| *** dsneddon has joined #oooq | 11:18 | |
| chandankumar | marios|rover: yes fs020 and standalone full tempest has almost same test list | 11:19 |
| arxcruz | chandankumar: weshay when you guys have time, can you explain a little bit better what we want to achieve at https://tree.taiga.io/project/tripleo-ci-board/us/1318 ? | 11:20 |
| ykarel | marios|rover, for standalone timeout bump was already proposed, but wrong place, see comment https://review.rdoproject.org/r/#/c/23042/ | 11:21 |
| ykarel | so possible if we get good result with that cache enable, timeout increase would not be needed | 11:21 |
| ykarel | and if we get sure about the fix, we can just promote ignore full tempest | 11:22 |
| ykarel | as fix will be in TripleO itself in this case | 11:22 |
| *** dsneddon has quit IRC | 11:22 | |
| ykarel | but let's wait for the result | 11:22 |
| ykarel | should be there in less than 3 hours | 11:22 |
| marios|rover | ykarel: ok | 11:23 |
| marios|rover | ykarel: yeah i don't want to bump it | 11:23 |
| marios|rover | but if its worth doing we need to do it asap | 11:23 |
| marios|rover | ykarel: to catch the next runs | 11:23 |
| *** saneax has quit IRC | 11:24 | |
| *** dsneddon has joined #oooq | 11:32 | |
| chandankumar | ykarel: please put depends-on in above test patch https://review.opendev.org/#/c/688684/ | 11:33 |
| ykarel | chandankumar, the data from ^^ would be available in the jobs running in that patch | 11:35 |
| chandankumar | ykarel: it is dump the tests run timing in a file | 11:36 |
| chandankumar | if the task got killed, we still have the data | 11:36 |
| ykarel | chandankumar, i have increased the timeout in test patch so will have data anyway, just wanted to save 30 minutes of run | 11:37 |
| ykarel | if depends-on is not that necessary | 11:37 |
| *** dsneddon has quit IRC | 11:37 | |
| weshay | marios|rover, https://review.opendev.org/#/c/688433/ | 11:38 |
| weshay | ykarel, thanks for catching that | 11:38 |
| ykarel | oh u are here, so i got reminded one more thing | 11:39 |
| * ykarel checks | 11:39 | |
| ykarel | wrt. contentdir patch | 11:39 |
| ykarel | weshay, re. https://review.opendev.org/#/c/688534/ i have seen issue outside of reproducer, so actual issue is somewhere else | 11:40 |
| *** amoralej is now known as amoralej|lunch | 11:40 | |
| ykarel | some theory: related to centos-release rpm, and availability of uname binary | 11:40 |
| ykarel | but need to dig more, if have more data where it seen | 11:41 |
| arxcruz | ykarel: ping | 11:41 |
| weshay | ykarel, ya.. I believe you.. I think altarch is written by something else as well | 11:41 |
| weshay | ykarel, is it blocking other ci? | 11:41 |
| ykarel | weshay, nope, i didn't see that in any CI job | 11:42 |
| ykarel | just noticed it in a vm on rdo cloud | 11:42 |
| *** jpena is now known as jpena|lunch | 11:42 | |
| marios|rover | weshay: ack checking | 11:45 |
| marios|rover | weshay: nit i think the default also needs to flip ? rfolco|ruck https://review.opendev.org/#/c/688433/1/roles/build-containers/tasks/main.yaml | 11:46 |
| *** ccamacho has quit IRC | 11:46 | |
| marios|rover | rfolco|ruck: i added the fs20 issue to the full-tempest bug... they are both suffering from timeout thanks ykarel i updated the title and description and added some comments https://bugs.launchpad.net/tripleo/+bug/1847585 | 11:47 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,New] | 11:47 |
| rfolco|ruck | marios|rover, cool | 11:48 |
| marios|rover | rfolco|ruck: ykarel is trying to improve it with https://review.opendev.org/688677 but even then... a 'good' run is almost 2.5 hours far too long for tempest ... see comment #2 and we can't even bump the timeout as temp measure see comment 4 | 11:48 |
| marios|rover | rfolco|ruck: i proposed the timeout bump but it fails cos 5 hours is max already | 11:48 |
| marios|rover | rfolco|ruck: like we don't want to but it was to get us passed the promotion cos red. | 11:49 |
| marios|rover | rfolco|ruck: chandankumar we still need to discuss those tempest tests do we need to cut some of them... e.g. for fs20 at least if the standalone can run them in better time for example | 11:49 |
| rfolco|ruck | marios|rover, concurrency=3 ? can we tweak it? | 11:50 |
| marios|rover | rfolco|ruck: don't know ... | 11:50 |
| marios|rover | rfolco|ruck: where is it on tempest/zuul confg? | 11:51 |
| chandankumar | rfolco|ruck: it is not related to concurreny there is something else going on I think | 11:51 |
| rfolco|ruck | :-( | 11:54 |
| chandankumar | rfolco|ruck: I am trying to get the tempest tests timing for each tests | 11:55 |
| chandankumar | then we have a clear picture what is going on | 11:55 |
| *** epoojad1 has quit IRC | 11:56 | |
| weshay | rfolco|ruck, take the bug triage time to sync w/ marios and I'll join perhaps a few minutes late | 11:58 |
| marios|rover | rfolco|ruck: send me meeting info? | 11:59 |
| marios|rover | don't see it on calendar | 11:59 |
| rfolco|ruck | ok I have an errand now should be back in 30 min marios|rover chandankumar weshay | 11:59 |
| rfolco|ruck | marios|rover, ovt | 12:00 |
| rfolco|ruck | pvt | 12:00 |
| *** dsneddon has joined #oooq | 12:00 | |
| marios|rover | rfolco|ruck: ack tx | 12:00 |
| *** epoojad1 has joined #oooq | 12:00 | |
| EmilienM | chandankumar, panda|off : /join #podman and we can chat there | 12:02 |
| chandankumar | EmilienM: ack | 12:02 |
| chandankumar | arxcruz: for above user story, we need to try all the 4 options and see which one is better and viable | 12:04 |
| chandankumar | brb | 12:04 |
| *** chem is now known as chem|lunch | 12:06 | |
| *** epoojad1 has quit IRC | 12:08 | |
| *** saneax has joined #oooq | 12:09 | |
| ykarel | marios|rover, https://bugs.launchpad.net/tripleo/+bug/1847585 is still not in CIX | 12:10 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,New] | 12:10 |
| ykarel | need to change status and milestone | 12:10 |
| ykarel | promotion-blocker tag is since 5 days but not in CIX | 12:10 |
| marios|rover | ykarel: ack | 12:13 |
| ykarel | ack Thanks | 12:14 |
| *** ratailor_ has quit IRC | 12:21 | |
| chandankumar | rfolco|ruck: we are cancelling bug traige meeting? | 12:23 |
| rfolco|ruck | chandankumar, we need to talk about promotion blockers, and I invite you to be there | 12:24 |
| rfolco|ruck | chandankumar, and maybe arx, since its tempest related | 12:24 |
| chandankumar | rfolco|ruck: ok | 12:24 |
| rfolco|ruck | arxcruz, https://meet.google.com/unv-qweu-tdp | 12:25 |
| rfolco|ruck | marios|rover, chandankumar ^ | 12:30 |
| *** ykarel is now known as ykarel|afk | 12:32 | |
| *** hamzy has quit IRC | 12:34 | |
| *** chem|lunch is now known as chem | 12:36 | |
| *** hamzy has joined #oooq | 12:37 | |
| *** saneax has quit IRC | 12:38 | |
| arxcruz | rfolco|ruck: marios|rover https://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/47fc7f9/logs/stackviz/#/testrepository.subunit/timeline | 12:39 |
| *** jpena|lunch is now known as jpena | 12:41 | |
| chandankumar | rfolco|ruck: joining back | 12:50 |
| arxcruz | marios|rover: rfolco|ruck https://review.rdoproject.org/r/#/c/23015/ | 12:51 |
| chandankumar | arxcruz: marios|rover rfolco|ruck https://review.opendev.org/#/c/688684/ | 12:53 |
| marios|rover | thnks arxcruz chandankumar | 12:53 |
| chandankumar | marios|rover: arxcruz rfolco|ruck rfolco|ruck http://logs.rdoproject.org/84/688684/1/openstack-check/tripleo-ci-rhel-8-standalone-rdo/7b84d49/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 12:56 |
| arxcruz | chandankumar: http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master/3fbae7e/logs/undercloud/home/zuul/tempest.log.txt.gz | 12:56 |
| weshay | rfolco|ruck, https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/run-test/templates/toci_gate_test.sh.j2#L26 | 13:01 |
| *** panda|off is now known as panda | 13:01 | |
| weshay | rfolco|ruck, you'll also need to handle the case where /opt/git does not exist | 13:01 |
| weshay | marios|rover, fyi ^ we missed that | 13:01 |
| *** dsneddon has quit IRC | 13:01 | |
| rfolco|ruck | weshay, find won't find it | 13:01 |
| rfolco|ruck | weshay, did it failed ? | 13:01 |
| weshay | yes | 13:01 |
| rfolco|ruck | fail* | 13:02 |
| rfolco|ruck | what if we ignore | 13:02 |
| weshay | and I think it's the wrong file | 13:02 |
| rfolco|ruck | || true | 13:02 |
| weshay | I think || true is fine | 13:02 |
| rfolco|ruck | : | 13:02 |
| rfolco|ruck | ko | 13:02 |
| rfolco|ruck | ok | 13:02 |
| *** ykarel|afk is now known as ykarel|away | 13:04 | |
| weshay | +(/opt/stack/tripleo-ci/toci_gate_test.sh:37): sudo find /opt/git -delete | 13:04 |
| weshay | find: ‘/opt/git’: No such file or directory | 13:04 |
| weshay | rfolco|ruck, ^ | 13:04 |
| rfolco|ruck | weshay, fixing | 13:05 |
| *** ykarel|away has quit IRC | 13:10 | |
| marios|rover | weshay: ack rfolco|ruck you looking at that? (the opt git fix?) | 13:14 |
| rfolco|ruck | marios|rover, yes fixing | 13:14 |
| marios|rover | rfolco|ruck: k thankx | 13:14 |
| rfolco|ruck | marios|rover, https://review.opendev.org/688707 Ignore error when /opt/git does not exist | 13:15 |
| weshay | rfolco|ruck, https://review.opendev.org/#/c/688707/1 | 13:19 |
| rfolco|ruck | aaah the template | 13:22 |
| *** amoralej|lunch is now known as amoralej | 13:26 | |
| weshay | tripleo community call https://meet.google.com/bqx-xwht-wky | 13:30 |
| *** epoojad1 has joined #oooq | 13:31 | |
| *** Goneri has joined #oooq | 13:44 | |
| rfolco|ruck | agenda for community call: https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw | 13:44 |
| *** ykarel has joined #oooq | 13:46 | |
| * chandankumar headed home | 13:50 | |
| *** soniya29 has quit IRC | 13:55 | |
| *** surpatil has quit IRC | 13:55 | |
| rfolco|ruck | marios|rover, do you know how to report once for train/master so we save some time and resources for promotions on both releases? | 13:55 |
| zbr | weshay: are you sure https://review.opendev.org/#/c/686192/2/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2 is still needed? | 13:57 |
| zbr | i am trying to fix it and already spent a huge number of hours with bash tricks. editing these files with sed is a recipe for failure | 13:57 |
| zbr | there are huge number of cases: file existing now, value missing or present, present without quoates, with simple or double quotes.... | 13:59 |
| marios|rover | rfolco|ruck: not really sure what it means 'report once for train/master' | 14:00 |
| marios|rover | rfolco|ruck: in the promoter? or something else | 14:01 |
| marios|rover | tripleo weekly irc starting #tripleo | 14:01 |
| zbr | does anyone knows the official fileformat used by sysconfig files? is real shell or ini like? official docs fail to mention that. | 14:02 |
| rfolco|ruck | marios|rover, train is master now, so we don't need to run jobs twice to promote both releases | 14:02 |
| rfolco|ruck | marios|rover, what weshay said in the mtg | 14:02 |
| marios|rover | rfolco|ruck: yeah so it will involve updating the zuul layout then? | 14:02 |
| rfolco|ruck | marios|rover, criteria? | 14:02 |
| marios|rover | rfolco|ruck: oh fine ok | 14:03 |
| marios|rover | rfolco|ruck: but | 14:03 |
| marios|rover | rfolco|ruck: that will have to be on the promoter itself | 14:03 |
| rfolco|ruck | not sure if that is a option | 14:03 |
| marios|rover | not the upstream critera | 14:03 |
| marios|rover | rfolco|ruck: cos it is frozen right | 14:03 |
| marios|rover | rfolco|ruck: sure will post that in a sec | 14:03 |
| marios|rover | rfolco|ruck: well | 14:03 |
| marios|rover | rfolco|ruck: also post it ? | 14:03 |
| marios|rover | rfolco|ruck: or just on the promoter | 14:03 |
| rfolco|ruck | marios|rover, yes hack n smile | 14:03 |
| marios|rover | rfolco|ruck: no i mean should we update the main criteria upstream too with a review | 14:04 |
| marios|rover | rfolco|ruck: cos this is meant to be temporary anyway right | 14:04 |
| marios|rover | rfolco|ruck: will post worst case we abandon it | 14:04 |
| rfolco|ruck | marios|rover, lets chat again now ? | 14:04 |
| marios|rover | rfolco|ruck: yeah sure | 14:04 |
| rfolco|ruck | we need to set a plan in case timeout is gone | 14:04 |
| rfolco|ruck | and in case its not gone | 14:04 |
| rfolco|ruck | and for train/master | 14:04 |
| marios|rover | looks like ykarel found the root cause anyway the cache thing helped a lot apparently | 14:04 |
| rfolco|ruck | nice | 14:05 |
| rfolco|ruck | ykarel++ | 14:05 |
| rfolco|ruck | ykarel += 1 | 14:05 |
| rfolco|ruck | meh nm | 14:05 |
| marios|rover | ykarel: ack lets use oooq ... but we could just use enabled cache for now? | 14:06 |
| ykarel | marios|rover, that patch doing multiple things related to memcache | 14:08 |
| ykarel | and i am not sure how far memcache is there by default | 14:08 |
| marios|rover | ykarel: i see so you mean enabling it impacts lots of things not just our tempest tests | 14:08 |
| ykarel | marios|rover, yes | 14:08 |
| ykarel | so those patches needs rework | 14:09 |
| rfolco|ruck | marios|rover, ykarel: did the test finish? | 14:09 |
| ykarel | rfolco|ruck, yes it finished in 1 hr 41 minutes | 14:09 |
| rfolco|ruck | wow | 14:09 |
| marios|rover | right? | 14:09 |
| marios|rover | i mean even the 'good' run from 10 days ago was like 2.5 hours | 14:10 |
| ykarel | marios|rover, standalone-tempest | 14:10 |
| rfolco|ruck | ykarel, but this is tempest only ? | 14:10 |
| marios|rover | ykarel: ah ok so a bit shorter than fs20? ack | 14:10 |
| ykarel | rfolco|ruck, yes tempest only | 14:10 |
| rfolco|ruck | ok good | 14:10 |
| ykarel | rfolco|ruck, https://logs.rdoproject.org/71/20171/61/check/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/dc02a8b/job-output.txt | 14:10 |
| rfolco|ruck | marios|rover, wanna chat bro | 14:11 |
| marios|rover | rfolco|ruck: lets talk after tripleo | 14:13 |
| marios|rover | rfolco|ruck: cos weshay is on and he is referring to us | 14:13 |
| rfolco|ruck | k | 14:13 |
| *** recheck has quit IRC | 14:20 | |
| *** ykarel is now known as ykarel|away | 14:20 | |
| *** recheck has joined #oooq | 14:20 | |
| *** recheck has quit IRC | 14:20 | |
| *** recheck has joined #oooq | 14:21 | |
| ykarel|away | marios|rover, rfolco|ruck /me need to go out now, tomorrow also would be available for only half day | 14:21 |
| *** recheck has quit IRC | 14:21 | |
| ykarel|away | you can consider revert or fix, /me will check later | 14:21 |
| *** recheck has joined #oooq | 14:21 | |
| ykarel|away | but remember in case of revert, both tht and puppet-tripleo patch need to be reverted | 14:22 |
| ykarel|away | puppet-tripleo depend on tht one | 14:22 |
| marios|rover | thanks ykarel|away | 14:23 |
| marios|rover | ykarel|away: comment #7 https://bugs.launchpad.net/tripleo/+bug/1847585 | 14:25 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,Triaged] | 14:25 |
| marios|rover | rfolco|ruck: o/ | 14:27 |
| *** recheck has quit IRC | 14:27 | |
| *** recheck has joined #oooq | 14:28 | |
| marios|rover | panda: if i update the criteria on the promoter do i need to restart the service? | 14:31 |
| marios|rover | to pick up the criteria i mean ? we will update train to be same as master | 14:31 |
| marios|rover | weshay: ^^ this is what you meant on the call earlier ? ^ | 14:31 |
| marios|rover | panda: i think it will pick up just sanity checking | 14:33 |
| panda | marios|rover: no need to restart | 14:33 |
| weshay | marios|rover, no | 14:33 |
| weshay | marios|rover, rfolco|ruck but only change the criteria w/ my coordination please :) | 14:34 |
| marios|rover | weshay: me and folco discussing now | 14:34 |
| marios|rover | weshay: do you have time to join us? | 14:34 |
| weshay | sure | 14:34 |
| marios|rover | weshay: otherwise, was that | 14:34 |
| marios|rover | k | 14:34 |
| marios|rover | sec | 14:34 |
| *** dtantsur is now known as dtantsur|brb | 14:40 | |
| *** dsneddon has joined #oooq | 14:42 | |
| chandankumar | weshay: ykarel|away with cache enabled the deployment reproducer is running tempest | 14:45 |
| chandankumar | weshay: ykarel|away http://38.145.34.6:9000/t/tripleo-ci-reproducer/stream/72f8575a71a44f99818fbf0b359187ba?logfile=console.log | 14:45 |
| *** dsneddon has quit IRC | 14:46 | |
| marios|rover | weshay: https://bugs.launchpad.net/tripleo/+bug/1847585 | 14:47 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,Triaged] | 14:47 |
| zbr | weshay: for which release(s) did we need the iptables hack? | 14:47 |
| chandankumar | marios|rover: reproducer is also up npw | 14:48 |
| weshay | https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-latest-released | 14:49 |
| weshay | marios|rover, rfolco|ruck baseurl=http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8080/rdo/centos7-train/11/c9/11c9839df8d9b5abb46f2c7bee2c8db975f77867_3f633a87 | 14:49 |
| weshay | rfolco|ruck, marios|rover https://trunk.rdoproject.org/api-centos-train/api/civotes_detail.html?commit_hash=11c9839df8d9b5abb46f2c7bee2c8db975f77867&distro_hash=3f633a877cc44c0fa774e06dbf99c7db29419b2f | 14:51 |
| marios|rover | weshay: ack | 14:58 |
| weshay | https://trunk.rdoproject.org/api-centos-train/api/civotes_detail.html?commit_hash=11c9839df8d9b5abb46f2c7bee2c8db975f77867&distro_hash=3f633a877cc44c0fa774e06dbf99c7db29419b2f | 14:58 |
| weshay | http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-scenario004-standalone-train/be6b4f8/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 14:58 |
| *** holser has quit IRC | 15:01 | |
| *** skramaja has quit IRC | 15:09 | |
| chandankumar | marios|rover: now we have results http://logs.rdoproject.org/37/23137/1/check/periodic-tripleo-ci-centos-7-standalone-full-tempest-master/23110aa/logs/undercloud/var/log/tempest/tempest_run.log.txt.gz | 15:13 |
| chandankumar | in timedout job | 15:13 |
| chandankumar | marios|rover: I will put this result in job compare to have a full idea | 15:14 |
| marios|rover | chandankumar: ack can you add comment on the bug ? | 15:15 |
| *** matbu has quit IRC | 15:21 | |
| *** holser has joined #oooq | 15:23 | |
| *** holser has quit IRC | 15:27 | |
| marios|rover | weshay: rfolco|ruck like that? https://review.rdoproject.org/r/#/c/23141/1/.zuul.yaml | 15:35 |
| weshay | dlrn_hash_tag: | 15:36 |
| marios|rover | weshay: ah k sec sorry | 15:36 |
| marios|rover | weshay: done | 15:37 |
| *** jfrancoa has quit IRC | 15:37 | |
| weshay | marios|rover, my example.. https://review.rdoproject.org/r/#/c/22878/3/.zuul.yaml didn't need a hash for mine... different case | 15:38 |
| marios|rover | weshay: you think i need the registry disabled too then ? | 15:38 |
| marios|rover | + iptables... adding | 15:38 |
| chandankumar | weshay: registry disabled patch merged in reproducer | 15:39 |
| weshay | chandankumar, thanks | 15:39 |
| weshay | marios|rover, chandankumar anyone know how to disable mirrors in a job? | 15:39 |
| marios|rover | weshay: sorry i do not | 15:39 |
| chandankumar | weshay: we discovered for periodic job force_periodic and rdo registry var should set in | 15:39 |
| chandankumar | weshay: jpena is the correct person | 15:40 |
| weshay | chandankumar, have a libvirt repro working... using the new repro.. but fails on the mirror | 15:40 |
| weshay | chandankumar, good.. thanks | 15:40 |
| marios|rover | weshay: updated https://review.rdoproject.org/r/#/c/22878/3/.zuul.yaml | 15:40 |
| marios|rover | weshay: sorry thats your one | 15:40 |
| marios|rover | weshay: here https://review.rdoproject.org/r/#/c/23141/3/.zuul.yaml | 15:40 |
| weshay | aye.. ok.. you should see it kick in rdo zuul | 15:40 |
| weshay | marios|rover, I see it | 15:41 |
| *** dsneddon has joined #oooq | 15:42 | |
| chandankumar | marios|rover: you can use rdo jobs instead of test project | 15:42 |
| marios|rover | weshay: ack | 15:43 |
| marios|rover | chandankumar: ok thanks | 15:43 |
| chandankumar | just modify projects.yaml and get the job done | 15:43 |
| *** matbu has joined #oooq | 15:44 | |
| *** jpena is now known as jpena|brb | 15:45 | |
| *** dsneddon has quit IRC | 15:46 | |
| *** jfrancoa has joined #oooq | 15:52 | |
| *** dsneddon has joined #oooq | 15:59 | |
| recheck | [gh-pytest-molecule] 1.2.2 → https://github.com/pycontribs/pytest-molecule/releases/tag/1.2.2 | 15:59 |
| rfolco|ruck | marios|rover, working on the rdo jobs patch ? let me know if you wanna some help or chat about | 16:03 |
| rfolco|ruck | marios|rover, need to talk 5 min before you leave | 16:03 |
| marios|rover | rfolco|ruck: https://review.rdoproject.org/r/#/c/23141/3/.zuul.yaml there | 16:03 |
| marios|rover | rfolco|ruck: am almost out now | 16:03 |
| rfolco|ruck | please don't go | 16:03 |
| rfolco|ruck | don't gooooo | 16:04 |
| rfolco|ruck | marios|rover, this is testproject | 16:04 |
| marios|rover | rfolco|ruck: 18:35 < marios|rover> weshay: rfolco|ruck like that? https://review.rdoproject.org/r/#/c/23141/1/.zuul.yaml | 16:04 |
| marios|rover | rfolco|ruck: pinged you a while ago man... | 16:04 |
| marios|rover | rfolco|ruck: yeah should it be rdo jobs? chandankumar mentioned that but why not testproject | 16:05 |
| rfolco|ruck | marios|rover, <chandankumar> marios|rover: you can use rdo jobs instead of test project | 16:05 |
| marios|rover | is there an advantage? | 16:05 |
| marios|rover | rfolco|ruck: ^ if there is sure post it there instead? | 16:05 |
| chandankumar | marios|rover: yes, it is easier | 16:05 |
| rfolco|ruck | marios|rover, ok whatever :) | 16:05 |
| marios|rover | chandankumar: easier how | 16:05 |
| chandankumar | marios|rover: you can do the stuff from termianl | 16:05 |
| chandankumar | locally | 16:05 |
| rfolco|ruck | marios|rover, just do what you find easier, both works | 16:05 |
| marios|rover | chandankumar: what stuff you mean to write the review? | 16:05 |
| weshay | 2019-10-15 15:54:14 | ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleotrain/centos-binary-glance-api:9fc9eb4112461e3f5db82e8c2dfeb3fb661011b0_062875d6 | 16:05 |
| marios|rover | chandankumar: not questioning it tryiung to understand if there is some advantage to rdo-jobs | 16:06 |
| *** akahat has quit IRC | 16:06 | |
| chandankumar | marios|rover: since most of the jobs defined there, we can manipulate there and run there | 16:06 |
| chandankumar | and then once testing finish, convert it to proper review | 16:07 |
| chandankumar | in a single repo itself | 16:07 |
| *** dsneddon has quit IRC | 16:07 | |
| chandankumar | this is the advantage | 16:07 |
| weshay | hrm.. maybe I'm wrong about the var for dlrn_hash | 16:08 |
| weshay | baseurl=http://mirror.regionone.rdo-cloud.rdoproject.org:8080/rdo/centos7-train/9f/c9/9fc9eb4112461e3f5db82e8c2dfeb3fb661011b0_062875d6 | 16:08 |
| weshay | http://logs.rdoproject.org/41/23141/3/check/periodic-tripleo-ci-centos-7-scenario010-standalone-train/c4592f2/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 16:08 |
| chandankumar | weshay: replace it with this https://trunk.rdoproject.org/centos7-train/9f/c9/9fc9eb4112461e3f5db82e8c2dfeb3fb661011b0_062875d6/ | 16:09 |
| chandankumar | to find the repo | 16:09 |
| chandankumar | or use skopeo inspect whether image exists or not | 16:09 |
| weshay | the override in zuul | 16:09 |
| weshay | is dlrn_hash: | 16:09 |
| weshay | or dlrn_hash_tag: | 16:09 |
| marios|rover | weshay: is it not working with dlrn_hash_tag? | 16:09 |
| weshay | https://review.rdoproject.org/r/#/c/23141/3..4/.zuul.yaml | 16:10 |
| weshay | marios|rover, nope.. but updated https://review.rdoproject.org/r/#/c/23141/3..4/.zuul.yaml | 16:10 |
| marios|rover | weshay: thank you will catch it again tomorrow then | 16:10 |
| weshay | marios|rover, jobs retarted :) | 16:10 |
| weshay | cool.. c ya | 16:10 |
| marios|rover | weshay:ack | 16:10 |
| chandankumar | marios|rover: see ya! | 16:10 |
| *** matbu has quit IRC | 16:11 | |
| *** kopecmartin is now known as kopecmartin|off | 16:11 | |
| marios|rover | o/ chandankumar | 16:12 |
| *** dtantsur|brb is now known as dtantsur | 16:13 | |
| chandankumar | marios|rover: weshay dlrn_hash_tag would be tripleo-ci-testting or current tripleo | 16:13 |
| chandankumar | dlrn_hash would be just hash | 16:13 |
| weshay | aye.. hard to keep it straight | 16:16 |
| weshay | thanks | 16:16 |
| weshay | chandankumar, in the repoducer.. we give a hash.. for hash tag | 16:16 |
| weshay | I need to go through the logic again | 16:17 |
| chandankumar | weshay: yes, we need to rework that part, /me waits for rlandy or sshnaidm|pto to come back | 16:17 |
| *** jpena|brb is now known as jpena | 16:18 | |
| *** dsneddon has joined #oooq | 16:24 | |
| *** marios|rover has quit IRC | 16:24 | |
| *** dsneddon has quit IRC | 16:29 | |
| *** jfrancoa has quit IRC | 16:32 | |
| weshay | it's dlrn_hash_tag | 16:33 |
| weshay | force_periodic is overriding it as designed | 16:33 |
| *** dsneddon has joined #oooq | 16:35 | |
| *** dtantsur is now known as dtantsur|afk | 16:37 | |
| *** dsneddon has quit IRC | 16:40 | |
| *** derekh has quit IRC | 16:47 | |
| *** holser has joined #oooq | 17:00 | |
| *** jpena is now known as jpena|off | 17:01 | |
| *** dsneddon has joined #oooq | 17:03 | |
| *** amoralej is now known as amoralej|off | 17:09 | |
| *** dsneddon has quit IRC | 17:13 | |
| *** ykarel|away is now known as ykarel | 17:15 | |
| *** dsneddon has joined #oooq | 17:17 | |
| *** dsneddon has quit IRC | 17:22 | |
| *** holser has quit IRC | 17:22 | |
| *** epoojad1 has quit IRC | 17:37 | |
| *** epoojad1 has joined #oooq | 17:38 | |
| rfolco|ruck | weshay, arxcruz do we know why jobs started timing out? it seems to be happening since last sprint | 17:51 |
| *** dsneddon has joined #oooq | 17:55 | |
| *** dsneddon has quit IRC | 18:06 | |
| *** holser has joined #oooq | 18:09 | |
| *** epoojad1 has quit IRC | 18:22 | |
| *** chandankumar is now known as raukadah | 18:24 | |
| *** dsneddon has joined #oooq | 18:46 | |
| *** ykarel is now known as ykarel|away | 18:47 | |
| weshay | rfolco|ruck, you there? | 18:58 |
| rfolco|ruck | weshay, yes, whats going on https://review.rdoproject.org/r/#/c/23141/ | 18:58 |
| rfolco|ruck | weshay, why only standalone | 18:58 |
| rfolco|ruck | weshay, ?? | 19:01 |
| weshay | rfolco|ruck, well.. I was having trouble forcing the hash.. and didn't want to kill rdo | 19:01 |
| weshay | rfolco|ruck, but found an interesting problem | 19:01 |
| rfolco|ruck | ps5 failed on | 19:02 |
| rfolco|ruck | HTTPError: 503 Server Error: Service Unavailable for url: http://mirror.regionone.rdo-cloud.rdoproject.org:8082/v2/ | 19:02 |
| weshay | aye | 19:02 |
| weshay | rfolco|ruck, let's chat so I can explain | 19:02 |
| weshay | in person | 19:02 |
| rfolco|ruck | ok | 19:02 |
| weshay | also noting it in etherpad | 19:02 |
| weshay | rfolco|ruck, https://meet.google.com/zyi-unuk-sfq | 19:03 |
| *** ykarel|away has quit IRC | 19:05 | |
| *** holser has quit IRC | 19:14 | |
| weshay | rfolco|ruck, http://codesearch.openstack.org/?q=hash_info.sh&i=nope&files=&repos= | 19:14 |
| rfolco|ruck | weshay, https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-ci-periodic-base/pre.yaml#L11 | 19:28 |
| rfolco|ruck | and https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/ci-scripts/tripleo-upstream/get-hash.sh#L34 | 19:28 |
| rfolco|ruck | so perhaps we can hack promote_name there | 19:29 |
| weshay | rfolco|ruck, ya.. $PROMOTE_NAME needs to equal the actual hash | 19:36 |
| weshay | not an arbitary name | 19:37 |
| rfolco|ruck | actually more than that | 19:37 |
| rfolco|ruck | 11/c9/11c9839df8d9b5abb46f2c7bee2c8db975f77867_2932899d | 19:37 |
| rfolco|ruck | https://trunk.rdoproject.org/centos7-master/11/c9/11c9839df8d9b5abb46f2c7bee2c8db975f77867_2932899d/commit.yaml | 19:37 |
| weshay | yes.. the hash formatted correctly | 19:37 |
| rfolco|ruck | 4 first chars split in sub dirs | 19:37 |
| weshay | we have a little tool for that.. | 19:37 |
| weshay | ya.. | 19:37 |
| weshay | exactly | 19:37 |
| rfolco|ruck | so this is hardcoded | 19:38 |
| weshay | ya | 19:38 |
| weshay | rfolco|ruck, when is that executed? | 19:38 |
| rfolco|ruck | I can propose a patch to get this from job definition var and include a if there in https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-ci-periodic-base/pre.yaml#L11 | 19:38 |
| weshay | do you know if that is executed after the repo's are setup? | 19:39 |
| rfolco|ruck | let me check that pre | 19:39 |
| weshay | perhaps we try to rerun it in post.. if /etc/yum.repos.d/delorean.repo exists | 19:40 |
| weshay | because there may be times when jobs don't get as far as creating the repo | 19:40 |
| weshay | but really who cares if it reports at in that case.. but better to run it twice | 19:40 |
| weshay | rfolco|ruck, you know what I mean? | 19:41 |
| rfolco|ruck | I am trying to parse your comments | 19:42 |
| weshay | rfolco|ruck, let me explain.. back to hangout for just a sec | 19:42 |
| weshay | it's important | 19:42 |
| rfolco|ruck | k | 19:42 |
| rfolco|ruck | http://logs.rdoproject.org/41/23141/7/check/periodic-tripleo-ci-centos-7-scenario010-standalone-train/b385d76/job-output.txt | 19:42 |
| rfolco|ruck | search for Populate hash | 19:43 |
| rfolco|ruck | you'll see when it happens... I am trying to find when the repo is setup | 19:43 |
| weshay | rfolco|ruck, ya.. we need to run it twice | 19:43 |
| weshay | rfolco|ruck, it's much later | 19:43 |
| *** tesseract has quit IRC | 19:49 | |
| *** holser has joined #oooq | 19:50 | |
| *** holser has quit IRC | 19:53 | |
| rfolco|ruck | weshay, can I read delorean.repo, get the hash and append a line to the end in the hash_info.sh here ? https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/ci-scripts/tripleo-upstream/get-hash.sh#L49 | 20:18 |
| rfolco|ruck | so no matter what you export at the top, the last one is what counts | 20:19 |
| * weshay looks | 20:19 | |
| rfolco|ruck | hmm I need to validate that as well if the path exists for commit.yaml in dlrn trunk | 20:20 |
| weshay | the delorean.repo does not exist at that time | 20:20 |
| rfolco|ruck | post task not pre | 20:20 |
| rfolco|ruck | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-ci-periodic-base/post.yaml#L13 | 20:21 |
| rfolco|ruck | when that shell runs it should exist | 20:21 |
| weshay | rfolco|ruck, ya.. we can override it in the same file for sure.. | 20:22 |
| weshay | I would echo a comment into it | 20:22 |
| rfolco|ruck | k | 20:22 |
| rfolco|ruck | lets see | 20:22 |
| weshay | so it just doesn't look like it's there twice for no reason | 20:22 |
| rfolco|ruck | it populates hash in pre | 20:23 |
| rfolco|ruck | it loads it in post | 20:23 |
| rfolco|ruck | hash_info I mean | 20:23 |
| weshay | ya | 20:23 |
| weshay | rfolco|ruck, we should log this file too.. http://logs.rdoproject.org/41/23141/7/check/periodic-tripleo-ci-centos-7-scenario010-standalone-train/b385d76/logs/quickstart_files/ | 20:24 |
| weshay | oh we do | 20:24 |
| weshay | http://logs.rdoproject.org/41/23141/7/check/periodic-tripleo-ci-centos-7-scenario010-standalone-train/b385d76/logs/get_hash_log.log | 20:24 |
| rfolco|ruck | k | 20:25 |
| weshay | rfolco|ruck, so ya.. so good comments will make that clear to a reader here: http://logs.rdoproject.org/41/23141/7/check/periodic-tripleo-ci-centos-7-scenario010-standalone-train/b385d76/logs/get_hash_log.log | 20:25 |
| rfolco|ruck | hash_info.sh you mean? | 20:25 |
| weshay | + curl -sLo /home/zuul/workspace/commit.yaml https://trunk.rdoproject.org/centos7-train/tripleo-ci-testing/commit.yaml | 20:25 |
| rfolco|ruck | the generated file we do not save | 20:25 |
| weshay | rfolco|ruck, aye.. the execution is enough | 20:26 |
| weshay | rfolco|ruck, even though it's sourced.. you can still http://paste.openstack.org/show/784006/ | 20:27 |
| zbr | weshay: fyi: centos-8 nodes can now be used, you will see a msg soon. | 20:29 |
| rfolco|ruck | zbr, ooohooo | 20:29 |
| zbr | is too late now to add jobs, but i will do tomorrow morning. starting with build containers? new release files? | 20:30 |
| weshay | rfolco|ruck, don't get too excited, we still don't have centos8 packages for rdo | 20:30 |
| weshay | :) | 20:30 |
| weshay | zbr, very cool.. what reviews? | 20:30 |
| rfolco|ruck | :( | 20:30 |
| weshay | zbr, don't add anything | 20:30 |
| weshay | zbr, we were told to hold off.. it's good to have the node.. | 20:31 |
| zbr | weshay: is true that we have centos-8 on openstack-zuul only, because is built from scratch, on rdo we do not have them. | 20:31 |
| zbr | because centos-8 did not release images. | 20:31 |
| weshay | zbr, too late to chat? | 20:32 |
| zbr | 5min should be ok. | 20:32 |
| weshay | ok.. ya will be quick | 20:32 |
| weshay | https://meet.google.com/icb-foqe-cvi | 20:32 |
| weshay | {7} tripleoclient.tests.v1.test_container_image.TestContainerImagePush.test_take_action_local_path [0.021741s] ... FAILED | 21:05 |
| weshay | mostly caused by mageUploaderException: No entry for centos-7-rax-dfw-0012323069.ctlplane in /etc/hosts | 21:06 |
| *** Goneri has quit IRC | 21:10 | |
| weshay | FYI https://bugs.launchpad.net/tripleo/+bug/1848275 | 21:12 |
| openstack | Launchpad bug 1848275 in tripleo "rpm DLRN builds failing due to ctlplane entry missing from /etc/hosts " [Critical,Triaged] | 21:12 |
| *** tosky has quit IRC | 23:38 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!