| *** tosky has quit IRC | 00:09 | |
| *** dalvarez has quit IRC | 01:00 | |
| *** vkmc|afk is now known as vkmc | 01:15 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004 (2 more messages) | 01:47 |
|---|---|---|
| *** vinaykns has joined #oooq | 02:19 | |
| vinaykns | anyone around..? | 02:23 |
| vinaykns | I need a quick help...I'm trying to deploy tripleo quickstart but it is getting failed as it couldn't start virtual bmc, as the configured resource limit was exceeded...any workaround..? | 02:23 |
| vinaykns | http://pastebin.test.redhat.com/649195 | 02:24 |
| *** sshnaidm has joined #oooq | 02:42 | |
| *** sshnaidm is now known as sshnaidm|pto | 02:42 | |
| sshnaidm|pto | weshay_pto, let's start merge this please https://review.openstack.org/#/c/597222 | 02:43 |
| *** skramaja has joined #oooq | 02:51 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004 (2 more messages) | 03:47 |
| *** udesale has joined #oooq | 03:53 | |
| *** jaganathan_ has quit IRC | 03:57 | |
| *** ykarel has joined #oooq | 04:05 | |
| *** vinaykns has quit IRC | 04:43 | |
| *** jaosorior has quit IRC | 05:03 | |
| *** ratailor has joined #oooq | 05:18 | |
| *** ykarel has quit IRC | 05:20 | |
| *** jtomasek has joined #oooq | 05:32 | |
| *** ykarel has joined #oooq | 05:40 | |
| quique|rover|off | sshnaidm|pto: Yep, let's merge it, my comments can be addressed later, we need that. | 05:40 |
| quique|rover|off | ykarel: Puff still the fix fo fs019 is not merge :-( all the night in the gates like a drunk guy | 05:41 |
| ykarel | :) | 05:41 |
| ykarel | how far is it in gate | 05:42 |
| chkumar|off | pour him more wine, it will get merged | 05:42 |
| ykarel | hmm still too far in gate | 05:43 |
| ykarel | 19th in queue | 05:44 |
| quique|rover|off | Don't know man, I am going to go to the hills and do farming, this is insane | 05:44 |
| ykarel | so seems rare chance it will merge today, | 05:44 |
| ykarel | unless and until gate is too happy | 05:44 |
| quique|rover|off | We are giving away like 3 days of master/roick/queens promotions for a little merge of a version pufff | 05:45 |
| * chkumar|off needs to dig fs020 neutron tempest issue unexpected error on rocky | 05:45 | |
| quique|rover|off | chkumar|off: Did it fail again ? | 05:45 |
| *** quique|rover|off is now known as quiquell|rover | 05:45 | |
| chkumar|off | quiquell|rover: yes https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/8a335fb/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-09-20_05_02_52 | 05:46 |
| chkumar|off | one way if i skip neutorn tests for fs020 we migth get a promotion provides fs019 passes | 05:46 |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004 (2 more messages) | 05:47 |
| quiquell|rover | chkumar|off: if we merge the stuf we will force run periodics, at least to discover the next issues | 05:47 |
| quiquell|rover | chkumar|off: Do you agree ? | 05:47 |
| chkumar|off | quiquell|rover: yes | 05:48 |
| chkumar|off | quiquell|rover: ykarel http://logs.openstack.org/45/560445/144/check/tripleo-ci-centos-7-containers-multinode-queens/dcb313c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz#_2018-09-19_23_11_16 | 05:48 |
| chkumar|off | 2018-09-19 23:11:16 | 2018-09-19 23:11:16,151 ERROR: ERROR running yum update in container 192.168.24.1:8787/tripleoqueens/centos-binary-horizon:current-tripleo | 05:48 |
| chkumar|off | it is from noop master job failure | 05:48 |
| chkumar|off | rest story is same | 05:49 |
| chkumar|off | quiquell|rover: I think I also got the soltuin for scenario 004 timed out | 05:51 |
| quiquell|rover | chkumar|off: It's not related to need of promotions ? | 05:52 |
| chkumar|off | quiquell|rover: nope | 05:52 |
| chkumar|off | quiquell|rover: it is also avoiding merging patches in gates | 05:53 |
| quiquell|rover | chkumar|off: DIdn't have time to even check CI | 05:53 |
| *** chkumar|off is now known as chkumar|ruck | 05:53 | |
| chkumar|ruck | quiquell|rover: let's we can focus on rocky | 05:53 |
| *** kopecmartin has joined #oooq | 06:08 | |
| *** holser_ has joined #oooq | 06:09 | |
| *** jaosorior has joined #oooq | 06:25 | |
| *** sanjayu_ has joined #oooq | 06:35 | |
| *** quiquell|rover is now known as quique|rover|brb | 06:36 | |
| *** jfrancoa has joined #oooq | 06:37 | |
| *** saneax has quit IRC | 06:38 | |
| *** gkadam has joined #oooq | 06:46 | |
| *** sanjayu_ has quit IRC | 06:51 | |
| *** kopecmartin has quit IRC | 06:54 | |
| *** quique|rover|brb is now known as quiquell|rover | 07:09 | |
| *** ykarel is now known as ykarel|lunch | 07:13 | |
| *** jtomasek has quit IRC | 07:15 | |
| *** bogdando has joined #oooq | 07:32 | |
| chkumar|ruck | brb | 07:37 |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004 (2 more messages) | 07:47 |
| bogdando | o/ PTAL https://review.openstack.org/#/c/593103/ | 07:51 |
| *** jfrancoa has quit IRC | 08:02 | |
| quiquell|rover | chkumar|ruck: why removed promotion-blocker from https://bugs.launchpad.net/tripleo/+bug/1793452 ? | 08:12 |
| openstack | Launchpad bug 1793452 in tripleo "[master] fs002 periodic job showing connection closed during overcloud prep image step" [Critical,Triaged] | 08:12 |
| *** jaosorior has quit IRC | 08:12 | |
| *** jfrancoa has joined #oooq | 08:18 | |
| *** ykarel|lunch is now known as ykarel | 08:20 | |
| *** tosky has joined #oooq | 08:21 | |
| chkumar|ruck | quiquell|rover: as per discussion we are going to include in connection closed bug and mark it as duplicate so | 08:25 |
| chkumar|ruck | let me add it back | 08:26 |
| quiquell|rover | chkumar|ruck: ack | 08:26 |
| chkumar|ruck | quiquell|rover: oh you filed the yum update bug good | 08:27 |
| quiquell|rover | chkumar|ruck: It's not that bad, no so ofthen | 08:27 |
| quiquell|rover | not so often | 08:27 |
| quiquell|rover | feels infra | 08:27 |
| chkumar|ruck | quiquell|rover: regarding rocky tempest failure i am getting no clue I am opening a bug | 08:28 |
| quiquell|rover | chkumar|ruck: Will look into it too, four eyes better than two | 08:28 |
| *** kopecmartin has joined #oooq | 08:41 | |
| *** jaosorior has joined #oooq | 08:58 | |
| *** sshnaidm|pto has quit IRC | 09:41 | |
| *** dtantsur|afk is now known as dtantsur | 09:43 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004 (2 more messages) | 09:47 |
| *** jaosorior has quit IRC | 09:53 | |
| marios | quiquell|rover: o/ was just checking out the gatechecks see if somethng i can help with | 10:00 |
| marios | quiquell|rover: do you want to abandon https://review.openstack.org/#/c/564285 ? | 10:00 |
| marios | quiquell|rover: i see you posted new one for pike? | 10:00 |
| quiquell|rover | marios: We have to create new one, since the Change-Id was mixed with new rocky | 10:00 |
| quiquell|rover | so we created new for rocky and new for pike | 10:01 |
| marios | quiquell|rover: oh wow you mean clash? | 10:01 |
| quiquell|rover | the good ones are configured at hubbot1 | 10:01 |
| marios | no way something else? someone manually created it with wrong id or soemthing? | 10:01 |
| quiquell|rover | hubbot1: config | 10:01 |
| hubbot1 | quiquell|rover: (config <name> [<value>]) -- If <value> is given, sets the value of <name> to <value>. Otherwise, returns the current value of <name>. You may omit the leading "supybot." in the name if you so choose. | 10:01 |
| quiquell|rover | hubbot1 config plugin.GateStatus.changeIds | 10:01 |
| hubbot1 | quiquell|rover: Error: 'supybot.plugin.GateStatus.changeIds' is not a valid configuration variable. | 10:01 |
| quiquell|rover | hubbot1 config plugin.GateStatus.changeIDs | 10:01 |
| hubbot1 | quiquell|rover: Error: 'supybot.plugin.GateStatus.changeIDs' is not a valid configuration variable. | 10:01 |
| quiquell|rover | argg don't remember the command | 10:02 |
| marios | quiquell|rover: ack so abandoning old one then (the the rcky one was abandoned at least so i update my bkmark) | 10:02 |
| chkumar|ruck | marios: https://review.openstack.org/#/c/603915/ need +2 and +w on thus | 10:03 |
| quiquell|rover | yep | 10:03 |
| quiquell|rover | jfrancoa: https://review.openstack.org/#/c/603915/ unblock gates | 10:03 |
| jfrancoa | quiquell|rover: I can +2 it, but I don't have context enough to merge it forward. Maybe marios can? | 10:06 |
| quiquell|rover | jfrancoa: fair enough | 10:06 |
| marios | jfrancoa: heh the strategy is to talk to me in tripleo and you in oooq | 10:07 |
| marios | jfrancoa: so we both +2 it | 10:07 |
| marios | waiting for zuul anyway | 10:07 |
| marios | quiquell|rover: chkumar|ruck ;) | 10:07 |
| *** jtomasek has joined #oooq | 10:09 | |
| marios | jfrancoa: so the context is this bug was filed a couple weeks ago https://bugs.launchpad.net/tripleo/+bug/1790857 for the queens gate but looks like the same issue affects o and p too | 10:10 |
| openstack | Launchpad bug 1790857 in tripleo "Queens tripleo-ci-centos-7-scenario004-multinode-oooq-container failing tempest manila_tempest_tests.tests.api.test_shares.SharesCephFSTest " [Critical,Fix released] - Assigned to Goutham Pacha Ravi (gouthamr) | 10:10 |
| jfrancoa | marios: aha, right. I saw this error some time ago, Thanks for the info, I wasn't sure what was it about | 10:11 |
| *** dalvarez has joined #oooq | 10:16 | |
| *** kopecmartin has quit IRC | 10:30 | |
| quiquell|rover | marios: What do we miss to merge https://review.openstack.org/#/c/604032 ? | 10:30 |
| marios | quiquell|rover: well it doesn't have any reviews yet, and was filed unler 10 mins ago? | 10:32 |
| marios | but otherwise not sure :) | 10:32 |
| marios | oh is that jistr one? | 10:32 |
| marios | https://review.openstack.org/#/c/590774/11 did you mean this quiquell|rover ? | 10:32 |
| *** jfrancoa has quit IRC | 10:33 | |
| quiquell|rover | marios: nope | 10:34 |
| quiquell|rover | marios: tempest one | 10:34 |
| marios | 13:30 < quiquell|rover> marios: What do we miss to merge https://review.openstack.org/#/c/604032 ? | 10:34 |
| marios | ? | 10:34 |
| quiquell|rover | marios: Puff my fault, I mean this https://review.openstack.org/#/c/603915/ | 10:34 |
| marios | quiquell|rover: oh well there was no zuul result when i voted | 10:34 |
| marios | quiquell|rover: still isn't | 10:35 |
| *** jfrancoa has joined #oooq | 10:35 | |
| quiquell|rover | marios: all passed except upgrades | 10:36 |
| quiquell|rover | marios: do you have the greasy monky plugin to view reviews ? is super cool | 10:36 |
| quiquell|rover | marios: show you the current state before finished | 10:36 |
| marios | quiquell|rover: no, is it reading from zuul :) | 10:36 |
| marios | quiquell|rover: cool | 10:36 |
| ssbarnea | quiquell|rover: seen your last comment, i was considering writing a recheck bot over a weekend. | 10:37 |
| ssbarnea | probably i will create my own "PA" bot to deal with nagging tasks. | 10:38 |
| quiquell|rover | marios: install tampermonkey at chrome and open this url https://github.com/mpeterson/openstack-greasemonkey-helpers/blob/master/src/openstack_gerrit_zuul_status.user.js | 10:40 |
| quiquell|rover | marios: then open a running review | 10:40 |
| marios | quiquell|rover: nice thanks will check it out (I checked zuul.openstack.org you're not lying but would still rather see the results on the review first should be few more mins? | 10:42 |
| marios | quiquell|rover: ok done :! | 10:43 |
| quiquell|rover | marios: Yep, let's just wait | 10:43 |
| ssbarnea | quiquell|rover: i think that this one will not merge until it gets an additional CR+2, right? https://review.openstack.org/#/c/595265/ | 10:43 |
| marios | quiquell|rover: I +w it just now | 10:43 |
| marios | and zuul JUST reported o_O | 10:43 |
| ssbarnea | same with https://review.openstack.org/#/c/576746/ | 10:44 |
| quiquell|rover | bogdando: ssbarnea has some reviews with +w but missing +2 | 10:45 |
| ssbarnea | quiquell|rover: i think wes is right on https://review.openstack.org/#/c/589068/12/playbooks/tripleo-ci/run-v3.yaml@42 -- can I fix it? is your change. | 10:48 |
| ssbarnea | substracting seconds from minutes | 10:48 |
| quiquell|rover | ssbarnea: This is not the right review, we are removing tiemouts all together | 10:48 |
| ssbarnea | one reason why is recommented to include UM in variable name :D | 10:48 |
| quiquell|rover | https://review.openstack.org/#/c/589068/ | 10:49 |
| *** sanjayu_ has joined #oooq | 10:49 | |
| ssbarnea | so, that one should be abandoned, right? | 10:49 |
| quiquell|rover | ssbarnea: We are reviving this ? | 10:50 |
| quiquell|rover | ssbarnea: But the comments are new | 10:50 |
| quiquell|rover | :-/ | 10:50 |
| ssbarnea | i am trying to review all CRs where I was added. | 10:50 |
| quiquell|rover | ssbarnea: ahh wait, take a look is a comment on old stuff | 10:51 |
| quiquell|rover | ssbarnea: totally uncontextual comment | 10:51 |
| quiquell|rover | ssbarnea: Commented code does not exists anymore | 10:51 |
| *** abishop has joined #oooq | 10:51 | |
| ssbarnea | yeah, not sure how I ended up looking at patch 2 when current version was 27 | 10:52 |
| ssbarnea | quiquell|rover: btw, is this normal? http://zuul.openstack.org/#588587 "16h" nothing happening. | 10:56 |
| quiquell|rover | ssbarnea: gate queue is pretty big right now | 10:57 |
| quiquell|rover | ssbarnea: we are all screw there | 10:57 |
| *** jaosorior has joined #oooq | 10:58 | |
| chkumar|ruck | who is the openstack CI god? | 10:58 |
| *** udesale has quit IRC | 10:58 | |
| ssbarnea | chkumar|ruck: have you tried typing iddqd in zuul, maybe it works? ;) | 11:08 |
| *** ratailor has quit IRC | 11:16 | |
| chkumar|ruck | ssbarnea: as per current design it only works with same branch | 11:18 |
| chkumar|ruck | once current run finisshes i will try that | 11:18 |
| chkumar|ruck | quiquell|rover: for rocky promotion we need to disable neutron tests in fs020 because of this bz https://bugzilla.redhat.com/show_bug.cgi?id=1631335 | 11:21 |
| openstack | bugzilla.redhat.com bug 1631335 in openstack-neutron "WSGI workers not started again after SIGHUP signal" [Unspecified,New] - Assigned to amuller | 11:21 |
| quiquell|rover | chkumar|ruck: Can you add it to the launchpad ? | 11:21 |
| quiquell|rover | chkumar|ruck: Why we have a bz for this ? lp is not enough ? | 11:22 |
| chkumar|ruck | quiquell|rover: it might eb downstream also so | 11:22 |
| quiquell|rover | chkumar|ruck: tthis ? https://bugs.launchpad.net/tripleo/+bug/1793482 | 11:24 |
| openstack | Launchpad bug 1793482 in tripleo "[Rocky] fs020 periodic job neutron tempest tests are failing by giving 503 Service Unavailable" [Critical,Triaged] | 11:24 |
| ssbarnea | do we have a codestyle guide for tripleo somewhere, or for ansible in general? I google a little bit and I found that tripleo-docs doesn't mention anything about style and found something called tripleo-incubator which seems to be a stlyeguide that covers only bash. | 11:24 |
| chkumar|ruck | quiquell|rover: yes | 11:24 |
| chkumar|ruck | quiquell|rover: I have added the comment there | 11:25 |
| quiquell|rover | chkumar|ruck: ups ok | 11:25 |
| chkumar|ruck | it needs investigation | 11:25 |
| quiquell|rover | chkumar|ruck: We want to do a promotion without fixing it ? | 11:26 |
| quiquell|rover | chkumar|ruck: Feels like it's not tempest only related | 11:26 |
| chkumar|ruck | quiquell|rover: I have no idea what to do, something serious might be happening there | 11:27 |
| *** kopecmartin has joined #oooq | 11:28 | |
| quiquell|rover | chkumar|ruck: Didn't have time to look at it... | 11:30 |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ (1 more message) | 11:47 |
| *** kopecmartin has quit IRC | 12:00 | |
| *** kopecmartin has joined #oooq | 12:00 | |
| *** kopecmartin has quit IRC | 12:01 | |
| *** weshay_pto is now known as weshay | 12:01 | |
| *** kopecmartin has joined #oooq | 12:12 | |
| *** rlandy has joined #oooq | 12:16 | |
| *** trown|outtypewww is now known as trown | 12:23 | |
| *** sanjayu_ has quit IRC | 12:34 | |
| agopi | good mornin weshay rlandy, can you please do a jjb push? To see if jobs are reconfigured with gerrit stuff? | 12:40 |
| agopi | the job I manually triggered didn't vote on the commit | 12:41 |
| agopi | *retriggered | 12:41 |
| *** gkadam has quit IRC | 12:46 | |
| *** gkadam has joined #oooq | 12:48 | |
| *** gkadam has quit IRC | 12:48 | |
| panda | my bluejeans interface is completely messed up | 12:52 |
| panda | It's like I'm running bluejeans on lynx. | 12:52 |
| weshay | panda, rlandy mtg | 13:02 |
| *** quiquell|rover is now known as quique|rover|lch | 13:15 | |
| *** jfrancoa has quit IRC | 13:19 | |
| *** jfrancoa has joined #oooq | 13:19 | |
| *** udesale has joined #oooq | 13:27 | |
| weshay | https://tree.taiga.io/ | 13:29 |
| weshay | https://trello.com/c/Ble66eRY/431-list-of-upstream-reviews-ruck-rover | 13:39 |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (1 more message) | 13:47 |
| *** quique|rover|lch is now known as quiquell|rover | 13:55 | |
| chkumar|ruck | quique|rover|lch: weshay arxcruz I am going to drop now, once https://review.openstack.org/604091 will merge, I hope we will back in bussinees | 13:55 |
| * chkumar|ruck tired | 13:55 | |
| *** chkumar|ruck is now known as chkumar|off | 13:56 | |
| quiquell|rover | chkumar|ruck: rest | 13:57 |
| chkumar|off | I might skip the tempest call | 13:57 |
| quiquell|rover | chkumar|off: thank for the review | 13:57 |
| rlandy | agopi: still need help? https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tripleo-environments-jjb-push/ shows a push today | 13:59 |
| agopi | oh thank you rlandy | 14:00 |
| agopi | phew | 14:00 |
| agopi | it looks like reset everything fine | 14:00 |
| agopi | weshay++ | 14:00 |
| hubbot1 | agopi: weshay's karma is now 9 | 14:00 |
| agopi | rlandy++ | 14:00 |
| hubbot1 | agopi: rlandy's karma is now 22 | 14:00 |
| arxcruz | chkumar|off: weshay will we have scrum today ? | 14:05 |
| weshay | arxcruz, no.. you guys are in the ether until the sprint review and planning | 14:05 |
| weshay | enjoy no mtgs | 14:05 |
| arxcruz | \o/ | 14:06 |
| arxcruz | weshay: https://tinyurl.com/y74d7q7b | 14:08 |
| marios | rfolco: o/ around? i just voted at https://review.openstack.org/#/c/589068/27 buthave a question about the collect_logs.sh please where is it invoked? (sorry for stupid, bu don't want to +2 if i don't at least (almost) understand it ;) ) | 14:28 |
| rfolco | marios, looking | 14:29 |
| rfolco | marios, fair, will respond there, thanks | 14:31 |
| rfolco | here is the answer :) | 14:32 |
| rfolco | https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/post.yaml#L8 | 14:32 |
| marios | rfolco: thanks, so the fact this is post has nothing to do with your review? | 14:33 |
| marios | rfolco: because it looks like it was currently being called there https://review.openstack.org/#/c/589068/27/playbooks/tripleo-ci/templates/toci_quickstart.sh.j2 (but you're revmoing it) | 14:33 |
| marios | rfolco: i mean the "bash $LOGS_DIR/collect_logs.sh" | 14:34 |
| marios | rfolco: thanks i'll check your comment if you also write one | 14:34 |
| marios | rfolco: cool thanks (grep tells me about zuul.d/post-run: post.yaml | 14:35 |
| marios | nice | 14:35 |
| *** rascasoft has quit IRC | 14:35 | |
| rlandy | panda: https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/run-v3.yaml#L30 | 14:36 |
| rlandy | what's the diff between primary and centos-7 | 14:36 |
| panda | rlandy: centos-7 is probably wrong, it's the node label, not the inventory name | 14:37 |
| rlandy | panda: so, it should be primary only? | 14:37 |
| panda | rlandy: I didn't remove it after realizing i could not specify just centos-7 for the playbook to run in it | 14:38 |
| rlandy | fine - I need to add upstream-centos-7 | 14:38 |
| rlandy | for ovb | 14:38 |
| *** jfrancoa has quit IRC | 14:38 | |
| rlandy | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/nodesets.yaml#L43 | 14:38 |
| rlandy | panda: ^^ | 14:38 |
| rlandy | will that break anything? | 14:39 |
| panda | rlandy: I think you can just remove centos-7 from the run-v3.yaml. as long as the host is called primary, the playbook will run on it | 14:40 |
| rlandy | panda: I need it to run on upstream-centos-7 | 14:41 |
| rlandy | I'll try that first | 14:41 |
| rlandy | panda: https://review.openstack.org/#/c/604109/ | 14:42 |
| panda | rlandy: if the job definiton is using the nodeset called: upstream-centos-7-2-nodes, then zuul will create a node using upstream-centos-7 definition, and calling it "primary" in the inventory. That's al lthe the playbook needs | 14:42 |
| rlandy | panda: https://logs.rdoproject.org/10/16310/3/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/9646bb3/job-output.txt.gz#_2018-09-20_14_12_16_195265 | 14:43 |
| rlandy | ^^ debugging that | 14:43 |
| rlandy | note how the playbook just stops | 14:43 |
| rlandy | it's actually before the nodeset definition | 14:43 |
| rlandy | https://logs.rdoproject.org/10/16310/3/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/9646bb3/zuul-info/ | 14:44 |
| rlandy | ^^ note no primary | 14:44 |
| panda | mmhhh | 14:45 |
| panda | we probably need to change the node definition then | 14:45 |
| panda | rlandy: in https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/nodesets.yaml#L46 name should be "primary" not upstream-centos-7 | 14:46 |
| panda | rlandy: in all other definition, name is always "primary" or "secondary" not the label | 14:47 |
| rlandy | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/nodesets.yaml#L104 | 14:47 |
| rlandy | ^^ I think we should use that nodeset | 14:47 |
| *** skramaja has quit IRC | 14:47 | |
| rlandy | I don;t want to mess with upstream | 14:47 |
| rlandy | it's used elsewhere | 14:47 |
| rlandy | sorry | 14:48 |
| rlandy | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/nodesets.yaml#L87 | 14:48 |
| rlandy | panda: ^^ | 14:48 |
| *** ratailor has joined #oooq | 14:49 | |
| panda | rlandy: yeah, I see, yes, that's the right nodeset. But also, if it's a single node, we probably don't need to add it to group "switch", that's used to call the multi-node-bridge | 14:50 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/nodesets.yaml#L20 | 14:50 |
| rlandy | panda: ^^ same as | 14:50 |
| rlandy | https://review.rdoproject.org/r/#/c/16337/3/zuul.d/zuul-v3-jobs.yaml | 14:51 |
| rlandy | trying that | 14:51 |
| panda | mmhh, yeah I remember we needed to add peers group empty because otherwise the playbook was topping with an error | 14:52 |
| panda | I'm not sure if adding primary to switch is really nedded, and we could make that group empty as well, but at this point let's leave it like this | 14:52 |
| *** quiquell|rover is now known as quique|rover|off | 15:05 | |
| *** ykarel is now known as ykarel|away | 15:05 | |
| *** jtomasek has quit IRC | 15:10 | |
| chkumar|off | quique|rover|off: weshay patches are in progress for fs020 https://bugs.launchpad.net/tripleo/+bug/1793482/comments/4 but as a wrokaround | 15:13 |
| openstack | Launchpad bug 1793482 in tripleo "[Rocky][master]fs020 periodic job neutron tempest tests are failing by giving 503 Service Unavailable" [Critical,Triaged] | 15:13 |
| weshay | chkumar|off, thank you sir.. I'm watching them all :) | 15:13 |
| weshay | chkumar++ | 15:13 |
| chkumar|off | weshay: I think we need to come up with a beeter way to handle skip list and keep monitoring through toold | 15:14 |
| chkumar|off | tools | 15:14 |
| weshay | chkumar|off, k.. interested in hearing your thoughts | 15:16 |
| weshay | chkumar|off, tell me tomorrow | 15:16 |
| weshay | go to sleep | 15:16 |
| *** ykarel|away has quit IRC | 15:18 | |
| *** kopecmartin has quit IRC | 15:18 | |
| rlandy | panda: any idea why the playbook would just stop in the middle like that? | 15:19 |
| *** jfrancoa has joined #oooq | 15:20 | |
| panda | rlandy: which log ? | 15:20 |
| rlandy | https://logs.rdoproject.org/10/16310/3/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/7efa6a9/job-output.txt.gz#_2018-09-20_15_11_19_897140 | 15:21 |
| rlandy | maybe a sf thing | 15:21 |
| rlandy | RUN START: [untrusted : git.openstack.org/openstack-infra/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master] | 15:22 |
| *** kopecmartin has joined #oooq | 15:22 | |
| panda | mmhh | 15:22 |
| rlandy | I am asking on #sf-dfg | 15:22 |
| panda | the weirdest thing is that it's acutally saying the the run playbook ended normally | 15:30 |
| *** agopi has quit IRC | 15:39 | |
| *** sshnaidm|pto has joined #oooq | 15:44 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 15:47 |
| *** ssbarnea|bkp2 has joined #oooq | 15:51 | |
| *** rascasoft has joined #oooq | 16:03 | |
| *** jfrancoa has quit IRC | 16:10 | |
| *** ratailor has quit IRC | 16:17 | |
| *** kopecmartin is now known as kopecmartin|away | 16:21 | |
| weshay | ssbarnea, did you want to go through that patch any more? | 16:21 |
| *** ykarel|away has joined #oooq | 16:25 | |
| weshay | marios, you still around? | 16:26 |
| weshay | marios, if not that is fine | 16:26 |
| * weshay reviewing https://review.openstack.org/#/c/602416/6/roles/standalone/tasks/main.yml | 16:26 | |
| weshay | wondering if that is doing both the workflow and the upgrade | 16:27 |
| weshay | panda, fyi. this is interesting https://review.openstack.org/#/c/602745/1 | 16:33 |
| *** agopi has joined #oooq | 16:33 | |
| *** holser_ has quit IRC | 16:33 | |
| *** bogdando has quit IRC | 16:35 | |
| panda | weshay: not clear to me what this option is preventing to run | 16:37 |
| panda | chem: what is the noop option actually preventing ? | 16:38 |
| weshay | panda, don't understand what you are saying | 16:38 |
| weshay | panda, that is a check option for ansible | 16:38 |
| weshay | have you not run that? | 16:38 |
| chem | panda: as weshay say, but more precisely it runs --check --diff ansible-playbook options | 16:39 |
| chem | panda: weshay: as I commented this has been drop to very low in the priority list and maybe something more useful for the user than for ci | 16:40 |
| weshay | panda, https://docs.ansible.com/ansible/2.5/user_guide/playbooks_checkmode.html | 16:40 |
| weshay | chem, what does --diff do? /me looking | 16:40 |
| panda | chem: weshay https://docs.ansible.com/ansible/2.5/user_guide/playbooks_checkmode.html | 16:41 |
| chem | weshay: show diff for files it would change | 16:41 |
| panda | "Check mode is just a simulation, and if you have steps that use conditionals that depend on the results of prior commands, it may be less useful for you. However it is great for one-node-at-time basic configuration management use cases." | 16:41 |
| weshay | oh.. | 16:41 |
| weshay | that is cool | 16:41 |
| *** trown is now known as trown|lunch | 16:42 | |
| *** radez has quit IRC | 16:42 | |
| panda | also, if we have custom modules, and they don't implement the check, thwy will not show the diff | 16:42 |
| *** radez has joined #oooq | 16:43 | |
| panda | chem: how you imagine to check if the workflow is working correctly ? would be a manual debugging of the tasks ? | 16:44 |
| *** sshnaidm|pto has quit IRC | 16:44 | |
| *** sanjayu_ has joined #oooq | 16:50 | |
| rlandy | 2018-09-20 16:49:32.621419 | primary | +(/home/zuul/src/git.openstack.org/openstack/tripleo-ci/toci_gate_test.sh:20): source /etc/ci/mirror_info.sh | 16:52 |
| rlandy | 2018-09-20 16:49:32.621585 | primary | /home/zuul/src/git.openstack.org/openstack/tripleo-ci/toci_gate_test.sh: line 20: /etc/ci/mirror_info.sh: No such file or directory | 16:52 |
| rlandy | where is that ... | 16:52 |
| *** udesale has quit IRC | 16:52 | |
| rlandy | panda: ^^ | 16:53 |
| panda | rlandy: dang. it's created by the mirror-info role I just mentioned. Maybe it's included by default in openstack upstream | 16:54 |
| * rlandy looks | 16:55 | |
| rlandy | we need to call that explicitly | 16:55 |
| panda | why we don't need it in tripleo-ci upstream ? I'm trying to find where it is implicitly called | 16:58 |
| panda | maybe it's in the node itself, already baked in the image | 16:59 |
| rlandy | panda: we could just include the role in https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/run-v3.yaml#L61 | 17:00 |
| rlandy | if the file is not found | 17:00 |
| panda | chem: so the idea is to actually make tripleo run ansible --check for all the mistral playbooks ? So it will not install anything, finish earlier so we can continue with the next phase of the upgrade ? | 17:01 |
| panda | rlandy: let's try, I just want this to be idempotent with what we have in upstream ci, that's why I'd like to understand where, when and host it's created there | 17:02 |
| rlandy | panda: try call it in run-v3? | 17:03 |
| chem | panda: the idea was to just "display" what would be run and then get the log back and check that all the expected hooks in the upgrade workflow were run ... a mouthful | 17:03 |
| chem | panda: currently the plan is to go with N->N upgrade for CI | 17:03 |
| chem | panda: so no yum upgrade, no container pulling, just run the upgrade from N to N | 17:04 |
| chem | panda: maybe we add some dummy role/resource that doesn't do much as well to speed even more the deployment, but first just try to do a N->N | 17:05 |
| panda | rlandy: yes | 17:05 |
| chem | panda: it's seem easier to implement and has a good chance of working under the timeout | 17:05 |
| rlandy | panda: https://review.openstack.org/#/c/604150/3/playbooks/tripleo-ci/run-v3.yaml | 17:07 |
| rlandy | let's try it | 17:07 |
| *** gkadam has joined #oooq | 17:14 | |
| panda | rlandy: we should be good, in upstream ci, that role is included by the base base base job configuration, so al long as we're calling that same role, we'll end up in the same spot. | 17:18 |
| rlandy | let's hope | 17:19 |
| panda | rlandy: so it's essential for rdo, it's harmless for openstack | 17:19 |
| panda | rlandy: we har inheriting from multinode base job, multinode base job is not inheriting from anything, but it's implicit that it's inheriting from the base base base job configured in project-config | 17:20 |
| panda | har | 17:26 |
| * panda facepalm | 17:26 | |
| panda | rlandy: http://logs.openstack.org/77/603277/4/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/2c7fbab/job-output.txt.gz#_2018-09-18_15_51_56_979960 | 17:27 |
| panda | rlandy: that's where we are calling mirror-info in upstream | 17:27 |
| rlandy | well if it's no harm. ok | 17:27 |
| panda | rlandy: one objection may be that the role should be added in our pre playbooks | 17:28 |
| panda | it shouldn't really be "run" time | 17:28 |
| panda | rlandy: but don't bother for now moving it, it's actually taking 2 seconds to run | 17:29 |
| rlandy | pnda; ok - I'll change that as soon as we see the run | 17:29 |
| rlandy | panda: | 17:29 |
| panda | rlandy: but we may need to add configure-mirrors too | 17:29 |
| rlandy | we may need to change something else | 17:30 |
| rlandy | let's see | 17:30 |
| panda | rlandy: I see it running right after mirror-info in upstream | 17:30 |
| rlandy | fine | 17:30 |
| * rlandy makes that change - to commit afterwards | 17:30 | |
| panda | rlandy: let's see if it's needed, I'm not sure | 17:31 |
| *** dtantsur is now known as dtantsur|afk | 17:40 | |
| rlandy | mirror_fqdn is undefine | 17:44 |
| rlandy | panda: ^^ | 17:44 |
| rlandy | need to fix that so can move roles | 17:44 |
| panda | rlandy: logs ? | 17:45 |
| panda | rlandy: I see most of the time it takes the same value as zuul_site_mirror_fqdn | 17:46 |
| rlandy | https://logs.rdoproject.org/10/16310/4/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/6367f0a/job-output.txt.gz#_2018-09-20_17_43_43_956074\ | 17:46 |
| panda | trying to find when this is defined upstream | 17:46 |
| rlandy | panda: we have two options to move the role ... | 17:46 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/openstack-zuul-jobs/legacy/pre.yaml | 17:47 |
| rlandy | or | 17:47 |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 17:47 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/nodepool-provider/pre.yaml | 17:47 |
| rlandy | panda: which one is your pick? | 17:47 |
| panda | rlandy: in project-config zuul/site-variables.yaml:zuul_site_mirror_fqdn: "mirror.{{ nodepool.region | lower }}.{{ nodepool.cloud | lower }}.openstack.org" | 17:47 |
| panda | rlandy: we need to change the value to contain rdo mirrors | 17:48 |
| panda | rlandy: I wouldn't use openstack-zuul-jobs/legacy/pre.yaml if we plan to remove it completely | 17:48 |
| rlandy | https://logs.rdoproject.org/10/16310/4/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/6367f0a/job-output.txt.gz#_2018-09-20_17_42_14_640595 | 17:48 |
| panda | not we had much success .. but the general idea is to remove it .. | 17:49 |
| rlandy | configure-mirrors is run | 17:49 |
| panda | rlandy: ok | 17:49 |
| *** rook has joined #oooq | 17:50 | |
| rlandy | 2018-09-20 17:42:16.091004 | primary | ok: "/tmp/tmpl1av7z79/6367f0afa552436bb29da6b0663aa2cb/untrusted/project_2/git.openstack.org/openstack-infra/zuul-jobs/roles/configure-mirrors/vars/CentOS.yaml" | 17:51 |
| panda | rlandy: continuin the discussion in sf-org | 17:53 |
| *** trown|lunch is now known as trown | 18:06 | |
| *** tosky has quit IRC | 18:32 | |
| *** ykarel|away has quit IRC | 18:41 | |
| *** ssbarnea|bkp2 has quit IRC | 18:48 | |
| agopi | https://code.engineering.redhat.com/gerrit/#/c/150545/ rlandy can you please review+merge when you get time | 18:51 |
| *** marios has quit IRC | 19:01 | |
| *** dtantsur|afk has quit IRC | 19:01 | |
| *** panda is now known as panda|dinner | 19:34 | |
| *** jtomasek has joined #oooq | 19:41 | |
| rlandy | agopi: done | 19:41 |
| agopi | thanks rlandy :) | 19:42 |
| *** jtomasek has quit IRC | 19:47 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 19:47 |
| *** dsneddon has quit IRC | 20:02 | |
| *** dsneddon has joined #oooq | 20:02 | |
| *** gkadam has quit IRC | 20:10 | |
| *** ssbarnea|bkp2 has joined #oooq | 20:14 | |
| *** jtomasek has joined #oooq | 20:17 | |
| *** panda|dinner is now known as panda | 20:24 | |
| *** jtomasek has quit IRC | 20:27 | |
| panda | rlandy: assuming the job will work, what are the next steps ? | 20:33 |
| panda | rlandy: moving the legacy jobs ? | 20:34 |
| rlandy | panda: we put back the review to add the browbeat job to actually trigger | 20:34 |
| rlandy | then we add a multinode job and base and try that | 20:35 |
| rlandy | and then we mass add v3 jobs | 20:35 |
| rlandy | watch them run | 20:35 |
| rlandy | and then delete legacy | 20:35 |
| panda | ok | 20:40 |
| panda | i like the mass add part :) | 20:40 |
| *** ssbarnea|bkp2 has quit IRC | 20:42 | |
| agopi | rlandy, since we're on the mass add part :P can you please look at https://review.openstack.org/#/c/603884/ | 20:42 |
| agopi | not sure why its failing | 20:42 |
| agopi | i just added a change to fs53 file | 20:43 |
| agopi | it says update_containers_append_tag is undefined | 20:43 |
| *** holser_ has joined #oooq | 20:49 | |
| *** trown is now known as trown|outtypewww | 20:56 | |
| *** agopi is now known as agopi|brb | 21:12 | |
| *** agopi|brb has quit IRC | 21:18 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 21:47 |
| rlandy | weshay: when you reviewed https://review.openstack.org/#/c/602673, did you try the reproducer? | 21:55 |
| * rlandy is thinking about what happens to the logs dir there | 21:55 | |
| weshay | hrm.. | 21:55 |
| weshay | do we want / need that for the repo? | 21:56 |
| * rlandy will have to try I guess | 21:56 | |
| weshay | or just ensure the zuul_changes is set | 21:56 |
| rlandy | well if there are no zuul changes defined? | 21:56 |
| * rlandy tries | 21:57 | |
| weshay | any of the zuul variables need to set in the script for sure | 21:57 |
| weshay | meh.. short answer no | 21:57 |
| weshay | I did not try | 21:57 |
| weshay | I've been focused on the gate and f28 | 21:57 |
| rlandy | Im trying with libvirt reproducer | 21:57 |
| weshay | oh what fun | 21:57 |
| panda | that's it! I can't take this manual running for the reproducer anymore, I'm adding a reproducer job in openstack infra RIGHT NOW! | 21:59 |
| *** holser_ has quit IRC | 22:00 | |
| *** toure is now known as toure|gone | 22:06 | |
| rlandy | panda: need to step out - will be back later | 22:37 |
| *** rlandy is now known as rlandy|bbl | 22:37 | |
| panda | rlandy|bbl: ok | 22:37 |
| *** ssbarnea|bkp2 has joined #oooq | 22:55 | |
| *** sanjayu_ has quit IRC | 23:23 | |
| *** panda is now known as panda|off | 23:39 | |
| hubbot1 | FAILING CHECK JOBS on stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-queens, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ (2 more messages) | 23:47 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!