| *** rfolco has joined #oooq | 00:07 | |
| *** tosky has quit IRC | 00:10 | |
| *** weshay is now known as weshay_pto | 00:38 | |
| *** rfolco has quit IRC | 00:50 | |
| *** ysandeep has quit IRC | 00:54 | |
| *** ysandeep has joined #oooq | 01:14 | |
| *** rfolco has joined #oooq | 01:29 | |
| *** rfolco has quit IRC | 02:12 | |
| *** ykarel|away has joined #oooq | 02:27 | |
| *** ykarel|away has quit IRC | 02:32 | |
| *** brault has quit IRC | 03:06 | |
| *** apetrich has quit IRC | 03:09 | |
| *** rlandy|bbl is now known as rlandy | 03:19 | |
| *** rlandy has quit IRC | 03:23 | |
| *** udesale has joined #oooq | 03:48 | |
| *** ykarel|away has joined #oooq | 03:51 | |
| *** ykarel|away is now known as ykarel | 03:51 | |
| raukadah | zbr_: sshnaidm mock is not needed when rpmbuild is in play | 04:24 |
|---|---|---|
| *** bhagyashris has joined #oooq | 04:33 | |
| *** soniya29 has joined #oooq | 05:05 | |
| *** bhagyashris has quit IRC | 05:05 | |
| *** epoojad1 has joined #oooq | 05:06 | |
| *** bhagyashris has joined #oooq | 05:26 | |
| *** raukadah is now known as chkumar|ruck | 05:45 | |
| *** marios|rover has joined #oooq | 06:02 | |
| *** udesale has quit IRC | 06:13 | |
| *** udesale has joined #oooq | 06:14 | |
| *** jtomasek has joined #oooq | 06:18 | |
| *** surpatil has joined #oooq | 06:23 | |
| *** ykarel_ has joined #oooq | 06:42 | |
| *** ykarel has quit IRC | 06:45 | |
| chkumar|ruck | marios|rover: Good morning sir | 06:49 |
| chkumar|ruck | marios|rover: https://review.opendev.org/#/c/697285/1 will fix queens promotion | 06:49 |
| marios|rover | chkumar|ruck: train | 06:49 |
| marios|rover | chkumar|ruck: looking at it now | 06:49 |
| chkumar|ruck | yes sorry train promotion | 06:49 |
| marios|rover | chkumar|ruck: morning to you too sir | 06:50 |
| chkumar|ruck | :-) | 06:50 |
| *** jfrancoa has joined #oooq | 06:53 | |
| *** ccamacho has quit IRC | 06:54 | |
| *** ykarel__ has joined #oooq | 07:01 | |
| *** udesale has quit IRC | 07:02 | |
| *** udesale has joined #oooq | 07:03 | |
| *** ykarel_ has quit IRC | 07:03 | |
| *** udesale has quit IRC | 07:06 | |
| *** udesale has joined #oooq | 07:06 | |
| *** ykarel__ is now known as ykarel | 07:08 | |
| chkumar|ruck | Tengu: Hello mate | 07:28 |
| chkumar|ruck | Tengu: regarding this card https://trello.com/c/MXbsfyLT/1242-cixlp1853978tripleociproa-periodic-rhel8-ovb-overcloud-deployment-failed-with-could-not-find-class-tripleoprofilebaseneutronovnm do we need to backport https://review.opendev.org/#/c/694525/ to train also ? | 07:29 |
| *** jbadiapa has joined #oooq | 07:31 | |
| Tengu | chkumar|ruck: heya! wait a second :) | 07:31 |
| Tengu | chkumar|ruck: I'd say "no", because https://review.opendev.org/#/c/693589/ is master only apparently. | 07:32 |
| chkumar|ruck | Tengu: ok | 07:37 |
| *** ykarel is now known as ykarel|lunch | 07:37 | |
| *** migi_sleep is now known as migi | 07:41 | |
| Tengu | chkumar|ruck: if you want to be sure, you might wait for Emilien - but I'm pretty sure it will be the same answer :) | 07:41 |
| *** saneax has joined #oooq | 07:50 | |
| marios|rover | chkumar|ruck: nice to see some green rhel check jobs ! finally ;) | 08:02 |
| *** epoojad1 has quit IRC | 08:03 | |
| chkumar|ruck | marios|rover: yes | 08:12 |
| *** tesseract has joined #oooq | 08:21 | |
| *** amoralej|off is now known as amoralej | 08:25 | |
| *** ykarel|lunch is now known as ykarel | 08:27 | |
| *** udesale has quit IRC | 08:31 | |
| *** udesale has joined #oooq | 08:32 | |
| *** tosky has joined #oooq | 08:36 | |
| *** SurajPatil has joined #oooq | 08:41 | |
| *** surpatil has quit IRC | 08:44 | |
| *** surpatil has joined #oooq | 08:46 | |
| *** SurajPatil has quit IRC | 08:49 | |
| *** jpena|off is now known as jpena | 08:50 | |
| *** apetrich has joined #oooq | 08:51 | |
| *** udesale has quit IRC | 08:58 | |
| *** soniya29 has quit IRC | 09:06 | |
| *** soniya29 has joined #oooq | 09:07 | |
| soniya29 | sshnaidm, Can you have look over this patch https://review.rdoproject.org/r/#/c/23933/? | 09:08 |
| sshnaidm | soniya29, sure | 09:09 |
| *** udesale has joined #oooq | 09:12 | |
| chkumar|ruck | marios|rover: ykarel and me find out the issue related to fs01 train rhel8 | 09:14 |
| soniya29 | sshnaidm, thanks | 09:14 |
| *** SurajPatil has joined #oooq | 09:15 | |
| chkumar|ruck | sshnaidm: marios|rover https://review.opendev.org/#/c/697423/ | 09:15 |
| marios|rover | chkumar|ruck: nice :D | 09:16 |
| marios|rover | chkumar|ruck: will fix the wrong packages issue? | 09:16 |
| marios|rover | chkumar|ruck: fs1 | 09:16 |
| chkumar|ruck | marios|rover: yes, let's test that | 09:16 |
| marios|rover | chkumar|ruck: hmmm but not sure we have release versions on the rcm guest | 09:16 |
| marios|rover | chkumar|ruck: ... i can check that let me bringup rdo vm though | 09:16 |
| *** surpatil has quit IRC | 09:17 | |
| *** surpatil has joined #oooq | 09:18 | |
| marios|rover | chkumar|ruck: (ah yeah we don't own the /images directory must be fine but vm spawning anyway ) | 09:18 |
| *** SurajPatil has quit IRC | 09:20 | |
| chkumar|ruck | Tengu: ^^ | 09:23 |
| Tengu | ah, might indeed make things easier :) | 09:26 |
| *** ccamacho has joined #oooq | 09:42 | |
| *** derekh has joined #oooq | 09:43 | |
| *** ccamacho has quit IRC | 09:54 | |
| *** saneax has quit IRC | 09:54 | |
| *** ykarel has quit IRC | 09:54 | |
| *** jtomasek has quit IRC | 09:54 | |
| *** hamzy has quit IRC | 09:54 | |
| *** jschlueter has quit IRC | 09:54 | |
| *** dmellado has quit IRC | 09:54 | |
| *** d0ugal has quit IRC | 09:54 | |
| *** EmilienM has quit IRC | 09:54 | |
| *** surpatil has quit IRC | 09:54 | |
| *** soniya29 has quit IRC | 09:54 | |
| *** rnoriega_ has quit IRC | 09:54 | |
| *** migi has quit IRC | 09:54 | |
| *** jpena has quit IRC | 09:54 | |
| *** jfrancoa has quit IRC | 09:54 | |
| *** marios|rover has quit IRC | 09:54 | |
| *** bhagyashris has quit IRC | 09:54 | |
| *** fmount has quit IRC | 09:54 | |
| *** chem has quit IRC | 09:54 | |
| *** dtantsur|afk has quit IRC | 09:54 | |
| *** radez has quit IRC | 09:54 | |
| *** tosky has quit IRC | 09:54 | |
| *** tesseract has quit IRC | 09:54 | |
| *** ysandeep has quit IRC | 09:54 | |
| *** yolanda has quit IRC | 09:54 | |
| *** dtrainor has quit IRC | 09:54 | |
| *** matbu has quit IRC | 09:54 | |
| *** baha has quit IRC | 09:55 | |
| *** irclogbot_1 has quit IRC | 09:55 | |
| *** jjoyce has quit IRC | 09:55 | |
| *** mjturek has quit IRC | 09:55 | |
| *** Goneri has quit IRC | 09:55 | |
| *** Tengu has quit IRC | 09:55 | |
| *** gouthamr has quit IRC | 09:55 | |
| *** sshnaidm has quit IRC | 09:55 | |
| *** amoralej has quit IRC | 09:55 | |
| *** beagles has quit IRC | 09:55 | |
| *** derekh has quit IRC | 09:55 | |
| *** udesale has quit IRC | 09:55 | |
| *** jbadiapa has quit IRC | 09:55 | |
| *** skramaja has quit IRC | 09:55 | |
| *** ksambor has quit IRC | 09:55 | |
| *** panda has quit IRC | 09:55 | |
| *** kopecmartin has quit IRC | 09:55 | |
| *** weshay_ has quit IRC | 09:55 | |
| *** chkumar|ruck has quit IRC | 09:55 | |
| *** tristanC has quit IRC | 09:55 | |
| *** arxcruz has quit IRC | 09:55 | |
| *** rodrigods has quit IRC | 09:55 | |
| *** apetrich has quit IRC | 09:55 | |
| *** zbr_ has quit IRC | 09:55 | |
| *** gchamoul has quit IRC | 09:55 | |
| *** ssbarnea has quit IRC | 09:55 | |
| *** weshay_pto has quit IRC | 09:55 | |
| *** ChanServ has quit IRC | 09:55 | |
| *** ykarel has joined #oooq | 10:04 | |
| *** dmellado has joined #oooq | 10:04 | |
| *** jschlueter has joined #oooq | 10:04 | |
| *** hamzy has joined #oooq | 10:04 | |
| *** jtomasek has joined #oooq | 10:04 | |
| *** saneax has joined #oooq | 10:04 | |
| *** ccamacho has joined #oooq | 10:04 | |
| *** epoojad1 has joined #oooq | 10:04 | |
| *** EmilienM has joined #oooq | 10:04 | |
| *** d0ugal has joined #oooq | 10:04 | |
| *** jpena has joined #oooq | 10:04 | |
| *** rnoriega_ has joined #oooq | 10:04 | |
| *** migi has joined #oooq | 10:04 | |
| *** surpatil has joined #oooq | 10:04 | |
| *** derekh has joined #oooq | 10:04 | |
| *** apetrich has joined #oooq | 10:04 | |
| *** tosky has joined #oooq | 10:04 | |
| *** tesseract has joined #oooq | 10:04 | |
| *** jbadiapa has joined #oooq | 10:04 | |
| *** ysandeep has joined #oooq | 10:04 | |
| *** Goneri has joined #oooq | 10:04 | |
| *** zbr_ has joined #oooq | 10:04 | |
| *** yolanda has joined #oooq | 10:04 | |
| *** skramaja has joined #oooq | 10:04 | |
| *** Tengu has joined #oooq | 10:04 | |
| *** dtrainor has joined #oooq | 10:04 | |
| *** gchamoul has joined #oooq | 10:04 | |
| *** ksambor has joined #oooq | 10:04 | |
| *** matbu has joined #oooq | 10:04 | |
| *** panda has joined #oooq | 10:04 | |
| *** ssbarnea has joined #oooq | 10:04 | |
| *** gouthamr has joined #oooq | 10:04 | |
| *** tristanC has joined #oooq | 10:04 | |
| *** kopecmartin has joined #oooq | 10:04 | |
| *** sshnaidm has joined #oooq | 10:04 | |
| *** baha has joined #oooq | 10:04 | |
| *** irclogbot_1 has joined #oooq | 10:04 | |
| *** chkumar|ruck has joined #oooq | 10:04 | |
| *** weshay_pto has joined #oooq | 10:04 | |
| *** weshay_ has joined #oooq | 10:04 | |
| *** amoralej has joined #oooq | 10:04 | |
| *** jjoyce has joined #oooq | 10:04 | |
| *** arxcruz has joined #oooq | 10:04 | |
| *** mjturek has joined #oooq | 10:04 | |
| *** orwell.freenode.net sets mode: +o arxcruz | 10:04 | |
| *** beagles has joined #oooq | 10:04 | |
| *** rodrigods has joined #oooq | 10:04 | |
| *** ChanServ has joined #oooq | 10:04 | |
| *** orwell.freenode.net sets mode: +o ChanServ | 10:04 | |
| *** jfrancoa has joined #oooq | 10:04 | |
| *** marios|rover has joined #oooq | 10:04 | |
| *** bhagyashris has joined #oooq | 10:04 | |
| *** fmount has joined #oooq | 10:04 | |
| *** chem has joined #oooq | 10:04 | |
| *** dtantsur|afk has joined #oooq | 10:04 | |
| *** radez has joined #oooq | 10:04 | |
| *** dtantsur|afk is now known as dtantsur | 10:06 | |
| chkumar|ruck | marios|rover: http://logs.rdoproject.org/25/23825/4/check/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-train/d8f1df2/job-output.txt | 10:09 |
| chkumar|ruck | marios|rover: ASK [fetch-images : Get image expected checksum] **** | 10:10 |
| marios|rover | chkumar|ruck: ack cool | 10:10 |
| chkumar|ruck | marios|rover: there is some issue there please check | 10:10 |
| marios|rover | chkumar|ruck: ah i thought you mean its passing looking | 10:10 |
| marios|rover | chkumar|ruck: no idea where is that coming from ? /me digging in queens issue at the moment i can have a look in a bit | 10:12 |
| chkumar|ruck | marios|rover: on running the same command on rdocloud vm | 10:12 |
| chkumar|ruck | exit status 22 | 10:12 |
| chkumar|ruck | ok no problem | 10:12 |
| marios|rover | chkumar|ruck: here looks like https://opendev.org/openstack/tripleo-quickstart/src/branch/master/roles/fetch-images/tasks/fetch.yml#L44 | 10:13 |
| *** ysandeep has quit IRC | 10:13 | |
| chkumar|ruck | marios|rover: we need to check whether image is there or not actually | 10:15 |
| chkumar|ruck | rdo_trunk namespace is problematic | 10:15 |
| *** ykarel is now known as ykarel|afk | 10:27 | |
| *** skramaja_ has joined #oooq | 10:32 | |
| *** skramaja has quit IRC | 10:33 | |
| *** skramaja_ is now known as skramaja | 10:33 | |
| panda | marios|rover: chkumar|ruck can you point me at any bug you have on centos7 queens ? | 11:07 |
| zbr_ | while we fixed one annoying dnf bug in ansible we missed yesterday maintenance releases, https://github.com/ansible/ansible/pull/65302 | 11:07 |
| *** ykarel|afk is now known as ykarel | 11:07 | |
| zbr_ | one f* comma | 11:08 |
| panda | zbr_: f* is a pointer, not a comma. | 11:08 |
| chkumar|ruck | panda: https://bugs.launchpad.net/tripleo/+bug/1855120 | 11:14 |
| openstack | Launchpad bug 1855120 in tripleo "periodic- featureset019-queens failing during overcloud deploy with error in tasks: ceph_base_ansible_workflow" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 11:14 |
| chkumar|ruck | panda: need one help | 11:14 |
| marios|rover | panda: o/ heh i was literally working on that bug just now | 11:15 |
| chkumar|ruck | panda: sorry not right now, job failed at different place | 11:16 |
| marios|rover | panda: only one am aware of though for Q 1855120 | 11:16 |
| *** rfolco has joined #oooq | 11:17 | |
| *** rfolco has quit IRC | 11:17 | |
| *** rfolco has joined #oooq | 11:18 | |
| panda | marios|rover: chkumar|ruck thanks. Anything I can do ? redhat8-train is promoted with the new code. only centos7 queens left to test, and if we don't get a good candidate soon, it's best that I start working on the alternative production testing method. | 11:22 |
| *** saneax has quit IRC | 11:22 | |
| marios|rover | panda: nice progress :) | 11:22 |
| chkumar|ruck | arxcruz: kopecmartin surpatil weshay_pto and I will be not around tomororw can we cancel the tomorrow's tempest meeting? | 11:23 |
| marios|rover | panda: so centos7 queens might promote if that passes https://review.rdoproject.org/r/#/c/23932/ does it help you? | 11:23 |
| *** saneax has joined #oooq | 11:23 | |
| surpatil | ok | 11:23 |
| chkumar|ruck | marios|rover: regarding redhat8-train got promoted day before yesterday na? | 11:24 |
| marios|rover | panda: chkumar|ruck: err.. why is there no queens log there after 3rd december? http://38.145.34.55/ | 11:25 |
| marios|rover | panda: chkumar|ruck: hmm so maybe queens wont promote then depending on https://review.rdoproject.org/r/#/c/23932/ :D | 11:25 |
| marios|rover | but why is it off? | 11:25 |
| panda | marios|rover: because it's in the new server | 11:26 |
| panda | marios|rover: I had to block it on the old to be handled in the new | 11:26 |
| marios|rover | panda: you moved it over on the 3rd and stopped it on the old one | 11:26 |
| marios|rover | panda: sure but i thought you would have told ruck rover about that? | 11:26 |
| marios|rover | panda: i thought you were working on train only | 11:26 |
| marios|rover | panda: so on the new server then... can you see latest logs? should be just fs19 missing? | 11:27 |
| chkumar|ruck | panda: Do we have the logs up for new server? | 11:27 |
| panda | marios|rover: that's what was decided on monday. RedHat8 train, and centos7 queens will be used to test the new promoter. I disabled them in the old, and moved it in the new, so the next good candidate that shows up, will be used for the new code. | 11:27 |
| panda | marios|rover: yes, only f019 is missing | 11:27 |
| panda | chkumar|ruck: yes, they are in the same position as the old. Only difference is taht they are run manually. | 11:28 |
| marios|rover | panda: hm not sure i agree with 'that what was decided on monday' my understanding at least was you would look at train first and see... but ok | 11:28 |
| marios|rover | panda: ack ok cool well hopefully we get that promoted soon then | 11:28 |
| marios|rover | panda: seems to have lined up nicely | 11:28 |
| marios|rover | panda: depends on that https://review.rdoproject.org/r/#/c/23932/ or whatever the fix for the bug will be | 11:29 |
| panda | marios|rover: I didn't want to risk having a good candidate slip away. | 11:29 |
| marios|rover | panda: yeah but i mean, without any coordination, i randomly decided 'well lets check queens' cos master/train were fcucked yesterday anyway (and day before i looked at rocky/stein)... anyway point is i filed that fs19 yesterday it was ongoing for few days an we didn't notice | 11:30 |
| marios|rover | panda: and then this morning ping gfidente ... all to line up the promotion for panda :D | 11:30 |
| marios|rover | panda: are you secretly stalking me? | 11:31 |
| marios|rover | panda: (and then you asked me if we had a queens bug the moment i was hitting return to post https://review.opendev.org/#/c/697442/ (my god you ARE stalking me ) | 11:32 |
| * marios|rover looks around nervously | 11:32 | |
| panda | marios|rover: who me ? who cares about an engineer that wakes up at 630, brings kids to school at 8 with his small recently repaired car ? | 11:32 |
| panda | O_O | 11:33 |
| marios|rover | ! OMG | 11:33 |
| openstack | marios|rover: Error: "OMG" is not a valid command. | 11:33 |
| marios|rover | :D | 11:33 |
| *** derekh has quit IRC | 11:33 | |
| *** SurajPatil has joined #oooq | 11:33 | |
| EmilienM | Tengu: what's up? | 11:34 |
| chkumar|ruck | !! | 11:34 |
| openstack | chkumar|ruck: Error: "!" is not a valid command. | 11:34 |
| chkumar|ruck | EmilienM: Hello | 11:34 |
| chkumar|ruck | EmilienM: I think I found the issue related to this bug https://bugs.launchpad.net/tripleo/+bug/1853978 | 11:34 |
| openstack | Launchpad bug 1853978 in tripleo "periodic train rhel8 ovb overcloud deployment failed with Could not find class ::tripleo::profile::base::neutron::ovn_metadata_agent_wrappers" [Critical,Triaged] | 11:34 |
| *** jbadiapa has quit IRC | 11:34 | |
| chkumar|ruck | EmilienM: it was using master overcloud image not train one | 11:35 |
| *** surpatil has quit IRC | 11:36 | |
| EmilienM | that's what I was saying I guess :D | 11:36 |
| *** SurajPatil has quit IRC | 11:38 | |
| *** epoojad1 has quit IRC | 11:41 | |
| chkumar|ruck | marios|rover: going to merge this https://review.rdoproject.org/r/#/c/23712/ if ok? | 11:47 |
| *** fmount has quit IRC | 11:48 | |
| marios|rover | chkumar|ruck: no ronelle is working there | 11:49 |
| marios|rover | chkumar|ruck: wait for her maybe | 11:49 |
| chkumar|ruck | marios|rover: ok | 11:49 |
| marios|rover | chkumar|ruck: i started it like last sprint but then was rover so rlandy took it | 11:49 |
| *** fmount has joined #oooq | 11:49 | |
| chkumar|ruck | ok thanks for the info | 11:49 |
| *** udesale has joined #oooq | 11:51 | |
| *** fmount has quit IRC | 11:51 | |
| *** fmount has joined #oooq | 11:53 | |
| marios|rover | votes on these please easy patches https://review.opendev.org/#/c/697413/1 https://review.opendev.org/#/c/695878/1 https://review.opendev.org/#/c/696870/ https://review.opendev.org/#/c/696874/ | 11:54 |
| *** soniya29 has joined #oooq | 11:57 | |
| panda | marios|rover: I don't even more a finger if you're not asking for at least 10 reviews at a time. | 11:58 |
| * marios|rover food biab | 12:03 | |
| zbr_ | podman rpm card is ready for reviews: https://tree.taiga.io/project/tripleo-ci-board/task/1349?kanban-status=1447275 -- please help here, especillay chkumar|ruck and arxcruz which know better the background. | 12:04 |
| arxcruz | zbr_: sure | 12:04 |
| chkumar|ruck | zbr_: you got comments on pr | 12:04 |
| arxcruz | i'll do it, but getting ready for meetings | 12:04 |
| zbr_ | i will have to skip todays scrum, so i am trying to update tickets and assure someone else can speak for me. | 12:05 |
| chkumar|ruck | marios|rover: I have a situation related to rhel8 train container build which got timed out right now | 12:06 |
| chkumar|ruck | ovn-northd container has taken 36 mins and leading to timedout | 12:07 |
| chkumar|ruck | and but in previous successful job it takes just mins | 12:08 |
| chkumar|ruck | rerunning the job | 12:08 |
| chkumar|ruck | marios|rover: if the job still timeout will open a bug | 12:09 |
| *** zbr_ is now known as zbr|out | 12:13 | |
| *** jpena is now known as jpena|lunch | 12:22 | |
| marios|rover | chkumar|ruck: ack | 12:25 |
| chkumar|ruck | EmilienM: zbr|out arxcruz panda https://github.com/containers/libpod/pull/4601#issuecomment-562105199 | 12:26 |
| EmilienM | fun :) | 12:27 |
| chkumar|ruck | EmilienM: more fun is coming soon | 12:27 |
| EmilienM | :-O | 12:27 |
| EmilienM | chkumar|ruck++ | 12:27 |
| arxcruz | fancy | 12:33 |
| chkumar|ruck | inteesting thing vote is not counted | 12:34 |
| rfolco | scrum? | 13:00 |
| rfolco | https://meet.google.com/oiv-geho-mai | 13:00 |
| *** holser has joined #oooq | 13:00 | |
| rfolco | sshnaidm, zbr|out arxcruz marios|rover chkumar|ruck panda rl...oops | 13:00 |
| *** rlandy has joined #oooq | 13:01 | |
| marios|rover | https://review.opendev.org/#/c/697442/ please vote there panda | 13:13 |
| *** jpena|lunch is now known as jpena | 13:14 | |
| panda | marios|rover: job in https://review.rdoproject.org/r/23932 fakes periodic, but desn't vote to dlrnapi | 13:20 |
| panda | marios|rover: So I'll manually vote to dlrnapi, then we can proceed with the promotion | 13:21 |
| marios|rover | panda: it should vote | 13:21 |
| marios|rover | panda: i mean i've had promotions before based on those | 13:21 |
| panda | marios|rover: don't see it here https://trunk.rdoproject.org/api-centos-queens/api/civotes_detail.html?commit_hash=982cf62717df4e6a9321f6aa7e309b2f2d5fbe25&distro_hash=5fce6f98d3d099915f6a32acb3b103de60d42e7e | 13:22 |
| marios|rover | panda: and you can see in the promoter logs when it gathers successful jobs, it references that change | 13:22 |
| panda | marios|rover: maybe it voted on a different hash ? | 13:22 |
| marios|rover | panda: you should be able to find it in the promoter logs | 13:22 |
| marios|rover | panda: like when it gathers the list of success... the fs19 should be from that change | 13:22 |
| panda | marios|rover: not if it's not in this list https://trunk.rdoproject.org/api-centos-queens/api/civotes_detail.html?commit_hash=982cf62717df4e6a9321f6aa7e309b2f2d5fbe25&distro_hash=5fce6f98d3d099915f6a32acb3b103de60d42e7e | 13:23 |
| marios|rover | panda: yeah you can see there | 13:23 |
| marios|rover | https://logs.rdoproject.org/32/23932/3/check/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-queens/ad35f18 | 13:23 |
| marios|rover | panda: ^^ 23932 | 13:23 |
| marios|rover | panda: isn't that what we need? | 13:24 |
| panda | marios|rover: damn, I thoght the list was in reverse time order | 13:24 |
| panda | marios|rover: so we are good. | 13:24 |
| *** Goneri has quit IRC | 13:24 | |
| marios|rover | panda: ack | 13:24 |
| panda | marios|rover: ignore me for the next 24h | 13:24 |
| marios|rover | panda: ack | 13:25 |
| panda | marios|rover: as punishment | 13:25 |
| marios|rover | panda: ack | 13:25 |
| *** bhagyashris has quit IRC | 13:27 | |
| *** udesale has quit IRC | 13:30 | |
| *** amoralej is now known as amoralej|lunch | 13:32 | |
| rlandy | marios|rover: chkumar|ruck: ok if I pull the trigger on https://review.rdoproject.org/r/#/c/23712/? | 13:32 |
| rlandy | feel free to revert if I messed anything up | 13:33 |
| marios|rover | rlandy: ack | 13:33 |
| rlandy | here we go - thanks | 13:34 |
| panda | rfolco: "P"+(![]+[])[+!+[]]+([]+[][[]]).slice(1,3)+(![]+[])[+!+[]] | 13:34 |
| rlandy | migi: hi | 13:35 |
| rlandy | migi: had a chat with weshay yesterday | 13:35 |
| rlandy | he clarified 'test from source' vs rpm with me | 13:35 |
| chkumar|ruck | panda: rlandy marios|rover sshnaidm we need this patch https://review.opendev.org/#/c/697423/ to clear the RHEL8 train promotion | 13:35 |
| rlandy | according to what he said you guys discussed, we should only have rpm testing in eth zuul-defined pipeline, | 13:35 |
| rfolco | panda, not very fan of js | 13:36 |
| chkumar|ruck | overcloud deployment is going on right now | 13:36 |
| rlandy | migi: have an interview in 30 mins - will ping you after that | 13:36 |
| rlandy | chkumar|ruck: if vexxhost the same? | 13:37 |
| marios|rover | chkumar|ruck: ack but update the test syntax error https://review.rdoproject.org/r/#/c/23825/ | 13:37 |
| panda | chkumar|ruck: I promoted train yesterday waiving almost all criteria, if you wonder why it passed. | 13:37 |
| marios|rover | panda: he is talking about centos | 13:38 |
| marios|rover | ? | 13:38 |
| marios|rover | or maybe not | 13:38 |
| marios|rover | panda: yeah its for centos bug | 13:38 |
| chkumar|ruck | rlandy: revert needs fix | 13:38 |
| rlandy | chkumar|ruck: see comment - missing dlrn_hash | 13:38 |
| *** udesale has joined #oooq | 13:39 | |
| panda | marios|rover: he said RHEL8 ... | 13:39 |
| marios|rover | panda: no actually its for rhel :D | 13:39 |
| marios|rover | panda: so SHOOT ME | 13:39 |
| panda | marios|rover: I'll ignore you for the next 24h as punishment. | 13:39 |
| marios|rover | brb | 13:39 |
| rlandy | chkumar|ruck: revert which patch? | 13:39 |
| chkumar|ruck | rlandy: check the error https://review.rdoproject.org/r/#/c/23825/ | 13:39 |
| chkumar|ruck | rlandy: we merged the component patch right now in config repo | 13:40 |
| chkumar|ruck | Job periodic-tripleo-ci-centos-7-standalone-compute-master not defined | 13:40 |
| rlandy | ok - reverting - will fix | 13:40 |
| chkumar|ruck | one min | 13:40 |
| chkumar|ruck | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/standalone-jobs.yaml#L541 is here | 13:41 |
| chkumar|ruck | rlandy: it was a merge clash | 13:43 |
| chkumar|ruck | no need to revert | 13:43 |
| rlandy | chkumar|ruck: k - what do you want me to do now? | 13:43 |
| *** soniya29 has quit IRC | 13:44 | |
| rlandy | chkumar|ruck: watching the get_hash in your test job | 13:44 |
| chkumar|ruck | rlandy: when the component patch merged, then the periodic job running there in rdojob with my patch get hitted with synced error | 13:44 |
| rlandy | ok | 13:44 |
| chkumar|ruck | so I told about revert but on rebaseing it is gone | 13:44 |
| chkumar|ruck | so we donot need | 13:45 |
| chkumar|ruck | to revert now | 13:45 |
| chkumar|ruck | sorry for the trouble | 13:45 |
| rlandy | np | 13:46 |
| chkumar|ruck | rlandy: updated the dlrn hashtag patch | 13:46 |
| chkumar|ruck | marios|rover: heading home | 13:48 |
| rlandy | revoted | 13:48 |
| marios|rover | chkumar|ruck: ack | 13:49 |
| *** ykarel is now known as ykarel|afk | 13:50 | |
| rfolco | rlandy, marios|rover panda chkumar|ruck please review https://review.rdoproject.org/r/23875 asap | 13:53 |
| sshnaidm | what is our internal interview channel? | 13:54 |
| sshnaidm | marios|rover, ^ | 13:54 |
| marios|rover | sshnaidm: top of the hackmd | 13:54 |
| sshnaidm | marios|rover, thanks) | 13:55 |
| *** derekh has joined #oooq | 13:56 | |
| marios|rover | arxcruz: join us | 14:00 |
| sshnaidm | arxcruz, arxcruz arxcruz | 14:01 |
| rlandy | marios|rover: broke something - reverting | 14:06 |
| *** marios|rover is now known as marios|rover|cal | 14:06 | |
| *** jtomasek has quit IRC | 14:08 | |
| *** amoralej|lunch is now known as amoralej | 14:11 | |
| *** jtomasek has joined #oooq | 14:11 | |
| panda | rfolco: so you're practically retiring the promoter server | 14:13 |
| panda | rfolco https://trunk-staging.rdoproject.org/api-centos-master/ endpoint should have centos7 ..... | 14:15 |
| panda | rfolco: I mean probably the dlrn endpoint is missing it, but it should have it | 14:15 |
| *** jbadiapa has joined #oooq | 14:15 | |
| *** jbadiapa has quit IRC | 14:16 | |
| rfolco | panda, the main dlrnapi does not have centos7 on its url | 14:16 |
| rlandy | marios|rover|cal: reverted | 14:17 |
| rfolco | panda, not sure what you mean | 14:17 |
| rfolco | its api-centos-master as well | 14:17 |
| panda | rfolco: I know, but it will not be much time until the endpoint needs to be renamed in centos7 | 14:18 |
| *** Goneri has joined #oooq | 14:18 | |
| panda | rfolco: when we'll have centos8 | 14:18 |
| panda | rfolco: anyway, I think it's time to retire the promoter server | 14:19 |
| panda | rfolco: just a sec ... a moment | 14:19 |
| panda | rfolco: ... hold on ... | 14:19 |
| panda | rfolco: removed promoter-server. | 14:19 |
| rfolco | panda, whhhaaaaat | 14:19 |
| rfolco | panda, what I did wrong to retire the promoter server ? | 14:21 |
| chkumar|ruck | rfolco: panda centos8 might be coming by end of next week | 14:21 |
| panda | chkumar|ruck: ??? | 14:21 |
| * panda starts screaming | 14:21 | |
| panda | rfolco: you are promoting on the job | 14:21 |
| panda | rfolco: so we don't need a separate server for that anymore | 14:22 |
| chkumar|ruck | panda: sorry may be coming | 14:22 |
| panda | chkumar|ruck: don't you have an interview, like, right now ? | 14:22 |
| chkumar|ruck | panda: nope | 14:22 |
| rfolco | panda, you find it bad ? | 14:23 |
| rfolco | panda, you have feelings about promoter looks like | 14:23 |
| panda | rfolco: I find it revolutionary, that's why I removed the promoter server, we don't need taht anymore | 14:23 |
| rfolco | panda, </sarcasm> | 14:24 |
| rfolco | panda, now talk seriously to me | 14:24 |
| rfolco | panda, tell me reasons your thing promote job is a bad idea | 14:25 |
| rfolco | think** | 14:25 |
| panda | rfolco: so, we remember discussing how to promote a few years ago | 14:25 |
| panda | rfolco: promoting as a job was one of the ideas | 14:25 |
| panda | rfolco: I'm trying to remember why it didn't win | 14:26 |
| panda | rfolco: one of the reasons was because at the time you couldn't specify multiple dependencies on the jobs | 14:26 |
| panda | rfolco: but now it's lifted | 14:26 |
| rfolco | panda, honestly I don't recall this discussion | 14:26 |
| panda | rfolco: there may be issue with control if we move everything to job | 14:26 |
| panda | rfolco: you were not there, when the ancient magic was created | 14:27 |
| migi | rfolco: ok let me know | 14:27 |
| arxcruz | just realize my computer time was 2 minutes different, and I wasn't using ntp to set the time... :( | 14:27 |
| migi | rlandy: ok let me know | 14:27 |
| migi | rfolco: sorry that was to rlandy | 14:27 |
| *** rlandy is now known as rlandy|mtg | 14:28 | |
| panda | rfolco: the promoter server was created after that discussion | 14:28 |
| rfolco | panda, I am totally open to review the design and move into the promoter. This was the fast path to get the component promotion without breaking the promoter code, which has its own struggles atm. | 14:29 |
| panda | rfolco: I'm saying the exact opposite, with the possibilited offered by zuul, we could think to move the promotion to zuul . | 14:30 |
| rfolco | panda, right, so no regrets for rm -rf promoter | 14:32 |
| rfolco | panda, end of story | 14:32 |
| *** jbadiapa has joined #oooq | 14:33 | |
| panda | rfolco: the biggest problem would be represented by credentials. As usual, either you keep everything in config, and you die when you need to change something, or you need an auth mechanism to pass tokens to a job in rdo-jobs | 14:33 |
| panda | rfolco: already done | 14:33 |
| *** zbr|out has quit IRC | 14:33 | |
| *** zbr has joined #oooq | 14:34 | |
| panda | and you need a change in config to change the criteria | 14:34 |
| panda | rfolco: ok ctrl+Z on the promoter server, we would probably need for a little longer now. | 14:34 |
| panda | rfolco: ohno, ctrl+Z doesn't work! | 14:35 |
| rfolco | panda, does dlrnapi preserve session/auth/token ? | 14:35 |
| ssbarnea | chkumar|ruck: one approve away from done https://github.com/containers/libpod/pull/4601 | 14:35 |
| rfolco | panda, I assume it doesn't | 14:35 |
| panda | rfolco: no it doesn't, pure basic auth | 14:36 |
| rfolco | panda, if it accepts a persistent login, the auth part could be a parent job in config, the rest in rdo-jobs | 14:36 |
| panda | rfolco: ssh to the images server too would be a problem | 14:36 |
| rfolco | panda, feature request to infra | 14:36 |
| chkumar|ruck | ssbarnea: approved | 14:37 |
| chkumar|ruck | ssbarnea: do I need to add /lgtm? | 14:37 |
| panda | rfolco: done and done. | 14:38 |
| ssbarnea | i think is ok, one of the cores will approve it. they are very active. | 14:38 |
| rfolco | panda, perhaps we can record this somewhere | 14:39 |
| panda | rfolco: in vinyl | 14:39 |
| panda | rfolco: for a better quality | 14:40 |
| rfolco | panda, lol, vinyl is better than taiga I think | 14:40 |
| *** zbr_ has joined #oooq | 14:41 | |
| ssbarnea | i find libpod review process really interesting, a lot of automation with git bots and more interesting they made reviewer allocation non-personal, the bot picks randomly cores. i am not sure if their ability to review and merge a big number of changes per day is result of this. anyway the number of open PRs seems to be kept under 30, which is good. that part of the pipe is working quite nicely. | 14:41 |
| ssbarnea | the downside is that I obseved that total coverage is not so good, i accidentaly broke copr build few days ago with one of my patches. the problem was that they did not have CI on it. Hopefully we can soon make RPM building part of their pipeline, so it does not get broken by another random change. | 14:43 |
| *** zbr has quit IRC | 14:44 | |
| *** zbr_ has quit IRC | 14:49 | |
| *** zbr has joined #oooq | 14:55 | |
| *** rlandy|mtg is now known as rlandy | 15:09 | |
| rlandy | marios|rover|cal: trying new patch | 15:09 |
| *** marios|rover|cal is now known as marios|rover | 15:12 | |
| marios|rover | rlandy: ack | 15:12 |
| migi | rlandy: ready to have chat or later? | 15:14 |
| rlandy | migi: in a few - just need to resubmit a patch | 15:15 |
| migi | ok | 15:15 |
| chkumar|ruck | marios|rover: needs your ack on these https://bugs.launchpad.net/tripleo/+bug/1854721 and https://bugs.launchpad.net/tripleo/+bug/1854899 | 15:15 |
| openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Triaged] | 15:15 |
| openstack | Launchpad bug 1854899 in tripleo "skipped neutron dns-integration related tempest test started failing with test_create_router_with_snat_explicit" [Critical,Triaged] | 15:15 |
| chkumar|ruck | marios|rover: few more bugs left, we will tackle next week and get it fixed | 15:16 |
| chkumar|ruck | rlandy: thank you for looking on no valid host found bug | 15:16 |
| chkumar|ruck | arxcruz: still around? | 15:20 |
| chkumar|ruck | arxcruz: do you have ceph bug handy | 15:20 |
| arxcruz | chkumar|ruck: no | 15:21 |
| chkumar|ruck | arxcruz: we can make it cix to get it fixed early | 15:21 |
| marios|rover | chkumar|ruck: https://bugs.launchpad.net/tripleo/+bug/1854899 don't think we want to close it | 15:21 |
| openstack | Launchpad bug 1854899 in tripleo "skipped neutron dns-integration related tempest test started failing with test_create_router_with_snat_explicit" [Critical,Triaged] | 15:21 |
| marios|rover | chkumar|ruck: sory i went to tripleo by mistake :D | 15:21 |
| marios|rover | chkumar|ruck: i think we wantt to keep https://bugs.launchpad.net/tripleo/+bug/1854899 until the tests are fixed? | 15:22 |
| chkumar|ruck | marios|rover: no problem we are hopping on multiple channel | 15:22 |
| chkumar|ruck | marios|rover: that works for me | 15:22 |
| marios|rover | chkumar|ruck: ah this is the one you said we didn't have the tests fixed... new feature that someone needs to do | 15:22 |
| chkumar|ruck | marios|rover: I discussed the idea with wes yesterday night, he told to wait for neutron component to come then implement | 15:23 |
| marios|rover | chkumar|ruck: i guess we can move it to 'won't fix' with a note | 15:23 |
| chkumar|ruck | marios|rover: yes | 15:23 |
| chkumar|ruck | marios|rover: by the way kolla fix got stuck becuase something got broken in their gate itself and few concerns on how we built tripleo container | 15:24 |
| chkumar|ruck | will share once wes will back | 15:24 |
| chkumar|ruck | from kolla people | 15:24 |
| chkumar|ruck | marios|rover: by the way it was a good week we try to smash so many bugs :-) Thank you for all the help and other #oooq team members :-) | 15:25 |
| marios|rover | chkumar|ruck++ | 15:25 |
| chkumar|ruck | marios|rover++ | 15:25 |
| panda | now kiss. | 15:29 |
| marios|rover | panda++ | 15:30 |
| marios|rover | you too | 15:30 |
| marios|rover | kiss me panda | 15:30 |
| rlandy | marios|rover: chkumar|ruck: going to try merge patch again - with fix | 15:30 |
| chkumar|ruck | panda: you too but no kiss not legal in india in public places | 15:31 |
| chkumar|ruck | rlandy: sure go ahead :-) | 15:31 |
| rlandy | putting two testproject jobs in place | 15:32 |
| chkumar|ruck | rlandy: any way you are going to hold the fort tomororw till I land in hydrabad :-) | 15:32 |
| panda | hail hydrabad | 15:32 |
| chkumar|ruck | https://en.wikipedia.org/wiki/Hyderabad | 15:32 |
| panda | (I'm watching containers being pushed to docker.io so I have a lot of time on my hands) | 15:33 |
| chkumar|ruck | biryani is so popular there | 15:33 |
| marios|rover | panda: so queens promoted or not? | 15:35 |
| marios|rover | i demand my queens promotion panda ! | 15:36 |
| rlandy | chkumar|ruck: safe travels | 15:36 |
| *** udesale has quit IRC | 15:37 | |
| rlandy | chkumar|ruck: marios|rover: well I guess that's poetic justice of you - I break the promotions, I own them for a day | 15:37 |
| chkumar|ruck | rlandy: thanks :-), travelling tomorrow afternoon | 15:37 |
| panda | marios|rover: long live the queens | 15:37 |
| chkumar|ruck | panda: you owe beers a hell of beer and milk to us | 15:37 |
| panda | marios|rover: I'm trying man, it's still uploading. I had to turn off the manifest push though | 15:37 |
| *** TrevorV has joined #oooq | 15:41 | |
| panda | chkumar|ruck: I'll send you a lifelong supply of masala chai | 15:45 |
| chkumar|ruck | panda: masala chai good try milk tea it is too awesome aka desi chai | 15:45 |
| marios|rover | panda: ack | 15:46 |
| chkumar|ruck | I can make it if I meet next soon | 15:46 |
| marios|rover | chkumar|ruck: drinking masala chai now i brought it from india when i came :) | 15:46 |
| chkumar|ruck | marios|rover: if you extend the trip when you come to india, I will take you to tea farm and we can try there | 15:47 |
| marios|rover | :) | 15:47 |
| chkumar|ruck | let's do a sprint call on chai pe charcha (charcha == discusssion) | 15:48 |
| chkumar|ruck | pe == on | 15:48 |
| panda | chkumar|ruck: deal, ask wes if we can expend travels for a chai taste round at your house. | 15:51 |
| *** aakarsh has joined #oooq | 15:51 | |
| chkumar|ruck | panda: we can ask for sure | 15:51 |
| chkumar|ruck | will make a retro topic | 15:51 |
| panda | marios|rover: [promotion failed in the middle, we have an inconsistent state with queens right now | 15:52 |
| panda | marios|rover: horizon failed to upload, but some caontianers are tagged as current-tripleo | 15:53 |
| panda | chkumar|ruck: ok now go to bed. | 15:53 |
| chkumar|ruck | few more mins sir | 15:53 |
| marios|rover | panda: oh great | 15:53 |
| panda | marios|rover: relaunching | 15:53 |
| marios|rover | panda: ack ;) | 15:53 |
| panda | chkumar|ruck: no, now. | 15:53 |
| panda | chkumar|ruck: or I won't tell you bedtime stories. | 15:54 |
| chkumar|ruck | panda: hehe | 15:54 |
| marios|rover | if that image doesn't give you nightmares ... | 15:54 |
| panda | hey, it works with my daughter | 15:54 |
| marios|rover | panda: telling you about proper testing of your patches | 15:54 |
| marios|rover | "bedtime stories" | 15:54 |
| chkumar|ruck | ok ok | 15:55 |
| panda | marios|rover: I wish these were my patches, this is pretty standard promotion. fortunately we usually refer to the hashes for the contaieners, the current-tripleo name is usually inconsistent | 15:56 |
| marios|rover | panda: should be fine if you rerun though no? | 15:56 |
| *** chkumar|ruck is now known as raukadah | 15:57 | |
| marios|rover | panda: but what failed? | 15:57 |
| raukadah | see ya | 15:57 |
| marios|rover | o/ raukadah | 15:57 |
| raukadah | \o | 15:57 |
| marios|rover | if you see any pandas on your way home avoid them raukadah | 15:57 |
| raukadah | bedtime stories == movies for me | 15:57 |
| raukadah | saw it in china | 15:57 |
| panda | marios|rover: yes. horizon failed to push. so everythihng else worked, the containres are uploaded, some have current-tripleo names, some don't | 15:57 |
| marios|rover | panda: ah cool... so registry error? i mean transient issue on the registry side maybe? | 15:57 |
| marios|rover | well not 'cool' but you know what i mean | 15:58 |
| panda | marios|rover: yes | 15:58 |
| raukadah | before bed time does anyone uses bip here? | 15:59 |
| rlandy | rfolco: I think we have a problem | 15:59 |
| rfolco | rlandy, open your heart | 16:00 |
| rlandy | rfolco: dlrn reporting on component jobs is going to go to the wrong DLRNAPI_SERVER | 16:00 |
| *** ykarel|afk is now known as ykarel|away | 16:01 | |
| rfolco | rlandy, why | 16:01 |
| rfolco | get-hash will run then create hash_info | 16:02 |
| rfolco | hash_info is sourced and it should have the right dlrnapi_server | 16:02 |
| rfolco | when the standalone component job runs | 16:02 |
| rfolco | am I missing anything ? | 16:03 |
| rlandy | https://review.rdoproject.org/zuul/stream/745bb2f94ee14229b66b6491e067edcf?logfile=console.log | 16:03 |
| rlandy | rfolco: ^^ | 16:03 |
| rlandy | seethe get_hash output here | 16:03 |
| rlandy | DLRNAPI_URL=https://trunk.rdoproject.org/api-centos-master | 16:03 |
| rlandy | only setting that if component_name is set | 16:04 |
| rlandy | in https://review.rdoproject.org/r/#/c/23958/2/playbooks/tripleo-ci-base-promote-consistent-to-tripleo-ci-testing/run.yaml | 16:04 |
| rfolco | rlandy, you have to merge my patch to fix that | 16:04 |
| rfolco | currently yes we are reporting to the main dlrnapi and that's bad | 16:05 |
| rlandy | rfolco: pls link your patch | 16:05 |
| *** ykarel|away has quit IRC | 16:05 | |
| rfolco | https://review.rdoproject.org/r/#/c/23875/9/playbooks/tripleo-ci-periodic-base/pre.yaml | 16:06 |
| rfolco | rlandy, ^ | 16:06 |
| rfolco | the one that pointed out this morning in the scrum | 16:06 |
| rfolco | you can move that bit to your patch if you want | 16:06 |
| rlandy | rfolco: you'll need component_name in that I think | 16:07 |
| rfolco | rlandy, hmm ok | 16:08 |
| rfolco | rlandy, lets do this | 16:08 |
| rfolco | rlandy, I'll move that bit to a new patch w/ component_name on it | 16:08 |
| rfolco | we merge it, and test | 16:08 |
| rfolco | rlandy, wdyt | 16:09 |
| rlandy | rfolco: one moment | 16:10 |
| rlandy | rfolco: https://review.rdoproject.org/r/23962 | 16:18 |
| rlandy | ok to merge? | 16:18 |
| rfolco | +1 | 16:19 |
| rfolco | I need to change this too in my patch | 16:19 |
| rfolco | one sec | 16:19 |
| rlandy | rfolco: give me a few to restest that | 16:19 |
| rlandy | and then we an look over your patch together | 16:20 |
| rlandy | if mine doesn;t work, yours won't | 16:20 |
| *** ykarel|away has joined #oooq | 16:20 | |
| panda | this is going to take a while | 16:23 |
| panda | a different container failed to push now. | 16:23 |
| rlandy | marios|rover: hey - is this a known standalone error? | 16:25 |
| rlandy | http://logs.rdoproject.org/06/23906/3/check/periodic-tripleo-ci-centos-7-standalone-master/e426603/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 16:25 |
| rlandy | or did I break it more? | 16:26 |
| marios|rover | rlandy: known | 16:27 |
| rlandy | bless you | 16:27 |
| *** skramaja has quit IRC | 16:27 | |
| marios|rover | rlandy: https://bugs.launchpad.net/tripleo/+bug/1855063 that one i think | 16:27 |
| openstack | Launchpad bug 1855063 in tripleo "Master standalone deploy failed with Table 'ovn_revision_numbers' already exists while performing neutron sync" [Critical,Confirmed] | 16:27 |
| *** soniya29 has joined #oooq | 16:32 | |
| * marios|rover starts shutdown sequence | 16:33 | |
| rfolco | marios|rover, cancel it now | 16:34 |
| rfolco | marios|rover, abort | 16:34 |
| rfolco | marios|rover, please don't go... don't goooooooo | 16:34 |
| marios|rover | rfolco: you must find a way. its not you. its me. | 16:34 |
| rfolco | marios|rover, you don't love me anymore. | 16:35 |
| rfolco | marios|rover, just leave. | 16:35 |
| rfolco | rlandy, https://review.rdoproject.org/r/#/c/23875/11/playbooks/tripleo-ci-periodic-base/pre.yaml | 16:35 |
| rfolco | rlandy, updated w/ component name | 16:35 |
| rlandy | one moment - fixing another issue | 16:36 |
| *** tesseract has quit IRC | 16:39 | |
| rlandy | rfolco: what s the dlrn api url for component? | 16:39 |
| rlandy | ERROR: Got error 404, probably endpoint https://trunk-staging.rdoproject.org/api-centos-master/component/compute is not available | 16:39 |
| rlandy | jpena: ^^? | 16:40 |
| jpena | rlandy: there's no per-component API, only one endpoint | 16:40 |
| jpena | in this case, it should be https://trunk-staging.rdoproject.org/api-centos-master-uc | 16:41 |
| rlandy | ok - let me try that | 16:41 |
| rlandy | rfolco: ^^ fixing | 16:41 |
| rfolco | rlandy, yes, https://trunk-staging.rdoproject.org/api-centos-master-uc | 16:42 |
| *** marios|rover is now known as marios|out | 16:43 | |
| rfolco | that is, the only change is "-staging" in the base url, the rest remains | 16:44 |
| rlandy | rfolco: https://review.rdoproject.org/r/23963 Fix dlrn server url for component promotions | 16:45 |
| *** jpena is now known as jpena|brb | 16:45 | |
| rfolco | rlandy, I am sorry, I missed that in the review | 16:47 |
| rfolco | +1 | 16:47 |
| rlandy | np - let's see how this goes | 16:47 |
| rlandy | rfolco: pls check lines 35-40 | 16:48 |
| rlandy | those are still right? | 16:48 |
| *** marios|out has quit IRC | 16:49 | |
| rfolco | rlandy, sec | 16:50 |
| rfolco | rlandy, yes, variables changed will result in https://trunk-staging.rdoproject.org/centos7/component/compute/consistent/commit.yaml | 16:53 |
| rlandy | migi: how long are you still around? | 16:53 |
| rlandy | migi: want to meet now? | 16:54 |
| migi | rlandy: yep, let's jump to bluejeans.com/mpryc | 16:54 |
| rlandy | k - joining | 16:54 |
| panda | ah, it's quay.io login that fails consistently | 17:03 |
| *** soniya29 has quit IRC | 17:19 | |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| migi | rlandy: https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=zuul.d/jobs-base.yaml;h=1d57956ffc92885d495f9e74eb06d2ff765a0b37;hb=HEAD | 17:24 |
| migi | rlandy: https://code.engineering.redhat.com/gerrit/#/c/176065/ | 17:24 |
| *** jpena|brb is now known as jpena | 17:25 | |
| rlandy | rfolco: ok - so we have https://trunk-staging.rdoproject.org/centos7/component/compute/tripleo-ci-testing/ | 17:46 |
| *** ccamacho has quit IRC | 17:46 | |
| rfolco | rlandy, cool | 17:47 |
| * rfolco goes to the pharmacy | 17:48 | |
| *** rfolco is now known as rfolco|brb | 17:48 | |
| rlandy | rfolco: but we have another problem | 17:48 |
| rlandy | it's promoting rhel-8 as well | 17:48 |
| rfolco|brb | rlandy, what you mean | 17:48 |
| rlandy | rfolco|brb: http://logs.rdoproject.org/53/21853/10/check/periodic-tripleo-centos-7-master-component-compute-promote-consistent-to-tripleo-ci-testing/c7bbddb/job-output.txt | 17:49 |
| rlandy | fixing rhel-8 | 17:50 |
| rfolco|brb | k brb | 17:50 |
| raukadah | rlandy: sshnaidm: rhel 8 jobs is now alive in check | 17:52 |
| sshnaidm | raukadah++ | 17:52 |
| rlandy | raukadah: rhel-8 train? | 17:52 |
| raukadah | master | 17:52 |
| raukadah | rlandy: we never runned train one in check | 17:52 |
| sshnaidm | raukadah, I think we do | 17:53 |
| sshnaidm | raukadah, in train patches | 17:53 |
| raukadah | is it? where, we need to check those then I think | 17:53 |
| sshnaidm | at least I configured it to run on build-containers in train branch | 17:53 |
| raukadah | sshnaidm: against kolla not running | 17:54 |
| sshnaidm | hmm | 17:54 |
| sshnaidm | raukadah, and tripleo-ci? | 17:54 |
| raukadah | they complained our build containers job reports bogus result need to check that result tomorrow | 17:54 |
| raukadah | sshnaidm: need to check that | 17:54 |
| sshnaidm | I saw it's running, weird | 17:54 |
| raukadah | because of this https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/main.yaml#L4 we need to get rid of rhel8 kolla patches now | 17:55 |
| raukadah | I am not sure what was wrong, will be looking tomorrow | 17:56 |
| *** derekh has quit IRC | 18:00 | |
| rlandy | raukadah: rfolco|brb: https://review.rdoproject.org/r/#/c/23964/ | 18:02 |
| rlandy | ^^ to prevent component promotions running rhel | 18:02 |
| *** amoralej is now known as amoralej|off | 18:05 | |
| raukadah | rlandy: one query | 18:05 |
| rlandy | Don't compare to empty string | 18:07 |
| rlandy | lol - linters | 18:07 |
| *** ykarel|away has quit IRC | 18:07 | |
| rlandy | raukadah: it doesn't compare | 18:07 |
| rlandy | that whole section runs on rhel only | 18:07 |
| rlandy | see the dlrn | 18:07 |
| rlandy | DLRNAPI_URL="https://trunk.rdoproject.org/api-rhel8-$RELEASE" | 18:08 |
| raukadah | rlandy: can we make linters happy | 18:10 |
| raukadah | ? | 18:10 |
| rlandy | raukadah: trying - next patch | 18:11 |
| rlandy | linters are happy now | 18:18 |
| rlandy | raukadah: ^^ | 18:18 |
| *** jpena is now known as jpena|off | 18:18 | |
| rlandy | i hope that won' blow up anything | 18:18 |
| raukadah | rlandy: done | 18:19 |
| raukadah | I am going to listen bed time stories , time to sleep see ya tomorrow | 18:20 |
| rlandy | thanks - good night | 18:21 |
| raukadah | rlandy: please revert back if doesnot work, and merge, we will try to help in morning | 18:26 |
| rlandy | raukadah: testing - thanks | 18:26 |
| *** Goneri has quit IRC | 18:35 | |
| *** rfolco|brb is now known as rfolco | 18:35 | |
| panda | quay.io is way slower than docker.io in upload. | 18:37 |
| *** Goneri has joined #oooq | 18:44 | |
| panda | and one upload failed ... :/ | 18:52 |
| *** dsneddon has joined #oooq | 18:59 | |
| *** dsneddon has quit IRC | 19:21 | |
| *** dsneddon has joined #oooq | 19:22 | |
| mjturek | rfolco: baha and I did a quick comparison of build-containers.sh (see here http://paste.openstack.org/raw/787193/ and here http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/db10ab5/logs/build_containers.sh.txt.gz) | 19:36 |
| mjturek | the only meaningfuyl difference is that our ppc job has the --use-buildah flag | 19:37 |
| mjturek | while the x86 one doesn't? | 19:37 |
| rlandy | rfolco: ok - I think promote to tripleo-ci-testing is ok now | 19:38 |
| rfolco | mjturek, looking | 19:39 |
| rfolco | mjturek, I'm sorry, it does not use buildah | 19:48 |
| mjturek | ahhh is there a job that does!? | 19:49 |
| mjturek | or should we not be using buildah? | 19:49 |
| rfolco | mjturek, the ones using buildah have been disabled for some reason I don't recall exactly | 19:50 |
| rfolco | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-centos-7-master-containers-buildah | 19:50 |
| rfolco | 2019-04-11T22:38:10 | 19:50 |
| rfolco | last run | 19:50 |
| *** aakarsh has quit IRC | 19:50 | |
| mjturek | so should we not be using buildah? | 19:50 |
| rfolco | so wait | 19:51 |
| rfolco | mjturek, I think we use buildah by default | 19:51 |
| rfolco | http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/db10ab5/logs/buildah-builds/ | 19:51 |
| rfolco | mjturek, without --use-buildah flag | 19:52 |
| mjturek | ahhh | 19:52 |
| baha | Do you know of anyone who could confirm that? | 19:52 |
| rfolco | kolla code | 19:52 |
| rfolco | rlandy, do you the history behind use-buildah ? | 19:52 |
| rfolco | rlandy, on centos7 we use buildah by default without any flags | 19:53 |
| *** holser has quit IRC | 19:53 | |
| rlandy | implemented in rhel jobs | 19:53 |
| rfolco | on rhel8 looks like we need set --use-buildah, and actually we use podman?! | 19:53 |
| rlandy | use_buildah is by default false | 19:53 |
| *** holser has joined #oooq | 19:54 | |
| rfolco | explain me this then | 19:55 |
| rfolco | http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/db10ab5/logs/buildah-builds/ | 19:55 |
| rlandy | https://github.com/openstack/tripleo-ci/blob/0ce5beaccfc402da0107b1507ca609703c3aae02/zuul.d/build-containers.yaml#L107 | 19:55 |
| rfolco | just source dirs | 19:55 |
| rlandy | only true where we define it to be | 19:55 |
| rfolco | this job is disabled - does not run for a while in master periodic pipeline | 19:56 |
| rfolco | must be in old releases only | 19:56 |
| rfolco | train/stein | 19:56 |
| rlandy | rfolco: which job are you asking about? | 19:56 |
| rlandy | periodic/check? | 19:57 |
| rlandy | centos/rhel? | 19:57 |
| rfolco | rlandy, our periodic build containers | 19:57 |
| rfolco | centos | 19:57 |
| rfolco | that one uses docker | 19:57 |
| rlandy | periodic-tripleo-centos-7-master-containers-buildah | 20:00 |
| rlandy | rfolco: what are you questioning - are you expecting that we would or woruld not use buildah in centos 7 periodic | 20:03 |
| rfolco | rlandy, asking if we use or not. Looks like we don't use. | 20:03 |
| rlandy | rfolco: that is correct | 20:04 |
| rlandy | we do not | 20:04 |
| rlandy | it is false by default unless we set it otherwise | 20:04 |
| rlandy | I hadthe discussion with weshay_ about changing that when I was on the rotation | 20:05 |
| rlandy | he didn't feel it was important at the time | 20:05 |
| rlandy | rhel we have no choice | 20:05 |
| rfolco | rhel is weird | 20:07 |
| rfolco | we build w/ buildah and push w/ podman | 20:07 |
| rfolco | mjturek, does this answer your question ? ^ | 20:08 |
| mjturek | rfolco kind of? | 20:08 |
| mjturek | I mean | 20:08 |
| rfolco | mjturek, sorry about the confusion | 20:08 |
| mjturek | I guess we shouldn't build with buildah? | 20:08 |
| rfolco | mjturek, we should in theory... but we didn't make the switch for some reason | 20:08 |
| rfolco | mjturek, don't recall if the runs were unstable or what | 20:09 |
| rfolco | mjturek, for now, make it false, use docker | 20:09 |
| *** dsneddon has quit IRC | 20:09 | |
| mjturek | rfolco okay doing so now | 20:09 |
| *** dsneddon has joined #oooq | 20:10 | |
| rfolco | mjturek, I am glad we are 6k+ miles away, so you can't hit me. I see the anger in your eyes now | 20:10 |
| mjturek | rfolco: nah man only beers for you :) | 20:10 |
| rfolco | :) | 20:10 |
| rfolco | rlandy, can you please review my patch now so we can test it and fix any remaining issues ? | 20:12 |
| rlandy | rfolco: yes - looking | 20:13 |
| rfolco | thx | 20:13 |
| rlandy | rfolco: so I think https://review.rdoproject.org/r/#/c/21853/ is working now | 20:13 |
| rlandy | for the two cases | 20:13 |
| rlandy | rfolco: is https://review.rdoproject.org/r/#/c/23875/11/playbooks/tripleo-ci-base-promote-component-to-current-tripleo/post.yaml needed? | 20:17 |
| rlandy | what does this parent off? | 20:17 |
| rlandy | base> | 20:17 |
| rfolco | rlandy, this post is a copy of the consistent-to-tripleo-ci-testing post | 20:23 |
| rfolco | rlandy, we could reuse but I preferred to keep it independent | 20:23 |
| rlandy | yeah - noticed the copy | 20:24 |
| rlandy | I think it's ok for now | 20:24 |
| rlandy | just responding o infra issue | 20:24 |
| rfolco | the closest to sprint end, the craziest and ugliest things get | 20:25 |
| panda | quay.io doesn't support V2 schema 2 yet. That means not multi-arch manifests. | 20:35 |
| panda | and the entire docker manifest set of commands | 20:36 |
| panda | so even manifest ispect doesn't work | 20:36 |
| panda | I think we have to disable it | 20:36 |
| rlandy | rfolco: sorry was covering ruck/rovers | 20:40 |
| rlandy | ok - back to your patch | 20:40 |
| rlandy | looks like there is a lot of duplication | 20:40 |
| rlandy | other than that minimal impact | 20:41 |
| rlandy | rfolco: where are the promotion criteria? | 20:45 |
| *** saneax has quit IRC | 20:53 | |
| rlandy | rfolco: commented and voted https://review.rdoproject.org/r/#/c/23875/ | 20:54 |
| panda | EmilienM: have you ever tried multi arch manifests on quay.io ? they don't seem to be supported. Quay V3 supports them but I don't know which version quay.io uses. | 21:00 |
| panda | queens promoted with the new code, but quay.io support introduces some problems. | 21:01 |
| EmilienM | panda: no | 21:01 |
| panda | EmilienM: ok, I'm disabling multi-arch for now then | 21:02 |
| *** holser has quit IRC | 21:04 | |
| *** jtomasek has quit IRC | 21:12 | |
| *** dsneddon has quit IRC | 21:18 | |
| *** dsneddon has joined #oooq | 21:18 | |
| *** jaosorior has joined #oooq | 21:22 | |
| *** rfolco has quit IRC | 21:25 | |
| *** TrevorV has quit IRC | 21:28 | |
| *** dsneddon has quit IRC | 21:33 | |
| *** dsneddon has joined #oooq | 21:34 | |
| *** Goneri has quit IRC | 21:38 | |
| *** dsneddon has quit IRC | 21:39 | |
| *** dsneddon has joined #oooq | 21:40 | |
| *** ysandeep has joined #oooq | 21:47 | |
| *** jaosorior has quit IRC | 21:53 | |
| rlandy | sshnaidm: you still around? | 22:12 |
| sshnaidm | rlandy, ya | 22:13 |
| rlandy | sshnaidm: gosh it's late | 22:13 |
| rlandy | sshnaidm: looks like ovb stacks are not deleting | 22:14 |
| sshnaidm | rlandy, where? | 22:14 |
| rlandy | rdocloud | 22:14 |
| rlandy | I know you pushed a change | 22:14 |
| rlandy | was that to fix the non-deleting stacks? | 22:14 |
| sshnaidm | rlandy, yep, maybe broke something.. | 22:14 |
| sshnaidm | rlandy, it was mostly for vexxhost | 22:14 |
| sshnaidm | rlandy, and changed a lil bit the cleaning script | 22:15 |
| rlandy | sshnaidm: I'm cleaning up but yeah - stacks are piling up | 22:15 |
| sshnaidm | rlandy, wait a sec | 22:15 |
| sshnaidm | rlandy, why are they not delete by jobs? | 22:15 |
| rlandy | sshnaidm: yeah - was looking foe the job logs from these stacks | 22:16 |
| rlandy | so we could see what happened | 22:16 |
| rlandy | they should have been deleted with the jobs | 22:17 |
| sshnaidm | rlandy, do you have one for example? | 22:17 |
| rlandy | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001openstack/tripleo-heat-templatesstable/rockyopenstack-check696847,184782019-12-05T17:40:20SUCCESS | 22:17 |
| rlandy | ^^ lastest success | 22:17 |
| rlandy | checking that | 22:17 |
| rlandy | 2019-12-05 20:00:07.792227 | primary | ERROR | 22:18 |
| rlandy | 2019-12-05 20:00:07.792560 | primary | { | 22:18 |
| rlandy | 2019-12-05 20:00:07.792660 | primary | "attempts": 2, | 22:18 |
| rlandy | 2019-12-05 20:00:07.792715 | primary | "delta": "0:00:01.845371", | 22:18 |
| rlandy | 2019-12-05 20:00:07.792785 | primary | "end": "2019-12-05 19:59:57.381690", | 22:18 |
| rlandy | 2019-12-05 20:00:07.792834 | primary | "msg": "non-zero return code", | 22:18 |
| rlandy | 2019-12-05 20:00:07.792882 | primary | "rc": 1, | 22:18 |
| rlandy | 2019-12-05 20:00:07.792929 | primary | "start": "2019-12-05 19:59:55.536319" | 22:18 |
| rlandy | 2019-12-05 20:00:07.792976 | primary | } | 22:18 |
| sshnaidm | link? | 22:20 |
| rlandy | http://logs.rdoproject.org/47/696847/1/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/a246b9a/job-output.txt | 22:21 |
| sshnaidm | rlandy, this error is ok | 22:23 |
| sshnaidm | rlandy, it should be there, it means it tried to show the stack and it doesn't exist | 22:23 |
| sshnaidm | primary | Stack not found: baremetal_95674 | 22:23 |
| sshnaidm | so it's deleted | 22:23 |
| rlandy | ok - so question remain why the stack stayed there | 22:24 |
| sshnaidm | rlandy, I think we need to mark stacks to be related to jobs.. because it's impossible to understand which stack where belongs to | 22:24 |
| rlandy | sshnaidm: with the change number? | 22:24 |
| sshnaidm | rlandy, yeah, change_patch_random, something like that | 22:25 |
| rlandy | good idea | 22:25 |
| sshnaidm | rlandy, I changed also default time to be 6 hours, not to delete 020 job | 22:25 |
| sshnaidm | it can take 5+ hours | 22:25 |
| rlandy | I hate that job | 22:26 |
| rlandy | what jobs needs to run so long | 22:26 |
| rlandy | we are already kicking the next promotion | 22:26 |
| sshnaidm | well, until we split it | 22:26 |
| rlandy | still question remains - why the stacks remained | 22:30 |
| rlandy | | stack_status_reason | Resource DELETE failed: ResourceInError: resources.baremetal_env.resources.openstack_baremetal_servers.resources[1].resources.baremetal_server: Went to status ERROR due to "Server baremetal-99948_1 delete failed: (500) Failed to terminate process 256110 with SIGKILL: Device or resource busy" | 22:31 |
| rlandy | that's new | 22:32 |
| sshnaidm | well, that explains | 22:32 |
| rlandy | meaning? | 22:32 |
| *** ysandeep has quit IRC | 22:32 | |
| sshnaidm | looks like rdo cloud problems | 22:33 |
| sshnaidm | that doesn't allow stacks to be deleted | 22:33 |
| rlandy | million downed ports to delete | 22:40 |
| sshnaidm | rlandy, running cleaning script.. | 22:46 |
| sshnaidm | rlandy, I hope all ports should go with stacks | 22:47 |
| sshnaidm | rlandy, if not - we miss something in ovb deletion process | 22:47 |
| *** jfrancoa has quit IRC | 22:53 | |
| *** dsneddon has quit IRC | 23:05 | |
| *** dsneddon has joined #oooq | 23:13 | |
| *** dsneddon has quit IRC | 23:23 | |
| sshnaidm | a lot of errors "Failed to terminate process 108515 with SIGKILL: Device or resource busy" | 23:23 |
| rlandy | will ping kforde tomorrow | 23:23 |
| sshnaidm | ok, have a nice weekend! | 23:24 |
| *** sshnaidm is now known as sshnaidm|afk | 23:24 | |
| rlandy | you too | 23:27 |
| *** tosky has quit IRC | 23:38 | |
| *** rlandy is now known as rlandy|afk | 23:45 | |
| *** dsneddon has joined #oooq | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!