| *** brault has joined #oooq | 00:01 | |
| *** brault has quit IRC | 00:06 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 00:21 |
|---|---|---|
| *** brault has joined #oooq | 00:23 | |
| *** brault has quit IRC | 00:28 | |
| *** tosky has quit IRC | 01:05 | |
| *** rlandy has quit IRC | 01:07 | |
| *** sshnaidm is now known as sshnaidm|off | 02:14 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 02:21 |
| *** ykarel|away has joined #oooq | 03:46 | |
| *** brault has joined #oooq | 03:49 | |
| *** brault has quit IRC | 03:54 | |
| *** skramaja has joined #oooq | 03:59 | |
| *** ykarel|away has quit IRC | 04:07 | |
| *** udesale has joined #oooq | 04:14 | |
| *** ykarel has joined #oooq | 04:15 | |
| *** ykarel_ has joined #oooq | 04:18 | |
| *** ykarel has quit IRC | 04:21 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci- (2 more messages) | 04:21 |
| *** ykarel_ has quit IRC | 04:28 | |
| *** ykarel has joined #oooq | 04:32 | |
| *** ykarel has quit IRC | 04:39 | |
| *** ykarel has joined #oooq | 04:42 | |
| *** brault has joined #oooq | 04:55 | |
| *** brault has quit IRC | 04:59 | |
| *** ykarel has quit IRC | 05:05 | |
| *** ykarel has joined #oooq | 05:26 | |
| *** ratailor has joined #oooq | 05:40 | |
| *** chkumar|out is now known as chandankumar | 05:40 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci- (2 more messages) | 06:21 |
| *** saneax has joined #oooq | 06:49 | |
| *** jfrancoa has joined #oooq | 06:54 | |
| *** Tengu has quit IRC | 06:57 | |
| *** kopecmartin|off is now known as kopecmartin | 07:07 | |
| *** Tengu has joined #oooq | 07:08 | |
| *** holser_ has joined #oooq | 07:13 | |
| *** quiquell|off is now known as quiquell | 07:16 | |
| quiquell | weshay: ack, will look into it | 07:17 |
| *** ykarel is now known as ykarel|lunch | 07:34 | |
| *** ccamacho has joined #oooq | 07:59 | |
| *** gkadam has joined #oooq | 08:09 | |
| marios | quiquell: any review requests? | 08:14 |
| marios | i have time for one/2 more | 08:14 |
| * marios goes review shopping | 08:14 | |
| *** gkadam_ has joined #oooq | 08:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci- (2 more messages) | 08:21 |
| *** jbadiapa has joined #oooq | 08:22 | |
| *** matbu has quit IRC | 08:22 | |
| *** gkadam has quit IRC | 08:23 | |
| *** ykarel|lunch is now known as ykarel | 08:23 | |
| *** matbu has joined #oooq | 08:25 | |
| quiquell | marios: o/ They are all from my part at WIP state | 08:28 |
| quiquell | marios: Do you need reviewing ? | 08:28 |
| *** brault has joined #oooq | 08:29 | |
| marios | quiquell: ack i checked this from bnemec he needs to add it to standalone scen 3 instead https://review.openstack.org/#/c/571321/5 | 08:30 |
| marios | quiquell: https://review.openstack.org/629777 this easy one maybe? | 08:31 |
| marios | quiquell: ah it has votes now sorry | 08:31 |
| marios | quiquell: otherwise same things merged yesterday thanks for help | 08:31 |
| marios | especially https://review.openstack.org/#/q/topic:scen2-standalone-vote+(status:open+OR+status:merged) | 08:31 |
| quiquell | A lot of stuff merged | 08:34 |
| quiquell | yey !!! | 08:34 |
| quiquell | the one about designate need more time for reviewing | 08:34 |
| marios | ssbarnea|rover: https://review.openstack.org/#/c/594077/5/scripts/oooq_common_functions.sh@126 i respectfully submit to you sir, that there is an error in your code. | 08:37 |
| marios | ssbarnea|rover: kindly review. in the event that I am mistaken i would be grateful that you'd inform me by return so that i can immediately rectify by revoting | 08:37 |
| marios | thanks | 08:37 |
| marios | :D | 08:37 |
| * marios recofee | 08:37 | |
| *** tosky has joined #oooq | 08:40 | |
| *** gkadam__ has joined #oooq | 08:45 | |
| *** gkadam_ has quit IRC | 08:47 | |
| *** jpena|off is now known as jpena | 08:54 | |
| *** panda|off is now known as panda | 08:55 | |
| panda | ... | 08:56 |
| *** chem has joined #oooq | 09:03 | |
| *** afazekas has quit IRC | 09:05 | |
| *** jrist has quit IRC | 09:05 | |
| *** jrist has joined #oooq | 09:06 | |
| ssbarnea|rover | marios: thanks! really good eye. another copy/paste bug. | 09:20 |
| * marios bows np | 09:21 | |
| *** rascasoft has joined #oooq | 09:23 | |
| ssbarnea|rover | marios: panda : ulimit error on mysql that i fixed yesterday, confirmed fix: https://review.openstack.org/#/c/629897/ | 09:25 |
| marios | panda: holser_ was asking me about taiga | 09:26 |
| marios | is free right? | 09:26 |
| marios | i mean you're not paying for it ? | 09:26 |
| marios | :D | 09:26 |
| marios | well i'm not anyway | 09:26 |
| marios | ! | 09:26 |
| panda | free as a bird | 09:26 |
| holser_ | thanks panda | 09:26 |
| marios | holser_: so on the sprint 4 i pointed you at we scroll down with priority ontop. left had side is the story, and then the rest of columns are tasks for that story | 09:27 |
| marios | make sense | 09:27 |
| marios | ? | 09:27 |
| marios | holser_: we move the tasks from parked --> done during the sprint | 09:28 |
| marios | ssbarnea|rover: ack on that noted but for later | 09:30 |
| *** bogdando has joined #oooq | 09:30 | |
| panda | ssbarnea|rover: I think 65536 is the default value for that parameter | 09:30 |
| ssbarnea|rover | panda: yep, this is why i proposed it. the -1 in puppet template is an invalid value. | 09:31 |
| panda | ssbarnea|rover: I mean, you could also have removed it | 09:31 |
| ssbarnea|rover | triggering an error inside mysql code, one that is ignored. | 09:31 |
| ssbarnea|rover | i know, but some may want to edit the puppet templates or modify them. safer to have a valid default value. | 09:32 |
| ssbarnea|rover | i guess nobody tested the defaults | 09:32 |
| ssbarnea|rover | the warn is even more confusing because the same command can really fail for other reasons but the last message on stderr is that one, which in fact is ignored. | 09:33 |
| ssbarnea|rover | i discovered this during debugging a job failure, and decided to address that aspect. | 09:34 |
| holser_ | marios thanks | 09:36 |
| holser_ | I am already looking at taiga | 09:36 |
| ssbarnea|rover | arxcruz|ruck: how we are supposed to addres disk full errors? last zuul msg on https://review.openstack.org/#/c/602248/ | 09:37 |
| ssbarnea|rover | on ze08.openstack.org | 09:37 |
| ssbarnea|rover | is up to us to fix or infra-root? | 09:37 |
| panda | holser_: if you have questions about taiga, ask marios. He'll be happy to help | 09:39 |
| panda | :D | 09:39 |
| holser_ | thanks I'll ask you both | 09:39 |
| holser_ | now I am learning it | 09:40 |
| arxcruz|ruck | ssbarnea|rover: checking with o-infra | 09:41 |
| arxcruz|ruck | ssbarnea|rover: where are you seeing the full disk error ? | 09:42 |
| ssbarnea|rover | gate-check job i posted above | 09:43 |
| arxcruz|ruck | ssbarnea|rover: https://review.openstack.org/#/c/602248/ ? | 09:44 |
| ssbarnea|rover | arxcruz|ruck: look at last zuul message: https://review.openstack.org/#/c/602248/ -- a very confusing message. | 09:44 |
| arxcruz|ruck | i'm not seeing this message | 09:44 |
| ssbarnea|rover | read the message! "DISK_FULL" | 09:45 |
| ssbarnea|rover | is not red | 09:45 |
| arxcruz|ruck | all i seeing is some elastic search bugs | 09:47 |
| marios | holser_: panda ack :D np maybe also consider joining one of our next calls (monday) to hear how we use it | 09:47 |
| arxcruz|ruck | ssbarnea|rover: man, I'm not seeing that error on my browser lol | 09:47 |
| *** derekh has joined #oooq | 09:48 | |
| arxcruz|ruck | ssbarnea|rover: https://pasteboard.co/HVVOPH6.png | 09:48 |
| ssbarnea|rover | arxcruz|ruck: yep! look at message, i know that the table view looks like this and does make no sense: zuul -1 but you see only green. | 09:49 |
| ssbarnea|rover | the table is broken, i pasted the issue on #zuul channel | 09:50 |
| ssbarnea|rover | is missing the job that failed due to full disk | 09:50 |
| ssbarnea|rover | always look at zuul message, the table can be misleading | 09:51 |
| arxcruz|ruck | ssbarnea|rover: is there an option to enable zuul message? I'm not seeing it | 09:53 |
| ssbarnea|rover | yes, the left-bottom corner "Toggle CI" | 09:54 |
| arxcruz|ruck | got it | 09:54 |
| *** jtomasek has quit IRC | 10:03 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ (2 more messages) | 10:21 |
| marios | one more vote needed here please https://review.openstack.org/#/c/628275/ thanks | 10:24 |
| *** skramaja has quit IRC | 10:25 | |
| panda | marios: the commit message is not very explanatory | 10:31 |
| marios | panda: you're right. in fact this review no longer makes sense i will revote | 10:33 |
| marios | thank you | 10:33 |
| panda | marios: really ? | 10:33 |
| panda | marios: you don't need the zuul.d/layout exclusion anymore ? | 10:33 |
| marios | panda: sec updating the commit | 10:33 |
| panda | I usually dont' like to block people on the commit messdage | 10:34 |
| marios | panda: yeah but the patch was originally to add the job | 10:34 |
| marios | which was added already seems without that zuul.d but it is different change | 10:34 |
| marios | to what is decribed | 10:34 |
| marios | i just came there from the topic branch trying to get them all merged | 10:34 |
| marios | https://review.openstack.org/#/q/topic:replace-scen3+(status:open+OR+status:merged) | 10:34 |
| quiquell | I am going to workflow it | 10:34 |
| panda | quiquell: STAWP | 10:34 |
| marios | quiquell: nop | 10:34 |
| marios | no | 10:34 |
| marios | i -2 it | 10:35 |
| marios | sec | 10:35 |
| panda | I'll -3 it | 10:35 |
| marios | ha | 10:35 |
| quiquell | haha | 10:35 |
| quiquell | We don't want it ? | 10:35 |
| panda | I have the secret button in gerrit, labeled "BOOM" | 10:35 |
| panda | quiquell: yes but not like this | 10:35 |
| quiquell | panda: why ? | 10:36 |
| panda | quiquell: how the reproducer track ? | 10:36 |
| marios | quiquell: panda k now vote go | 10:36 |
| marios | ! | 10:36 |
| quiquell | panda: different implementation putting "files" at common place ? | 10:36 |
| quiquell | panda: are we talking about https://review.openstack.org/#/c/628275/ ? | 10:36 |
| panda | quiquell: not yet, just put the right commit message | 10:36 |
| marios | https://review.openstack.org/#/c/628275/5 | 10:36 |
| marios | HERE ^ | 10:36 |
| marios | STAWP FIGHTING | 10:37 |
| panda | hmpf, no fight today. | 10:37 |
| quiquell | Ahh the commit message ok | 10:37 |
| quiquell | :-) | 10:37 |
| panda | quiquell: on an unrelated request | 10:37 |
| panda | quiquell: how the reproducer tracking ? | 10:37 |
| panda | quiquell: do we need to revisit tasks ? | 10:38 |
| marios | note that the job is currently non voting (blocked on https://bugs.launchpad.net/tripleo/+bug/1811004 so it is expected to fail on tempest here) | 10:39 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,Triaged] | 10:39 |
| quiquell | panda: LÃLet me check theuser story | 10:39 |
| *** ratailor has quit IRC | 10:44 | |
| quiquell | panda: We don't want to add running jobs and dlrn hashes from the ansible role ? | 10:46 |
| quiquell | panda: I cannot find a taiga task i create about repro using sf RPMs | 10:49 |
| quiquell | panda: shit I think weshay has trash it it inlcluded the Dockerfiles for sf RPMs... can we find deleted tasks ? | 10:51 |
| *** ykarel_ has joined #oooq | 10:51 | |
| *** ykarel has quit IRC | 10:54 | |
| *** udesale has quit IRC | 10:54 | |
| quiquell | panda: looking for "Use sf RPMs to at a zuul base docker image" | 10:57 |
| *** ykarel has joined #oooq | 11:02 | |
| panda | quiquell: I'll try with the API | 11:02 |
| quiquell | panda: have just re-created it as a US | 11:03 |
| *** ykarel_ has quit IRC | 11:03 | |
| quiquell | panda: https://tree.taiga.io/project/tripleo-ci-board/us/599ç | 11:03 |
| panda | quiquell: did you need some description | 11:03 |
| panda | quiquell nothing even in the snapshots. | 11:05 |
| quiquell | panda: added description | 11:06 |
| panda | quiquell: do you remember the name of the task ? | 11:07 |
| panda | quiquell: I have last week's snapshots | 11:07 |
| quiquell | panda: Na don't worry has just created the US, is a US more than a task | 11:08 |
| panda | quiquell: so the deleted tasks didn't have any important content ? | 11:08 |
| panda | quiquell: do we need to reevaluate the tasks at this point ? | 11:09 |
| quiquell | panda: I don't think we are going to do the sf stuff this sprint | 11:09 |
| quiquell | panda: Going to create a design session on the repro for next week as asked by weshay | 11:10 |
| quiquell | panda: let's talk about it | 11:10 |
| panda | quiquell: early next week please :) | 11:10 |
| panda | or we can do it on sunday | 11:11 |
| panda | night | 11:11 |
| panda | a D&D session and the BAM, repro design | 11:11 |
| quiquell | panda: damn creatd the event at google | 11:12 |
| quiquell | panda: not at bluejeans | 11:12 |
| panda | quiquell: I can reach google in 20 minutes train from here, no problem for me | 11:15 |
| panda | quiquell: I'll se you there. | 11:15 |
| quiquell | panda: monday next week will be sshnaidm|off is not here today | 11:15 |
| *** dtantsur|afk is now known as dtantsur | 11:26 | |
| *** holser_ is now known as holser|food_figh | 11:38 | |
| panda | holser|food_figh: me too ! me too! where ? | 11:38 |
| holser|food_figh | panda - here :D | 11:39 |
| holser|food_figh | bbi1h | 11:39 |
| *** saneax has quit IRC | 11:48 | |
| jfrancoa | quiquell: hey, do you remember about the standalone_upgrade issue we were seeing the other day? that it didn't include the patch under test? https://bugs.launchpad.net/tripleo/+bug/1811371 | 12:04 |
| openstack | Launchpad bug 1811371 in tripleo "standalone_upgrade CI job not applying patch under test" [High,Triaged] | 12:04 |
| jfrancoa | quiquell: now the question is how to properly fix it | 12:04 |
| quiquell | jfrancoa: so we install the repo but we don't do the update ? | 12:06 |
| quiquell | jfrancoa: but the RPM was in the RPM lis t | 12:06 |
| quiquell | ssbarnea|rover, arxcruz|ruck: ^ can you help jfrancoa | 12:09 |
| quiquell | jfrancoa: but its explicitly disabled, you have to talk with Alex | 12:12 |
| quiquell | looks like its by design | 12:12 |
| *** d0ugal has quit IRC | 12:15 | |
| panda | quiquell: the release script is written in python3 ? | 12:20 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 12:21 |
| quiquell | tox-py3 is working there so it should be compatible | 12:22 |
| quiquell | panda: also fedora28 is working | 12:22 |
| panda | quiquell: ok | 12:22 |
| panda | arxcruz|ruck: what's the repo for os tempest ? | 12:31 |
| panda | arxcruz|ruck: found openstack-ansible-os_tempest | 12:33 |
| *** jpena is now known as jpena|lunch | 12:38 | |
| panda | bogdando: in https://review.openstack.org/629859 , what is gear ? | 12:40 |
| bogdando | I don't know panda | 12:44 |
| bogdando | just saw an issue, then fixed it :D | 12:44 |
| fultonj | panda: i'm trying to figure out why a boolean I'm passing to a j2 template has the opposite value of what i'm expecting | 12:49 |
| fultonj | https://review.openstack.org/#/c/617368/18/roles/standalone/templates/standalone.sh.j2@1 | 12:49 |
| fultonj | if i set a boolean in a oooq featureset, then can i override it in a playbook in oooq-extras ? | 12:50 |
| fultonj | panda: ^ that's my general question | 12:50 |
| *** holser|food_figh is now known as holser_ | 12:58 | |
| *** dtantsur is now known as dtantsur|brb | 12:59 | |
| *** d0ugal has joined #oooq | 13:02 | |
| arxcruz|ruck | quiquell: jfrancoa hey, what's up? | 13:09 |
| jfrancoa | arxcruz|ruck: hey | 13:09 |
| arxcruz|ruck | panda: for the tempest role that we will use in the future? yes, openstack-ansible-os_tempest | 13:09 |
| arxcruz|ruck | hey jfrancoa | 13:09 |
| arxcruz|ruck | ssbarnea|rover: so, how will we fix the logs? | 13:09 |
| jfrancoa | arxcruz|ruck: well...it's mainly an issue with the stanadlone upgrades job, that it's not installing the new tripleo-heat-templates package with the patch under test. anyway I'll discus it with sofer first to see if we can find a good way to solve it | 13:10 |
| arxcruz|ruck | jfrancoa: okay | 13:11 |
| weshay | arxcruz|ruck, ssbarnea|rover you guys have eyes on master promotion? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/866b0c7/job-output.txt.gz | 13:14 |
| fultonj | if i set a boolean in a oooq featureset, then can i override it in a playbook in oooq-extras ? | 13:15 |
| fultonj | i guess i'll find out https://review.openstack.org/#/c/615987/5..6/config/general_config/featureset057.yml | 13:15 |
| fultonj | i'll see how ^ affects https://review.openstack.org/#/c/617368/18/roles/standalone/templates/standalone.sh.j2@1 | 13:15 |
| ssbarnea|rover | weshay: working now to findout why a build was killed running out of disk space for logs (5GB!) | 13:15 |
| * fultonj waits | 13:15 | |
| fultonj | just thought I'd ask before waiting 3 hours to see what the compute did in case anyone knew already | 13:16 |
| panda | fultonj: wait | 13:17 |
| panda | fultonj: I have slow digestion | 13:17 |
| fultonj | he :) | 13:18 |
| fultonj | np | 13:18 |
| *** rlandy has joined #oooq | 13:18 | |
| panda | fultonj: so the default value is in the featureset file and you're trying to override it somewhere else ? | 13:20 |
| fultonj | panda: yes | 13:21 |
| fultonj | panda: it could really be anywhere though, whatever will work | 13:21 |
| panda | fultonj: ok, that's actually not possibile by design, with some notable exceptions | 13:21 |
| fultonj | ok, good to know. thanks | 13:21 |
| panda | fultonj: featureset where designed to have the last word on what value a variable has | 13:21 |
| fultonj | what's going on here is that i have a j2 template which is always using the default values from the feature set | 13:21 |
| fultonj | https://review.openstack.org/#/c/617368/18/roles/standalone/templates/standalone.sh.j2@1 | 13:21 |
| fultonj | and if, as you say, featuresets have the last word by design, then that's fine | 13:22 |
| fultonj | what i did, was remove the defaults from the feature set... | 13:22 |
| fultonj | and set them directly in my playbook | 13:22 |
| panda | fultonj: and it's still not working ? | 13:22 |
| fultonj | https://review.openstack.org/#/c/617368/18..19/playbooks/split-controlplane-standalone.yml | 13:22 |
| fultonj | panda: it will probably work now | 13:23 |
| fultonj | i set them directly in my playbook 2 minutes ago | 13:23 |
| fultonj | glad to know i'm heading in the right direction now, thanks | 13:23 |
| fultonj | i'll see how it goes on the next run | 13:24 |
| fultonj | panda++ | 13:24 |
| hubbot1 | fultonj: panda's karma is now 10 | 13:24 |
| panda | fultonj: well split_control_plane_controller: true would be a perfect candidate for a featureset file | 13:24 |
| fultonj | panda: ah | 13:25 |
| panda | fultonj: because it's the variable that activates the test on edge | 13:25 |
| panda | but if you need to override it often, ... | 13:25 |
| fultonj | panda: so i have split_control_plane 57 and based on that variable it makes an edge compute or an edge controller that exports it's params for the edge compute | 13:25 |
| fultonj | i wasn't sure if i should have ONE feature set with a switch OR two feature sets | 13:25 |
| panda | fultonj: features are *by no writte definion) flags that activate main things you have to test in your job | 13:26 |
| panda | so if you already have split_control_plane as main variable to enable the edge testing, it's enough | 13:27 |
| panda | marios: so $job = standalone-scenario001 ? | 13:27 |
| fultonj | https://review.openstack.org/#/c/615987/5..7/config/general_config/featureset057.yml | 13:28 |
| *** zul has joined #oooq | 13:28 | |
| fultonj | panda: if it's enough, i'd rather keep it as is so i don't copy/paste code | 13:28 |
| marios | panda: well yes i hope so , your patch made it so anyway [1] https://review.rdoproject.org/r/#/c/17744/ # config | 13:29 |
| fultonj | i usually think it's better to make something more flexible than make two copies with a minor change | 13:29 |
| fultonj | but i want to follow whatever patterns the CI experts think best for their own reasons | 13:29 |
| marios | panda: but not sure what the question/context is behind your query here | 13:29 |
| panda | fultonj: sorry, what's the configuration that activates the split plane standalone then ? | 13:29 |
| panda | marios: in user story #585 | 13:30 |
| fultonj | 57 | 13:30 |
| fultonj | in "split plane standalone" you have two types of nodes | 13:30 |
| fultonj | 57 makes both types based on a boolean | 13:30 |
| fultonj | true --> type1 | false --> type2 | 13:30 |
| marios | panda: oh i see yes | 13:31 |
| panda | fultonj: the featureset design was a rather long and difficult discussion. We landed with what we have to satisty a zillion different parameters. | 13:31 |
| chandankumar | weshay: 1:1 | 13:31 |
| marios | panda: i thought i'd start with that since it is the thing which started the discussion in the first place | 13:31 |
| panda | it's the perfect trade off because nobody liked it. | 13:31 |
| weshay | chandankumar, alan and I are in | 13:31 |
| marios | panda: plan is do one see how it looks/works then go on more if time | 13:31 |
| fultonj | panda: if this latest patchset works i'll be happy with it | 13:32 |
| panda | fultonj: but howdo you activate your playbook instead of the standard playbook ? mmhhh I would need a summary of your changes . are they in the user story ? | 13:32 |
| panda | fultonj: and for the next three week I'm officialy dedicated in giving you a hand, so hit me | 13:33 |
| fultonj | panda: wow, thanks | 13:33 |
| fultonj | panda: fwiw, i'll be in be brno next week doing podman things (diff project) | 13:33 |
| panda | marios: I don't think that use is for a particular job, we need to create templates with group of jobs | 13:34 |
| panda | marios: things is I don't know what these groups are | 13:34 |
| fultonj | panda: fultonj: but howdo you activate your playbook instead of the standard playbook... | 13:34 |
| fultonj | https://review.openstack.org/#/c/615988/9/zuul.d/standalone-jobs.yaml | 13:34 |
| panda | fultonj: oh, here it is, you're overriding the playbooks | 13:35 |
| marios | panda: ack we have a template already for the standalones https://github.com/openstack-infra/tripleo-ci/blob/63eb796c99fbc54b5589f5198f6dced7c157944f/zuul.d/standalone-jobs.yaml#L77-L78 trying to understand how/where we should override (in top most layoutn.yaml?) | 13:36 |
| panda | fultonj: so you're actually not using our main workflow | 13:36 |
| panda | marios: yep, those weree the question I wanted to ask to alex | 13:36 |
| panda | marios: also it seems he already has the implementation in the epic description | 13:36 |
| panda | so not sure what is left to do | 13:37 |
| marios | panda: well, post it for starters. then get it merged | 13:37 |
| marios | then do more jobs | 13:37 |
| marios | $$$ | 13:37 |
| panda | fultonj: got time to chat ? | 13:37 |
| panda | marios: subject ? | 13:37 |
| fultonj | panda: yes | 13:37 |
| marios | panda: wot? | 13:38 |
| panda | marios: what's "it" in "post it" ? | 13:38 |
| marios | panda: a review in tripleo-ci under zuul.d to add files: for the standalone scenario1 | 13:38 |
| marios | (at least) | 13:38 |
| marios | panda: to include all the files from each of the other repos | 13:38 |
| marios | common/tht/oooq etc | 13:38 |
| panda | marios: and then add the other scenario jobs to the same template ? | 13:39 |
| panda | marios: and the remove the template that we already have ? | 13:40 |
| marios | panda: right and each of their files | 13:40 |
| panda | marios: take a look at the code in the epic https://tree.taiga.io/project/tripleo-ci-board/epic/568 | 13:40 |
| panda | marios: seems to be all there alread ... | 13:41 |
| marios | panda: ok we should get it into the tripleo-ci repo right? sanity check those list of files. make sure it works as we expect across the repos with test changes | 13:42 |
| marios | ?? | 13:42 |
| marios | panda: etc | 13:42 |
| marios | panda: for starters | 13:42 |
| marios | panda: but still not 100% clear to me | 13:42 |
| panda | marios: I guess so, but it seems 80% done at this point | 13:42 |
| panda | marios: yep, we need to shower alex with questions | 13:42 |
| marios | panda: no i disagree | 13:42 |
| marios | panda: i mean it is 80% understood maybe | 13:43 |
| marios | panda: 0% done | 13:43 |
| marios | nothing posted even one review | 13:43 |
| marios | and definitely not merged | 13:43 |
| marios | panda: am i wrong | 13:43 |
| * panda pokes marios in the eyes "three stooges'" style | 13:43 | |
| marios | :D | 13:43 |
| *** jpena|lunch is now known as jpena | 13:44 | |
| panda | marios: wanna proceed ? or we clarify with alex first ? | 13:44 |
| panda | fultonj: almost ready | 13:45 |
| marios | panda: both i will proceed until i have a specific question, go ahead if you want to harrass alex now :) | 13:45 |
| panda | marios: alone ? never. | 13:45 |
| bogdando | rlandy: hi. Could you please update the last depends-on in https://review.rdoproject.org/r/#/c/18058 so we could test the fix https://review.openstack.org/#/c/630165/ ? | 13:45 |
| * rlandy looks | 13:46 | |
| panda | fultonj: bluejeans ? | 13:46 |
| bogdando | I cannot send my amendment to gerrit as git-review tool tries to access rpmfactory.beta.rdoproject.org:29418/testproject | 13:46 |
| bogdando | and I have no creds setup there it seems | 13:46 |
| panda | fultonj: 3492508669 | 13:47 |
| panda | bogdando: it's the old repo | 13:47 |
| bogdando | oh, right so it has a wrong gitreview file | 13:47 |
| panda | bogdando: review.rdoproject.org, and github credentials | 13:47 |
| rlandy | bogdando: https://review.rdoproject.org/r/testproject | 13:47 |
| rlandy | bogdando: if you still can't access, nhicher added my creds | 13:47 |
| bogdando | the repo just has a wrong .gitreview | 13:48 |
| fultonj | panda: yep, joining | 13:48 |
| rlandy | bogdando: also - https://review.openstack.org/#/c/629186/7/config/general_config/featureset020.yml disables idempotent check for master so we can keep the pipelines up | 13:50 |
| rlandy | will put in a separate review for your testing | 13:51 |
| bogdando | rlandy: not good | 13:51 |
| bogdando | let's wait for that fix and move it back) | 13:51 |
| bogdando | ok, thanks, rlandy | 13:52 |
| rlandy | bogdando: that patch has another fix the pipelines needs | 13:52 |
| rlandy | I'll get you a new test patch | 13:52 |
| rlandy | one sec | 13:52 |
| rlandy | panda, weshay: https://review.openstack.org/#/c/629186/ can I get another review on this | 13:53 |
| rlandy | needs to merge for pipeline | 13:53 |
| rlandy | ssbarnea|rover: arxcruz|ruck: ^^ fyi | 13:53 |
| rlandy | fs020 fix | 13:53 |
| rlandy | need to w+ that | 13:53 |
| arxcruz|ruck | yes please :D | 13:54 |
| quiquell | rlandy: I can workflow it | 13:54 |
| rlandy | quiquell: cool - sorry forgot you have the power | 13:54 |
| arxcruz|ruck | quiquell: do it! just do it! | 13:54 |
| quiquell | done | 13:55 |
| arxcruz|ruck | quiquell: https://www.youtube.com/watch?v=ZXsQAXx_ao0 | 13:55 |
| rlandy | bogdando: https://review.openstack.org/#/c/630229 | 13:57 |
| rlandy | ^^ you can use that as a depends-on patch | 13:58 |
| rlandy | quiquell: sorry - saw your review from yesterday - got caught up in pipeline fixes | 13:58 |
| rlandy | will look today | 13:58 |
| quiquell | rlandy: No problem merged already, just make pre.yaml compatible with humans as we where talking the other day | 13:59 |
| ssbarnea|rover | weshay: arxcruz|ruck: i finished documenting the issue with disk full that affected one of the gate check jobs, but i need to push more to making things happen. still, i want to know if there is something else more important to look at before doing this. | 14:00 |
| weshay | ssbarnea|rover, atm.. pushing on the promotion jobs | 14:01 |
| ssbarnea|rover | weshay: ok, looking now at them. btw, try to wf https://review.openstack.org/#/c/629897/ if possible. | 14:02 |
| bogdando | rlandy: thanks, but could you please update https://review.rdoproject.org/r/#/c/18058 for me, with that patch and https://review.openstack.org/#/c/630165/ ? | 14:07 |
| bogdando | rlandy: nvm, done https://review.rdoproject.org/r/#/c/18058/ | 14:09 |
| bogdando | wow, a 2h from a submission to the check done oO | 14:10 |
| bogdando | zuul makes magic | 14:10 |
| *** jtomasek has joined #oooq | 14:10 | |
| rlandy | bogdando: no - I just rebased a patch | 14:11 |
| rlandy | rechecked | 14:11 |
| bogdando | okay | 14:11 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 14:21 |
| rfolco | arxcruz|ruck, I hit this bug today... any clues ? https://bugs.launchpad.net/tripleo/+bug/1792375 | 14:24 |
| openstack | Launchpad bug 1792375 in tripleo "[master] multinode noop job failure giving IOError: [Errno 26] Text file busy: '/home/zuul/workspace/.quickstart/bin/python'" [Critical,Fix released] - Assigned to Sorin Sbarnea (ssbarnea) | 14:24 |
| rfolco | ssbarnea|rover, you probably have an idea as you fixed it last time ^ | 14:25 |
| *** ykarel is now known as ykarelaway | 14:27 | |
| marios | panda: ftr the stuff in the epic was missing tripleo-quickstart and upppet-tripleo | 14:28 |
| marios | (e.g. featureset 052 is pretty important !) | 14:28 |
| arxcruz|ruck | rfolco: no idea, it seems not related to quickstart/tripleo but i'll take a look | 14:28 |
| arxcruz|ruck | ssbarnea|rover: nope, we covered all the bugs :) thanks for the update, made my friday easier i haven't time to do anything inthe morning with my baby sick :) | 14:29 |
| arxcruz|ruck | need to leave for 20 min, brb | 14:33 |
| panda | marios: did you chat with alex then ? | 14:34 |
| marios | panda: no | 14:34 |
| marios | panda: just stared at it a bit | 14:34 |
| fultonj | panda++ thanks | 14:35 |
| hubbot1 | fultonj: panda's karma is now 11 | 14:35 |
| marios | panda: proposing what i think makes sense lets see https://review.openstack.org/#/q/topic:centralise_scenario_standalone+(status:open+OR+status:merged) | 14:35 |
| ssbarnea|rover | rfolco: comments are right on that, this error is likely due to messing with virtualenv. | 14:36 |
| ssbarnea|rover | rfolco: any chance of having two jobs running at the same time? on jenkins it was more common issue. | 14:37 |
| panda | rlandy: quiquell I remember we were running undercloud idempotency only in single node undercloud install. It was an important test, but we were not convering that in all jobs | 14:37 |
| panda | also, now that ansible is used, not sure how much we want to test idempotency, maybe it's still important | 14:37 |
| rlandy | panda: we cover it in fs020 | 14:37 |
| rlandy | promotions | 14:37 |
| rfolco | ssbarnea|rover, this is an update job... so it runs update on top of an existing one... does this ring any bell? | 14:38 |
| panda | oh yeah because we don't have a sinlge node undercloud | 14:38 |
| panda | marios: did you update the user story with that ? :) | 14:38 |
| panda | marios: also, do you have answers to my questions in the epic ? | 14:38 |
| panda | marios: also this | 14:38 |
| panda | marios: and also that. | 14:38 |
| marios | panda: not yet, i am creating a task for it currently | 14:39 |
| quiquell | panda: Puff I recall something, but not much... I remember fixing something enabling/disabling it | 14:39 |
| ssbarnea|rover | rfolco: i do not know toci well, but does it run some python code in background by any chance? | 14:41 |
| quiquell | ssbarnea|rover: te-broker as doing something in background | 14:41 |
| ssbarnea|rover | this error is almost always cased by another python process running which prevents virtualenv from being updated. | 14:41 |
| quiquell | the testenv part | 14:41 |
| ssbarnea|rover | if they share the same virtualenv you are doomed | 14:42 |
| rfolco | and marios had hit that bug [1] few days ago | 14:42 |
| rfolco | https://bugs.launchpad.net/tripleo/+bug/1792375 | 14:42 |
| openstack | Launchpad bug 1792375 in tripleo "[master] multinode noop job failure giving IOError: [Errno 26] Text file busy: '/home/zuul/workspace/.quickstart/bin/python'" [Critical,Fix released] - Assigned to Sorin Sbarnea (ssbarnea) | 14:42 |
| rfolco | (after being fixed I mean) | 14:43 |
| *** kopecmartin is now known as kopecmartin|off | 14:45 | |
| ssbarnea|rover | rfolco: we still have toci_quickstart.sh files? both one used? | 14:47 |
| panda | I think only the old reproducer and not much else is using the old file at this point | 14:47 |
| rfolco | ssbarnea|rover, yes... if you look at the logs, the error happens in the 2nd time quickstart.sh runs, that is, the update part. | 14:47 |
| rfolco | ssbarnea|rover, so probably the VIRTUAL_ENV is gone and we are not detecting an existing venv in the update job?? | 14:49 |
| rfolco | talking about this https://review.openstack.org/#/c/602347/13/quickstart.sh | 14:49 |
| marios | panda: https://tree.taiga.io/project/tripleo-ci-board/task/603 | 14:49 |
| marios | panda: here you are lord panda | 14:49 |
| rfolco | panda, rlandy can you please get a new pass on f28_promotion topic? https://review.rdoproject.org/r/#/q/topic:f28_promotion+status:open | 14:52 |
| rfolco | panda, rlandy I can bj and go through these changes if you like | 14:53 |
| rlandy | rfolco: that would be good - can you give me 15 mins? | 14:53 |
| panda | marios: you forgot to kneel | 14:53 |
| panda | rfolco: ready whenever you are | 14:53 |
| panda | and rlandy is also | 14:53 |
| *** ykarelaway has quit IRC | 14:54 | |
| rfolco | panda, will wait for rlandy's green light | 14:54 |
| * marios deep bow to lord panda | 14:54 | |
| rfolco | marios, wash your mouth when saying lord's name, the patron saint of desperate causes of ci, the master panda | 14:56 |
| panda | how do you call it when you do a fake bow and then a uppercut ? | 14:56 |
| panda | crossbow! | 14:56 |
| panda | I'm on fire today. | 14:57 |
| ssbarnea|rover | rfolco: i am working on the venv part, trying to understande exactly what happens. | 14:57 |
| * marios maintain deep bow | 14:57 | |
| marios | back hurting | 14:57 |
| rfolco | ssbarnea|rover, yeah, just make sure its not often on updates job... not critical if its intermittent | 14:58 |
| panda | marios: if you want different check/gate combination, you'll need to create a different template | 14:59 |
| panda | marios: now sure we want to do that | 14:59 |
| panda | OOOHH so decentralized means on different repos | 15:00 |
| ssbarnea|rover | rfolco: is recurring often enough. Mainly the "New python executable in" is key because this means "python was updated since venv was created" | 15:00 |
| panda | I thought we were talking about files on the same repo | 15:00 |
| panda | all in tripleo-ci | 15:00 |
| panda | marios: so we will need a template for each repo ? | 15:00 |
| marios | panda: yeah i think we already do something similar in fact for example https://github.com/openstack-infra/tripleo-ci/blob/63eb796c99fbc54b5589f5198f6dced7c157944f/zuul.d/multinode-jobs.yaml#L12 | 15:00 |
| ssbarnea|rover | in some cases this is ok, in others it means "you have a broken virtualenv" now. | 15:00 |
| marios | panda: tripleo-multinode-container-minimal | 15:00 |
| marios | panda: which is consumed e.eg. https://github.com/openstack/tripleo-common/blob/b19daa966f6f52059c410eff6c6c4724db986aee/zuul.d/layout.yaml#L11 | 15:01 |
| marios | panda: right? | 15:01 |
| rfolco | ssbarnea|rover, yep, appreciate your eyes on it | 15:01 |
| marios | panda: anyway lets see what comments there are on the reviews. and if they are correct then we should see the jobs running in each of the https://review.openstack.org/#/q/topic:centralise_scenario_standalone+(status:open+OR+status:merged) | 15:01 |
| marios | panda: since this changes zuul.d/ | 15:02 |
| panda | marios: I never start a task from a review ... I know it's tempting, but laying code before the design is too close to trial and error in a phase where it shouldn't be needed. | 15:03 |
| quiquell | weshay: found the issue with launchpad | 15:03 |
| weshay | woot | 15:04 |
| quiquell | from httplib2 import _md5, re_url_scheme, re_slash | 15:04 |
| quiquell | ImportError: cannot import name re_slash | 15:04 |
| quiquell | root@telegraf:/# | 15:04 |
| quiquell | In the docker images at infra... | 15:04 |
| marios | panda: then consider this the design https://review.openstack.org/#/q/topic:centralise_scenario_standalone+(status:open+OR+status:merged) together with https://tree.taiga.io/project/tripleo-ci-board/task/603 ? | 15:04 |
| marios | panda: i don't see the problem ... earlier you were saying 'so this is basically done now? 80%" | 15:05 |
| marios | ? | 15:05 |
| weshay | arxcruz|ruck, panda so what needs to change w/ standalone001 re: disk full?' | 15:05 |
| * weshay reading irc in infra | 15:05 | |
| panda | weshay: ssbarnea|rover is following that | 15:06 |
| weshay | thanks | 15:06 |
| weshay | ssbarnea|rover, whats up w/ that? | 15:06 |
| *** dtantsur|brb is now known as dtantsur | 15:06 | |
| panda | marios: that was a question. For example, I'd like to know what repos we want to be centralized | 15:07 |
| marios | panda: "all the tripleo ones" | 15:08 |
| marios | panda: others typically have us in experimental but that can be another task on the story, sanity check mistral/aodh/ceilo/heat etc | 15:08 |
| panda | marios: that's for another life | 15:08 |
| panda | marios: I don't have the list of all the repos taht use tripleo-ci in mind | 15:09 |
| marios | panda: the repos i list in the task https://tree.taiga.io/project/tripleo-ci-board/task/603 | 15:09 |
| marios | panda: these are the ones that have standalone cenario 1-4 | 15:09 |
| quiquell | weshay: this should fix it https://review.rdoproject.org/r/18224 | 15:10 |
| panda | marios: updated description in 603 | 15:10 |
| arxcruz|ruck | back | 15:10 |
| arxcruz|ruck | weshay: collect logs reach 7gb for this job, and infra set a max of 5gb | 15:11 |
| marios | panda: nop https://github.com/openstack/tripleo-quickstart-extras/blob/master/zuul.d/layout.yaml | 15:11 |
| arxcruz|ruck | weshay: ssbarnea|rover is with a patch to skip files bigger than 1gb on zuul collect logs to avoid that | 15:11 |
| marios | panda: removed it but otherwise thanks | 15:11 |
| arxcruz|ruck | but it will require some discussion | 15:11 |
| marios | panda: indeed bullet list is nice | 15:12 |
| marios | panda: ppp | 15:12 |
| weshay | ssbarnea|rover, arxcruz|ruck do we know how the logs got that big to begin w/? | 15:12 |
| weshay | arxcruz|ruck, ssbarnea|rover what patch was https://s3.sbarnea.com/ss/190111-Change_Ia5bcd556_GATE_CHECK_for_TripleO__review.openstack_Code_Review_.png found in? | 15:12 |
| arxcruz|ruck | weshay: well, probably we are saving the images as well | 15:12 |
| marios | panda: but you raise a good point. perhaps we *should* have it in extras too? adding anote | 15:12 |
| marios | panda: cos that is where standalone roles lives soo maybe kinda important! | 15:13 |
| weshay | arxcruz|ruck, sounds right.. but we shouldn't be | 15:13 |
| weshay | I don't see images in other logs | 15:13 |
| arxcruz|ruck | weshay: https://review.openstack.org/#/c/630224/ | 15:13 |
| panda | marios: that's it ? no other tripleo repos ? | 15:14 |
| panda | marios: I thought tripleo was HUGE. | 15:14 |
| panda | marios: python-tripleoclient ? | 15:15 |
| weshay | arxcruz|ruck, ssbarnea|rover I like the idea of updating openstack-infra/zuul-jobs but I'm also interested in know how our job did that and making sure it does not repeat | 15:15 |
| panda | marios: tripleo-image-elements ? tripleo-puppet-elements ? ansible-role-tripleo* ? tripleo-ui ? tripleo-upgrade ? | 15:15 |
| marios | panda: yeah we can add those but right now, only those 4 have them | 15:16 |
| *** d0ugal has quit IRC | 15:16 | |
| arxcruz|ruck | weshay: we don't have access to the logs, because it wasn't uploaded :( | 15:16 |
| ssbarnea|rover | weshay: sure, now the reality is that is impossible to guarantee that such accidents will not happen. any looping-bug is likely to produce an "infinite" log file on the affected service. | 15:16 |
| panda | marios: not saying we need to do it now, just trying to take a look at the big picture. | 15:16 |
| marios | panda: ansible-role-tripleo no but yeah the rest we can add | 15:16 |
| marios | panda: sure :D | 15:16 |
| weshay | ssbarnea|rover, what file was that big though? | 15:16 |
| weshay | do we know? | 15:16 |
| ssbarnea|rover | i seen it too often, the only thing we can to is to avoid crashing CI when we encounter such a case, "truncate". | 15:17 |
| ssbarnea|rover | nope | 15:17 |
| panda | marios: so we can eventually prepare our review knowing that something else needs to be added | 15:17 |
| panda | marios: added later | 15:17 |
| marios | panda: acutally yeah here too https://github.com/openstack/ansible-role-tripleo-modify-image/blob/master/zuul.d/layout.yaml | 15:17 |
| marios | panda: ok, | 15:17 |
| marios | is lord panda now appeased? | 15:17 |
| ssbarnea|rover | the worker is killed when this happens AFAIK, so no chance to find out. but we should work on infra to see which options we have. | 15:18 |
| panda | marios: you added them to the list ? :) | 15:18 |
| ssbarnea|rover | this issue is not unique to tripleo | 15:18 |
| panda | marios: not yet. each repo will have differnt file patterns right ? | 15:18 |
| marios | panda: done | 15:18 |
| panda | so we need to create a template for each repo ? | 15:18 |
| marios | panda: they have to be added into the main review | 15:18 |
| panda | marios: I'll copy this list to the epic | 15:18 |
| panda | marios: they ? what they ? | 15:19 |
| marios | panda: here https://review.openstack.org/#/c/630239/2/zuul.d/standalone-jobs.yaml | 15:19 |
| marios | panda: 17:18 < panda> marios: not yet. each repo will have differnt file patterns right ? | 15:19 |
| marios | panda: they the file patterns we have to copy the files for each repo that has them (but mainly its the tripleo ones that do that) | 15:19 |
| marios | panda: like modify inmage is lhttps://github.com/openstack/ansible-role-tripleo-modify-image/blob/master/zuul.d/layout.yaml not overriding | 15:20 |
| panda | marios: I mean, will we need for example a tripleo-standalone-scenarios-full-tht and a tripleo-standalone-scenarios-full-puppet-tripleo ? | 15:20 |
| marios | panda: no | 15:20 |
| marios | panda: we should not | 15:20 |
| marios | panda: it should be as posted here https://review.openstack.org/#/q/topic:centralise_scenario_standalone+(status:open+OR+status:merged) | 15:20 |
| ssbarnea|rover | rfolco: weshay : regarding venv bug: reality is that recreating a virtualenv on top of existing one is risky business (can fail for several reasons, even most time works fine). | 15:20 |
| arxcruz|ruck | ssbarnea|rover: maybe fungi have the logs? | 15:20 |
| marios | panda: i.e. we define the template once, in ci. then wire it up in the projects as one liner here https://review.openstack.org/#/c/630248/1/zuul.d/layout.yaml | 15:21 |
| marios | panda: (repeat for each project) ^^ | 15:21 |
| weshay | ssbarnea|rover, help me understand how the venv relates to the logging of 1G+ files | 15:21 |
| rfolco | ssbarnea|rover, enforcing a fresh one everytime is bad idea? | 15:22 |
| panda | marios: ok maybe I'm missing something . The same files combination in one template in tripleo-ci will satisfy all the other repository that use tripleo-ci ? | 15:22 |
| *** quiquell is now known as quiquell|off | 15:22 | |
| marios | panda: i hope so. if it doesn't then i have understood wrongly | 15:22 |
| marios | panda: and i can correct the reviews accordingly | 15:22 |
| panda | marios: when I reach this level of dumbness I usually need a chat ... | 15:22 |
| ssbarnea|rover | rfolco: yep, i am againts recreation automaticale. with controlable param ok, but not default. | 15:23 |
| panda | marios: can we chat ? I have no idea what I'm talking about at this point | 15:23 |
| marios | panda: what have we been doign for the last 2 hours? | 15:23 |
| marios | panda: not right now i have to do somethign else first please | 15:23 |
| ssbarnea|rover | rfolco: i am inclined to propose to add some debug when failure happens and attempt to recreate it only if in-place fails. | 15:23 |
| panda | marios: ok | 15:23 |
| marios | panda: have a look at the reviews and we'll take it from there | 15:23 |
| marios | panda: the proof is in the reviews themselves | 15:23 |
| marios | panda: they lgtm | 15:23 |
| ssbarnea|rover | if create fails > rm dir > run create again. | 15:23 |
| marios | panda: because | 15:23 |
| marios | panda: if you check zuul queue | 15:24 |
| marios | panda: you can see the 'right ' combination being run for the standalone-scenarios | 15:24 |
| ssbarnea|rover | I could even do better: if create fails > print running py processes > rename folder > recreate > activate. | 15:24 |
| rfolco | ssbarnea|rover, if venv is not found, create a new venv (enforce a new venv name) | 15:24 |
| marios | panda: e.g. check http://zuul.openstack.org/status for https://review.openstack.org/#/c/630248/1 you can see that the scenario standalone jobs are being queued up to run | 15:25 |
| marios | panda: because in the template we have zuuld/* and this change is to zuul.d/ | 15:25 |
| marios | panda: does it help | 15:25 |
| ssbarnea|rover | rfolco: you dont get it, in this place we already mentioned the desired venv location, we need to respect it. | 15:25 |
| marios | panda: once zuul reports on https://review.openstack.org/#/q/topic:centralise_scenario_standalone+(status:open+OR+status:merged) it should be clear | 15:25 |
| marios | panda: if it worked or not | 15:25 |
| rlandy | rfolco: ready to meet | 15:25 |
| rfolco | ssbarnea|rover, send the patch, we discuss there :) | 15:26 |
| rfolco | rlandy, panda can we meet for few min ? | 15:26 |
| panda | rfolco: for you ? of course .. not. | 15:27 |
| rfolco | :( | 15:27 |
| rfolco | I beg | 15:27 |
| rfolco | on knees | 15:27 |
| rlandy | wow | 15:27 |
| rfolco | oh lord of ci | 15:27 |
| panda | please don't call me lord | 15:28 |
| panda | his majesty is more than enough | 15:28 |
| rfolco | patron saint of desperate mortals | 15:28 |
| rfolco | ok lets go | 15:28 |
| rfolco | https://bluejeans.com/5878458097 | 15:29 |
| panda | rfolco: give me 3 minutes | 15:29 |
| panda | marios: this is what I meant https://github.com/openstack/tripleo-heat-templates/blob/master/zuul.d/layout.yaml#L12 https://github.com/openstack/tripleo-common/blob/master/zuul.d/layout.yaml#L16 | 15:30 |
| panda | marios: same job, different files: | 15:30 |
| panda | that's why this is an epic and not just a task | 15:31 |
| marios | panda: yes thats fine | 15:31 |
| marios | panda: you ahve them all in one place | 15:31 |
| marios | panda: i still dont see the problem | 15:31 |
| marios | panda: isnt that EXACTLY what alex has in his example | 15:31 |
| marios | https://tree.taiga.io/project/tripleo-ci-board/epic/568 | 15:31 |
| *** ykarel has joined #oooq | 15:31 | |
| marios | panda: all the files from the different repos in once place | 15:32 |
| marios | 17:30 < panda> marios: this is what I meant https://github.com/openstack/tripleo-heat-templates/blob/master/zuul.d/layout.yaml#L12 https://github.com/openstack/tripleo-common/blob/master/zuul.d/layout.yaml#L16 | 15:32 |
| marios | panda: yes one is for tht and the other is for tripleo-common | 15:32 |
| marios | panda: the whole point here is to move them to tripleo-ci | 15:32 |
| marios | panda: what is the problem | 15:32 |
| marios | panda: i am missing somethign please | 15:32 |
| rfolco | rlandy, ready ? | 15:32 |
| marios | panda: yes it IS an epic | 15:33 |
| marios | panda: when did i say it wasn't | 15:33 |
| marios | panda: we have an EPIC which is for replace ALL the jobs acros all the repos | 15:33 |
| marios | panda: and a task which is for standalone scenario | 15:33 |
| panda | marios: I didn't say you did say that | 15:33 |
| marios | panda: so what is the problem ? | 15:33 |
| marios | panda: you've been giving me the spanish inquisition about this for 2 hours :D | 15:33 |
| rfolco | rlandy, where you? | 15:33 |
| panda | marios: that we'll need one template for each repo | 15:33 |
| weshay | ssbarnea|rover, arxcruz|ruck when you guys have time.. later if you are busy, I'd like to hear more details on the log issue | 15:33 |
| weshay | on blue | 15:33 |
| weshay | I don't get it yet | 15:34 |
| panda | marios: taht's why I want to chat, I'm giving you the wrong impression | 15:34 |
| panda | marios: this is design for me, poking holes and asking the what-ifs | 15:34 |
| marios | panda: ok monday though please not right now need to context switch for now. can you please post any comments on the reviews thanks https://review.openstack.org/#/q/topic:centralise_scenario_standalone | 15:34 |
| panda | marios: not rejecting your ideas, just worrying if it's going to cover all the cases | 15:35 |
| panda | marios: ok, I'll update the card/epic/tasks and report a summary on the review. | 15:35 |
| marios | panda: thanks for review time - its always easier to be specific. if it is wrong we'll soon find out once zuul reports it will be clear if it worked or not. otherwise if it is somehow wrong please point to what is broken so we can fix it. | 15:36 |
| marios | panda: i still consider this to be aprt of the design. it took me ~ half hour ish to post that topic branch | 15:36 |
| marios | panda: it will demonstrate if the idea works | 15:36 |
| marios | panda: if it does we can expand that to icnlude all the repos | 15:37 |
| marios | panda: if it doesn't we are still in design | 15:37 |
| ssbarnea|rover | weshay: joined your bj now. | 15:38 |
| marios | biab | 15:38 |
| rlandy | rfolco: here - sorry | 15:39 |
| rfolco | rlandy, https://bluejeans.com/5878458097 | 15:39 |
| weshay | ssbarnea|rover, k.. joining | 15:39 |
| chandankumar | panda: Hello | 15:43 |
| panda | chandankumar: hi | 15:44 |
| chandankumar | panda: please have a look at this error http://logs.openstack.org/00/627500/33/check/tripleo-ci-centos-7-scenario001-standalone/91af297/job-output.txt.gz#_2019-01-11_12_05_10_215836 | 15:44 |
| chandankumar | panda: and I didnot understand last comment here https://review.openstack.org/627500 | 15:45 |
| chandankumar | panda: need help here | 15:48 |
| weshay | ssbarnea|rover, arxcruz|ruck master container build failed again on 2019-01-11 15:14:29 | "2019-01-11 15:12:54,412 ERROR: 17482 -- standard_init_linux.go:203: exec user process caused \"no such file or directory\"", | 15:49 |
| weshay | :( | 15:49 |
| panda | chandankumar: sorry the -1 was unintentional | 15:49 |
| panda | chandankumar: the fact is that I checked and that configuration should work | 15:49 |
| chandankumar | panda: But I am not sure why it is not working | 15:50 |
| ssbarnea|rover | weshay: https://review.openstack.org/#/c/602248/ | 15:50 |
| panda | chandankumar: neither I am, I think we're missing a zuul configuration that is necessary for the project to be able to add the repo to required projets | 15:51 |
| panda | chandankumar: we should ask in infra | 15:51 |
| chandankumar | ssbarnea|rover: need help here http://logs.openstack.org/00/627500/33/check/tripleo-ci-centos-7-scenario001-standalone/91af297/job-output.txt.gz#_2019-01-11_12_05_10_215836 | 15:51 |
| chandankumar | panda: you mean the required porject parts? | 15:51 |
| panda | chandankumar: yes | 15:51 |
| chandankumar | panda: let me add that | 15:51 |
| panda | chandankumar: it's already there ... | 15:51 |
| *** trown|outtypewww is now known as trown|lunch | 15:53 | |
| weshay | ssbarnea|rover, arxcruz|ruck fyi.. working w/ rdo team on this https://tree.taiga.io/project/tripleo-ci-board/epic/601 | 15:54 |
| weshay | no action required other than your thoughts and input into design | 15:54 |
| *** d0ugal has joined #oooq | 15:56 | |
| *** jfrancoa has quit IRC | 15:58 | |
| panda | rlandy: rfolco https://etherpad.openstack.org/p/fedora28-tripleo-containers-build | 15:59 |
| *** dtrainor_ has quit IRC | 16:01 | |
| *** dtrainor has joined #oooq | 16:07 | |
| weshay | marios, task 582, get scen003 back to green is marked ready for review https://tree.taiga.io/project/tripleo-ci-board/us/533 | 16:10 |
| weshay | rfolco, ^ | 16:10 |
| weshay | I don't see a review | 16:10 |
| marios | weshay: checking | 16:10 |
| weshay | thanks | 16:10 |
| marios | weshay: thanks was mistake i put into progress | 16:10 |
| rfolco | weshay, can you join my blue pls ? | 16:11 |
| marios | weshay: i added some things to the bug at this point getting ready to reach out to deployment possibly but not sure if it container related. https://bugs.launchpad.net/tripleo/+bug/1811004 - comment #5/#6/#7/#8 add logstash png (happens on 8thJan), mysql, journal errors | 16:11 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,Triaged] | 16:11 |
| rfolco | weshay, with panda and rlandy - regarding containers build method | 16:12 |
| marios | weshay: appeared exactly 8th jan so something happened somehwere ... :D | 16:12 |
| weshay | rfolco, sure | 16:13 |
| marios | have a good one folks me prepares to disembark | 16:17 |
| *** gkadam__ has quit IRC | 16:18 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 16:21 |
| fultonj | panda: so we got the booleans working... yay! | 16:23 |
| fultonj | https://review.openstack.org/#/c/617368/19/playbooks/split-controlplane-standalone.yml@63 | 16:23 |
| fultonj | however... the file didn't get transfered ^ | 16:23 |
| fultonj | panda: so i guess that's the next mystery for next time | 16:24 |
| fultonj | next week or something | 16:24 |
| *** brault has quit IRC | 16:25 | |
| panda | fultonj: I have a hunch | 16:34 |
| panda | unch ? | 16:34 |
| panda | unc ? | 16:34 |
| fultonj | lunch... hmmm | 16:34 |
| panda | hunc ? | 16:34 |
| fultonj | sorry | 16:34 |
| panda | whatever | 16:34 |
| panda | fultonj: there is no tag defined | 16:34 |
| fultonj | panda: i had a tag before so i removed | 16:35 |
| fultonj | panda: i will add a tag back | 16:35 |
| fultonj | panda: but how do i get my new tag to be run? | 16:35 |
| fultonj | a hunch? https://pictures.abebooks.com/isbn/9781570821738-us.jpg | 16:36 |
| *** d0ugal has quit IRC | 16:36 | |
| panda | fultonj: do you need a new tag ? | 16:37 |
| panda | fultonj: standalone will not work ?\ | 16:37 |
| panda | rfolco: have you ever invited fireds to your home that stays and talk to each other even when you go to bed ? | 16:38 |
| panda | :D | 16:38 |
| panda | rfolco: I think it happened to your channel | 16:38 |
| fultonj | that's happened to me panda | 16:38 |
| *** holser_ has quit IRC | 16:38 | |
| fultonj | i will add the standalone tag | 16:39 |
| fultonj | thanks panda | 16:39 |
| *** bogdando has quit IRC | 16:39 | |
| rfolco | panda, lol | 16:43 |
| rfolco | panda, I am relieved ... thought you were talking about another "event" that happened in my channel today | 16:43 |
| panda | rfolco: you have no idea | 16:43 |
| *** ccamacho has quit IRC | 16:44 | |
| panda | rfolco: what we do in your channel while you're not there | 16:44 |
| rfolco | :) | 16:45 |
| panda | fultonj: so to give you context, if you run a playbook specifying the tags, only the tags will run | 16:46 |
| fultonj | panda: makes sense | 16:46 |
| panda | fultonj: unless you specify also "untagged" then all the tasks that dont' have a tag will run too | 16:46 |
| fultonj | i'm familiar with ansible tagging | 16:46 |
| panda | fultonj: and I don't see untagged anywhere | 16:46 |
| fultonj | and it looks like the playbook is run with the tag standalone | 16:46 |
| panda | fultonj: yep | 16:46 |
| panda | fultonj: http://logs.openstack.org/88/615988/10/check/tripleo-ci-centos-7-split-controlplane-standalone/a3fe717/job-output.txt.gz#_2019-01-11_13_54_04_415686 | 16:47 |
| fultonj | right! | 16:47 |
| *** ykarel is now known as ykarel|away | 16:55 | |
| weshay | rfolco, you are going to update the sprint 4 ss for stuart and send a summary to openstack-discuss[tripleo] ya? | 16:58 |
| rfolco | weshay, yes, just finishing my lunch, will do. | 17:04 |
| ssbarnea|rover | rfolco: good news, not only that I was able to find a way to reproduce the venv recreation bug, but also found a working and testable solution | 17:07 |
| weshay | quiquell++ | 17:07 |
| weshay | quiquell|off, I see bugs :) | 17:07 |
| weshay | lolz.. crap almost | 17:08 |
| rfolco | ssbarnea|rover, you da man | 17:08 |
| rfolco | ssbarnea|rover++ | 17:08 |
| hubbot1 | rfolco: ssbarnea|rover's karma is now 2 | 17:08 |
| rfolco | ssbarnea++ | 17:08 |
| panda | rfolco: you can't ++ a non existing nick | 17:10 |
| ssbarnea|rover | rfolco: weshay : please help me test/review https://review.openstack.org/#/c/630300/ -- clone it and run "tox -e molecule" | 17:10 |
| ssbarnea|rover | it should replicate the bug and test the bugfix. | 17:10 |
| ssbarnea|rover | if you look at the test, you will find me doing an insane sleep in background in order to proof what happens with lingering processes. it triggers the issue, qs detects it and applies workaround. | 17:12 |
| ssbarnea|rover | workaround = renames old venv, and creates a new one. also displays the running python processes, so we would know what happened. | 17:13 |
| weshay | k | 17:13 |
| rfolco | panda_the_lord++ | 17:13 |
| ssbarnea|rover | wow, this means that the "|rover" and "|ruck" karma is separated from the normal one. The funny part would be to see top karma on "<nick>|away" ones. | 17:15 |
| weshay | ssbarnea|rover, so.. sorry dumb manager question.. just run the playbook? | 17:15 |
| weshay | to test | 17:15 |
| panda | ssbarnea|rover: well, when you are rover, you're actually a different person | 17:15 |
| ssbarnea|rover | weshay: well, you could run directly the playbook. but it may mess your system (prob not in this case) | 17:16 |
| ssbarnea|rover | quite glad i managed to nail this bug before the end of the week. now I can focus on some kube' beef during the weekend. | 17:19 |
| weshay | ssbarnea|rover, arxcruz|ruck have you guys seen this? http://logs.openstack.org/61/628161/13/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/0feef33/job-output.txt.gz#_2019-01-11_15_59_38_474544 | 17:19 |
| arxcruz|ruck | weshay: no, opening a bug right now | 17:20 |
| ssbarnea|rover | now i did... | 17:20 |
| arxcruz|ruck | ssbarnea|rover: sorry, did you open it already? | 17:20 |
| weshay | it must be how it's called in multinode-overcloud-upgrade.yml | 17:20 |
| ssbarnea|rover | weshay: wonders of well desined roles: http://codesearch.openstack.org/?q=target_upgrade_version&i=nope&files=&repos= -- variable default defined in different repo :D | 17:21 |
| arxcruz|ruck | weshay: https://bugs.launchpad.net/tripleo/+bug/1811450 | 17:22 |
| openstack | Launchpad bug 1811450 in tripleo "Undefined variable target_upgrade_version in tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades" [High,Triaged] | 17:22 |
| ssbarnea|rover | weshay: out of curiosity: this job is doing and upgrade from master to master? | 17:23 |
| weshay | ssbarnea|rover, ya | 17:24 |
| weshay | I can explain later | 17:24 |
| weshay | it's built to test the workflow, not the ugprade | 17:25 |
| weshay | the standalone upgrade.. tests the upgrade of the bits | 17:25 |
| ssbarnea|rover | I have the impression that ansible-linter is now able to warn about variables used inside a role that are not declared as required. | 17:25 |
| weshay | it's an upstream thang | 17:25 |
| weshay | you dont have time to do both in a single job | 17:25 |
| *** panda is now known as panda|off | 17:25 | |
| ssbarnea|rover | problem avoidable by adding master as default value inside this role, at leas this is the first thing I would do. | 17:25 |
| ssbarnea|rover | right? | 17:26 |
| weshay | ssbarnea|rover, no.. | 17:26 |
| weshay | it's calculated | 17:26 |
| weshay | sec.. | 17:27 |
| * weshay gets the script | 17:27 | |
| *** trown|lunch is now known as trown | 17:28 | |
| ssbarnea|rover | codesearch tells me that for this job, we don't calculate anything. there are only two shells scripts that can define it. and apparently none is used. | 17:29 |
| *** jtomasek has quit IRC | 17:30 | |
| weshay | rlandy, do you recall the name of the output log file that had the output of the update/upgrade calculation | 17:31 |
| weshay | I see playbook_executions.log | 17:31 |
| weshay | but not the other one | 17:31 |
| weshay | ah.. | 17:32 |
| weshay | emit releases | 17:32 |
| rlandy | weshay: releases.sh? | 17:32 |
| weshay | ssbarnea|rover, arxcruz|ruck I think we're missing http://logs.openstack.org/75/628275/5/check/tripleo-ci-centos-7-containers-multinode/7e3fe3b/logs/emit_releases_file.log | 17:32 |
| * rlandy gets | 17:32 | |
| weshay | in that job | 17:32 |
| rlandy | it's not always run | 17:32 |
| weshay | python /home/zuul/src/git.openstack.org/openstack/tripleo-ci/scripts/emit_releases_file/emit_releases_file.py --stable-release master --featureset-file /home/zuul/src/git.openstack.org/openstack/tripleo-quickstart/config/general_config/featureset010.yml --output-file /home/zuul/workspace/logs/releases.sh --log-file /home/zuul/workspace/logs/emit_releases_file.log | 17:33 |
| weshay | arxcruz|ruck, ssbarnea|rover ^ | 17:33 |
| weshay | for example | 17:33 |
| *** rlandy is now known as rlandy|brb | 17:36 | |
| weshay | ssbarnea|rover, arxcruz|ruck I suspect we're missing something here: http://logs.openstack.org/61/628161/13/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/0feef33/job-output.txt.gz#_2019-01-11_14_27_51_584568 | 17:39 |
| weshay | for that job | 17:39 |
| *** tosky has quit IRC | 17:40 | |
| *** dtantsur is now known as dtantsur|afk | 17:48 | |
| *** jtomasek has joined #oooq | 17:59 | |
| *** derekh has quit IRC | 18:01 | |
| *** rlandy|brb is now known as rlandy | 18:07 | |
| weshay | ssbarnea|rover, arxcruz|ruck panda|off rlandy https://review.openstack.org/630363 | 18:07 |
| weshay | I think that's right | 18:07 |
| weshay | guess we'll see | 18:07 |
| *** jtomasek has quit IRC | 18:08 | |
| *** jtomasek_ has joined #oooq | 18:09 | |
| weshay | ssbarnea|rover, gate job failure on your change.. I haven't see this before http://logs.openstack.org/21/629121/5/gate/tripleo-ci-centos-7-containers-multinode/1f4b2c8/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-01-11_17_27_54 | 18:11 |
| ssbarnea|rover | weshay: about fs051 seems legit, still does not avoid any similar bug from re-occuring. better to merge both patches. | 18:12 |
| weshay | what's the other patch? | 18:12 |
| weshay | default master? | 18:13 |
| weshay | I don't think we want to do that | 18:13 |
| weshay | hrm. .maybe we do | 18:13 |
| ssbarnea|rover | weshay: i do like defaults. | 18:13 |
| weshay | ssbarnea|rover, let me think about it.. I like defaults as well.. | 18:14 |
| weshay | I guess I need to hear from quiquell|off, panda|off and rlandy | 18:14 |
| weshay | if they remember that work lolz | 18:14 |
| rlandy | weshay: I +2'ed | 18:14 |
| rlandy | what dod I miss? | 18:14 |
| weshay | ssbarnea|rover, we have an epic for zuulv3 that is not done | 18:14 |
| rlandy | did | 18:14 |
| weshay | we will return to it | 18:14 |
| *** jtomasek_ has quit IRC | 18:15 | |
| weshay | rlandy, setting a default ansible variable for the variables that emit_releases is setting | 18:15 |
| * weshay is not clear if that is a good idea | 18:15 | |
| ssbarnea|rover | i think, I seen something similar to the passwords key error. bot sure if it was same key but. | 18:15 |
| weshay | ssbarnea|rover, for example | 18:15 |
| weshay | if someone deployed pike.. than ran an upgrade w/o it set.. | 18:15 |
| weshay | it would upgrade to master | 18:16 |
| weshay | and FAIL hard | 18:16 |
| * rlandy reads back | 18:16 | |
| weshay | vs.. incorporating that script.. the user would get a nice.. NOT SUPPORTED DUDE | 18:16 |
| weshay | message | 18:16 |
| weshay | at least I think that is the idea | 18:16 |
| ssbarnea|rover | weshay: yep, afaik that failed attempt to upgrade to master is more useful than an undefined key. | 18:16 |
| weshay | ssbarnea|rover, not taking someone all the way through the install and upgrade | 18:17 |
| ssbarnea|rover | obviously that next step would be to make it "guess" the value. | 18:17 |
| weshay | better to fail at the emit_releases phase | 18:17 |
| weshay | ssbarnea|rover, that's what emit releases deos | 18:17 |
| weshay | does | 18:17 |
| ssbarnea|rover | but is late friday for me, no time or energy to argue :D | 18:17 |
| weshay | ssbarnea|rover, maybe making emit_releases a ansible module | 18:17 |
| weshay | meh | 18:18 |
| weshay | there is work there for sure | 18:18 |
| ssbarnea|rover | there are other things I would like to move upstream, like jinja2 extra test into ansible-lint, so we don;t have to maintain it. | 18:18 |
| ssbarnea|rover | emit-releases will never reach "ansible" repos, is too specific. | 18:19 |
| weshay | ssbarnea|rover, we can have custom modules in tq | 18:19 |
| ssbarnea|rover | yep, is ok. | 18:19 |
| ssbarnea|rover | weshay: re passwords: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22KeyError:%20'passwords'%5C%22 | 18:20 |
| ssbarnea|rover | first occurence almost one week ago | 18:20 |
| weshay | ssbarnea|rover, ya.. that job has been down for months | 18:21 |
| weshay | I think that's why this never was added | 18:21 |
| weshay | the job didn't work at the time | 18:21 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 18:21 |
| *** ykarel|away has quit IRC | 18:21 | |
| weshay | ssbarnea|rover, that's a good topic for a standup | 18:23 |
| weshay | re: the default | 18:23 |
| weshay | ssbarnea|rover, it's complicated and a little old.. so let's refresh everyone w/ the context in a standup and see what folks think | 18:23 |
| weshay | k | 18:23 |
| weshay | ? | 18:23 |
| ssbarnea|rover | sure | 18:24 |
| ssbarnea|rover | weshay: out of curiosity, what do you use for py3 on centos7? | 18:39 |
| ssbarnea|rover | installing with scl seems inconvenient as tox does not find them by default | 18:39 |
| weshay | ah.. I don't | 18:39 |
| ssbarnea|rover | and i need to manually activate | 18:39 |
| ssbarnea|rover | i am happy user of pyenv on macos, with a very long list of versions. also tox plays very well with it, | 18:42 |
| ssbarnea|rover | maybe i should do the same on centos | 18:42 |
| *** jpena is now known as jpena|off | 18:44 | |
| *** hubbot1 has quit IRC | 18:49 | |
| *** dmellado has quit IRC | 18:51 | |
| *** gouthamr has quit IRC | 18:51 | |
| *** openstackstatus has quit IRC | 19:43 | |
| *** openstackstatus has joined #oooq | 19:43 | |
| *** ChanServ sets mode: +v openstackstatus | 19:43 | |
| *** gouthamr has joined #oooq | 19:53 | |
| *** dmellado has joined #oooq | 19:58 | |
| *** hubbot1 has joined #oooq | 20:05 | |
| weshay | rfolco, when are you gone? | 20:06 |
| weshay | tomorrow? | 20:06 |
| rfolco | weshay, wed | 20:06 |
| weshay | k | 20:06 |
| rfolco | work on mond and tue | 20:06 |
| weshay | rlandy, so there is a little more to it.. http://logs.openstack.org/63/630363/1/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades/8ef38d3/job-output.txt.gz#_2019-01-11_18_59_50_966054 | 20:12 |
| weshay | 2019-01-11 18:59:50.966054 | primary | RuntimeError: Overcloud upgrade has to be mixed upgrades | 20:12 |
| weshay | ssbarnea|rover, fyi ^ | 20:12 |
| * weshay looks at the config for emit_releases | 20:13 | |
| weshay | should we call master to master mixed | 20:13 |
| rlandy | that should be dealt with | 20:13 |
| rlandy | error | 20:13 |
| * rlandy looks | 20:13 | |
| *** openstack has joined #oooq | 20:15 | |
| *** ChanServ sets mode: +o openstack | 20:15 | |
| rlandy | raise RuntimeError("Overcloud upgrade has to be mixed upgrades") | 20:16 |
| rlandy | why though?? | 20:16 |
| rlandy | probably because wer eonly dealing with overcloud upgrade | 20:16 |
| weshay | right | 20:17 |
| weshay | checking it out | 20:18 |
| weshay | I don't think we want to set mixed_upgrade for that job | 20:20 |
| weshay | because emit releases will deploy rocky and upgrade to stein | 20:20 |
| weshay | where this job is meant to be a no-op upgrade.. but go through the paces of the upgrade.. | 20:20 |
| weshay | may need to consult matbu or chem | 20:21 |
| weshay | http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/roles/run-test/templates/oooq_common_functions.sh.j2#n5 | 20:21 |
| rlandy | fs050 works | 20:21 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci- (2 more messages) | 20:21 |
| rlandy | we need marios | 20:21 |
| weshay | rlandy, that is just the undercloud though | 20:22 |
| rlandy | th error condition seems wrong though | 20:22 |
| rlandy | but I am not very clued in on these jobs | 20:22 |
| weshay | rlandy, so I *think* 51 is a special case | 20:23 |
| * weshay is going to try a change to https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/emit_releases_file/emit_releases_file.py#L143 | 20:23 | |
| rlandy | I'm comparing | 20:23 |
| weshay | not sure if we have a good comparison atm | 20:23 |
| rlandy | it's not a good comparison | 20:24 |
| rlandy | fs are very diff | 20:24 |
| rlandy | I just think we're failing it unnecessarily | 20:24 |
| rlandy | weshay: think you're right | 20:25 |
| rlandy | just change the condition to get by the error | 20:25 |
| rlandy | and then we can see what it spits out | 20:25 |
| rlandy | I think we just didn't consider the case | 20:25 |
| weshay | ya | 20:28 |
| weshay | rlandy, ya.. the epic is still open | 20:28 |
| weshay | we'll have the fun of coming back to it | 20:28 |
| weshay | need the containers for python3 running first though | 20:29 |
| rlandy | oh gee | 20:29 |
| rlandy | the fun never ends | 20:30 |
| weshay | right.. | 20:30 |
| *** fultonj has quit IRC | 20:58 | |
| *** holser_ has joined #oooq | 21:08 | |
| *** rlandy has quit IRC | 21:31 | |
| *** tosky has joined #oooq | 21:42 | |
| *** trown is now known as trown|outtypewww | 21:52 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci- (2 more messages) | 22:21 |
| *** holser_ has quit IRC | 23:16 | |
| *** rascasoft has quit IRC | 23:19 | |
| *** rascasoft has joined #oooq | 23:26 | |
| *** rascasoft has quit IRC | 23:37 | |
| *** rascasoft has joined #oooq | 23:44 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
