| *** agopi has quit IRC | 00:42 | |
| weshay | rlandy, it does seem suspect now doesn't it | 01:22 |
|---|---|---|
| *** apetrich has quit IRC | 02:27 | |
| *** rlandy has quit IRC | 02:42 | |
| *** skramaja has joined #oooq | 03:13 | |
| *** ykarel has joined #oooq | 04:00 | |
| *** udesale has joined #oooq | 04:00 | |
| *** ykarel has quit IRC | 04:06 | |
| *** ykarel has joined #oooq | 04:07 | |
| *** saneax has joined #oooq | 04:14 | |
| *** holser_ has joined #oooq | 04:59 | |
| *** ratailor has joined #oooq | 05:04 | |
| *** ykarel has quit IRC | 05:07 | |
| *** ykarel has joined #oooq | 05:25 | |
| *** jbadiapa has quit IRC | 05:32 | |
| *** jaosorior has quit IRC | 05:45 | |
| *** jaosorior has joined #oooq | 05:47 | |
| *** holser_ has quit IRC | 06:02 | |
| *** ccamacho has joined #oooq | 06:28 | |
| *** ratailor_ has joined #oooq | 06:34 | |
| *** ratailor has quit IRC | 06:35 | |
| *** ratailor_ has quit IRC | 06:36 | |
| *** ratailor has joined #oooq | 06:36 | |
| *** ratailor_ has joined #oooq | 06:38 | |
| *** ratailor__ has joined #oooq | 06:40 | |
| *** jfrancoa has joined #oooq | 06:40 | |
| *** ratailor has quit IRC | 06:41 | |
| *** ratailor__ has quit IRC | 06:41 | |
| *** ratailor__ has joined #oooq | 06:41 | |
| *** ratailor_ has quit IRC | 06:43 | |
| *** amoralej|off is now known as amoralej | 07:23 | |
| *** jtomasek has joined #oooq | 07:24 | |
| *** gkadam has joined #oooq | 07:25 | |
| *** tosky has joined #oooq | 07:29 | |
| *** jbadiapa has joined #oooq | 07:30 | |
| *** jtomasek has quit IRC | 07:33 | |
| *** bogdando has joined #oooq | 07:33 | |
| *** jtomasek has joined #oooq | 07:33 | |
| jfrancoa | morning folks, could anybody give me a hand to debug the legacy major upgrades job? it's failing when setting up the workspace and it doesn't even generate the reproduce-quickstart script: https://logs.rdoproject.org/16/563616/10/openstack-check/legacy-tripleo-ci-centos-7-container-to-container-upgrades-queens/8fe9266/logs/devstack-gate-setup-workspace-new.txt | 07:43 |
| ssbarnea | jfrancoa: i had a quick look but I don't know yet whats wrong with it. | 08:00 |
| jfrancoa | ssbarnea: I can't tell either, maybe the "fatal: Unable to create '/opt/stack/new/manila-ui/.git/index.lock': File exists."? I was about to open another similar job to compar logs with | 08:01 |
| ssbarnea | jfrancoa: yep, good idea. a new one should not reproduce it. i guess cleanup part should prevent the lock from remaining there. | 08:03 |
| marios | o/ folks jfrancoa do we have a bug for it yet? | 08:08 |
| marios | "No such file or directory (2)\nrsync: rename failed for \"/tmp/tmp2yxuinem/8fe9266d77bd4151a845914fe67a97cd/work/logs/logs/ara/index.html.gz\" (from logs/ara/.~tmp~/index.html.gz): No such file or directory (2)\nrsync: rename" | 08:09 |
| marios | jfrancoa: from https://logs.rdoproject.org/16/563616/10/openstack-check/legacy-tripleo-ci-centos-7-container-to-container-upgrades-queens/8fe9266/job-output.txt.gz#_2018-08-20_15_42_34_110440 | 08:09 |
| jfrancoa | marios: o/ nope, it's the first execution for legacy-container-to-container since this patch got merged yesterday https://review.rdoproject.org/r/15557 (it was supposed to fix the job, but something new appeared) | 08:09 |
| marios | but anyway would be nice to have a bug for all that info | 08:09 |
| marios | if we don't already | 08:09 |
| marios | to coordinate the debug & fix | 08:09 |
| marios | jfrancoa: ack o] | 08:09 |
| jfrancoa | marios: ok, I'll open it. Yes, I saw that error, but it appears in the log collecting part | 08:10 |
| jfrancoa | marios: something is wrong before that doesn't allow the job to even start up | 08:10 |
| ssbarnea | marios: maybe you can help here, I see "sort: write failed: standard output: Broken pipe" on a failed queens job at http://logs.rdoproject.org/48/590748/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/cca5ef7/job-output.txt.gz | 08:13 |
| ssbarnea | funny part is that i read about the "tail -n +1" trick, which aparently does not work, at least here. | 08:14 |
| jfrancoa | marios: bug opened https://bugs.launchpad.net/tripleo/+bug/1788128 , I'll keep investigating it meanwhile | 08:21 |
| openstack | Launchpad bug 1788128 in tripleo "[CI upgrades] legacy-tripleo-ci-centos-7-container-to-container-upgrades-queens failing" [Undecided,New] | 08:21 |
| marios | jfrancoa: thank you | 08:21 |
| *** jfrancoa has quit IRC | 08:30 | |
| *** dsneddon has quit IRC | 08:44 | |
| *** jfrancoa has joined #oooq | 08:46 | |
| *** dsneddon has joined #oooq | 08:48 | |
| *** jfrancoa has quit IRC | 08:52 | |
| *** jfrancoa has joined #oooq | 08:52 | |
| *** holser_ has joined #oooq | 08:53 | |
| *** sshnaidm|off is now known as sshnaidm | 09:21 | |
| *** apetrich has joined #oooq | 09:23 | |
| *** chem has joined #oooq | 09:29 | |
| *** dtantsur|afk is now known as dtantsur | 09:59 | |
| *** ykarel is now known as ykarel|lunch | 10:29 | |
| *** Tengu has quit IRC | 10:32 | |
| *** Tengu has joined #oooq | 10:33 | |
| *** ratailor__ has quit IRC | 10:37 | |
| *** ratailor has joined #oooq | 10:37 | |
| *** ratailor_ has joined #oooq | 10:40 | |
| *** ratailor_ has quit IRC | 10:40 | |
| *** ratailor has quit IRC | 10:43 | |
| *** amoralej is now known as amoralej|lunch | 11:03 | |
| *** ykarel|lunch is now known as ykarel | 11:10 | |
| *** udesale has quit IRC | 11:25 | |
| *** trown|outtypewww is now known as trown | 11:50 | |
| ssbarnea | marios: i observe that a huge number of update jobs are timing out, see http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates | 11:54 |
| marios | ssbarnea: there is a but for that | 11:55 |
| ssbarnea | based on duration, I am inclined to believe is a performance issue, should we increase the expiration time a little bit to address it? | 11:55 |
| marios | heh bug | 11:55 |
| marios | sec | 11:55 |
| marios | * https://bugs.launchpad.net/tripleo/+bug/1786764 tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare | 11:55 |
| openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 11:55 |
| marios | fixes https://review.openstack.org/#/c/592577/ https://review.openstack.org/#/c/593169/ | 11:55 |
| ssbarnea | success ones are very close to the timeout, this is why i asked | 11:55 |
| *** sanjayu_ has joined #oooq | 12:06 | |
| ssbarnea | sshnaidm or panda : can we get +W on https://review.openstack.org/#/c/592577/ ? | 12:08 |
| *** saneax has quit IRC | 12:10 | |
| *** amoralej|lunch is now known as amoralej | 12:11 | |
| panda | ssbarnea: approved, please change your nick | 12:11 |
| sshnaidm | panda, I have a comment about it | 12:12 |
| weshay | ssbarnea need to renick as ssbarnea|ruck | 12:13 |
| ssbarnea | sadly i am afraid changing nick is not possible due to two reasons: i am using riot/synapse and we are required to use registered nicks on our channels due to recent spam. i tried to look for this this morning but didn't find a solution. | 12:13 |
| weshay | oh.. ha | 12:13 |
| weshay | that is a good reason | 12:14 |
| ssbarnea | i am going to ask on matrix channel about this, maybe there is a solution.... | 12:14 |
| bogdando | sshnaidm: hi! answered https://review.openstack.org/#/c/594055/4/roles/undercloud-deploy/tasks/post-install.yml | 12:14 |
| ssbarnea | but we may have a workaround: using topic to mention roles | 12:14 |
| weshay | ssbarnea, you could open another irc client.. and log into freenode and rh as ssbarnea|ruck | 12:14 |
| weshay | or I could | 12:15 |
| *** weshay is now known as weshay|ssbarnea_ | 12:15 | |
| ssbarnea | i will try, just after the meeting | 12:15 |
| *** weshay|ssbarnea_ is now known as weshay | 12:15 | |
| weshay | ha.. too long | 12:15 |
| weshay | ssbarnea, k | 12:15 |
| weshay | thanks | 12:15 |
| sshnaidm | panda, hmm.. gate jobs still run even if you remove +w | 12:17 |
| bogdando | weshay: sshnaidm ssbarnea hi folks, so with all of these https://review.openstack.org/#/q/topic:localcon+(status:open+OR+status:merged) e2e with virthost localhost seems like working, at least now I have overcloud deployment in progress | 12:18 |
| bogdando | gonna update the extras patch now | 12:18 |
| sshnaidm | bogdando, I mean ansible-fetch is always downloads to localhost | 12:18 |
| sshnaidm | bogdando, when I run quickstart from localhost with virthost it will download to /home/stack? but I don't have it in my laptop | 12:19 |
| bogdando | sshnaidm: well, crap :) missed that, right | 12:19 |
| sshnaidm | bogdando, actually you need delegate_to | 12:19 |
| bogdando | undercloud groups [0]? | 12:20 |
| weshay | ykarel, sshnaidm no luck w/ the patch eh | 12:20 |
| bogdando | right, so... fixing | 12:20 |
| weshay | marios|rover, ssbarnea trying to get https://review.openstack.org/#/c/593383/ merged | 12:20 |
| marios | weshay: ack just finished call gimme 2 mins | 12:21 |
| weshay | no prob | 12:21 |
| marios | oh its +a already | 12:21 |
| weshay | ya | 12:21 |
| marios | weshay: yeah so we are waiting for latest recheck from sagi | 12:21 |
| ykarel | weshay, hmm :( it's fails in gate with timeout in one or other job | 12:21 |
| ykarel | last time it failed in post | 12:21 |
| sshnaidm | weshay, which patch? about hash? | 12:21 |
| marios | weshay: https://trello.com/c/vudM8h1r/700-cixlp1787817tripleociproa-emitreleasespy-is-looking-for-a-promoted-version-of-rocky-and-failing | 12:21 |
| marios | weshay: we just discussed on the call ^ | 12:21 |
| marios | brb | 12:22 |
| weshay | this just merged https://review.openstack.org/#/c/593169/ | 12:22 |
| weshay | so maybe things will get better now | 12:22 |
| sshnaidm | marios, weshay I noticed a lot of timeouts lately | 12:22 |
| weshay | sshnaidm, ya | 12:22 |
| dtantsur | quick q folks: is it possible to use --release rocky or --release tripleo-ci/rocky already? | 12:22 |
| weshay | a | 12:22 |
| weshay | r | 12:22 |
| weshay | a | 12:22 |
| weshay | dtantsur, no promotions on rocky | 12:22 |
| weshay | unless we fake it | 12:22 |
| dtantsur | okie, so master for now | 12:23 |
| ssbarnea | sshnaidm: this is why I asked about the timeouts, as I seen them too, and because passing jobs duration was very close to the failed one I wondered if adding 10mins should not fix it. | 12:23 |
| sshnaidm | weshay, ara is wip, didn't forgot | 12:26 |
| weshay | sshnaidm, negative | 12:26 |
| weshay | sorry | 12:26 |
| weshay | ssbarnea, not allowed | 12:26 |
| marios | sshnaidm: yah so htere is at least one known thing for that with the updates job at * https://bugs.launchpad.net/tripleo/+bug/1786764 tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare | 12:26 |
| openstack | Launchpad bug 1786764 in tripleo "tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates times out on prepare" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 12:26 |
| marios | fixes https://review.openstack.org/#/c/592577/ https://review.openstack.org/#/c/593169/ | 12:26 |
| sshnaidm | ssbarnea, fyi ^^ | 12:26 |
| ssbarnea | ok, read the comment, good point, this will cause some issues | 12:28 |
| ssbarnea | last time i checked it was not possible to use a variable previous defined in the same step in ansible. not sure if it would work in defaults, need to check. | 12:29 |
| ssbarnea | I guess update_containers_repo: "{{ gating_repo_name }},delorean-current" should work... | 12:29 |
| sshnaidm | bogdando, do you join? | 12:31 |
| bogdando | sshnaidm: yes | 12:32 |
| *** ssbarnea is now known as ssbarnea|ruck | 12:35 | |
| *** rlandy has joined #oooq | 12:41 | |
| ssbarnea|ruck | marios: weshay : regarding timeouts, does it make sense to increase timeout from 180min to 200min? | 12:49 |
| rlandy | marios|rover: hi - looking at the reviews on https://trello.com/c/lrSUMaqw/885-translate-tripleosh-bootstrap-subnodes-into-a-series-of-tasks-s17 | 12:49 |
| rlandy | I marked out the two that have merged | 12:50 |
| rlandy | one is waiting on sshnaidm's -2 | 12:50 |
| weshay | ssbarnea|ruck, looks like rdo phase 1 is about to pass https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-queens-current-tripleo/163/ | 12:50 |
| rlandy | is this ready to go? https://review.openstack.org/#/c/587012/ | 12:50 |
| rlandy | still see the pike failure. | 12:50 |
| sshnaidm | ssbarnea|ruck, it's not a solution | 12:52 |
| weshay | ssbarnea|ruck, next step for queens.. https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/all%20top%20level%20multijobs/job/rdo-promote-queens-rdo_trunk/ | 12:53 |
| marios | rlandy: ack sagi review is next then | 12:53 |
| sshnaidm | rlandy, which patch do you mean? | 12:53 |
| marios | sshnaidm: here https://trello.com/c/lrSUMaqw/885-translate-tripleosh-bootstrap-subnodes-into-a-series-of-tasks-s17 https://review.rdoproject.org/r/#/c/15000 | 12:53 |
| rlandy | sshnaidm, https://review.rdoproject.org/r/#/c/15000 | 12:53 |
| marios | rlandy: tripleo-ci-centos-7-containers-multinode-pike success @ https://review.openstack.org/#/c/587012/ | 12:54 |
| sshnaidm | rlandy, oh, so can we merge it? | 12:54 |
| rlandy | marios: what do you want to do about w+1 on https://review.openstack.org/#/c/583195? | 12:54 |
| rlandy | sshnaidm:isk yet | 12:54 |
| rlandy | idk | 12:54 |
| rlandy | asking you and marios | 12:54 |
| sshnaidm | marios, ^^ | 12:54 |
| rlandy | just trying to keep a track of the reviews | 12:54 |
| rlandy | and which ones can be merged | 12:54 |
| ssbarnea|ruck | if normal job duration is very close to the limit, it makes sense. if we get more than 2-3% timeout failures that are only natural variations, we should increase it. I am not saying that timeout increase is always good, but we should have a 10% safety net, and right now i am inclined to believe we don't have. | 12:55 |
| marios | rlandy: i think it can merge https://review.openstack.org/#/c/583195 pike is green now too | 12:55 |
| *** jfrancoa has quit IRC | 12:56 | |
| sshnaidm | ssbarnea|ruck, normal job duration is not close to the limit | 12:56 |
| rlandy | marios: ok - I'll w+1 it then | 12:57 |
| marios | rlandy: thank you | 12:57 |
| * marios gets ready to run for the hills having broke all the things | 12:58 | |
| rlandy | marios: best to do it when you are rover | 12:58 |
| marios | rlandy: well it just so happens i am! | 12:59 |
| rlandy | then you understand what went wrong | 12:59 |
| marios | which reminds me do you remember the 3 node issue like could not locate file in lookup: ControllerApi/centos-7-rax-dfw-0001265357/UpgradeInitDeployment"} | 12:59 |
| marios | rlandy: i as poking at bug list and checking out https://bugs.launchpad.net/tripleo/+bug/1786520?comments=all | 12:59 |
| openstack | Launchpad bug 1786520 in tripleo "3node jobs failing due to missing file UpgradeInitDeployment" [Critical,Triaged] | 12:59 |
| marios | rlandy: i found a few more recent examples (the job is non voting so not urgent) ... so will mark the bug triaged at least and point to the ones i found | 13:00 |
| marios | in a minute | 13:00 |
| marios | (tm) | 13:00 |
| marios | rlandy: but wanted to ask you as it sounded familiar/similar to an issue you were looking at a while back? | 13:00 |
| rlandy | marios: yeah - that has come up before | 13:00 |
| rlandy | I was looking at it a while back | 13:00 |
| rlandy | was supposed to ping slagle on it | 13:00 |
| rlandy | will do that after the meeting | 13:01 |
| rlandy | required the nodes to match from https://github.com/openstack/tripleo-quickstart/blob/master/config/nodes/2ctlr.yml | 13:02 |
| rlandy | marios: ^^ see notes in https://trello.com/c/95AvKjpX/145-migrate-3-jobs-to-tripleo-ci-base-parent | 13:03 |
| rlandy | https://review.openstack.org/#/c/581376/ | 13:05 |
| marios | rlandy: added comment with recent examples just now https://bugs.launchpad.net/tripleo/+bug/1786520?comments=all | 13:05 |
| openstack | Launchpad bug 1786520 in tripleo "3node jobs failing due to missing file UpgradeInitDeployment" [Critical,Triaged] | 13:05 |
| marios | rlandy: looking | 13:05 |
| rlandy | marios: you could add slagle to that | 13:06 |
| ssbarnea|ruck | sshnaidm: I am looking at http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario001-multinode-oooq-container --- and I agregated the data, avg duration of successful one 10013s, avg of timeout ones 12167s. | 13:09 |
| marios | ssbarnea|ruck: sshnaidm i updated with today's status re known lp bugs affecting ci/promotion at https://review.rdoproject.org/etherpad/p/ruckrover-sprint18 | 13:11 |
| marios | might be useful for status/we have tripleo weekly and community call today | 13:11 |
| marios | update with the ones i may have missed | 13:11 |
| rlandy | marios: I'll ping slagle about it on #tripleo | 13:12 |
| marios | rlandy: thanks | 13:12 |
| marios | rlandy: or maybe even save it for the weekly in an hour | 13:13 |
| marios | but w/e/ | 13:13 |
| rlandy | marios: would peref not to hash this out on everyone's time | 13:13 |
| marios | k | 13:13 |
| *** jfrancoa has joined #oooq | 13:14 | |
| bogdando | https://review.openstack.org/#/c/593106/ looks good to go, sshnaidm, weshay | 13:14 |
| bogdando | ditto https://review.openstack.org/#/c/593104/ | 13:15 |
| sshnaidm | bogdando, and what's wrong with https://github.com/openstack/tripleo-quickstart/blob/6926a86c3088f76e00397e58862070ba5e2851e6/roles/environment/vars/centos-7.yml#L1 ? | 13:16 |
| *** skramaja has quit IRC | 13:16 | |
| bogdando | sshnaidm: prolly nothing, but when I run it from tht container wrapper, it seeks for centos.yml | 13:16 |
| bogdando | and fails to apply host facts | 13:17 |
| bogdando | from that* container wrapper | 13:17 |
| sshnaidm | bogdando, then maybe to rename it to centos.yml? | 13:17 |
| bogdando | sshnaidm: not sure :) | 13:17 |
| bogdando | may be there is cases for centos-7 ? | 13:17 |
| bogdando | I can't be sure | 13:18 |
| sshnaidm | bogdando, and what exactly in this wrapper does look for it? | 13:18 |
| rlandy | marios: ok - back to the reviews on your card ... https://review.openstack.org/#/c/588623/ - can this be abandoned? | 13:18 |
| ykarel | bogdando, u checked my comment there? as per the code i linked it doesn't seem to fail, or i looked wrong? | 13:18 |
| sshnaidm | bogdando, generally lookup by OS and version, and then by OS, OS family, etc | 13:19 |
| rlandy | marios; just trying to asses which reviews are still left to merge from your cards | 13:20 |
| rlandy | weshay: pls see discussion on #tripleo re: 3 node job. Do we have your ack to remove it? | 13:24 |
| rlandy | marios picked up on the errors again | 13:24 |
| *** trown is now known as trown|brb | 13:27 | |
| marios|rover | rlandy: sec :) just switch chans | 13:29 |
| marios|rover | rlandy: yeah i think so for https://review.openstack.org/#/c/588623/ i added a coment to that effet | 13:30 |
| marios|rover | effect | 13:30 |
| marios|rover | rlandy: oh well i didn't write "lets abandon this " but i said it on the last call we had and wes -1 it | 13:30 |
| marios|rover | rlandy: so consider it abandoned for now i guess | 13:30 |
| bogdando | ykarel, sshnaidm: FWIW, that failure could be a red herring raised to me and confused :) | 13:31 |
| rlandy | marios|rover: abandoned t is | 13:31 |
| bogdando | so if you think we do not need that centos.yml , let me know | 13:31 |
| *** trown|brb is now known as trown | 13:31 | |
| bogdando | it seems it takes the config from a fallback | 13:31 |
| bogdando | but lemme deouble-check for the wrapper container | 13:31 |
| bogdando | which facts it produces | 13:32 |
| bogdando | (once I have it working back haha) | 13:32 |
| *** apetrich has quit IRC | 13:33 | |
| rlandy | marios|rover: weshay: to get back to the 3node job, do you still want to raise it at the meeting? | 13:33 |
| ykarel | bogdando, it's good to get the root cause to get that patch, i agree adding centos.yml will not harm but it's good to have a reason there | 13:33 |
| rlandy | https://review.openstack.org/#/c/572713/ may help | 13:33 |
| marios | rlandy: up to weshay i don't have it in my list at https://review.rdoproject.org/etherpad/p/ruckrover-sprint18 | 13:33 |
| rlandy | if what we really want to test is custom roles | 13:33 |
| rlandy | marios: ok - I think you should be all set on this card - if the last review gets a zuul +2 https://trello.com/c/GvjcnJB2/850-translate-tripleosh-bootstrap-subnodes-into-a-series-of-tasks | 13:37 |
| *** ccamacho has quit IRC | 13:38 | |
| marios | rlandy: thank you ! | 13:39 |
| rlandy | marios:- ok last review from your second card ... https://review.openstack.org/#/c/587012/ - does this need to be recheck after pike fix merged? | 13:48 |
| sshnaidm | weshay, did you disconnect? | 13:54 |
| marios | rlandy: well tripleo-ci-centos-7-containers-multinode-pike is green afaics at https://review.openstack.org/#/c/587012/ | 13:59 |
| marios | rlandy: it has the pike fix as depends on | 14:00 |
| marios | rlandy: not sure about the need to depend on sagi fix though do we need that sshnaidm at https://review.openstack.org/#/c/587012/ depends on please do we need the rdo one there? | 14:00 |
| *** agopi has joined #oooq | 14:01 | |
| *** jbadiapa has quit IRC | 14:05 | |
| *** ccamacho has joined #oooq | 14:08 | |
| rlandy | apetrich disappear? | 14:11 |
| *** jbadiapa has joined #oooq | 14:11 | |
| ykarel | marios, i can see ovb jobs are still failing even after those nova patches | 14:15 |
| ykarel | but issue seems different this time | 14:15 |
| marios | ykarel: ack maybe add pointer and comment on the bug please? | 14:17 |
| ykarel | marios, okk will do | 14:17 |
| *** jfrancoa has quit IRC | 14:18 | |
| weshay | ssbarnea|ruck, https://review.openstack.org/#/q/status:open++topic:standalone-f28 | 14:20 |
| weshay | sorry | 14:20 |
| weshay | sshnaidm, https://review.openstack.org/#/q/status:open++topic:standalone-f28 | 14:20 |
| ykarel | marios, added https://bugs.launchpad.net/tripleo/+bug/1787910/comments/10 | 14:20 |
| openstack | Launchpad bug 1787910 in tripleo "OVB overcloud deploy fails on nova placement errors" [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:20 |
| ssbarnea|ruck | at least I am glad that nickname rename worked, in my client i still see my account name without the suffix. | 14:21 |
| *** chem has quit IRC | 14:23 | |
| *** jfrancoa has joined #oooq | 14:34 | |
| weshay | ssbarnea|ruck, https://review.rdoproject.org/grafana/dashboard/db/tripleo-ci?orgId=1 | 14:41 |
| marios | weshay: joining community call in 2 mins | 14:45 |
| ssbarnea|ruck | https://docs.google.com/spreadsheets/d/1ixS-pK_L56nBc0cogfLl1uRVAMQBcf2oIFWKR8MJ6ZE/edit#gid=1754788380 | 14:45 |
| *** ccamacho has quit IRC | 14:53 | |
| marios | ykarel: thanks | 14:54 |
| rlandy | panda: ping re: https://review.rdoproject.org/zuul/api/config-errors | 14:59 |
| rlandy | rf0lc0: ^^ | 14:59 |
| rf0lc0 | ambiguous project | 15:00 |
| rlandy | panda: rf0lc0: I need to add the nodesets from https://github.com/openstack-dev/devstack/blob/master/.zuul.yaml | 15:00 |
| rlandy | rf0lc0: I think I have a fix for that | 15:00 |
| rlandy | rf0lc0: https://paste.fedoraproject.org/paste/X2B86wFwvsaGt5tookMfzA | 15:01 |
| rlandy | ^^ to submit | 15:01 |
| rlandy | rf0lc0: my question is about the nodesets | 15:02 |
| rlandy | rf0lc0: I am adding the nodesets to rdo-jobs | 15:02 |
| rlandy | question: what label to use? | 15:02 |
| rlandy | label: ubuntu-xenial | 15:02 |
| rlandy | ^^ for that | 15:02 |
| rlandy | panda: ^^??? | 15:02 |
| rlandy | openstack-single-node and openstack-two-node | 15:03 |
| panda | rlandy: sorry I'm lost | 15:04 |
| rlandy | panda: https://review.rdoproject.org/zuul/api/config-errors came from the reconfigure after adding tripleo-ci-base | 15:04 |
| rlandy | I have a review to fix openstack/heat-templates issue | 15:05 |
| rlandy | I am trying to fix the nodesets now | 15:05 |
| rlandy | I am adding them to rdo-jobs | 15:05 |
| rlandy | as we did before | 15:05 |
| rlandy | the nodesets are openstack-single-node and openstack-two-node | 15:06 |
| rlandy | panda: following so far??? | 15:06 |
| panda | rlandy: more or less!!! | 15:06 |
| rf0lc0 | openstack-single-node\" was not found... | 15:06 |
| rlandy | panda: rf0lc0: ok - so the question is ... | 15:07 |
| *** sanjayu_ has quit IRC | 15:07 | |
| rlandy | what to use for node label? | 15:07 |
| rf0lc0 | so #1) include nodeset in the same place where we did for tripleo-ci: include: job | 15:07 |
| rlandy | - name: controller | 15:07 |
| rlandy | label: ubuntu-xenial | 15:07 |
| panda | where are we using devstack ? | 15:07 |
| rf0lc0 | label should match with what we have in nodepool.yaml (diskimage) | 15:07 |
| rlandy | rf0lc0: we added the nodesets to rdo-jobs before | 15:07 |
| rlandy | openstack-dev/devstack | 15:08 |
| dtantsur | folks, it seems that even if I explicitly configure a non-containerized undercloud, a containerized one is installed... | 15:08 |
| dtantsur | could you point me where to fix it? | 15:08 |
| panda | dtantsur: what release ? | 15:09 |
| dtantsur | panda: master | 15:09 |
| panda | dtantsur: I don't think we ever supported that, at a point we switched and may have never thought of adding support for non containerized. | 15:10 |
| dtantsur | panda: so, no more development? only devstack? :( | 15:11 |
| panda | dtantsur: the containers should be updated with whatever repo you throw at them | 15:12 |
| dtantsur | panda: it's not repo, it's git checkout | 15:12 |
| dtantsur | which is implied by the word "development" | 15:13 |
| dtantsur | sorry if I sound harsh, but I raised it to some tripleo folks several times, and I'm afraid that now I'm screwed | 15:13 |
| rlandy | rf0lc0: panda: nvm about above - will be fixed by another team | 15:19 |
| panda | dtantsur: as far as I'm hearing, people are building packages from repos and inject the to the container. BUt you may need to ask what other developers are doing at the moment. | 15:20 |
| rf0lc0 | rlandy, ack | 15:20 |
| dtantsur | panda: asking around. but this workflow is too complex for anything beyond one line fix :( | 15:20 |
| panda | rlandy: oh, that's why I was lost :) | 15:21 |
| *** amoralej is now known as amoralej|off | 15:26 | |
| weshay | dtantsur, so dprince put in the patches that disabled non-containerized deployments | 15:29 |
| weshay | you could search his history | 15:29 |
| dtantsur | why wouldn't he indeed :) | 15:30 |
| weshay | dtantsur, I have not seen any posted developer workflows on how folks are patching their containers | 15:30 |
| dtantsur | weshay: if you check #tripleo, you'll realize that some people do ugly hacks, some haven't tried it yet :) | 15:30 |
| weshay | but dprince, mandre, slow etc.. I'm sure do it all the time.. I hope.. I would like to see it posted in the public s | 15:31 |
| weshay | lolz | 15:31 |
| weshay | tripleo is one big ugly hack though :) | 15:31 |
| *** jbadiapa has quit IRC | 15:33 | |
| *** jfrancoa has quit IRC | 15:35 | |
| weshay | marios|rover, if you have time in your day https://review.openstack.org/#/c/592577/ | 15:45 |
| *** ykarel is now known as ykarel|away | 15:45 | |
| marios | weshay: ack nice | 15:53 |
| sshnaidm | rlandy, marios so, we are gonna merge it, right? https://review.rdoproject.org/r/#/c/15000/ | 15:53 |
| marios | sshnaidm: yes | 15:53 |
| marios | sshnaidm: https://review.openstack.org/587369 merged so is safe now | 15:53 |
| marios | sshnaidm: it doesn't matter if your code creates the loop device first | 15:53 |
| ssbarnea|ruck | weshay: reg timeout, i guess i need to wait to get 592577 tested and merged before retriggering other jobs. Asking before that one timed out again after retriggering it this morning: https://review.openstack.org/#/c/590748/ | 15:56 |
| sshnaidm | marios, merged | 15:56 |
| sshnaidm | marios, need to take a look at new jobs to see it's ok (it should be) | 15:57 |
| marios | weshay: meant to ask you | 15:57 |
| marios | weshay: this component testint ci project call we missed | 15:57 |
| marios | weshay: we meant to be on it? | 15:57 |
| marios | ruck/rover i mean | 15:57 |
| marios | weshay: its your call afaics? (creator?) | 15:58 |
| * marios end of day fast approach | 15:58 | |
| marios | ttyl folks have a good one | 15:58 |
| marios | ack sshnaidm thanks | 15:59 |
| *** jtomasek_ has joined #oooq | 16:25 | |
| *** bogdando has quit IRC | 16:25 | |
| *** jtomasek has quit IRC | 16:26 | |
| *** panda is now known as panda|off | 16:33 | |
| *** dtantsur is now known as dtantsur|afk | 16:38 | |
| *** vinaykns has joined #oooq | 16:40 | |
| *** trown is now known as trown|lunch | 16:46 | |
| *** ykarel|away has quit IRC | 16:49 | |
| weshay | rlandy, I don't see a syntax error here https://review.openstack.org/#/c/594308/4 | 16:54 |
| rlandy | weshay: see #rdo ... once https://softwarefactory-project.io/r/#/c/13474/ merges, I will recheck https://review.openstack.org/#/c/594308 | 16:55 |
| rlandy | oh it merged | 16:55 |
| rlandy | rechecking | 16:56 |
| rlandy | hmmm | 16:57 |
| rlandy | weshay: sorry - you are right | 16:58 |
| rlandy | still broken | 16:58 |
| chandankumar | weshay: myoung|training http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2018-08-21.log.html#t2018-08-21T16:49:12 discussion | 17:06 |
| chandankumar | we will send the email tomorrow | 17:06 |
| chandankumar | for better visibilty | 17:06 |
| weshay | chandankumar++ thanks | 17:07 |
| chandankumar | weshay: you need to buy some time to be tere | 17:07 |
| weshay | sorry? | 17:07 |
| weshay | oh.. u are suggesting I need to join that meeting? | 17:07 |
| weshay | if only we could buy time :) | 17:07 |
| chandankumar | weshay: yes I wanted you to be there during the discussion | 17:08 |
| chandankumar | with arxcruz and tosky | 17:08 |
| chandankumar | more people leads to more confidence | 17:08 |
| weshay | aye | 17:09 |
| weshay | so far it sounds like you and arxcruz did a great job | 17:09 |
| chandankumar | when the job is done then we can measure the result we are bad at doing things | 17:09 |
| chandankumar | without planning | 17:10 |
| weshay | not sure what you mean | 17:10 |
| rf0lc0 | rlandy, quick question... can you please explain me or point me to the code where browbeat-minimal.yml is 'loaded' from browbeat repo as it was a tqe one ? | 17:18 |
| rlandy | rf0lc0: not sure what you mean | 17:20 |
| rlandy | loaded | 17:20 |
| rlandy | the requirements are changed to include that playbook | 17:20 |
| rlandy | and then then playbook is called | 17:20 |
| rf0lc0 | rlandy, browbeat-minimal.yml is not our playbook, correct? | 17:21 |
| rlandy | as in https://review.openstack.org/#/c/581488/29/playbooks/baremetal-full-deploy.yml | 17:21 |
| rlandy | rf0lc0: correct - lives in browbeat | 17:21 |
| rf0lc0 | rlandy, ok, I see... I am just thinking if we are opening a door to hell by freely allowing people to include external playbooks | 17:22 |
| rlandy | rf0lc0: that door is already open ... updates | 17:23 |
| rlandy | and tripleo-ha-utils | 17:23 |
| rlandy | hell is with us already | 17:23 |
| *** tosky has quit IRC | 17:24 | |
| rf0lc0 | ok the door is open, we'll just open windows now | 17:26 |
| weshay | rf0lc0, as long as the playbook is post deployment it's ok | 17:36 |
| weshay | so something like browbeat is ok | 17:37 |
| rf0lc0 | weshay, hmm good point. Would be a good way to limit.... just not sure how we define as post deployment playbook.... will look at the code | 17:38 |
| weshay | rf0lc0, it is pretty obvious when someone changes the playbooks | 17:39 |
| weshay | more obvious than a single variale | 17:39 |
| weshay | maybe do not allow changing of tags | 17:39 |
| rf0lc0 | rlandy, maybe this is a check for your reparenting patch ^ | 17:40 |
| weshay | hooray | 17:40 |
| rf0lc0 | rlandy, do not allow overriding/modifying tags in the zuul config | 17:40 |
| *** trown|lunch is now known as trown | 17:53 | |
| rlandy | tags? | 17:57 |
| myoung|training | chandankumar: from reading the IRC logs that looks like great news! (e.g. "odyssey4me: yep, we're absolutely on board and quite excited to be working together with the tripleo folks :)") | 17:59 |
| rf0lc0 | rlandy, I still have the same question... so let me rephrase :) | 18:01 |
| rf0lc0 | rlandy, I can include <my_playbook.yml> as long as I add <my_project> to required-projects ? | 18:02 |
| *** apetrich has joined #oooq | 18:02 | |
| rlandy | rf0lc0: you would need something like: https://github.com/openstack/tripleo-quickstart/blob/master/quickstart-extras-requirements.txt#L3 | 18:03 |
| rf0lc0 | hmm ok, so this means custom extra playbooks needs a +2 from ci team, while custom tempest doesn't weshay rlandy | 18:05 |
| rf0lc0 | coz importing playbooks from outside is not just a matter of adding a new config to zuul job | 18:06 |
| weshay | ya. that is true.. good | 18:06 |
| rf0lc0 | unless you really mean we need to support this ^ | 18:06 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/build-image.yaml#L50 | 18:06 |
| weshay | no.. the only thing I want to be customizable is validation | 18:06 |
| rlandy | anyone know where ^^ nodeset is defined??? | 18:07 |
| rlandy | legacy-centos-7 | 18:07 |
| rlandy | not in https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/nodesets.yaml | 18:07 |
| rf0lc0 | zuul-jobs ? | 18:07 |
| rf0lc0 | checking | 18:07 |
| rf0lc0 | https://github.com/openstack-infra/openstack-zuul-jobs/blob/2a424bac23521ee047fb8f38c60189a860e74e79/zuul.d/nodesets.yaml#L108 | 18:08 |
| rlandy | rf0lc0: weshay: no not in zuul-jobs | 18:08 |
| rf0lc0 | rlandy, ^ | 18:08 |
| rlandy | ha - ok | 18:08 |
| rf0lc0 | o-z-j :) | 18:08 |
| rf0lc0 | sorry | 18:08 |
| rlandy | will have to define it in rdo-jobs | 18:08 |
| rf0lc0 | yeah, redefine and adjust label | 18:09 |
| rf0lc0 | to upstream-centos-7 | 18:09 |
| *** jbadiapa has joined #oooq | 18:09 | |
| rlandy | rf0lc0: pls check me https://review.rdoproject.org/r/15780 | 18:12 |
| rlandy | made too many mistakes already today :( | 18:12 |
| rf0lc0 | rlandy, one step back... why don't you change nodesets in the tripleo-ci's build.yaml | 18:14 |
| rlandy | rf0lc0: to what? | 18:14 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/build-image.yaml | 18:14 |
| rf0lc0 | rlandy, let me double check if I am non-sense | 18:14 |
| rf0lc0 | well, I am not quite sure about the diff bettween old legacy nodes and new ones... but... instead of continuing using legacy-centos-7 (which in out case will point to upstream-centos-7) anyway.... lets move to single-centos-7-node instead | 18:17 |
| rf0lc0 | rlandy, what you find ? | 18:17 |
| rf0lc0 | if we change nodesets there to use single-centos-7, they will use upstream-centos-7 label (dib) anyway | 18:18 |
| rlandy | rf0lc0: yep - paul just commented ... https://review.rdoproject.org/r/#/c/15780/ | 18:18 |
| rlandy | will change | 18:18 |
| rlandy | rf0lc0: stupid question - where is the pipeline stanza? | 18:34 |
| rf0lc0 | rlandy, I think he means where the job runs, not where it is defined | 18:34 |
| rf0lc0 | like check: - my_awesome_job | 18:35 |
| rf0lc0 | runs in check pipeline for a given project | 18:35 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/build-image.yaml#L6 | 18:35 |
| rlandy | where exactly? | 18:36 |
| *** leifmadsen has quit IRC | 18:37 | |
| *** hrybacki has quit IRC | 18:42 | |
| *** jrist has quit IRC | 18:46 | |
| rf0lc0 | sorry rlandy got a little emergency here, driving my son to the doctor | 19:05 |
| rf0lc0 | brb | 19:05 |
| rlandy | rf0lc0: k - pls comment when you are back | 19:12 |
| rlandy | we need to make a decision about this | 19:12 |
| rlandy | weshay: what is the status of the jobs defined in https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/build-image.yaml? they run in upstream and rdocloud? | 19:17 |
| weshay | rlandy, I think we can nuke those tbh | 19:19 |
| weshay | unless ian uses them for | 19:20 |
| * weshay looks | 19:20 | |
| weshay | rlandy, http://logs.openstack.org/45/583145/10/check/tripleo-buildimage-overcloud-full-centos-7/6907c9d/ | 19:21 |
| weshay | rlandy, that is checking dib element changes | 19:21 |
| rlandy | weshay: really - because they are causing some real headaches in the zuul3 transition | 19:22 |
| rlandy | they look to be running though | 19:22 |
| rlandy | in both envs | 19:22 |
| weshay | rlandy, have a sec to explain the issue in blue? | 19:22 |
| weshay | maybe I can help | 19:22 |
| rlandy | weshay: sure | 19:22 |
| ssbarnea|ruck | weshay: sadly https://review.openstack.org/#/c/592577/ failed on one job that matters, not sure why yet, but it has something to do with "subnode-2" | 19:23 |
| weshay | rlandy, https://bluejeans.com/4113567798 | 19:24 |
| rlandy | https://review.openstack.org/#/c/594308/ | 19:24 |
| weshay | ssbarnea|ruck, rlandy https://review.openstack.org/#/c/594314/3/config/release/tripleo-ci/master.yml | 19:25 |
| weshay | ssbarnea|ruck, that probably needs to land first | 19:25 |
| ssbarnea|ruck | 'update_containers' is undefined | 19:27 |
| *** jjoyce has quit IRC | 19:33 | |
| *** rodrigods has quit IRC | 19:39 | |
| *** jjoyce has joined #oooq | 19:39 | |
| *** Tengu has quit IRC | 19:47 | |
| rlandy | weshay: you mentioned you sent me a topic - which one? | 20:09 |
| *** jaosorior_ has joined #oooq | 20:14 | |
| *** jaosorior has quit IRC | 20:17 | |
| *** marios|rover has quit IRC | 20:19 | |
| *** gkadam has quit IRC | 20:27 | |
| weshay | rlandy, https://review.openstack.org/#/q/topic:standalone-f28+(status:open+OR+status:merged) | 20:33 |
| rlandy | weshay: I think changes to tripleo-client trigger the build - testing that | 20:42 |
| *** gkadam has joined #oooq | 20:43 | |
| *** holser_ has quit IRC | 20:44 | |
| rlandy | ssbarnea|ruck: I assume these timeout failures are what you were discussing earlier: https://review.openstack.org/#/c/581488 - unrelated to patch? | 20:55 |
| rlandy | weshay: you want me to review the f28 stuff? test it out? | 21:04 |
| rlandy | just know about it in case you win the lottery? | 21:04 |
| *** trown is now known as trown|outtypewww | 21:07 | |
| weshay | rlandy, the centos7 version is ready | 21:12 |
| weshay | and deploys | 21:12 |
| weshay | rlandy, fedora is WIP as tripleo-common for python3 is not built yet | 21:13 |
| rlandy | weshay:ok - just lost track of what reviews you wanted me to look at | 21:13 |
| rlandy | https://review.openstack.org/#/c/594548 should test the build-image job | 21:14 |
| weshay | k | 21:14 |
| rlandy | here's hoping | 21:14 |
| rlandy | if browbeat ever runs, I will hold a party | 21:14 |
| rlandy | runs in CI, I mean | 21:15 |
| weshay | hrm.. | 21:15 |
| weshay | rlandy, actually f28 is not working w/ the stock image.. but I have a special one saved | 21:16 |
| weshay | + chown -R stack:stack /home/stack/ | 21:16 |
| weshay | + dd if=/dev/zero of=/swapfile bs=1024 count=4194304 | 21:16 |
| weshay | dd: error writing '/swapfile': No space left on device | 21:16 |
| weshay | 3054685+0 records in | 21:16 |
| rlandy | ?? | 21:19 |
| *** gkadam has quit IRC | 21:26 | |
| *** sshnaidm is now known as sshnaidm|afk | 21:51 | |
| *** tosky has joined #oooq | 22:07 | |
| *** vinaykns has quit IRC | 22:36 | |
| rlandy | http://logs.openstack.org/48/594548/1/check/tripleo-buildimage-overcloud-full-centos-7/ff19db5/ worked | 22:44 |
| *** jtomasek_ has quit IRC | 22:47 | |
| *** tosky has quit IRC | 23:07 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!