Friday, 2018-09-21

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60327500:00
*** mcornea has quit IRC00:00
*** agopi|brb has joined #tripleo00:05
*** bfournie has joined #tripleo00:09
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256000:10
*** ooolpbot has quit IRC00:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)00:10
*** apevec has quit IRC00:11
*** med_ has quit IRC00:17
*** rbrady has quit IRC00:28
*** med_ has joined #tripleo00:33
*** Chaserjim has joined #tripleo00:48
*** med_ has quit IRC00:48
*** Chaserjim has quit IRC00:52
*** Chaserjim has joined #tripleo00:53
*** Chaserjim has quit IRC01:05
*** Chaserjim has joined #tripleo01:06
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256001:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)01:10
*** ooolpbot has quit IRC01:10
openstackgerritSteve Baker proposed openstack/tripleo-heat-templates master: standalone: map opendaylight services to None  https://review.openstack.org/60424101:11
*** Chaserjim has quit IRC01:14
*** Chaserjim has joined #tripleo01:18
*** mschuppert has quit IRC01:27
*** Chaserjim has quit IRC01:27
*** mrsoul has quit IRC01:27
*** Chaserjim has joined #tripleo01:29
*** Chaserjim has quit IRC01:45
*** rlandy|bbl is now known as rlandy01:46
*** akhila has quit IRC01:51
openstackgerritMerged openstack/tripleo-quickstart-extras master: Skip some neutron_tempest_plugin tests for master and rocky and telemetry for master  https://review.openstack.org/60409101:53
openstackgerritMerged openstack/tripleo-common master: Clean up node registration output.  https://review.openstack.org/56981101:58
openstackgerritMerged openstack/puppet-tripleo stable/queens: Prepare 8.3.6 (Queens) release  https://review.openstack.org/60387901:58
openstackgerritMerged openstack/puppet-tripleo stable/pike: Prepare 7.4.16 (Pike) release  https://review.openstack.org/60388001:58
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256002:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)02:10
*** errr has quit IRC02:12
*** errantekarmico has quit IRC02:34
*** errr has joined #tripleo02:37
*** dxiri has joined #tripleo02:46
*** apetrich has quit IRC02:48
*** ykarel has joined #tripleo03:00
*** cylopez has joined #tripleo03:00
*** rodrigods has quit IRC03:02
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256003:10
*** ooolpbot has quit IRC03:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)03:10
*** errantekarmico has joined #tripleo03:10
*** cylopez has quit IRC03:37
*** sdake has quit IRC03:38
*** moshele has joined #tripleo03:42
*** sdake has joined #tripleo03:43
*** rcernin has quit IRC03:48
*** rcernin has joined #tripleo03:49
*** aedc has joined #tripleo03:56
*** aevoo has quit IRC03:56
*** moshele has quit IRC03:58
*** ramishra has joined #tripleo04:07
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256004:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)04:10
*** skramaja has joined #tripleo04:20
*** abishop has quit IRC04:28
*** udesale has joined #tripleo04:29
*** sanjayu_ has joined #tripleo04:30
*** Petersingh has joined #tripleo04:30
*** kopecmartin|away has quit IRC04:31
*** sshnaidm|pto has joined #tripleo04:34
*** dxiri has quit IRC04:35
*** sanjayu_ has quit IRC04:37
*** abishop has joined #tripleo04:37
*** kopecmartin has joined #tripleo04:37
*** rfolco has quit IRC04:41
*** Petersingh is now known as Petersingh|afk04:43
*** abishop has quit IRC04:46
*** abishop has joined #tripleo04:46
*** bugzy_ has joined #tripleo05:06
*** etingof has quit IRC05:08
*** saneax has joined #tripleo05:09
*** bugzy has quit IRC05:09
*** rlandy has quit IRC05:10
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256005:10
*** ooolpbot has quit IRC05:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)05:10
*** ratailor has joined #tripleo05:11
*** ykarel has quit IRC05:21
*** Petersingh|afk is now known as Petersingh05:24
bandinijaosorior: I presume we are still in merge lockdown in CI?05:27
*** yprokule has joined #tripleo05:43
*** quique|rover|off is now known as quiquell|rover05:43
quiquell|roverGood morning05:43
jaosoriorbandini: yeah05:45
jaosoriorbandini: the patches that were supposed to fix CI merged though. So, maybe next promotion is the trick :D05:45
jaosoriorquiquell|rover: good morning05:45
quiquell|roverjaosorior: Damn, first  review still not merged :-(05:45
jaosoriorquiquell|rover: which?05:45
quiquell|roverjaosorior: https://review.openstack.org/#/c/603323/05:46
quiquell|roverfix for fs01905:46
jaosoriorbummer05:46
*** boazel has joined #tripleo05:47
quiquell|roverIf we get it merged we run promotions, to reduce a little timeouts ASAP05:48
quiquell|roverjaosorior: There are also stack at nodepool that we cannot get rid of... so also some OVB jobs at promotion can fail :-(05:48
quiquell|roverjaosorior: this is impossible man05:48
jaosoriorquiquell|rover: so.. RDO cloud issues too?05:49
quiquell|roverjaosorior: yep05:49
jaosoriorquiquell|rover: anybody in this timezone we could poke?05:50
quiquell|roverjaosorior: kforde is doing great efforts, even talk with developed and they saw nothing05:50
*** shyamb has joined #tripleo05:51
*** ykarel has joined #tripleo05:51
bandinijaosorior: ack, will wait then, thanks05:53
jaosoriorquiquell|rover: what was the site with the sova status?05:54
quiquell|roverjaosorior: we have this now http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=105:57
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds  https://review.openstack.org/60426805:57
jaosoriorquiquell|rover: thanks05:58
quiquell|roverjaosorior: sova -> http://cistatus.tripleo.org/05:58
quiquell|roverjaosorior: we use both, will add more feature to the tripleo ci grafana05:58
openstackgerrityatin proposed openstack/tripleo-quickstart-extras master: Revert "add delorean-current to repolist for updates"  https://review.openstack.org/59580305:59
openstackgerritMatthew Thode proposed openstack/diskimage-builder master: enable caching for gentoo builds  https://review.openstack.org/60426805:59
quiquell|roverjaosorior: We hav also a dashboard to explore jobs upstream and RDO http://dashboard-ci.tripleo.org/d/-UEjGKFmz/exploration?orgId=105:59
ykareljistr, ^^ just to get the time difference05:59
*** shyamb has quit IRC06:01
*** shyamb has joined #tripleo06:02
*** yprokule has quit IRC06:02
jaosoriorquiquell|rover: seems a bunch of them are failing in this task: undercloud-setup : Run the TripleO-CI VXLAN networking script06:03
jaosoriorNot clear to me what the issue is from the logs https://logs.rdoproject.org/17/600517/25/openstack-check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades-pike-branch/bf98e7e/logs/undercloud/home/zuul/vxlan_networking.sh.log.txt.gz06:03
quiquell|roverykarel: we have a promotion for pike puff06:03
quiquell|roverjaosorior: Let me check06:04
ykarelquiquell|rover, nice06:04
ykarelwe need master/rocky also06:04
*** janki has joined #tripleo06:04
quiquell|roverykarel: rocky has fail even at container build :-(06:05
ykarel:(06:05
ykarelreason?06:05
quiquell|roverykarel: sure we have more broken stuff now06:05
quiquell|roverJust starting to check06:05
ykarelokk06:05
quiquell|roverration blood/coffe is not good yet06:05
jaosoriorgood idea06:06
* jaosorior fetches coffee06:06
mandrejaosorior: good morning, is it fine to restore patches you've abandoned due to gate issue yday?06:06
*** sshnaidm|pto has quit IRC06:06
quiquell|rovermandre: we need only one more06:06
quiquell|rovermandre: to unblock fs019 at promotions, sorry06:07
quiquell|roverjaosorior: ^ is that ok ?06:07
*** mschuppert has joined #tripleo06:07
ykarelinfact we need two, as that one is dependent on ara one06:07
jaosoriormandre: lets not start restoring patches yet. I restored some in the morning thinking the situation was better. But it isn't yet.06:08
quiquell|roverykarel: ara one is at the gates, is good tool06:08
mandrequiquell|rover, jaosorior: alright, I'll wait a little longer ToT06:08
quiquell|roverykarel: but feels like we have timeouts because we don't have promotions...06:08
ykarelquiquell|rover, yes i rechecked both today morning06:08
quiquell|rovermandre: thanks for the patience :-)06:08
ykarelquiquell|rover, yes we need to fix those timeouts06:08
ykarelif we fix the timeouts, we can get patches / promotions evertything on track06:09
*** jfrancoa has joined #tripleo06:10
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256006:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)06:10
*** ooolpbot has quit IRC06:10
quiquell|roverykarel: :modprobe: ERROR: could not insert 'ipmi_devintf': Operation not permitted\nINFO:kolla.common.utils.nova-compute:modprobe: ERROR: could not insert 'ipmi_msghandler': Operation not permitted06:11
quiquell|roverykarel: rocky container build issue06:11
quiquell|roverthat's new, non related to nodepool stacks06:11
quiquell|roverlooks like we have new rpm version doing new stuff that need permissions06:11
quiquell|roverInstalling : kernel-3.10.0-862.11.6.el7.x86_6406:12
*** gkadam has joined #tripleo06:13
*** shyamb has quit IRC06:13
*** gkadam has quit IRC06:13
ykarelquiquell|rover, checking06:13
ykarelseems we had this kernel already06:14
*** shyamb has joined #tripleo06:14
quiquell|roverYep is the same06:14
quiquell|roverI think this is new Installing : c-ares-1.10.0-3.el7.x86_6406:15
*** Aelia has quit IRC06:17
quiquell|rovernah is at nova-compute06:18
quiquell|roversame version06:18
ykarelquiquell|rover, no that's not issue06:19
quiquell|roverykarel: maybe something new at kolla nova-compute Docker ?06:19
ykarelquiquell|rover, no06:19
ykarelquiquell|rover, actual issue is ERROR:kolla.common.utils.keystone-ssh:Error: Status 404 trying to push repository tripleorocky/centos-binary-keystone-ssh: "404 page not found\n"06:19
ykarelhttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-centos-7-rocky-containers-build/fb62f45/logs/kolla/logs/000_FAILED_keystone-ssh.log06:19
ykarelmay be infra06:19
quiquell|roverykarel: could be the stacks06:19
ykarelno06:20
ykarelit can be container registry06:20
jaosoriorquiquell|rover: I don't see how the stacks issue could related to not being able to push a container.06:20
ykarelyes correct06:20
jaosoriorthat does sound more like a container registry issue06:20
quiquell|roverjaosorior: Yep you are right06:20
jaosorioris the container registry being handled by infra? or what are we using?06:21
*** shyamb has quit IRC06:21
ykarelwe use rdo container registry in container build job06:21
*** shyamb has joined #tripleo06:22
jaosoriorykarel: are you aware of any issues there?06:22
*** jtomasek has joined #tripleo06:22
*** ksambor has joined #tripleo06:22
ykareljaosorior,seems some network issue06:23
*** chkumar|off is now known as chkumar|ruck06:23
quiquell|roverchkumar|ruck: good morning we have pike promotion :-)06:23
quiquell|roverchkumar|ruck: And you don't want to know more06:23
ykareljaosorior, as i can inspect that image currently: skopeo inspect docker://trunk.registry.rdoproject.org/tripleorocky/centos-binary-keystone-ssh:tripleo-ci-testing06:23
chkumar|ruckquiquell|rover: Good morning :-)06:24
*** quiquell|rover is now known as quique|rover|brb06:25
chkumar|ruckquique|rover|brb: ykarel https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/2619576/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-21_02_42_2206:26
chkumar|ruck"jq: error: Could not open file /var/lib/heat-config/deployed/274c5fae-b5d4-4364-aa5b-fa3159d5e261.notify.json: No such file or directory"06:26
chkumar|ruckykarel: is it a known issue?06:27
*** Petersingh is now known as Petersingh|afk06:28
quique|rover|brbchkumar|ruck: https://review.openstack.org/#/c/60417106:28
quique|rover|brbslagle: found the issue and has a fix06:28
ykarelchkumar|ruck, i think a bug is ther for it06:28
quique|rover|brbrascasoft: ^06:28
chkumar|ruckquique|rover|brb: ykarel https://bugs.launchpad.net/tripleo/+bug/179234306:29
openstackLaunchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,In progress] - Assigned to James Slagle (james-slagle)06:29
ykarelyes06:29
jaosoriorshould I merge slagle's fix then?06:29
openstackgerritMartin Schuppert proposed openstack/tripleo-common stable/queens: Fix nova-placement healthcheck  https://review.openstack.org/60427206:29
*** yprokule has joined #tripleo06:30
ratailorjaosorior, should I restore now https://review.openstack.org/#/c/603557/ ?06:30
jaosoriorratailor: not yet.06:30
*** akrivoka has joined #tripleo06:30
chkumar|ruckjaosorior: yes that one is needed06:30
jaosoriorchkumar|ruck: done06:31
ratailorjaosorior, ack, please ping me or restore when you think its fine to do so.06:31
jaosoriorratailor: I have a LOT of tabs open with all the patches I abandoned. I'll restore them once we can.06:31
*** jaganathan has joined #tripleo06:32
ratailorjaosorior, Thanks!06:32
*** Petersingh|afk is now known as Petersingh06:32
*** boazel_ has joined #tripleo06:33
openstackgerritMerged openstack/tripleo-quickstart-extras master: Track ansible overcloud deployment with ARA  https://review.openstack.org/59722206:34
*** slaweq has joined #tripleo06:35
ykarelone merged ^^06:35
*** boazel has quit IRC06:35
quique|rover|brbGood one for debugging06:35
quique|rover|brbceph version please...06:35
jaosoriorchkumar|ruck, quique|rover|brb, ykarel: By the way... yesterday I was told that the whole abandon game isn't that necessary anymore; we can actually ask folks in #openstack-infra to prioritize patches in the gate, and zuul admins can do that.06:36
quique|rover|brbjaosorior: That's awesome, but abandom also frees resources06:36
*** etingof has joined #tripleo06:36
jaosoriorquique|rover|brb: it does. But yeah, there is yet another useful resource we can use (asking zuul admins for priority for a patch)06:36
quique|rover|brbjaosorior: Would be nice, to auto reprioritize is review has "Closes-Bug" on the comment06:37
*** marios has joined #tripleo06:37
quique|rover|brbI drop now06:37
ykarelquique|rover|brb, the required patch failed 2018-09-21 06:24:33.256152 | primary | subnode-2                  : ok=23   changed=8    unreachable=0    failed=106:37
ykarelat tempest it failed06:37
jaosoriorykarel: which one?06:37
ykarel60332306:38
ykarelupdate docker ceph image06:38
quique|rover|brbykarel: pufff06:38
quique|rover|brbykarel: I am too old for this06:38
quique|rover|brbykarel: what job ?06:38
*** marios has quit IRC06:38
ykareltripleo-ci-centos-7-containers-multinode06:38
*** shyamb has quit IRC06:38
ykarelquique|rover|brb, sorry think that passed, just false reporting06:39
ykarelchecking again06:39
jaosoriorykarel: if you look above there's another funky issue with stackviz (which is also a redherring but would be nice to fix)06:39
jaosoriorAn exception occurred during task execution. To see the full traceback, use -vvv. The error was: AttributeError: 'NoneType' object has no attribute 'startswith'06:39
*** rdopiera has joined #tripleo06:40
quique|rover|brbykarel: weird I see failed=1 but result code 006:40
*** marios has joined #tripleo06:40
quique|rover|brbat subnode-206:40
jaosoriorthe failures got ignored I think06:40
ykarelquique|rover|brb, jaosorior yes something is wrong06:40
*** aufi has joined #tripleo06:40
ykarelmay be it's the task thats being rescued06:40
quique|rover|brbykarel: let's see if it passes the gate06:40
ykarelquique|rover|brb, yes hoping that06:40
quique|rover|brbOk now I really drop06:40
ykarelquique|rover|brb, ack06:41
ykarel2018-09-21 04:45:34.026259 | primary | fatal: [subnode-2]: FAILED! => {"changed": false, "msg": "modprobe: ERROR: could not insert 'kvm_intel': Operation not supported\n", "name": "kvm-intel", "params": "nested=1", "rc": 1, "state": "present", "stderr": "modprobe: ERROR: could not insert 'kvm_intel': Operation not supported\n", "stderr_lines": ["modprobe: ERROR: could not insert 'kvm_intel': Operation not supported"], "stdout": "",06:41
ykarel"stdout_lines": []}06:41
ykarelthis task is being rescued, not sure it's handled correctly06:42
ykarelmay be ansible bug06:42
jankihjensas, hey06:42
jaosoriorykarel: what task is it?06:43
ykareljaosorior, http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/set-libvirt-type/tasks/main.yml#n1406:44
jaosorioruhm06:44
jaosoriorI wonder if it's an actual error from modprobe, or an issue with the ansible modprobe plugin (or whatever that's called...resource?)06:45
ykareljaosorior, for that's reason it's rescued, but still ansible says failed=106:45
ykareldon't know it's expected behavior for ansible, but looks wrong06:46
jaosoriorykarel: what do you mean by "it's rescued"? (lack of terminology on my side)06:46
ykareljaosorior, rescue block if any tasks fail in block:06:47
ykarelhttp://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/set-libvirt-type/tasks/main.yml#n2506:47
jaosoriorykarel: aaah got it06:47
jaosoriorthanks06:47
ykareltry: catch thing on ansible06:47
jaosoriorykarel: right, I missed that bit.06:48
jaosoriorykarel: anyway, seems to have passed.06:48
ykarelyes06:48
ykarelbut confused little in first look06:48
jaosoriorit does raise concern.06:48
jaosoriornot sure if it's expected06:49
jaosoriorsure... it worked. But might need further investigation06:49
ykarelack06:51
quique|rover|brbykarel, chkumar|ruck: fs016 and fs017 with ceilomter fix06:54
quique|rover|brbhttps://review.rdoproject.org/r/#/c/13943/06:54
jaosoriorquique|rover|brb: seems that it can't deallocate ports06:56
jaosoriorhttps://logs.rdoproject.org/43/13943/40/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-master/3163398/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-09-20_15_57_5506:57
jaosoriorquique|rover|brb: is this the same issue that was trying to be fixed?06:57
*** shyamb has joined #tripleo06:58
ykareljaosorior, yes06:58
ykareljaosorior, sileht proposed https://review.openstack.org/#/c/604095/ yesterday06:58
ykareljaosorior, quique|rover|brb this fixed poi jobs, but not tripleo06:58
ykarelso something is still missing to fix that06:59
jankipanda|off, hi06:59
jankijaosorior, hi06:59
jaosoriorykarel: are we sure that run took that change into use?06:59
jaosoriorjanki: hello06:59
jankijaosorior, how are you? GM06:59
jaosoriorjanki: I'm good, coffee is kicking in07:00
ykareljaosorior, yes07:00
jaosoriorthe existencial dread of breaking CI is bearable07:00
jaosoriorand it's friday07:00
*** saneax has quit IRC07:00
jaosoriorso, all is good :D07:00
ykareljaosorior, so earlier poi jobs also failed like: https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6333/weirdo-project/logs/testr_results.html.gz07:00
*** saneax has joined #tripleo07:00
ykareland now they are passing:- https://centos.logs.rdoproject.org/weirdo-generic-puppet-openstack-scenario001/6345/weirdo-project/logs/testr_results.html.gz07:01
jaosoriorjanki: what about you?07:01
jaosoriorykarel: i don't understand telemetry enough to understand what the fix did :/07:02
*** rcernin has quit IRC07:02
jankijaosorior, still jet-legged. and coffee is no longer effective :(07:05
jaosoriorjanki: haha yeah, that was my first 5 days back in Finland. Now I feel quite normal :D07:06
chkumar|ruckykarel: does telemetry tempest config changes does not fixed it?07:06
jaosoriorchkumar|ruck: apparently not.07:06
ykarelchkumar|ruck, the fix was mostly related to tempest, so i think u can detect what missed07:07
*** leanderthal has joined #tripleo07:07
ykarelchkumar|ruck, it fixed poi but not tripleo07:07
chkumar|ruckykarel: checking07:07
jankijaosorior, i am going to Amsterdam tomorrow so it will worsen for me :(07:07
jankijaosorior, anyways can you pls review https://review.openstack.org/#/c/602196/. its imp and on priority07:08
leanderthalo/07:08
ykarelchkumar|ruck, thought there is a tht patch whtat testproject was testing, promotion job also failed(includes telemetry-tempest-plugin change)07:09
jaosoriorjanki: done07:09
openstackgerritinspurericzhang proposed openstack/puppet-tripleo master: Remove the duplicated word  https://review.openstack.org/60428107:09
ykarelchkumar|ruck, may be both tht and telemetry-tempest-plugin change required, if that's the case recheck testproject would help07:09
jankijaosorior, thanks :)07:09
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256007:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)07:10
*** ooolpbot has quit IRC07:10
chkumar|ruckykarel: does it also include tht + telemetry tempest plugin fix?07:10
chkumar|rucki mean test project07:10
ykareli think not, but good to check07:10
ykarelas it ran yesterday07:11
ykarel14 hr ago, patch merged 13 hr ago, so no07:11
mschuppertjaosorior: if you have some time could you please review https://review.openstack.org/#/c/582623/ its a rebase which already had +2s07:12
chkumar|ruckykarel: https://review.openstack.org/#/c/603590/7 this fix is no linger needed07:12
chkumar|rucklonger07:12
*** amoralej|off is now known as amoralej07:12
*** saneax has quit IRC07:12
chkumar|ruckykarel: where is the tht patch?07:13
ykarelchkumar|ruck, yes, i was talking about tht patch07:13
ykarelsee Depednds-on07:13
*** shyamb has quit IRC07:13
*** shyamb has joined #tripleo07:13
*** saneax has joined #tripleo07:14
*** quique|rover|brb is now known as quiquell|rover07:14
chkumar|ruckykarel: let me test without depends on07:15
ykarelchkumar|ruck, no07:15
ykarelchkumar|ruck, that's already tested on periodic pipeline07:15
quiquell|roverjaosorior: don't know what to do with this one https://review.openstack.org/#/c/603802/07:15
*** saneax has quit IRC07:15
chkumar|ruckykarel: https://review.openstack.org/#/c/603590/7 is replaced by this fix https://review.openstack.org/#/c/604095/1/telemetry_tempest_plugin/config.py07:15
ykarelchkumar|ruck, i mean telemetry-tempest-plugin is already tested, u can confirm that first07:15
quiquell|roverIf the connection closed appears in promotions it will hurt us07:15
jaosoriorquiquell|rover: well, I did restore the patch07:16
*** saneax has joined #tripleo07:17
quiquell|roverjaosorior: Can we abandom this https://review.openstack.org/#/c/604073/   ?07:18
quiquell|roverjaosorior: or repriorize to get the ceph version at the top ?07:18
quiquell|roverjaosorior: Or it would hurt more ?07:18
chkumar|ruckquiquell|rover: ykarel https://logs.rdoproject.org/43/13943/40/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-master/3163398/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-09-20_15_48_53 that telemetry tempest changes is not there07:18
mschuppertjaosorior: thx a lot07:18
chkumar|ruck2018-09-20 15:48:53 | python2-telemetry-tests-tempest-0.1.1-0.20180913085316.49ed45e.el7.noarch -> https://github.com/openstack/telemetry-tempest-plugin/commit/49ed45e01d7f8ea46fade58e2d7361ac92a497a607:18
quiquell|roverchkumar|ruck: Give me the review07:19
jaosoriorquiquell|rover: done07:19
quiquell|roverjaosorior: thanks07:19
chkumar|ruckhttps://review.openstack.org/#/c/604095 is still not picked by the job07:19
*** florianf|afk is now known as florianf07:19
ykarelchkumar|ruck, that's i am trying to say :) see periodic job man07:19
jaosoriorquiquell|rover: oh, that restarted the ceph patch jobs :/07:19
*** gkadam has joined #tripleo07:19
quiquell|roverjaosorior: that's what I mean by hurt...07:19
jaosoriorquiquell|rover: well... now we know07:19
*** shyamb has quit IRC07:20
*** shyamb has joined #tripleo07:20
quiquell|roverjaosorior: but restarted only the gates, they have just started so we are better07:20
quiquell|roverchkumar|ruck: You are talking about disabling tempest tests ?07:20
chkumar|ruckquiquell|rover: nope, it is about fs16 fs17 telemetry tempest issue07:21
quiquell|roverchkumar|ruck: Do you have the review from yesterday disable them ?07:21
chkumar|ruckquiquell|rover: skip list is not picked by periodic jobs in the morning, it got merged early morning07:21
chkumar|ruckquiquell|rover: https://review.openstack.org/60409107:21
quiquell|roverchkumar|ruck: Ahh ok but itÂ's merged07:21
*** shardy has joined #tripleo07:23
quiquell|roverchkumar|ruck: we have rocky periodic jobs with "SKIPPED" result code :-/07:28
*** jpich has joined #tripleo07:31
chkumar|ruckarxcruz: Hello07:32
chkumar|ruckarxcruz: please have a look at this bug https://bugs.launchpad.net/tripleo/+bug/179366507:32
openstackLaunchpad bug 1793665 in tripleo "Fs020 periodic jobs fails to generate tempest results once tempest run finishes" [Critical,Triaged]07:32
chkumar|ruckarxcruz: I have seen this issue in multiple jobs07:32
chkumar|ruckquiquell|rover: but why skipped07:33
quiquell|roverchkumar|ruck: no clue07:33
chkumar|ruckquiquell|rover: can you pass me the link of graphna which only shows timedout one07:34
*** holser_ has joined #tripleo07:34
*** etingof has quit IRC07:35
chkumar|ruckquiquell|rover: queens fs020 failure https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-queens/2dbcad2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-09-21_06_56_2307:36
*** bogdando has joined #tripleo07:37
*** cylopez has joined #tripleo07:37
*** fhubik has joined #tripleo07:37
quiquell|roverchkumar|ruck: http://dashboard-ci.tripleo.org/d/-UEjGKFmz/exploration?orgId=1&var-influxdb_filter=result%7C%3D%7CTIMED_OUT07:39
*** shyamb has quit IRC07:39
*** jfrancoa has quit IRC07:39
chkumar|ruckquiquell|rover: thanks :-)07:39
quiquell|roverchkumar|ruck: yw, changing the dashboard-ci to split periodic fails between releases07:39
*** tosky has joined #tripleo07:40
*** aufi_ has joined #tripleo07:44
*** aufi has quit IRC07:47
ykarelquiquell|rover> chkumar|ruck: we have rocky periodic jobs with "SKIPPED" result code :-/, if u mean featureset jobs, then those were skipped because containers-build job failed07:47
ykarelthat we were discussion in the morning07:47
ykareltripleorocky/centos-binary-keystone-ssh one07:48
quiquell|roverykarel: Ok, they run in parallel and get skipped if it fails ok07:48
*** ykarel is now known as ykarel|lunch07:48
chkumar|ruckykarel: do we have a bug to trakc?07:48
arxcruzchkumar|ruck: this is when some test fail right ?07:48
*** jpena|off is now known as jpena07:48
arxcruzchkumar|ruck: i'm creating an environment to debug07:48
chkumar|ruckarxcruz: yes07:48
ykarel|lunchquiquell|rover, no they don't run parallel, first container-build job runs, if it passes then only child jobs run07:48
quiquell|roverykarel|lunch: Then why do we have skipped if they have not even started as the container-build has fail ?07:49
ykarel|lunchchkumar|ruck, no we don't have bug for that failure, looks like infra related, i asked jpena on #rdo, let's see what he replies, if he thinks it's network then i think there are already some bugs open aainst it07:49
ykarel|lunchquiquell|rover, that should be a ZUUL way or how we define the job, parent child jobs07:50
quiquell|roverykarel|lunch: ack07:50
chkumar|ruckykarel|lunch: ack07:50
chkumar|ruckquiquell|rover: does ara also dumps the ansible run db in ci logs?07:51
quiquell|roverchkumar|ruck: yep we have the ansible.sqlite07:52
*** janki has quit IRC07:53
*** etingof has joined #tripleo07:54
dtantsurstevebaker: thanks! it's a bit unclear how to use this role though07:58
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044508:00
openstackgerritQuique Llorente proposed openstack/tripleo-common master: Set SSH server keep alive options  https://review.openstack.org/60417108:02
quiquell|roverjaosorior: Can you +2 +w this again ? it was failing at infra issue and did a rebase08:02
quiquell|roverhttps://review.openstack.org/#/c/604171/08:02
*** jfrancoa has joined #tripleo08:09
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256008:10
*** ooolpbot has quit IRC08:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)08:10
chkumar|ruckquiquell|rover: does periodic jobs also timed out?08:10
chkumar|ruckquiquell|rover: for example this one https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/088c2cd/job-output.txt.gz#_2018-09-21_03_32_43_06736208:11
quiquell|roverchkumar|ruck: RDO has less timeout restrictions I think08:11
quiquell|roverchkumar|ruck: This is our internal timeout, yep, they have timeout :-/08:12
*** shyamb has joined #tripleo08:13
quiquell|roverPuff ceph gating affected by infra issues08:19
quiquell|rover:-/08:19
quiquell|roverchkumar|ruck, ykarel|lunch, jaosorior: http://logs.openstack.org/23/603323/5/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/bb0e20b/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz#_2018-09-21_07_32_2808:20
quiquell|rovermassive failure08:20
*** two_oes has joined #tripleo08:20
chkumar|ruckquiquell|rover: hmm08:21
chkumar|ruckquiquell|rover: mirror timedout08:21
quiquell|roverchkumar|ruck: this review is doomed08:22
quiquell|roverchkumar|ruck: asked at #openstack-infra08:22
chkumar|ruckquiquell|rover: I am able to do curl on the timed out urls08:22
quiquell|roverchkumar|ruck: yep but infra nodes not08:22
*** gfidente has joined #tripleo08:25
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO  https://review.openstack.org/60429308:26
*** Petersingh is now known as Petersingh|lunch08:27
jaosorior:(08:32
quiquell|roverjaosorior: yep...08:34
quiquell|roverjaosorior: It's affecting all the revies though08:34
openstackgerritQuique Llorente proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429808:39
*** derekh has joined #tripleo08:39
*** shyamb has quit IRC08:41
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates master: Add deprecated_server_resource_name for ObjectStorage role  https://review.openstack.org/60429908:43
*** fhubik is now known as fhubik|brb08:44
*** dalvarez has joined #tripleo08:47
quiquell|roverjaosorior: Can re re-run only the gate jobs ?08:48
quiquell|roverjaosorior: maybe if we reprioritize or the like08:48
quiquell|roverjaosorior: so we don't have to run check again08:48
jaosoriorI don't know :/08:50
quiquell|roverjaosorior: friday is slow08:51
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Upgrade docker ceph container  https://review.openstack.org/60332308:54
*** saneax has joined #tripleo08:58
*** ramishra has quit IRC08:59
quiquell|roverjaosorior: Looks like infra roots can do that09:01
quiquell|rover#zuul answer09:01
jaosoriorquiquell|rover: where did you get that info? (would be good to know where to search)09:04
quiquell|roverjaosorior: #zuul channel09:04
*** jtomasek has quit IRC09:05
*** ramishra has joined #tripleo09:06
*** shyamb has joined #tripleo09:08
*** jaosorior is now known as jaosorior_lunch09:09
*** salmankhan has joined #tripleo09:10
*** ooolpbot has joined #tripleo09:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION09:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256009:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)09:10
*** ooolpbot has quit IRC09:10
*** ykarel|lunch is now known as ykarel09:11
quiquell|roverykarel: ceph failed at infra :-(((09:11
*** salmankhan has quit IRC09:13
ykarelquiquell|rover, :(, reason timeout?09:15
quiquell|roverykarel: mirror timeouts, it's infra09:16
quiquell|roverykarel: other reviews are affected too09:16
ykareldamn :(09:16
quiquell|roverykarel: yes man...09:16
chkumar|ruckquiquell|rover: do we need to requeue all the affected one?09:17
*** salmankhan has joined #tripleo09:17
quiquell|roverchkumar|ruck: the needed ones at least09:17
quiquell|roverI think https://review.openstack.org/#/c/604171/09:18
chkumar|ruckyes this one is needed09:18
quiquell|roveranything else ?09:19
chkumar|ruckdoes ceph-ansible is merged?09:19
quiquell|roverceph-ansible is updated correctly in the mirror now09:19
quiquell|roverproblem is ceph container09:19
chkumar|ruckhttps://review.openstack.org/#/c/603802/09:19
chkumar|ruckpython-tripleoclient connteciton close fix09:20
chkumar|ruckquiquell|rover: ykarel holy shit09:20
*** jtomasek has joined #tripleo09:20
quiquell|roverchkumar|ruck: EmilienM said something about it not merging it09:20
quiquell|roverchkumar|ruck: what's up ?09:21
chkumar|ruckfs016 fs017 passed in noop job without depends on09:21
quiquell|roverchkumar|ruck: kidding09:21
chkumar|ruck*testproject09:21
chkumar|ruckhttps://logs.rdoproject.org/43/13943/41/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset017-master/1b72bec/09:21
chkumar|ruckhttp://logs.rdoproject.org/43/13943/41/check/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master/5152805/09:21
chkumar|ruckhttps://review.rdoproject.org/r/#/c/13943/09:21
ykarelchkumar|ruck, possibly because of skip list, not because of fix, right?09:21
quiquell|roverchkumar|ruck: Can be related for newer RPMs ?09:21
chkumar|ruckcheck the last run09:21
chkumar|ruckykarel: oh forgot to check that09:22
chkumar|ruckquiquell|rover: yes, tests got skipped09:22
quiquell|roverso it's a false positive ?09:22
quiquell|rovera yes :-)09:22
quiquell|roverno problem we are happy with it too :-)09:22
*** Petersingh|lunch is now known as Petersingh09:22
chkumar|ruckquiquell|rover: let me propose a fix to unskip it and try that09:22
quiquell|roverchkumar|ruck: yep, just revert it09:24
chkumar|ruckquiquell|rover: if we revert neutron will start failing09:24
ykarela partial revert just to test,09:25
openstackgerritChandan Kumar proposed openstack/tripleo-quickstart-extras master: Remove telemetry from tempest skip list  https://review.openstack.org/60431109:26
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates master: Run online migrations via "external-upgrade run"  https://review.openstack.org/60382609:28
quiquell|roverok reviews back in top of queue09:28
quiquell|roverlet cross fingers09:28
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs.  https://review.openstack.org/60412109:28
quiquell|roverchkumar|ruck: Do you have the lp for the neutron issue? I am going to dig while stuff get merged09:29
chkumar|ruckquiquell|rover: yes09:30
chkumar|ruckquiquell|rover: https://bugs.launchpad.net/neutron/+bug/178013909:31
openstackLaunchpad bug 1780139 in tripleo "Sending SIGHUP to neutron-server process causes it to hang" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando)09:31
chkumar|ruckquiquell|rover: they have created a new bug out of previous one09:31
openstackgerritJiri Tomasek proposed openstack/tripleo-ui stable/rocky: Use swiftUrl as a base for exported plan download URL  https://review.openstack.org/60431409:31
chkumar|ruckquiquell|rover: previous bug was this one https://bugs.launchpad.net/tripleo/+bug/179348209:32
openstackLaunchpad bug 1780139 in tripleo "duplicate for #1793482 Sending SIGHUP to neutron-server process causes it to hang" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando)09:32
openstackgerritGael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Use the static yaml inventory instead of the ini inventory  https://review.openstack.org/60431609:33
openstackgerritYael Azulay proposed openstack/puppet-tripleo master: Changes are needed for performance and stability at large intervals  https://review.openstack.org/60318309:34
openstackgerritJose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Link subnodes-setup into multinode jobs.  https://review.openstack.org/60412109:36
quiquell|roverchkumar|ruck: fs020 has a fix in progress by bogdando09:40
*** shyamb has quit IRC09:40
bogdandorockt backports are blocked by the missing master merges09:40
bogdandohttps://review.openstack.org/#/c/596274/ and https://review.openstack.org/#/c/596275/ need to go first09:41
chkumar|ruckquiquell|rover: bogdando I think we need to priortize these reviews also ^^09:41
chkumar|ruckjaosorior_lunch: ^^09:44
openstackgerrityatin proposed openstack/tripleo-quickstart master: [DNM] Testing perf for container updates with cached delorean-current  https://review.openstack.org/60431809:51
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer  https://review.openstack.org/60423209:55
*** shyamb has joined #tripleo10:03
*** ade_lee has quit IRC10:06
*** ooolpbot has joined #tripleo10:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION10:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256010:10
*** ooolpbot has quit IRC10:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)10:10
*** iranzo has joined #tripleo10:11
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Use rdo mirror for pikes buildlogs at promotions  https://review.openstack.org/60432410:18
openstackgerritSergii Golovatiuk proposed openstack/python-tripleoclient master: Add timeouts to package_update workflows  https://review.openstack.org/60432510:22
*** quiquell|rover is now known as quique|rover|afk10:26
quique|rover|afkgfidente: master ceph container is going to change ?10:27
gfidentequique|rover|afk yes it is10:27
gfidentebut for now 3.1 is okay10:27
quique|rover|afkgfidente: it's going to break ceph-ansible ?10:27
* quique|rover|afk please tell no10:27
*** quique|rover|afk is now known as quiquell|rover10:29
quiquell|rovergfidente: is there any way we can gate that ?10:29
quiquell|rovergfidente: before tripleo gates, ceph-ansible gates or the like10:30
gfidenteit works the other way around10:30
gfidentewe upgrade ceph-ansible first which can deploy 3.x images10:30
quiquell|rovergfidente: maybe ceph-ansible has to complain if images are not supported10:31
quiquell|rovergfidente: to fail fast10:31
*** shyamb has quit IRC10:32
*** shyamb has joined #tripleo10:33
*** shyamb has quit IRC10:42
openstackgerrityatin proposed openstack/tripleo-quickstart master: [DNM] Testing perf for container updates with cached delorean-current  https://review.openstack.org/60431810:55
openstackgerrityatin proposed openstack/ansible-role-tripleo-modify-image master: [DNM] Test perf with bind mount instead of Copying  https://review.openstack.org/60433010:56
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer  https://review.openstack.org/60423210:57
*** maufart__ has joined #tripleo10:59
*** aufi_ has quit IRC11:02
*** udesale has quit IRC11:05
openstackgerritSorin Sbarnea proposed openstack/tripleo-quickstart master: Make setup repo task output visible when errored  https://review.openstack.org/59935811:08
quiquell|rovermschuppert: What was the state of the rootwrap issue ?11:09
quiquell|rovermschuppert: we needed a promotion ?11:09
*** abishop has quit IRC11:09
*** ooolpbot has joined #tripleo11:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION11:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256011:10
*** ooolpbot has quit IRC11:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)11:10
*** abishop has joined #tripleo11:10
*** aufi_ has joined #tripleo11:10
mschuppertquiquell|rover: yes the promotion11:10
quiquell|rovermschuppert: Let's check if the change is in DLRN11:11
quiquell|rovermschuppert: It was this https://review.openstack.org/#/c/593114 ?11:11
mschuppertquiquell|rover: mine was  https://review.openstack.org/#/c/594145/ , yes that one was for the new filter11:12
*** maufart__ has quit IRC11:13
*** udesale has joined #tripleo11:13
quiquell|rovermschuppert: It's merged Depends-On not neede11:14
mschuppertquiquell|rover: yes, the depends-on was from our tests11:14
quiquell|rovermschuppert: Ahh that was the issue :-(11:14
quiquell|rovermschuppert: Depends-On does not work on merged I think11:15
ykarelis this normal a job running parallel in check and gate pipeline? 603323 is running in both11:15
ykareljpena, ^^11:15
mschuppertquiquell|rover: yes it does not11:15
quiquell|roverykarel: they  promoted to the top in the middle of check11:15
quiquell|roverykarel: Don't know if missing +1v will promote even having +2v11:15
*** Petersingh is now known as Petersingh|afk11:19
ykarelquiquell|rover, i don't think it will merge without +1, +2: quiquell|rover, i don't think it can merge without +1, as per https://github.com/openstack-infra/project-config/blob/master/zuul.d/pipelines.yaml#L75, i have never seen both pipeline running together, so curious if that's a normal behavior, may be jpena have some idea on it11:19
jpenaykarel: is that upstream or in review.rdo?11:19
ykareljpena, upstream11:19
ykarelhttp://zuul.openstack.org/11:19
quiquell|roverjpena: they move our revie in the top, it started gate jobs in the middle of check jobs running11:20
quiquell|roverykarel: yep, it will wait for check after gate jobs11:20
jpenaactually, it's not testing in the gate11:20
quiquell|roverjpena: I see jobs running in the gate11:20
jpenait's speculatively including it as part of another gate job, for 58858711:21
quiquell|roverjpena: in the status page11:21
jpenaI'm not sure about the details, but I think it's got something to do with that speculative execution. Maybe someone at #openstack-infra will know better11:22
*** shyamb has joined #tripleo11:22
*** jpena is now known as jpena|lunch11:23
ykareljpena|lunch, ack Thanks for looking11:23
*** mschuppert is now known as mschuppert|lunch11:24
quiquell|roverykarel: ceph version has fail at scenario008 at tempest :-/11:25
quiquell|roverykarel:  at check11:25
ykarelbut that's non-voting, right11:26
quiquell|roverykarel: ahh yep11:26
quiquell|roverok we are fine in the gates11:27
*** janki has joined #tripleo11:28
chkumar|ruckykarel: do you the logs?11:29
chkumar|rucktempest ceph failure11:29
quiquell|roverchkumar|ruck: non voting, I am in alert mode sorry11:29
chkumar|rucknp11:30
*** udesale has quit IRC11:34
*** udesale has joined #tripleo11:34
*** rh-jelabarre has joined #tripleo11:42
weshaychkumar|ruck, get out of here11:44
weshayquiquell|rover, how can I help11:44
weshayit seems we have not yet fixed the telemetry tests in master11:45
*** dprince has joined #tripleo11:45
*** chkumar|ruck is now known as chkumar|off11:46
quiquell|roverweshay: we are just struggling to get a pair of review merged11:47
quiquell|roverweshay: to have promotions11:48
*** apetrich has joined #tripleo11:48
weshayjaosorior_lunch, you there?11:48
quiquell|roverweshay: they are in the gates now... let's see if we don't hit another infra issue11:48
weshayquiquell|rover, which patches are you watching?11:48
*** morazi has joined #tripleo11:48
weshayI see11:48
weshayon the etherpad11:49
weshaylines 41,4211:49
weshay?11:49
quiquell|roverhttps://review.openstack.org/60332311:49
quiquell|roverhttps://review.openstack.org/60417111:49
quiquell|roverwe need those to fix promotions at master,rocky and queens if nothing new appears, we found some infra issue in RDO but feels intermintent11:50
weshayof course11:50
*** cylopez has quit IRC11:50
quiquell|roversomething related, pushing keystone only docker image at upload job11:50
*** Petersingh|afk is now known as Petersingh11:51
quiquell|roverabout telemetry and neutron, some stuff is being tested, let's just skip it with chkumar|off changes11:51
quiquell|roverthre is issue in pike but that's not very important right now11:52
weshayhrm.. ya.. rdo is at a 54% pass rate11:52
weshayquiquell|rover, ya.. ignore anything other than master and rocky11:52
*** salmankhan1 has joined #tripleo11:52
quiquell|roverweshay: also rocky noop changes was broken by the master tht noop change11:53
quiquell|roverweshay: hubbot does not work well with change-id with multiple reviews11:53
weshayyou'll have to explain that one to me11:53
weshayah.. meh11:53
weshayhubbot--11:53
quiquell|roverweshay: the master tht change share the change-id with rocky's one11:53
weshaywhat?11:53
weshayhow is that possible11:53
weshaybrb.. coffee11:54
quiquell|roverweshay: cherry-pick without renewing the change-id https://review.openstack.org/#/q/I67ba1d4f80d8cc34fd3735d647b62e3179aaaa1411:54
*** salmankhan has quit IRC11:54
*** salmankhan1 is now known as salmankhan11:54
*** leanderthal has quit IRC11:54
quiquell|roverweshay: have recreate both, but don't know if we want to replace tqe master noop with tht master noop11:54
*** etingof has quit IRC11:55
weshayquiquell|rover, I'm fucking locked up in meetings all day11:58
weshayI need like 15min w/ you11:58
weshaylolz11:58
*** aedc has quit IRC11:58
*** aedc_ has joined #tripleo11:58
quiquell|roverweshay: there is CIX now11:58
weshayya.. joining11:58
*** cylopez has joined #tripleo11:58
weshayjaosorior_lunch, you may want to join this meeting11:59
*** mschuppert|lunch is now known as mschuppert12:00
*** yprokule has quit IRC12:06
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates stable/pike: Fix Swift storage device setup  https://review.openstack.org/60307112:09
*** fhubik|brb is now known as fhubik12:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256012:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)12:10
*** ooolpbot has quit IRC12:10
*** jaosorior_lunch is now known as jaosorior12:11
jaosoriorweshay: sure12:11
*** trown|outtypewww is now known as trown12:14
*** yprokule has joined #tripleo12:14
*** rfolco has joined #tripleo12:15
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Use openshift-ansible container instead of RPMs  https://review.openstack.org/58386812:16
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Add heat param for openshift prerequisites playbook  https://review.openstack.org/60433812:16
*** raildo has joined #tripleo12:16
openstackgerritJiri Stransky proposed openstack/tripleo-heat-templates stable/queens: [Queens/Pike] Run online data migrations on converge, allow disabling them  https://review.openstack.org/60434212:20
openstackgerritSergii Golovatiuk proposed openstack/python-tripleoclient master: Add 2h timeout when waiting for websocket messages on package_update  https://review.openstack.org/60432512:26
*** jpena|lunch is now known as jpena12:26
*** udesale has quit IRC12:26
*** toure|gone is now known as toure12:28
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer  https://review.openstack.org/60423212:29
*** boazel_ has quit IRC12:29
*** shyamb has quit IRC12:29
*** rlandy has joined #tripleo12:32
*** quiquell|rover is now known as quique|rover|lch12:32
*** gkadam has quit IRC12:34
*** ratailor has quit IRC12:34
*** mhenkel_ has joined #tripleo12:35
openstackgerritMerged openstack/tripleo-quickstart-extras master: Upgrade docker ceph container  https://review.openstack.org/60332312:37
openstackgerritMerged openstack/tripleo-common master: Set SSH server keep alive options  https://review.openstack.org/60417112:37
openstackgerritMerged openstack/tripleo-heat-templates master: Add host_routes to port_from_pool.j2  https://review.openstack.org/60317212:37
jaosoriorquique|rover|lch: ^^12:38
*** salmankhan1 has joined #tripleo12:40
*** salmankhan has quit IRC12:41
*** salmankhan1 is now known as salmankhan12:41
*** skramaja has quit IRC12:42
*** EmilienM is now known as EvilienM12:44
EvilienMgood morning...12:44
trownanyone able to review a few tripleo-common patches? https://review.openstack.org/#/c/602112/ and the ones below ... last patches needed to add validation for openshift deploy (other than https://review.openstack.org/#/c/600067/16 )12:44
trownquick hide12:44
*** agopi|brb is now known as agopi12:45
jaosoriorquique|rover|lch chkumar|off: With the new ceph container and the ssh fix, should we run the promotion job?12:45
*** cylopez has quit IRC12:47
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix several config file location  https://review.openstack.org/60435112:48
*** cylopez has joined #tripleo12:48
*** lblanchard has joined #tripleo12:49
quique|rover|lchjaosorior: I think so, let's reduce the time window for new stuff breaking promotions12:49
quique|rover|lchjaosorior: I can change time trigger of periodic-24h12:49
quique|rover|lchjaosorior: but maybe you can just kick in them12:49
jaosoriorquique|rover|lch: how?12:50
openstackgerritMerged openstack/tripleo-quickstart-extras master: Revert "Revert "Add ability to have dlrn build all the packages at once""  https://review.openstack.org/60340612:50
*** thrash|g0ne is now known as thrash12:51
*** ssbarnea|bkp has joined #tripleo12:51
*** aedc_ is now known as aedv12:51
*** aedv is now known as aedc12:51
quique|rover|lchjaosorior: They are going to kick in at 13:30 utc so we are good12:52
quique|rover|lchjaosorior: in half an hour12:52
jaosoriorcool cool!12:52
*** panda|off is now known as panda12:53
quique|rover|lchjaosorior: Ahh wait... didn't refresh the repo12:53
*** leanderthal has joined #tripleo12:53
*** quique|rover|lch is now known as quiquell|rover12:55
quiquell|roverjaosorior: https://review.rdoproject.org/r/1636012:55
dtantsurshardy: hi! you asked for this bug: https://bugs.launchpad.net/tripleo/+bug/179313412:56
openstackLaunchpad bug 1793134 in tripleo "Switch undercloud from using Nova capabilities" [High,Triaged] - Assigned to Dmitry Tantsur (divius)12:56
shardydtantsur: hi!  Thanks for raising that, will check it12:57
openstackgerritGael Chamoulaud proposed openstack/tripleo-validations master: Fix a minor typo in the ceilometerdb-size validation  https://review.openstack.org/60435313:02
*** tzumainn has joined #tripleo13:03
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates stable/rocky: Add host_routes to port_from_pool.j2  https://review.openstack.org/60435413:03
*** cylopez has left #tripleo13:06
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256013:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)13:10
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: [WIP]Add root device hints validation  https://review.openstack.org/60435613:12
*** quiquell|rover is now known as quique|rover|afk13:13
*** ramishra has quit IRC13:13
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: Mount /usr/share/ceph-ansible into mistral-executor  https://review.openstack.org/60435713:14
openstackgerritMartin Schuppert proposed openstack/tripleo-heat-templates master: WIP - Add nova file_backed_memory and memory_backing_dir support for qemu.conf  https://review.openstack.org/60436013:18
openstackgerritGiulio Fidente proposed openstack/tripleo-common master: Do not include ceph-ansible in the container image.  https://review.openstack.org/60436113:19
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Use rdo mirror for pikes buildlogs at promotions  https://review.openstack.org/60432413:20
*** amoralej is now known as amoralej|lunch13:23
*** mjturek has joined #tripleo13:29
openstackgerritJose Luis Franco proposed openstack/tripleo-quickstart-extras master: WIP: Add subnode-setup.yml playbook.  https://review.openstack.org/60412013:30
*** jistr is now known as jistr|call13:31
*** Petersingh is now known as Petersingh|afk13:31
*** Petersingh|afk is now known as Petersingh13:37
*** mcornea has joined #tripleo13:38
*** quique|rover|afk is now known as quiquell|rover13:41
*** florianf is now known as florianf|brb13:41
dtantsurshardy: also https://review.openstack.org/#/c/576856/ is ready for another review pass, when you some time13:43
dtantsurslagle: ^^13:43
*** iranzo has quit IRC13:43
*** Petersingh is now known as Petersingh|away13:48
*** mjturek has quit IRC13:49
*** dansmith is now known as SteelyDan13:50
*** errantekarmico has quit IRC13:51
*** mjturek has joined #tripleo13:55
*** mhenkel_ has quit IRC13:56
mwhahahaso hows CI today13:56
*** ade_lee has joined #tripleo13:57
quiquell|rovermwhahaha: needed reviews merged13:58
quiquell|rovermwhahaha: waitting for promotions13:58
*** amoralej|lunch is now known as amoralej13:59
jaosoriorquiquell|rover: which reviews are left? thought the ones needed had just merged.13:59
quiquell|roverjaosorior: I think we are good, tempest, telemetry is in progress I think14:00
quiquell|roverjaosorior: And I don't know if we have green light for https://review.openstack.org/#/c/603802/14:01
weshayquiquell|rover, in the gate right?14:01
quiquell|roverweshay: merged14:01
weshayk14:01
jaosoriorquiquell|rover: if that's needed for CI, you could ask in infra for folks to prioritize it in the gate.14:01
quiquell|roverEvilienM: Do you remember why we cannot merge this https://review.openstack.org/#/c/603802/ ?14:02
openstackgerrityatin proposed openstack/tripleo-quickstart master: [DNM] Testing perf for container updates with cached delorean-current  https://review.openstack.org/60431814:02
mwhahahaits in the gate?14:04
quiquell|roverjaosorior: Can we merge this https://review.rdoproject.org/r/#/c/16361/14:04
quiquell|roverjaosorior: is the revert to run openstack-periodic24hr14:04
jaosoriorquiquell|rover: +1ed it. I don't have +2 rights there :(14:05
quiquell|roverjpena: ^ can you ?14:05
*** openstackgerrit has quit IRC14:07
*** florianf|brb is now known as florianf14:09
*** ooolpbot has joined #tripleo14:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION14:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256014:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)14:10
*** ooolpbot has quit IRC14:10
*** aedc_ has joined #tripleo14:12
*** aedc has quit IRC14:13
*** radeks has joined #tripleo14:13
*** quiquell|rover is now known as quique|rover|off14:14
*** openstackgerrit has joined #tripleo14:17
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet  https://review.openstack.org/60325314:17
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks  https://review.openstack.org/60325414:17
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs  https://review.openstack.org/60325514:17
*** boazel has joined #tripleo14:19
shardydtantsur: sorry been in a meeting, yup re-testing & reviewing the workflows patch is on my todo list, been travelling again this week but hopefully will get to in Monday14:20
dtantsurcool!14:20
jaosorioroff for the day, have a good weekend everyone!14:25
spotzSeeya jaosorior14:25
*** janki has quit IRC14:33
*** mjturek has quit IRC14:39
*** pmannidi has joined #tripleo14:42
*** gkadam has joined #tripleo14:43
*** aufi_ has quit IRC14:44
*** mjturek has joined #tripleo14:44
*** ksambor has quit IRC14:45
*** rdopiera has quit IRC14:46
*** paramite has quit IRC14:47
*** mjturek has quit IRC14:49
openstackgerritBrent Eagles proposed openstack/puppet-tripleo master: WIP: Add support for configuration undercloud with puppet  https://review.openstack.org/60437714:50
*** cylopez has joined #tripleo14:53
*** sshnaidm|pto has joined #tripleo14:56
*** aedc_ has quit IRC14:58
*** radeks has quit IRC14:58
*** aedc_ has joined #tripleo14:58
*** radeks has joined #tripleo14:58
*** dxiri has joined #tripleo14:59
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer  https://review.openstack.org/60423214:59
*** radeks has quit IRC15:01
*** radeks has joined #tripleo15:01
*** sshnaidm|pto has quit IRC15:02
*** ykarel is now known as ykarel|away15:04
weshaygfidente, ping15:06
gfidenteweshay hey15:07
*** saneax has quit IRC15:07
weshaygfidente, sucks there are no jobs there15:07
gfidenteweshay there wer?15:07
gfidente*were15:07
weshayno15:07
gfidenteand also, there where?15:07
gfidenterdo ceph-ansible you mean right?15:08
*** mjturek has joined #tripleo15:08
pandawhere wolves ? there wolves .15:08
gfidenteweshay so my point is I need to add ceph-ansible as dep15:08
gfidentelike we did for other stuff like ansible-pacemaker15:08
gfidentebut we don't ship ceph-ansible ourselves so there need to be pointers to the ceph repos15:09
gfidente(in the ci job)15:09
weshaygfidente, ah.. nice alfreado owns those jobs I think15:09
gfidenteweshay ack thanks I'll add him on the review15:10
*** gkadam has quit IRC15:10
*** ooolpbot has joined #tripleo15:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION15:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256015:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)15:10
*** ooolpbot has quit IRC15:10
*** ykarel|away has quit IRC15:10
*** jistr|call is now known as jistr15:11
*** leanderthal has quit IRC15:18
*** bnemec is now known as beekneemech15:21
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154015:24
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks  https://review.openstack.org/60325415:29
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs  https://review.openstack.org/60325515:29
*** ykarel has joined #tripleo15:34
nhicherweshay, rfolco, rlandy: if you have time to review https://review.openstack.org/#/c/596432 it will be great =)15:35
*** jfrancoa has quit IRC15:38
*** cylopez has left #tripleo15:39
*** yprokule has quit IRC15:40
*** med_ has joined #tripleo15:40
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154015:40
*** pmannidi has quit IRC15:41
*** aedc_ has quit IRC15:41
*** aedc_ has joined #tripleo15:41
*** fhubik has quit IRC15:42
*** marios is now known as marios|home15:49
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: Add DockerPuppetMountHostPuppet parameter  https://review.openstack.org/55084815:49
*** marios|home is now known as marios15:49
*** radeks has quit IRC15:50
*** radeks has joined #tripleo15:50
openstackgerritTrevor Vardeman proposed openstack/tripleo-common master: Hardcoding x86_64 in template overrides for the  opendaylight repository  https://review.openstack.org/60440115:50
*** med_ has quit IRC15:50
*** jpich has quit IRC15:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044516:00
*** florianf is now known as florianf|afk16:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO  https://review.openstack.org/60429816:00
*** med_ has joined #tripleo16:03
*** rh-jelabarre has quit IRC16:07
*** ooolpbot has joined #tripleo16:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION16:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256016:10
*** ooolpbot has quit IRC16:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)16:10
*** aedc_ has quit IRC16:16
*** holser_ has quit IRC16:18
*** aedc has joined #tripleo16:20
openstackgerritAlex Schultz proposed openstack/tripleo-puppet-elements stable/queens: Remove Openstack and Ceph packages  https://review.openstack.org/60396516:21
*** aedc has quit IRC16:22
*** agopi is now known as agopi|long_lunch16:22
*** aedc has joined #tripleo16:23
*** janki has joined #tripleo16:25
*** rh-jelabarre has joined #tripleo16:29
*** mburned has joined #tripleo16:31
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker: deploy podman until we get podman working without docker  https://review.openstack.org/60423516:32
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker: deploy podman until we get podman working without docker  https://review.openstack.org/60423516:32
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman  https://review.openstack.org/60051716:32
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154016:33
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add extra tasks to run lint and upgrade after deployment.  https://review.openstack.org/60241616:33
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-quickstart-extras master: Add extra tasks to run lint and upgrade after deployment.  https://review.openstack.org/60241616:35
*** salmankhan has quit IRC16:36
openstackgerritMerged openstack/python-tripleoclient master: Fix typo in upgrade playbook's name.  https://review.openstack.org/60336316:36
openstackgerritMerged openstack/tripleo-heat-templates master: Fix syntax for set_fact module.  https://review.openstack.org/60235316:36
*** vkmc is now known as vkmc|afk16:37
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154016:38
*** dtantsur is now known as dtantsur|afk16:41
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154016:43
*** radeks has quit IRC16:44
openstackgerritAthlan-Guyot sofer proposed openstack-infra/tripleo-ci master: Adding new standalone-upgrade jobs and lint playbook in standalone.  https://review.openstack.org/60412616:45
openstackgerritMerged openstack/python-tripleoclient master: Add a guard to break if no connection  https://review.openstack.org/60380216:46
openstackgerritMerged openstack/tripleo-heat-templates master: Bind mount the database client settings in sahara_db_sync  https://review.openstack.org/59498016:46
*** mjturek has quit IRC16:46
*** derekh has quit IRC16:48
*** abishop_ has joined #tripleo16:49
*** abishop has quit IRC16:49
*** jpena is now known as jpena|off16:54
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] add a job to test the reproducer  https://review.openstack.org/60423216:57
*** gkadam has joined #tripleo17:03
*** spsurya has quit IRC17:05
*** bogdando has quit IRC17:06
*** ooolpbot has joined #tripleo17:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION17:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256017:10
*** ooolpbot has quit IRC17:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)17:10
*** med_ has quit IRC17:12
openstackgerritDan Macpherson proposed openstack/ansible-role-openstack-operations master: Adding Backup and Restore Operations  https://review.openstack.org/60443917:14
*** med_ has joined #tripleo17:18
*** ssbarnea|bkp has quit IRC17:18
*** akhila has joined #tripleo17:20
*** dxiri has quit IRC17:26
*** dxiri has joined #tripleo17:27
*** akhila has quit IRC17:29
openstackgerritAkhila Kishore proposed openstack/puppet-tripleo master: Add puppet base class for barometer service.  https://review.openstack.org/60444217:33
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add MTU to neutron networks and nic-config templates  https://review.openstack.org/59959317:37
*** tzumainn has quit IRC17:45
*** trown is now known as trown|lunch17:46
*** med_ has quit IRC17:54
*** janki has quit IRC17:57
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Dell EMC Sc: Add support for excluded_domain_ips  https://review.openstack.org/60444417:58
openstackgerritMerged openstack/tripleo-ui stable/rocky: Use swiftUrl as a base for exported plan download URL  https://review.openstack.org/60431417:59
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Dell EMC SC: Add support for excluded_domain_ips  https://review.openstack.org/60444417:59
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add MTU to neutron networks and nic-config templates  https://review.openstack.org/59959318:00
*** gfidente is now known as gfidente|afk18:05
*** mjturek has joined #tripleo18:06
openstackgerritRajini Karthik proposed openstack/tripleo-heat-templates master: Dell EMC Sc: Add support for excluded_domain_ips  https://review.openstack.org/60444718:07
*** panda has quit IRC18:08
*** ykarel is now known as ykarel|away18:09
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256018:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)18:10
*** ooolpbot has quit IRC18:10
*** panda has joined #tripleo18:10
*** morazi has quit IRC18:11
*** tzumainn has joined #tripleo18:27
*** jtomasek has quit IRC18:32
*** ykarel|away has quit IRC18:54
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker: deploy podman until we get podman working without docker  https://review.openstack.org/60423519:09
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256019:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)19:10
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman  https://review.openstack.org/60051719:15
*** shardy has quit IRC19:16
*** morazi has joined #tripleo19:17
*** agopi|long_lunch is now known as agopi19:17
*** shardy has joined #tripleo19:17
*** gfidente|afk has quit IRC19:25
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer  https://review.openstack.org/60249219:28
openstackgerritJames Slagle proposed openstack/tripleo-common stable/rocky: Set SSH server keep alive options  https://review.openstack.org/60445519:32
*** gkadam has quit IRC19:35
*** hjensas has quit IRC19:36
*** dprince has quit IRC19:36
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer  https://review.openstack.org/60249219:43
*** med_ has joined #tripleo19:45
*** radeks has joined #tripleo19:45
*** radeks_ has joined #tripleo19:46
*** rlandy is now known as rlandy|brb19:47
*** radeks has quit IRC19:50
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks  https://review.openstack.org/60446019:52
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode  https://review.openstack.org/60446119:52
*** slaweq has quit IRC19:54
*** slaweq has joined #tripleo19:57
*** mjturek has quit IRC20:00
openstackgerritSam Doran proposed openstack/ansible-role-tripleo-cookiecutter master: Fix incorrect file name  https://review.openstack.org/60446320:03
openstackgerritSam Doran proposed openstack/ansible-role-tripleo-cookiecutter master: Change YAML file extensions to .yaml from .yml  https://review.openstack.org/60446320:05
*** hjensas has joined #tripleo20:05
*** mjturek has joined #tripleo20:07
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154020:09
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256020:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)20:10
*** ooolpbot has quit IRC20:10
*** mjturek has quit IRC20:12
*** rlandy|brb is now known as rlandy20:16
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt  https://review.openstack.org/59154020:19
*** aedc has quit IRC20:21
*** aedc has joined #tripleo20:21
*** med_ has quit IRC20:23
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: enable fedora-28 for the reproducer  https://review.openstack.org/60249220:26
*** boazel has quit IRC20:27
*** boazel has joined #tripleo20:33
*** radeks_ has quit IRC20:38
mwhahahaslagle: i'm getting http://paste.openstack.org/show/730572/ when i try and do a deployment with current-tripleo-dev20:40
mwhahahathoughts?20:40
mwhahahathe containers are likely old (due to lack of promotion)20:40
*** boazel has quit IRC20:42
slaglemwhahaha: not sure, haven't seen that one before20:45
mwhahahai just rebuilt this underclodu today cause it started yesterday/today20:46
mwhahahait was working on wednesday20:46
slagledid you check what merged?20:46
mwhahahai'm not even sure where that code is or what it's erroring on20:48
slaglei think the yaml it's showing there is yaml from what's in the config-download dir20:51
slaglesomething is invalid yaml there20:52
slaglenot sure what's trying to parse it at that point20:52
*** thrash is now known as thrash|g0ne20:53
slaglei'd check the mistral executor log to see if the GetOvercloudConfig action logged a tb20:53
slagleit might be a problem with HostnameMap or an unnamed SoftwareDeployment20:54
slaglei've seen those cause bad yaml in config-download before20:55
mwhahahaslagle: oh I know what's wrong20:57
mwhahahaI'm using the standalone env which I don't want when using an undercloud20:58
mwhahahaSo the ports are bad20:58
slaglein DeployedServerPortMap?20:58
mwhahahaYeah I don't want that20:59
mwhahahaSo I need to copy the bits I want out into a new env file20:59
*** akrivoka has quit IRC21:00
openstackgerritNicolas Hicher proposed openstack-infra/tripleo-ci master: testenv-worker: add keystone v3 support  https://review.openstack.org/60447021:00
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: fedora28 standalone job definition  https://review.openstack.org/59537421:06
openstackgerritSam Doran proposed openstack/tripleo-ansible master: Add playbook to generate config files inside a container  https://review.openstack.org/60447421:07
mwhahahaseems like we're missing some validation somewhere, that probably shouldn't make it to the ansible phase before failing21:10
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256021:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)21:10
*** shardy has quit IRC21:15
*** shardy has joined #tripleo21:15
*** raildo has quit IRC21:15
openstackgerritBob Fournier proposed openstack/os-net-config master: Gracefully handle interface name TypeErrors  https://review.openstack.org/58298921:19
slaglemwhahaha: was there a traceback from the mistral action?21:21
slaglemwhahaha: i'd guess it was failing in validate_config from utils/config.py21:22
slaglewe try to load all the yaml we write out to validate it, so it shouldn't have gotten to running ansible-playbook yet21:22
mwhahahayes21:23
mwhahahain utils/config.py 19321:23
slagleyea that's validate_config21:24
mwhahahayea but my thought was we shouldn't even get to that part21:24
mwhahahacause we're missing something21:24
mwhahahawe're just blindly passing text through somewhere21:24
slagleyea in Heat21:24
slaglei tried to do this in heat a while back, but couldn't figure it out21:25
mwhahahahttp://paste.openstack.org/show/730573/ is the specific traceback21:26
openstackgerritMichael Johnson proposed openstack/diskimage-builder master: Fix DIB ubuntu-minimal running on bionic (18.04)  https://review.openstack.org/60447821:26
mwhahahain this case i had defined using deployed-neutron-port.yaml but didn't pass in a DeployedServerPortMap21:27
mwhahahaperhaps that should be required and not have a default?21:27
mwhahaha(i actually didn't want to use deployed server so that was my fault)21:28
*** med_ has joined #tripleo21:31
slaglemwhahaha: removing the default makes sense. for cases where you do want it, but the lookup fails for some reason, this is what I tried: https://review.openstack.org/#/c/52081621:31
slaglebut it just didn't work, b/c, well my brain doesnt think in "Heat"21:32
mwhahaha:D21:32
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: break out release config by distro type  https://review.openstack.org/60238721:33
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: config: ignore missing server_id from the stack  https://review.openstack.org/60448321:35
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet  https://review.openstack.org/60325321:37
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks  https://review.openstack.org/60325421:37
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs  https://review.openstack.org/60325521:37
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks  https://review.openstack.org/60446021:37
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode  https://review.openstack.org/60446121:37
openstackgerritEmilien Macchi proposed openstack/tripleo-common master: config: ignore missing server_id from the stack  https://review.openstack.org/60448321:37
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet  https://review.openstack.org/60325321:41
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: docker_puppet_tasks  https://review.openstack.org/60325421:41
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: check mode: paunch configs  https://review.openstack.org/60325521:41
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Name unnamed tasks  https://review.openstack.org/60446021:41
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Fix tasks in check mode  https://review.openstack.org/60446121:41
*** mjturek has joined #tripleo21:41
EvilienMweshay: hey21:42
EvilienMI tried to think how we could test blacklisting in tripleo ci21:42
EvilienMhttps://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/server_blacklist.html21:42
EvilienMfirst, I don't think it's something worth testing in check/gate21:42
EvilienMbut maybe in periodic21:42
EvilienMI was wondering to re-use an ovb featureset & override some ansible vars in the job definition in zuul layout21:43
*** med_ has quit IRC21:44
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: break out release config by distro type  https://review.openstack.org/60238721:44
EvilienMbasically I would override 2 things :21:45
EvilienMlike "enable_blacklisting" and "blacklisted_nodes", these params would live in oooq-extras maybe21:45
*** boazel has joined #tripleo21:48
*** ssbarnea|bkp has joined #tripleo21:52
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Update standalone container prep steps  https://review.openstack.org/60448721:53
*** tzumainn has quit IRC21:53
*** rlandy has quit IRC21:56
openstackgerritAlex Schultz proposed openstack/tripleo-quickstart-extras master: Update standalone container prep steps  https://review.openstack.org/60448722:00
*** boazel has quit IRC22:04
*** zul has quit IRC22:04
openstackgerritEmilien Macchi proposed openstack/ansible-role-container-registry master: Install & configure podman in a new task  https://review.openstack.org/60287922:08
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: docker: deploy podman until we get podman working without docker  https://review.openstack.org/60423522:08
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: WIP - Switch fs027 to deploy with podman  https://review.openstack.org/60051722:08
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256022:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)22:10
*** ooolpbot has quit IRC22:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Ensure some directories are created with the correct SELinux label  https://review.openstack.org/60053222:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Conditionally mount the TLS-related certs on the haproxy container  https://review.openstack.org/60382722:10
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Ensure some directories are created with the correct SELinux label  https://review.openstack.org/60053222:11
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow to run bootstrap containers in privileged mode.  https://review.openstack.org/60053322:11
*** aedc has quit IRC22:11
*** mjturek has quit IRC22:12
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Remove parameter reference to non-exisingt parameter  https://review.openstack.org/60449022:17
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Set proper setype for service directories  https://review.openstack.org/60053422:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Allow to deactivate SELinux separation for selected containers  https://review.openstack.org/60053522:21
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Support podman when tagging container for Pacemaker  https://review.openstack.org/60418022:23
*** mcornea has quit IRC22:27
*** tosky has quit IRC22:58
*** tosky has joined #tripleo22:59
*** ssbarnea|bkp has quit IRC23:03
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/179256023:10
openstackLaunchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,Triaged] - Assigned to Jiří Stránský (jistr)23:10
*** ooolpbot has quit IRC23:10
*** morazi has quit IRC23:10
openstackgerritJanki Chhatbar proposed openstack/tripleo-heat-templates master: Allow overlay tunnel endpoints on IPv6 address  https://review.openstack.org/60353223:11
*** jrist has quit IRC23:12
*** tosky has quit IRC23:13
*** janki has joined #tripleo23:13
openstackgerritMerged openstack/tripleo-heat-templates master: Enable Sahara API health check  https://review.openstack.org/56472323:27
*** janki has quit IRC23:28
openstackgerritMerged openstack/tripleo-heat-templates master: Enable health check for Ironic inspector services  https://review.openstack.org/56395023:31
*** lblanchard has quit IRC23:33
*** lblanchard has joined #tripleo23:34
*** mschuppert has quit IRC23:35
*** mschuppert has joined #tripleo23:36
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Fix list concatenation of routes in bond-with-vlan  https://review.openstack.org/60449323:45
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Add MTU to neutron networks and nic-config templates  https://review.openstack.org/59959323:47
*** medberry has joined #tripleo23:53
*** dsneddon has quit IRC23:58
*** dsneddon has joined #tripleo23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!