Thursday, 2019-02-07

*** holser_ has quit IRC00:20
*** tosky has quit IRC00:43
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario001 (3 more messages)00:51
*** holser_ has joined #oooq02:25
*** holser_ has quit IRC02:47
*** holser_ has joined #oooq02:50
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario003 (3 more messages)02:52
*** holser_ has quit IRC02:52
*** holser_ has joined #oooq02:54
*** holser_ has quit IRC03:03
*** holser_ has joined #oooq03:04
*** holser_ has quit IRC03:10
*** holser_ has joined #oooq03:12
*** holser_ has quit IRC03:15
*** holser_ has joined #oooq03:17
*** skramaja has joined #oooq03:20
*** holser_ has quit IRC03:20
*** vinaykns has quit IRC03:25
*** rlandy|bbl is now known as rlandy03:43
*** rlandy has quit IRC04:06
*** gkadam has joined #oooq04:06
*** ykarel has joined #oooq04:14
*** skramaja_ has joined #oooq04:24
*** skramaja has quit IRC04:24
*** udesale has joined #oooq04:25
*** jtomasek has joined #oooq04:48
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004 (2 more messages)04:52
*** ykarel has quit IRC05:03
*** jtomasek has quit IRC05:07
*** jtomasek has joined #oooq05:16
*** ykarel has joined #oooq05:19
*** jtomasek has quit IRC05:24
*** chandankumar has joined #oooq05:45
*** chandankumar is now known as chkumar|ruck05:46
*** ratailor has joined #oooq05:56
*** quiquell|off is now known as quiquell|rover06:36
*** saneax has joined #oooq06:37
*** jfrancoa has joined #oooq06:39
quiquell|roverchkumar|ruck: o/06:40
chkumar|ruckquiquell|rover, \o/06:40
chkumar|ruckquiquell|rover, master container build again failed as RDO master was not consistent, https://review.rdoproject.org/r/18733 right now to make consistent, we need to for next run to pick rabbitmq changes06:41
quiquell|roverchkumar|ruck: also we have issues with noop06:45
chkumar|ruckquiquell|rover, master noop jobs? upgrades and scenario007 failing06:48
*** chkumar|ruck has quit IRC06:48
*** chandankumar has joined #oooq06:48
*** chandankumar is now known as chkumar|ruck06:49
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004 (2 more messages)06:52
*** quiquell|rover is now known as quique|rover|r--06:53
quique|rover|r--Was wrong noop are ok06:55
quique|rover|r--Scenario007 still failing?06:57
quique|rover|r--We had to deactivate tacker06:57
*** kopecmartin|off is now known as kopecmartin07:01
*** jtomasek has joined #oooq07:12
*** dtrainor has joined #oooq07:15
*** quique|rover|r-- is now known as quiquell|rover07:18
quiquell|roverchkumar|ruck: So periodic-tripleo-centos-7-master-containers-build has fail at nova start ?07:18
quiquell|roverI don't see rabbitmq container07:19
quiquell|roverAhh looks like stuff merged after periodic07:20
quiquell|rover/usr/local/bin/kolla_start: line 18: /usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory07:20
chkumar|ruckquiquell|rover, changes are not yet picked up there07:22
quiquell|roveryep just checked :-/07:22
quiquell|roverOk let's wait for next run07:22
chkumar|rucklet me fix my irc07:22
ykarelquiquell|rover, it will fail even in next run, as job will again run with broken containers, i shared issue with chkumar|ruck on #rdo07:24
*** quiquell|rover is now known as quiquell|rover|b07:33
*** quiquell|rover|b is now known as quique|rover|brb07:33
*** quique|rover|brb is now known as quiquell|rover07:58
*** ykarel is now known as ykarel|lunch08:16
*** ccamacho has joined #oooq08:20
quiquell|roverchkumar|ruck: weshay say something about we have to look at https://bugs.launchpad.net/tripleo/+bug/180297108:27
openstackLaunchpad bug 1802971 in neutron "tempest volume_boot_pattern and basic_ops running concurrently causing timeouts" [Undecided,In progress] - Assigned to Slawek Kaplonski (slaweq)08:27
chkumar|ruckquiquell|rover, I have assigned it in the mornign08:28
chkumar|ruckquiquell|rover, I am going to hunt it donw08:28
chkumar|ruckquiquell|rover, there are two bugs something like that08:28
quiquell|roverchkumar|ruck: ack08:28
*** dsneddon__ has quit IRC08:30
*** dsneddon__ has joined #oooq08:31
mariosquiquell|rover: chkumar|ruck o/ folks do you know off hand what ansible we have in upstream jobs, is it ansible 2.6 (yurii is asking from upgrades ... issue with https://review.openstack.org/#/c/633731/ http://logs.openstack.org/31/633731/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/f18bf70/job-output.txt.gz#_2019-01-29_17_47_15_26267308:44
*** jpena|off is now known as jpena08:45
quiquell|rovermarios: maybe this ? https://github.com/openstack/tripleo-quickstart/blob/master/requirements.txt#L308:46
mariosquiquell|rover: nice thanks08:46
quiquell|roverchecking job08:46
*** tosky has joined #oooq08:46
*** bogdando has joined #oooq08:47
quiquell|roverHumm but I see this RPM installed08:48
quiquell|roverInstalled: ansible-2.6.11-1.el7.ans.noarch08:48
quiquell|roverhttp://logs.openstack.org/31/633731/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/f18bf70/logs/undercloud/var/log/yum.log.txt.gz08:48
*** zbr|ssbarnea has joined #oooq08:49
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004 (2 more messages)08:52
mariosquiquell|rover: trying to get yurii to join here08:52
*** chkumar|ruck has quit IRC08:53
*** yprokule has joined #oooq08:54
mariosyprokule: o 10:48 < quiquell|rover> http://logs.openstack.org/31/633731/3/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/f18bf70/logs/undercloud/var/log/yum.log.txt.gz08:54
quiquell|rovermarios: is the ansible from requirements.txt we run tq on venv there08:54
mariosquiquell|rover: i see so 2.5.7 should be in play then08:55
mariosquiquell|rover: even if 2.6 installed on the box08:55
mariosbut either way, yprokule the review talks about 2.508:55
mariosyprokule: so if it is 2.5.7 or 2.6 should be enough?08:55
yprokulemarios: according to docs yes08:56
yprokulemarios: but as U can see smth is wrong08:56
mariosyprokule: well hold on.. the version bit is about the loop08:57
mariosyprokule: i mean since 2.5 bit in the commit msg08:57
mariosyprokule: but the error is for dict2items which occurs only once https://git.openstack.org/cgit/openstack/tripleo-upgrade/commit/?id=4c1e3b510b09fcc3cfa39f894663d3b0a937048308:57
mariosyprokule: the error is specific about the dict2items filter not the loop template error while templating string: no filter named 'dict2items'. String08:58
mariosyprokule: and looks like dict2items needs 2.608:58
marioshttps://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#dict-filter08:58
quiquell|rovermarios: yep08:59
quiquell|rovermarios: so we are usin 2.5 from pip there08:59
quiquell|rovermarios: the one as requirements.txt08:59
mariosyprokule: quiquell|rover thanks. so lets propose an increase to 2.6? in requirements.txt?08:59
mariosyprokule: want to do the honours?08:59
yprokulemarios: dict2items is mentioned in 2.5 porting guide08:59
mariosat least we can point at it and people can say if it is a bad idea08:59
quiquell|rovermarios: Not that easy08:59
mariosyprokule: but the docs say new in 2.6 /me shrugs08:59
mariosquiquell|rover: k09:00
quiquell|roverzbr|ssbarnea: do you rememeber what was the issue with bumping ansible to 2.6 ?09:00
yprokulehttps://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.5.html#with-dict09:00
mariosquiquell|rover: tell us why is a bad idea09:00
mariosyprokule: /me shrugs again https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#dict-filter :D09:00
quiquell|rovermarios: Le me find stuff09:00
mariosyprokule: oh hold on the thing i point  to is about dict09:00
mariosyprokule: not dict2items09:01
yprokulemarios: yah, 'docs up2date' :)09:01
mariosyprokule: no you're right09:01
quiquell|rovermarios: but the filter that is missing is dict2items09:01
quiquell|roveryou were good there09:01
mariosyprokule: yeah i still don't get why its not there if we have 2.5.709:01
quiquell|rovermarios: I am going to reproduce this09:01
mariosyprokule: quiquell|rover so sounds like we need a bug, and a proposal to bump the requirement. at least we can use that to check if it is the version yprokule09:04
marioseven if we don't merge it yet. and at least we can get feedback on why/if it is a bad idea to bump it. if it doesn't fix the issue something else is the problem09:04
quiquell|rovermarios: ok let me put a review on it, but I think panda|off and zbr|ssbarnea did something related09:04
mariosquiquell|rover: ack. yprokule do you want to do the honours? or do you prefer for quiquell|rover to take care of it?09:05
yprokulemarios: I'd like quiquell|rover take care of it09:05
quiquell|rovermarios: well better if we take care of it, it could happend that a lot of stuff is needed to change at tq09:05
mariosquiquell|rover: ok thanks09:05
yprokulemarios: quiquell|rover: thanks09:06
quiquell|roverGoing to bump to 2.6.1109:06
quiquell|roverThe one at RPMs09:06
mariosquiquell|rover: (agreed about the lot of stuff/but i wan't talking about merging it, just proposing the patch. might be better coming from the person who needs it for $reason)09:06
mariosquiquell|rover:and figure youre already busy09:06
mariosbut /me shrug09:06
mariosyprokule: :D09:07
quiquell|rovermarios: ahh ok then let's yprokule do it and se if it works09:07
quiquell|roverThen we decide09:07
mariosquiquell|rover: exactly09:07
yprokulequiquell|rover: marios I have no idea about upstream things - so far be it from me to advise what U need to use09:08
yprokulequiquell|rover: marios with ir we use 2.7.509:08
*** skramaja_ is now known as skramaja09:09
mariosyprokule: are you ok to post a review to change the line in https://github.com/openstack/tripleo-quickstart/blob/master/requirements.txt#L309:09
yprokulemarios: can we bump version just for upgrade jobs ?09:09
mariosyprokule: then we can use that review as depends-on in your/sergii patch09:09
mariosyprokule: and see if it fixes the issuee. this if first step09:10
*** ykarel|lunch is now known as ykarel09:10
mariosif it does then we can start a campaign to see what people think about bumping ansible09:10
mariosyprokule:  ^09:10
mariosyprokule: well its something we might explore if we need to (i.e. specific jobs) but i woulnd't support it unless there is some serious reason09:11
mariosyprokule: why have different versions of ansible across jobs09:11
*** zbr has joined #oooq09:11
*** zbr|ssbarnea has quit IRC09:12
*** saneax has quit IRC09:15
arxcruzquiquell|rover: ping09:16
arxcruzquiquell|rover: nevermind09:17
*** panda|off is now known as panda09:17
quiquell|roverarxcruz: ok :-(09:18
mariosskramaja:09:18
yprokulemarios: quiquell|rover https://review.openstack.org/#/c/635459/09:18
quiquell|roverpanda: o/09:18
mariosskramaja: sorry wrong chan ;)09:18
pandaquiquell|rover: what's down ?09:18
quiquell|roverpanda: Do you remember issue with bumping ansible at tq to 2.6.11 ?09:18
quiquell|roverpanda: 2.6.1109:18
quiquell|roverpanda: https://review.openstack.org/#/c/635459/09:18
pandaquiquell|rover: never easy to tell, we have a lot of places where we can break stuff09:21
pandayprokule: is there a workaround ?09:21
*** saneax has joined #oooq09:22
pandayprokule: quiquell|rover it's also a major version update, many things can go wrong09:22
pandawe never do this in haste I'm afraid09:23
*** ccamacho has quit IRC09:23
zbrquiquell|rover: regarding bumping ansible, i am not aware of any issue, nobody reported one but sshnaidm|afk blocked my attempt asking for extensive testing.09:23
yprokulepanda: there is review from ssbarnea so I can abandon mine09:23
quiquell|rover zbr also panda have concerns09:23
zbrjust update the minor version on mine, i am *for* doing the bump, is already far too old. i think even zuul is now using 2.609:25
pandazbr: how many jobs did you test on the bump ?09:25
zbrpanda: i do remember adding somethign with depends-on but i am unable to find it now. sorry but I cannot help more with this today as i am trying to catchup with sprint tasks.09:28
*** saneax has quit IRC09:28
zbrbut anyone interested should update https://review.openstack.org/#/c/596282/ and adde xtra tests to it.09:28
*** saneax has joined #oooq09:29
quiquell|roverpanda: do you know why scenario007 is nonvoting but part of the promotion ?09:30
zbrthe funny story is that we have a an undocumented upgrade policy: not doing it until we are hit by something so big that we are pressured to do it without the extensive testing. /end-joke09:30
quiquell|roverpanda: Is not a problem to use non voting jobs at promotion ?09:30
zbrin less than a month it will 2.5 will be EOL anyway.09:31
*** derekh has joined #oooq09:32
*** chandankumar has joined #oooq09:34
*** chandankumar is now known as chkumar|ruck09:35
*** jtomasek has quit IRC09:36
*** jtomasek has joined #oooq09:37
*** zbr has quit IRC09:40
*** zbr|ssbarnea has joined #oooq09:40
marioszbr|ssbarnea: read this https://tree.taiga.io/project/tripleo-ci-board/us/652 design notes has pointers /info lets catch up in a bit ? panda me and zbr|ssbarnea having a call in 15 if you fancy radio ci09:46
*** jbadiapa has quit IRC09:47
*** jbadiapa has joined #oooq09:48
*** chandankumar has joined #oooq09:50
*** chkumar|ruck has quit IRC09:51
*** chandankumar is now known as chkumar|ruck09:51
*** holser_ has joined #oooq09:54
chkumar|ruckquiquell|rover: ykarel the container build job passed http://logs.rdoproject.org/49/18749/1/check/periodic-tripleo-centos-7-master-containers-build/8d099aa/job-output.txt.gz09:58
quiquell|roverchkumar|ruck: let's look at centos-binary-nova-compute-ironic container09:58
quiquell|roverchkumar|ruck: And see if rabbitmq is there09:59
pandaquiquell|rover: scenario007 is blocked at the moment, it's changing its purpose09:59
pandamarios: bj ?10:00
marioshttps://redhat.bluejeans.com/7661925373/10:00
mariosjoining zbr|ssbarnea ^10:00
quiquell|roverpanda: then we have to remove it from criteria at promotion ?10:00
quiquell|roverpanda: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini#L2710:01
quiquell|rovermarios: looks like noop review is not exercising standalone scenarios will look at it10:01
zbr|ssbarneasorry i think i need to reboot, headphones not working10:03
*** zbr|ssbarnea has quit IRC10:04
chkumar|ruckquiquell|rover: there is only puppet-rabbitmq within container10:07
chkumar|ruckquiquell|rover: http://paste.openstack.org/show/744660/10:07
*** zbr|ssbarnea has joined #oooq10:07
chkumar|ruckquiquell|rover: if we check here http://logs.rdoproject.org/49/18749/1/check/periodic-tripleo-centos-7-master-containers-build/8d099aa/logs/undercloud/var/log/extra/podman/containers/rabbitmq/stdout.log.txt.gz10:09
chkumar|ruckrabbitmq-server is there10:09
quiquell|roverso serer is serving with the right version10:09
chkumar|ruckquiquell|rover: or I am missing something there?10:09
quiquell|roverchkumar|ruck: I think we have two different bug let me check something10:10
quiquell|roverchkumar|ruck: nah we are all good sorry10:11
quiquell|roverundercloud install has go well10:12
chkumar|ruckquiquell|rover: ok10:12
chkumar|ruckquiquell|rover: so what next there?10:12
chkumar|ruckquiquell|rover: do we want to kick promotion right now ?10:12
quiquell|roveror we wait10:13
quiquell|roverIt's going to finish now with a good rocky10:14
chkumar|ruckyup10:17
quiquell|roverok promoted queens and pike10:21
quiquell|roveronly master is missing10:21
chkumar|ruck:-)10:21
mariosquiquell|rover: SORRY WAS IN CALL10:26
mariossorry for shouting too10:26
quiquell|rovermarios: well you are with headphones still :-)10:27
mariosright10:27
quiquell|rovermarios: that's why you shout10:27
arxcruzsshnaidm|afk: on ovb job, port 5000 is for what ?10:28
mariosquiquell|rover: so which noop change?10:29
mariosquiquell|rover: something missing in zuul layout? don't follow10:29
quiquell|roverlooks like we have two reviews to check jobs at master10:29
quiquell|roverone at tqe and one at tht10:30
quiquell|rovertqe one is not running standalone sceanrios but tht is alright10:30
quiquell|roverI can change that10:30
quiquell|roverBut maybe we have to just use tht10:30
quiquell|roverDon't know why we have two there10:30
pandanever seen someone as allergit to meetings as marios :). It's like "30 minutes in, I'm getting ithes" "1h, omg, I'm having a rush" "2h, I'm going to die!"10:31
mariospanda: :D well was there something more? missing?10:32
quiquell|roverI have to pee10:32
quiquell|roverI have to poo10:32
zbr|ssbarnearfolco: upstream remarked that this change created a job that is waiting for ~70 hours in the queue https://review.openstack.org/#/c/634410/4 -- not clear why.10:37
zbr|ssbarneathey are asking if is ok to abort that change.10:37
zbr|ssbarneainitially they suspected that is waiting for f28 nodes but this is not the case, i got one allocated right away today, is not that.10:37
rfolcozbr|ssbarnea, yep, its ok to abort10:38
zbr|ssbarnearfolco: please abort it, i cannot do it.10:38
ykarelchkumar|ruck, quiquell|rover those were not correct logs to check10:38
ykarelthat was current-tripleo container, should check https://logs.rdoproject.org/49/18749/1/check/periodic-tripleo-centos-7-master-containers-build/8d099aa/logs/kolla/logs/rabbitmq.log10:38
ykareland here rabbitmq-server is installed and pushed successfully10:38
rfolcozbr|ssbarnea, new patchset ?10:39
chkumar|ruckykarel: what about this warning10:39
chkumar|ruckInstalling : rabbitmq-server-3.6.5-1.el7.noarch                       114/11410:39
chkumar|ruckINFO:kolla.common.utils.rabbitmq:10:39
chkumar|ruckINFO:kolla.common.utils.rabbitmq:Failed to get D-Bus connection: Operation not permitted10:39
chkumar|ruckINFO:kolla.common.utils.rabbitmq:warning: %post(rabbitmq-server-3.6.5-1.el7.noarch) scriptlet failed, exit status 110:39
chkumar|ruckINFO:kolla.common.utils.rabbitmq:[91mNon-fatal POSTIN scriptlet failure in rpm package rabbitmq-server-3.6.5-1.el7.noarch10:39
chkumar|ruckykarel: do we need to worry about that?10:39
ykarelchkumar|ruck, don't think so, i remember i have seen it earlier, but you can check previous logs10:40
chkumar|ruckykarel: ok10:40
mariospanda: rfolco fyi added https://tree.taiga.io/project/tripleo-ci-board/task/726 & https://tree.taiga.io/project/tripleo-ci-board/task/72710:44
marios(storyless)10:44
rfolcomarios, ackless10:45
quiquell|roverykarel: damn, ok yep -> RUN yum -y install erlang-hipe hostname rabbitmq-server pacemaker pacemaker-remote pcs libqb resource-agents && yum clean all && rm -rf /var/cache/yum10:45
quiquell|roverykarel: isall good now10:45
pandamarios: still haven't been able to get the reproducer running.10:50
pandamarios: why is it storyless ?10:51
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004 (2 more messages)10:52
mariospanda: doesn't have to be i just took an AI last time we spoke as a team to create those two as storyless10:52
mariospanda: we can delete and add to the repro story too10:52
mariospanda: the main goal was not to forget it10:52
jfrancoaquiquell|rover: marios have you seen this https://review.openstack.org/#/c/634243/ ???  => tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades SUCCESS10:56
jfrancoajfrancoa: and it's not a trick :-D, we just need to merge that patch10:57
mariosjfrancoa: nice looking10:57
*** dtrainor has quit IRC10:58
quiquell|roverjfrancoa: +2 cannot workflow that10:58
jfrancoaquiquell|rover: gracias!10:59
quiquell|roverjfrancoa: we are still having the oc-release-uc-release.yaml files ?10:59
jfrancoaquiquell|rover: not in this job, it uses single release files10:59
zbr|ssbarneamarios: panda : see my last comment on https://tree.taiga.io/project/tripleo-ci-board/us/651 -- any idea what could cause "No such file or directory" while trying to run yum?11:00
jfrancoaquiquell|rover: but having this job working, it could be a nice first step to enable some real upgrade job (rocky to master instead of master to master )  in RDO Cloud11:00
quiquell|roverI see11:01
quiquell|rovercool cool11:01
quiquell|rover:-)11:01
marioszbr|ssbarnea: nope didn't come across that11:02
zbr|ssbarneaweird because this happens when command trying to be called by subprocess does not exist, in this case being yum.11:02
zbr|ssbarneacould be messed PATH, I could try to use a full path to it.11:03
pandazbr|ssbarnea: yum in fedora28 ?11:10
zbr|ssbarneapanda: yeah, yum is available in fedora28.11:10
zbr|ssbarneapanda: that is the line where is used https://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py#L18311:12
zbr|ssbarneait should also work with dnf, too.11:12
pandazbr|ssbarnea: if it's a path problem why is it working on centos ?11:13
zbr|ssbarneapanda: i do not know, that is why I asked about it.11:13
pandazbr|ssbarnea: anyway, you're not doing something for the user story11:16
pandazbr|ssbarnea: if you're talking about upstream job that build containers in fedora28 host, this is covered in US #65011:16
*** sshnaidm|afk is now known as sshnaidm11:17
sshnaidmquiquell|rover, hi11:18
zbr|ssbarneapanda: damn, this set of dependencies makes taiga so confusing. Mainly if a story has a dependency on another we should better block it.11:19
pandazbr|ssbarnea: it's not blocked11:20
pandazbr|ssbarnea: it's a soft dependency11:20
zbr|ssbarneaso soft that when I added the job i ended up in what is desribed at https://tree.taiga.io/project/tripleo-ci-board/task/72511:20
zbr|ssbarneapanda: so the bug about tripleo-repos not supporting f28, where it fits? to which task/story?11:22
pandazbr|ssbarnea: there's a lot more taht can be done in the promotion pipeline to prepare the configuration to receive a working job11:22
pandazbr|ssbarnea: #650, but you should check with rfolco if he's hitting it11:22
pandazbr|ssbarnea: rfolco I think at this point we need to modify the owner11:23
rfolcopanda, one single owner you mean?11:24
rfolcozbr|ssbarnea, what exactly tripleo-repos does not do on f28?11:25
rfolcomy repos look ok11:25
pandarfolco: zbr|ssbarnea the owner is just the person that ensures the design is done, and the tasks are updated regularly, I think it cab ne one11:25
rfolcodone, removed myself11:25
pandarfolco: zbr|ssbarnea I think we should actually swap you two11:26
zbr|ssbarnearfolco: when run manually by me it "worked" (setting up centos7 repos, obviously). but on this job it choked: http://zuul.openstack.org/stream/39a006ef9f254716849f008781ea386d?logfile=console.log11:26
rfolcopanda, ok so sorry sorin, so I owned 65011:27
*** holser_ has quit IRC11:28
rfolcoSSS(TM) - So Sorry Sorin11:28
zbr|ssbarneai looked at the code inside tripleo-repos and it seems that it needs several changes for accomodating the new distro as now is hardcoded to assume centos7.  and i do not think that it makes sense to specify the distro on command line, as we are already running on that the platform, so we can read it. right?11:28
pandaI'm sorin to hear that11:28
rfolcolol11:28
*** chkumar|ruck has quit IRC11:29
zbr|ssbarnea believe me that in UK in need to mispell my name and say it very slowly every time someone is asking my name. like "Soreeeen" because if I spell it corectly they only hear "Sorry" and ask the quetion again, imagining that I didnt hear them.11:30
*** chandankumar has joined #oooq11:30
rfolcohaha11:30
zbr|ssbarneai do also have a 2nd name that I could use... but this may prove even more problematic "Ionutz" which sounds like 'You' nutz!'11:30
*** chandankumar is now known as chkumar|ruck11:30
rfolcozbr|ssbarnea, this apparently works for me11:31
rfolcosudo yum install -y trunk.rdoproject.org/fedora/current/python3-tripleo-repos-0.0.1-0.20190114223631.6470f0c.fc28.noarch.rpm11:31
rfolcosudo -E tripleo-repos -d fedora -b master current11:31
pandarfolco: you tested this locally or on a WIP job in zuul ?11:31
rfolcopanda, manual test on rdo instance11:32
*** udesale has quit IRC11:33
pandarfolco: ok are you using official images ?11:33
*** udesale has joined #oooq11:34
pandarfolco: zbr|ssbarnea's issue may still be valid, if the worflow doesn't work with what we have upstream11:34
zbr|ssbarnearfolco: what I want to do now is to modify it not to assume default centos7 distro and to use platform.linux_distribution()11:34
zbr|ssbarneaand I am saying this because adding --distro to the jobs would require extra work and add extra complexity. making it smart should be better. not removing the option, just making the default smart.11:35
*** holser_ has joined #oooq11:35
zbr|ssbarneapanda: do you agree this is desirable?11:36
rfolcopanda, using rdo trunk packages that are setup by tripleo-repos, and then alex containers11:36
*** holser_ has quit IRC11:37
rfolcopanda, from what I saw, repos are equivalent to what a f28 standalone job configures11:37
rfolcopanda, I have a doctor appt now, should be back before scrum...11:37
pandarfolco: ok11:37
*** rfolco is now known as rfolco_doctor11:38
pandazbr|ssbarnea: what are you looking at ?11:39
pandazbr|ssbarnea: what needs --distro ?11:39
zbr|ssbarneahttps://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py#L8011:40
rfolco_doctorsudo -E tripleo-repos -d fedora -b master current11:40
rfolco_doctor-d11:40
pandazbr|ssbarnea: the thing is, you're a bit late in the design week to make this kind of remarks, we need to start implementation now, and we shouldn't move too much or implementation will be a pain11:40
zbr|ssbarneayeah, i know but i think this change would be easier than the other approach because I found at least once case where we do not call it with --distro. i think that is the fast way to fix.11:42
sshnaidmquiquell|rover, chkumar|ruck I think I solve problems with ports leftovers here: https://review.rdoproject.org/r/#/c/18740/11:47
sshnaidmquiquell|rover, chkumar|ruck I didn't remove all ports in ovb, so some of them remained and took IPs11:47
*** ccamacho has joined #oooq11:49
chkumar|rucksshnaidm: ok, let me see the review11:50
chkumar|ruckarxcruz: hello11:51
chkumar|ruckarxcruz: you are looking at this bug na https://bugs.launchpad.net/tripleo/+bug/1802971 ?11:51
openstackLaunchpad bug 1802971 in neutron "tempest volume_boot_pattern and basic_ops running concurrently causing timeouts" [Undecided,In progress] - Assigned to Slawek Kaplonski (slaweq)11:51
arxcruzchkumar|ruck: yes11:51
arxcruzand I found a very critical problem11:51
arxcruza VERY CRITICAL11:51
chkumar|ruckarxcruz: thanks!11:51
arxcruzA11:51
arxcruzVERY11:51
arxcruzCRITICAL11:51
chkumar|ruckarxcruz: what is that?11:51
arxcruzchkumar|ruck: i don't know why/how trigger the problem11:51
*** amoralej is now known as amoralej|lunch11:51
arxcruzbut after a few tests running on tempest11:52
arxcruzthe controller just stop respond11:52
arxcruzso tests start to fail11:52
arxcruzi can't access the controller through ssh11:52
arxcruznothing work, like it's in freeze state11:52
sshnaidmchkumar|ruck, the particular part is here: lines 33-46 https://review.rdoproject.org/r/#/c/18740/3/roles/ovb-manage/tasks/ovb-delete-stack.yml11:52
arxcruzI had to reboot it11:52
quiquell|roverO damnint the checkout-override stuff !!!11:52
pandaquiquell|rover:  ?11:52
quiquell|roverpanda: I am preparing a patch for config/rdo-jobs to have the checkout-override at branch_override jobs11:53
quiquell|roverpanda: I have forget about it11:53
*** dtrainor has joined #oooq11:53
chkumar|rucksshnaidm: yes, looks nice!11:55
sshnaidmquiquell|rover, chkumar|ruck I merge https://review.rdoproject.org/r/#/c/18740/ which may impact OVB jobs, please tell me if you see something wrong12:00
chkumar|ruckquiquell|rover: sure12:00
*** dtrainor_ has joined #oooq12:00
chkumar|rucksshnaidm: after this do we still need ovb tenant cleanup script?12:01
quiquell|roversshnaidm, chkumar|ruck: BOOOM!!!12:01
sshnaidmchkumar|ruck, yes, but there won't be so many ports, just failed stacks12:01
chkumar|ruckok12:01
*** dtrainor has quit IRC12:02
zbr|ssbarneapanda: one thing that was not clear to me is why "fedora" without number was considered acceptable distro when default was "centos7"..12:08
pandazbr|ssbarnea: to make progress. Not even the help text on the argument was changed12:09
pandazbr|ssbarnea: still says "we only support centos"12:09
zbr|ssbarneapanda: depends where you look: https://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py#L15612:10
zbr|ssbarneamaybe i should do a  s/supported/tolerated/12:15
*** dtrainor_ has quit IRC12:17
*** dtrainor has joined #oooq12:17
*** ratailor has quit IRC12:21
*** dtrainor has quit IRC12:23
*** dtrainor has joined #oooq12:23
weshaysshnaidm, re: https://review.openstack.org/#/c/633843/ you still -1 on https://review.openstack.org/#/c/633843/ ?12:42
sshnaidmweshay, mm.. not sure we need to dump it in multiple places, seems like dup effort12:43
sshnaidmweshay, but won't object to it too12:44
*** jpena is now known as jpena|lunch12:44
weshaysshnaidm, ya.. I still think it may be helpful12:44
sshnaidmweshay, but make sure it passes reproducer CI12:45
weshaysshnaidm, quiquell|rover marios this should be ready to merge https://review.openstack.org/#/c/634329/12:45
weshayor final reviews12:45
*** dtrainor_ has joined #oooq12:46
quiquell|roverweshay: ack12:48
*** dtrainor has quit IRC12:48
quiquell|roverweshay: we don't want to use profiles to have only one bindep.txt ?12:48
weshayquiquell|rover, we might be able to have a fedora-28 vs.. fedora-27 profile12:49
weshayquiquell|rover, think mvp12:50
*** skramaja_ has joined #oooq12:50
*** skramaja has quit IRC12:50
weshayquiquell|rover, also think mvp w/ https://review.rdoproject.org/r/#/c/18664/40/playbooks/tripleo-ci-reproducer/pre.yaml12:50
weshayquiquell|rover, I did not see any great examples on delineating the version of a specific distro12:50
quiquell|roverweshay: ack, has put a comment there so we don't have zuul, maybe this is not mvp12:50
weshaybut I suspect we can12:50
weshayquiquell|rover, ya.. the code defaults back to non-zuul vars12:51
quiquell|roverweshay: you mean fedora28 instead of fedora ?12:51
weshayyes12:51
quiquell|roverweshay: I mean python2 python3 profiles12:51
quiquell|roverinstead of having _python2 _python3 suffixes12:51
weshayyes.. I know12:51
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004-multinode-oooq- (2 more messages)12:52
quiquell|roverweshay: ok ok projects are opnstack have only one bindep.txt I think12:52
sshnaidmweshay, commented12:52
*** sshnaidm is now known as sshnaidm|afk12:52
weshayquiquell|rover, anyone have an example on defining the distro version in bindep?12:52
weshayplatform:ubuntu-precise]12:53
weshayis one12:53
weshaypython3-all-dev [platform:dpkg !platform:ubuntu-precise]12:53
weshayquiquell|rover, it doesn't really matter RIGHT now.. but i'll get those squashed into one12:53
chkumar|ruckquiquell|rover: weshay on osa side they use like this https://github.com/openstack/openstack-ansible-tests/blob/master/run_tests_common.sh#L9112:54
chkumar|rucki mean bindeps12:54
quiquell|roverweshay: I see stuff like this [platform:suse !platform:opensuseproject-42]12:54
*** jtomasek has quit IRC12:55
quiquell|roverbut not for centos or fedora12:55
*** jtomasek has joined #oooq12:55
* weshay tries12:57
zbr|ssbarneaweshay: if I rememeber well version is not yes supported by bindep!12:57
zbr|ssbarneai wanted to addit myself few months back, but didn't had time. but i think there is a hack that can be used.12:58
quiquell|roverzbr|ssbarnea: they have version for ubuntu http://git.openstack.org/cgit/openstack/openstack-ansible-ops/tree/multi-node-aio/bindep.txt#n5212:58
mariosweshay: ack was getting some lunch will check in a sec12:58
weshayk12:58
mariosweshay: ack i reviewed it earlier. would like another pass but if you want to merge i'll revote?13:00
marioslets see if others can come up with better review13:00
mariosweshay: i can revote bfeore i leave if needed13:00
weshaymarios, aye.. just explaining why that if statement is there multiple times13:01
weshaythe repro and quickstart have different install paths13:01
mariosweshay: yeah thanks i just saw your reply. it was a minor comment really i coulndn't find something to -1 for13:01
marioshence my wish for better reviews13:01
marios:D13:01
weshayaye..13:01
weshayI'd like to get the reproducer into the hands of the team this sprint to play with...13:02
*** gkadam has quit IRC13:02
weshaysshnaidm|afk, quiquell|rover marios panda .. so we will keep working on improvements.. but too many folks have been stuck or blocked on it13:02
weshayhttps://review.rdoproject.org/r/#/c/18664/13:02
quiquell|roverweshay: we don't want to pin bindep ?13:03
*** rlandy has joined #oooq13:04
zbr|ssbarneaquiquell|rover: yep, now I remember, bindep has tags like foo-123 but you cannot add conditions like ver>=123, which is an issue if you have something that makes new releases often, like fedora.13:04
quiquell|roverzbr|ssbarnea: so we have fedora-28 ?13:05
zbr|ssbarneaquiquell|rover: almost sure that yes. let me use codesearch...13:05
quiquell|roverzbr|ssbarnea: ack I did codeserch of bindep.txt files13:05
quiquell|roverdind't found any13:05
quiquell|roverwith fedora-.. tag13:05
zbr|ssbarneaquiquell|rover: try it, the tags are generated at runtime13:06
rlandypanda: hello13:06
rlandypanda: going through cards on scenario00913:06
pandarlandy: anything not clear ?13:07
rlandypanda: https://tree.taiga.io/project/tripleo-ci-board/task/720?kanban-status=1447274 links a review for 1213:07
pandammhh, keep mixing them13:07
rlandypanda: I ma ready to start testing13:07
pandarlandy: updated13:08
rlandypanda: also - there are a bunch of reviews in there - no explanation - and the cards are still in parked13:08
pandarlandy: isn't the description on the tasks itself enough, should I expand a bit more ?13:09
weshayquiquell|rover, I'm goinmg to test it.. give me a few13:09
quiquell|roverweshay: works13:10
quiquell|roverweshay: I am testing at a fedora28 and fedora-28 platform works there13:10
rlandypanda: should the cards be in parked if there are ready reviews?13:10
quiquell|roverAt least we can skip fedora-27 with !fedora-2713:10
weshayk.. quiquell|rover anyone mind if I do that on top of the current reviews?13:10
pandarlandy: probably not, I'll put them in progress, assign them to you13:10
weshaytrying to get to something our folks can use13:11
rlandypanda: you can leave them assigned to you13:11
rlandysince you did the work13:11
quiquell|roverweshay: no problem let's try to flatten the bindeps too13:11
quiquell|roverweshay: btw why do we do a forloop instead of $(bindep ...) at install ?13:11
*** sshnaidm|afk is now known as sshnaidm13:11
rlandypanda: thank you for picking up all this work13:13
weshayquiquell|rover, which file?13:13
weshaythe ansible?13:13
rlandypanda: one other question - why did you decide to go with a new fs?13:13
pandazbr|ssbarnea: rfolco_doctor marios I'd like to QE one of the user stories in the f28 containers13:13
rlandypanda: one last request ....13:13
quiquell|roverweshay: the review is iteraing at all bindeps package not installed and doing  package install13:13
weshayquiquell|rover, oh in the shell?13:14
quiquell|roverweshay: why don' twe past the list to dnf/yum ?13:14
rlandypanda: I need someone with diff gerrit users to test the reproducer13:14
quiquell|roverweshay: maybe is because one fails it does not continue ?13:14
pandarlandy: because in this particular case, most of the configuration is in the featureset, we are not deploying much in the scenario13:14
weshayquiquell|rover, want to chat for a sec?13:14
quiquell|roverweshay: I remember something like this before13:14
rlandypanda: would you mind giving it a try and telling me if the user/keys works13:14
pandarlandy: I'll fire up the reproducer after the meeting13:14
pandarlandy: yep, sure13:14
quiquell|roverweshay: ack13:14
weshayyou mean.. dnf install `bindep -b -f adf.txt`13:14
weshay?13:14
quiquell|roverweshay: yep13:15
rlandypanda: thank you - I only have one user name so no way to tell13:15
weshayhrm.. k k13:15
weshaygood suggestion13:15
weshayin my channel if there is anything else13:15
quiquell|roverif we put everyitng at bindep.txt we can do dnf install `bindep -b` bindep.txt is the default13:15
weshayk13:15
*** amoralej|lunch is now known as amoralej13:15
pandarlandy: so fs052 wasn't enough for scenario 009, and the alternative was adding irrelevant stuff on the featureset052 itself13:16
rlandypanda: understood - just trying to put the pieces together - thanks13:16
quiquell|roverweshay: bindep -b $(pythom_cmd)13:16
zbr|ssbarneapanda: sure, I wanted to ask you to be QE.13:16
quiquell|roverweshay: and profiles at bindep.txt but we can do this later on13:16
mariospanda: k, are you asking which one? does the centos congtainer build periodic one count?13:17
mariospanda: i was thinking of asking you anyway13:17
quiquell|roverzbr|ssbarnea: Do you remember an issue we have with differences between yum/dnf about installing a list of packages ?13:17
weshayquiquell|rover, oh..13:17
quiquell|roverzbr|ssbarnea: I remember one of them continues if one fails and the other not13:17
weshaylolz..13:17
weshayhrm..13:17
quiquell|roverweshay: yep...13:17
quiquell|roverweshay: I remembe one weird option so you can continue in both13:18
weshayhow far back in fedora are we going to support?13:18
weshayI guess.. 28 is EOL in may..13:18
weshayso 27?13:18
zbr|ssbarneaquiquell|rover: yes, is not a bug is a feature, and is related to use use of *. even worse behaviour changed between yum/dnf.13:18
quiquell|roverzbr|ssbarnea: but if package has full namewe are good ?13:19
zbr|ssbarneaquiquell|rover: yes, full name makes it mandatory for install. "<pkg> install foo bar" works. but "<pkg> install foo bar*" would succeed even if there is no package matching bar* pattern.13:20
quiquell|roverzbr|ssbarnea: ack, then we are good13:20
zbr|ssbarneato be correct: result may depend on pkg type/version.13:20
zbr|ssbarneashortly, avoid wildcard if possible or when using them do not mention more than one package.13:21
quiquell|roverweshay: +2 with a little comment13:22
weshayk13:22
weshaythanks for reviewing13:22
weshayzbr|ssbarnea, platform:centos-7?13:22
quiquell|roverrlandy: your pre is looking good now13:22
zbr|ssbarneaweshay: yes, should work.13:22
rlandyquiquell|rover: hey ... thanks for your comments on https://review.rdoproject.org/r/#/c/18664/... will fix zuul vars ... wrt post failure on tripleo-ci-reproducer-centos-7-libvirt - do you see that in other jobs>13:22
rlandyit passed yesterday13:22
zbr|ssbarneaweshay: in fact very easy to test which tags are availanel, run bindep on that platform and it can report them.13:23
quiquell|roverrlandy: yep I am playing with retries at libvirt role right now, could be that when the hypervisor is bad and force_tcg is activated could take more time13:23
quiquell|roverrlandy: but not sure at least we dont' have the virt-foobar command issue13:23
rlandyk, I see13:23
quiquell|roverrlandy: have to test13:24
rlandyquiquell|rover: k, need a new patch anyways for the zuul var13:25
quiquell|roverrlandy: you are going to use "git" command or just pass zuul from the job definition ?13:25
quiquell|roverrlandy: I would prefer second one so we don't depend on this to be cloned13:26
rlandyquiquell|rover: passing zuul from job seems cleaner13:26
quiquell|roverrlandy: yep13:26
quiquell|rovercool cool13:26
rlandyquiquell|rover: one day, we will merge this :)13:26
quiquell|roverrlandy: with a big red button13:27
weshayquiquell|rover, heh13:28
weshayhttps://github.com/openstack/tripleo-quickstart/blob/master/other-requirements.txt13:28
weshaybindep -b                                                                         î‚  review/wes_hayutin/generate-zuul-based-reproducer âś— thinkdoe ⌚ 6:25:1613:28
weshayBoth bindep.txt and other-requirements.txt files exist, choose one.13:28
weshaybindep is three years old?13:28
pandait's already 25:16 ? Damn I'm late.13:29
zbr|ssbarneapanda: marios: it seems that my f28 fix to tripleo-repos from https://review.openstack.org/#/c/635475/3 works -> http://zuul.openstack.org/stream/9ae5295ca4264d2ba67f80f15ca77f54?logfile=console.log -- please take a look.13:29
sshnaidmchkumar|ruck, quiquell|rover ovb jobs run collect logs twice now, can you please submit a bug? I'll look at it today. For example: http://logs.rdoproject.org/80/634380/10/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/705a814/job-output.txt.gz13:29
chkumar|rucksshnaidm: ok submitting13:30
chkumar|ruckweshay: 1:1 time13:31
weshaychkumar|ruck, I'm in13:31
chkumar|ruckweshay: I am there also, am I audible?13:32
marioszbr|ssbarnea:  ack13:33
zbr|ssbarneamarios: http://zuul.openstack.org/status?635458 -- almost too good to be true: tripleo-build-containers-fedora-28 [success].13:38
quiquell|roverchkumar|ruck: I am with the mirroring of centos7-rt do you have time for the logs at OVB ?13:39
chkumar|ruckquiquell|rover: currently in 1:1 with wes13:41
quiquell|roverchkumar|ruck: ack13:41
chkumar|ruckweshay: https://bugs.launchpad.net/tripleo/+bug/181391313:46
openstackLaunchpad bug 1813913 in tripleo "periodic-tripleo-fedora-28-master-containers-build failing with unauthorized access to rdo registry" [Wishlist,Triaged] - Assigned to Gabriele Cerami (gcerami)13:46
*** ykarel is now known as ykarel|away13:47
chkumar|ruckweshay: https://bugs.launchpad.net/tripleo/+bug/181485113:48
openstackLaunchpad bug 1814851 in tripleo "[master][promotion] centos 7 container build job deploying undercloud with usr/lib/rabbitmq/bin/rabbitmq-server: No such file or directory" [High,Fix released] - Assigned to Quique Llorente (quiquell)13:48
*** jpena|lunch is now known as jpena13:48
chkumar|ruckweshay: https://review.rdoproject.org/r/#/c/18733/13:54
*** rfolco_doctor is now known as rfolco13:58
*** ykarel|away has quit IRC14:00
rfolcoping scrum marios, quiquell, sshnaidm, weshay, panda, rlandy, arxcruz, mwhahaha, rfolco, chkumar, ssbarnea, kopecmartin14:01
sshnaidmweshay, panda need to include US https://tree.taiga.io/project/tripleo-ci-board/us/669 in the sprint14:01
sshnaidmrfolco, ^^14:02
*** skramaja has joined #oooq14:03
*** skramaja_ has quit IRC14:03
chkumar|rucksshnaidm: quiquell|rover I have created a bug https://bugs.launchpad.net/tripleo/+bug/1815048 (Will fill the info once I reach home)14:06
openstackLaunchpad bug 1815048 in tripleo "[ovb][collect_logs] ovb collect logs job runs twice " [Critical,Triaged]14:06
sshnaidmchkumar|ruck, ack14:06
quiquell|roverchkumar|ruck: ack thanks14:06
*** quiquell|rover is now known as quique|rover|eat14:06
*** vinaykns has joined #oooq14:07
quique|rover|eatsshnaidm: they are in a meeting14:11
quique|rover|eatsshnaidm: dib copy the repos the elements use them and then dib delete the repos14:11
sshnaidmquique|rover|eat, I understand, but we don't have this repo yet to copy it14:12
quique|rover|eatsshnaidm: just adding the repo to "repo:" at release file is not enough ?14:13
quique|rover|eatWell go for lunch14:13
quique|rover|eathttps://review.openstack.org/#/c/635157/5/config/release/tripleo-ci/CentOS-7/master.yml14:13
quique|rover|eatthis is the main review14:14
quique|rover|eatGo back in a few14:14
sshnaidmquique|rover|eat, ok, didn't see this review, should be fine14:14
*** jtomasek has quit IRC14:18
*** dtrainor_ has quit IRC14:22
*** holser_ has joined #oooq14:29
*** irclogbot_0 has joined #oooq14:29
rlandymarios: can I get the quick training on scenarios to standalone?14:34
*** ykarel|away has joined #oooq14:37
*** quique|rover|eat is now known as quiquell|rover14:38
mariosrlandy: of cource14:40
marioscourse14:40
mariosrlandy: call after scrum14:40
marioscc panda  ^14:40
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004-multinode-oooq- (2 more messages)14:52
mariossshnaidm: story 6 6 6 !14:52
*** d0ugal has quit IRC14:52
*** d0ugal has joined #oooq14:53
pandamarios: 666 is the epic14:54
mariosright14:55
*** jtomasek has joined #oooq14:56
pandamarios: I will change its subject to NECRONOMICON14:56
*** dtrainor_ has joined #oooq14:57
arxcruzweshay: need to talk with you about https://bugs.launchpad.net/tripleo/+bug/180297115:03
openstackLaunchpad bug 1802971 in neutron "tempest volume_boot_pattern and basic_ops running concurrently causing timeouts" [Undecided,In progress] - Assigned to Slawek Kaplonski (slaweq)15:03
mariosrlandy: in 25 mins ok?15:04
mariosrlandy: need go afk for few15:05
weshayarxcruz, hold for a sec.. have a 1-1 w/ apetrich15:05
rlandymarios: sure15:05
weshayif apetrich is ready15:05
weshayI need a quick bio15:05
arxcruzwho cares about apetrich ?15:05
arxcruz:*15:05
arxcruzhehehe15:05
apetricharxcruz, no one15:05
arxcruzapetrich: exactly :P15:05
mariosarxcruz: be careful what you say, he will squish you. literally15:05
apetrichweshay, take a bio and I'm here15:05
pandamarios: yeah, go put your ointment, it's been a long meeting, you're probably suffering a lot :D15:06
mariospanda: ;)15:06
mariospanda: will you join us?15:06
pandamarios: yes15:06
pandamarios:knowledge is idempotent at worst15:06
mariosso its officially a conspiracy15:06
weshayapetrich, I'm in15:07
sshnaidmrfolco, how do I move task from US to US? without a board15:07
rfolcosshnaidm, you have to use the board.... we may implement this in panda's taigacli so it makes easier15:09
rfolcosshnaidm, for now drag and drop is the only way15:10
sshnaidmrfolco, ok, so need to add US 731 to the board15:10
sshnaidmrfolco, https://tree.taiga.io/project/tripleo-ci-board/us/73115:10
rfolcosshnaidm, ok go to backlog, drag and drop the us you created to sprint, then move tasks between us'ies15:11
sshnaidmrfolco, where is backlog? how do I add it to sprint?15:13
rfolcosshnaidm, can you bj ? I can show you real quick15:13
sshnaidmrfolco, https://bluejeans.com/u/sshnaidm/15:13
*** ccamacho has quit IRC15:24
weshayapetrich, ugh.. sorry.. I don't see the receipt in oracle15:25
weshayI mean, it's attached15:25
weshaybut nothing comes up..15:25
weshayrenders15:25
weshayapetrich, can you please check that.. and reattach if needed?15:25
apetrichweshay, sure15:25
apetrichlet me check the local version15:25
weshayapetrich, it could just be my computer15:25
apetrichmy printing is behaving bad15:25
weshaytry viewing through oracle15:25
apetrichit is probably here15:26
weshayarxcruz, come on in15:26
apetrichso the pdf works locally15:26
*** jtomasek has quit IRC15:27
*** quiquell|rover is now known as quiquell|off15:27
mariosrlandy: panda https://redhat.bluejeans.com/7661925373/ joining15:29
apetrichweshay, the oracle one doesn't rend to me either. uploading again15:30
*** jfrancoa has quit IRC15:31
rlandyjoining15:32
apetrichweshay, works for me now. but it required that I enabled flash15:32
*** zbr has joined #oooq15:35
weshayarxcruz, https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/e779016/15:35
weshaybaseurl=http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8080/rdo/centos7/bf/97/bf97151ce15feccd98be3b43c06656939841eb9b_8fb6131315:36
*** zbr|ssbarnea has quit IRC15:37
weshayknown-good https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/561b325/logs/undercloud/var/log/extra/rpm-list.txt.gz15:38
mariosrlandy: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/standalone-jobs.yaml15:41
weshayquiquell|off, first pass at a single bindep file https://review.openstack.org/63555015:57
weshayrlandy, based on my updated review.. ^ we need to change the file name bindep_python2.txt to bindep.txt in https://review.rdoproject.org/r/#/c/18664/15:58
weshayrlandy, can you do that? or want me to?15:58
rlandyweshay: sec - in meeting15:59
rlandyfine if you want to update15:59
*** ratailor has joined #oooq15:59
rlandyweshay: will also need change in scripts15:59
weshayk.16:00
weshayrlandy, /me starts w/ https://review.rdoproject.org/r/#/c/18664/16:00
rlandyweshay: sorry - out of meeting - will update scripts16:03
rlandyweshay: tmate is still open if you want to use that for /18664/16:04
weshayrlandy, hrm.. I don't think I can connect to a ssh url that is old16:04
weshayrlandy, I alrady finished /18664/16:05
rlandyk - script update in two minutes16:05
*** apetrich has quit IRC16:06
weshayrlandy, so based on sagi's comment re: wget16:06
weshaywget https://github.com/rdo-infra/ansible-role-tripleo-ci-reproducer/bindep_python2.txt16:06
weshaycurl -o bindep.txt  https://github.com/rdo-infra/ansible-role-tripleo-ci-reproducer/bindep.txt16:07
weshayline 15816:07
rlandyweshay: sagi or sorin?16:08
weshaysagi mentioned that wget is not default on distro's .. however curl is16:08
zbrtrue, do not use wget16:08
zbrwget is nice, but is not a default tool. curl is more standard.16:09
zbrnow my question is why trying to grab files from git in code? or this is only for us to test now manually?16:09
weshayzbr chicken and egg16:10
weshayget use to me using that expression16:10
rlandyfile needs to be there before you git clone16:10
weshay:)16:10
rlandyso you can run ansible16:10
rlandyto git clone16:10
rlandyto get the file16:10
rlandygit run the job16:10
rlandyhad gadyah16:11
zbrweshay: bad practice (you already know why), but for the moment we could cut some corners to ship something.16:11
weshayzbr, I can chat about it for a sec16:12
weshaywe're enabling folks who want to not use the script to not use it16:12
weshayzbr, folks like you can just kick the playbooks16:12
weshayif you already have things installed16:13
weshayand configured16:13
rlandyweshay: https://review.openstack.org/#/c/631067/ updated16:15
zbrbtw, are we ok to make it bindep.txt ?16:16
weshayhttps://raw.githubusercontent.com/rdo-infra/ansible-role-tripleo-ci-reproducer/master/bindep.txt16:17
weshayrlandy, ^ sorry16:17
weshayzbr already did16:17
weshayzbr, https://review.rdoproject.org/r/#/c/18664/16:17
zbri don't want to start a flame.. but it fails on my f28 machine complaining about missing docker. but I do have docker installed and running, docker-ce16:18
rlandyk - fixing16:19
zbrand i am afraid that bindep is too limited to be able to deal with alternative package names and similar stuff. good old bash rules :D16:19
*** ccamacho has joined #oooq16:24
*** skramaja has quit IRC16:24
weshayzbr, well.. seems to be working fine atm16:24
weshayzbr, /me assumes it was written by folks in openstack16:25
weshaynot sure16:25
zbrweshay: i am trying to say that if we use bindep and run it always, we will prevent preople from using the tool if they have docker installed from upstream.16:26
zbreven if docker itself would be working16:26
zbrjust because it locks to the rpm name, which is different.16:26
zbrand there is no way to make bindep to do a "foo or bar" kind of dependency16:26
*** ratailor has quit IRC16:27
zbrthis is why I am saying that testing that docker works with bash "docker ps", or "docker info" is more reliable.16:27
weshaywhat?16:27
weshayblue16:27
zbrsure16:27
weshayI'm not understanding16:27
*** udesale has quit IRC16:33
*** agopi has quit IRC16:34
*** chkumar|ruck has quit IRC16:38
*** chandankumar has joined #oooq16:39
*** saneax has quit IRC16:40
*** apetrich has joined #oooq16:40
*** yprokule has quit IRC16:49
rlandypanda: https://review.openstack.org/#/c/635461/ - let's see what this does16:50
rlandyI left the multinode job commented16:50
rlandyin case we want to compare at some stage16:50
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004-multinode-oooq- (2 more messages)16:52
pandarlandy: \o/ I'm having fun going up and down the scenario resource registry, and learn more about standalone deployment16:54
pandahope it lasts16:54
rlandypanda: ugh - still kicks a ton of tests due to depends-on16:54
rlandypanda: you had another fix rather than editing the zuul layout?16:55
rlandychanged check in toci maybe?16:55
rlandypanda: otherwise I need to edit zuul layout in three repos16:56
pandarlandy: no, I just edited the zuul layout, but only in tripleo-ci, maybe I should do it also in tht, mmmh16:56
rlandypanda: I will do the edit in all three16:57
rlandyand tq16:57
rlandydue to fs addition16:57
rlandynobody merge this stuff - please!!!16:57
weshaypanda, point me at your taiga tool please in github17:02
weshayor gitlab.. what ever17:02
weshaygit17:02
pandaweshay: https://github.com/gabrielecerami/tripleo-ci-team-management17:05
weshaythanks17:06
rlandypanda: woohoo - one test only17:08
pandarlandy: let the fun begin17:09
weshaythis is cool https://github-history.netlify.com/openstack/tripleo-quickstart/commits/master/quickstart.sh17:12
rlandypanda; ok - step one - the job kicked17:15
pandarlandy: step 2 - the job fails17:15
rlandyso negative17:16
pandarlandy: step 3 you fix the failure17:16
rlandyyou never know17:16
pandarlandy: goto 117:16
rlandyrinse and repeat17:16
*** bogdando has quit IRC17:19
zbrsomeone mentioned a bug in paramiko few days ago, can I get a link to it, I am a bit curious about it as I got another issue related to paramiko and keys.17:19
rlandypanda: going to enable the multinode job for scenario009 so we have some comparison17:19
*** jtomasek has joined #oooq17:19
weshayzbr, ya.. so if you have an older rsa key.. like me..17:26
weshayzbr, https://storyboard.openstack.org/#!/story/200484217:26
*** dtrainor_ has quit IRC17:27
zbrweshay: thanks, read it. in my case is different: it fails due to encrypted id_rsa file even if the agent has it loaded. funny part depends on client implementation: ansible has no problem with that (another library i maintain also works well) but docker-py chockes. all are using paramiko. sadly i didn't had time to look more into it, maybe over the weekend.17:36
zbrthere is a funny named bug on paramiko which say a lot about it: https://github.com/paramiko/paramiko/issues/387 -- just reading the title is enough.17:37
*** trown is now known as trown|lunch17:38
sshnaidmrlandy, can you please review in your time? https://review.openstack.org/#/c/635582/17:44
*** ykarel|away has quit IRC17:52
rlandysshnaidm: sure17:58
*** jpena is now known as jpena|off17:59
*** kopecmartin is now known as kopecmartin|off18:00
rlandyweshay: https://review.rdoproject.org/r/#/c/18664/ - should we merge that?18:02
weshayrlandy, yes18:02
*** derekh has quit IRC18:03
weshayI +218:03
rlandysshnaidm: ^^ can you vote?18:04
sshnaidmrlandy, need to review it firstly18:07
weshayzbr, should I add https://github.com/paramiko/paramiko/issues/387 to the doc?18:13
*** jtomasek has quit IRC18:14
zbrweshay: not sure to which doc, also i tried to take something useful out of it yesterday but i failed.18:15
weshaysshnaidm, any idea why this would not work?18:16
weshay# unfortunately with the right path requirements.txt is still not found18:16
weshay    # - name: Install python dependencies18:16
weshay    #  pip:18:16
weshay    #    requirements: requirements.txt18:16
weshay    #    extra_args: --user --force18:16
weshay    #  chdir: "{{ top_working_dir }}"18:16
weshaycan't find the file18:17
sshnaidmweshay, I'm looking now at patch and not sure I understand what is going on there..18:17
weshayrlandy, may be busy but maybe I can provide some context if it would help18:17
weshaymost of the recent work there was to enable a use case w/o passwordless sudo and make the playbook idempotent18:18
weshayI can blue if you like18:18
rlandyI'm here18:19
rlandysshnaidm: zuul claimed it could not find the requirements file18:20
rlandyit worked with the cat'ed out file18:20
weshaywell..ansible18:20
weshayrlandy, and line 4418:20
weshayproves it's there and readable18:20
sshnaidmrlandy, it's wrong path18:20
weshayhow so?18:20
weshaysshnaidm, clearly line 49 is the same path as 6718:21
sshnaidmweshay, zuul.project.src is src/review.rdoproject.org/rdo-infra/ansible-role-tripleo-ci-reproducer, you need /home/zuul before18:21
weshayat least asfaict18:21
rlandysshnaidm: w tried full path as well18:21
rlandypanda: ok - step 2 - we have a failure ... http://logs.openstack.org/64/635464/3/check/tripleo-ci-centos-7-scenario009-standalone/0582435/job-output.txt.gz18:21
rlandyw/o the chdir18:22
sshnaidmrlandy, weshay well, I don't think it's ready to merge, let's find out what is problem with path at least18:22
weshayansible doc say use chdir, but meh.. the doc could be wrong.. but we tried both18:22
weshaysshnaidm, we tried man18:22
weshaynot saying we're going to leave it this way18:22
weshaysshnaidm, if you want to tmate I'm down18:23
weshaybut rlandy I spent.. hours18:23
sshnaidmweshay, no, please don't merge not working code18:23
weshayit works fine :)18:23
sshnaidmweshay, there is a lot of magic there..18:23
rlandywell if sshnaidm has a solution - I'd like to hear it18:23
weshayheh.. sshnaidm let's chat18:23
* weshay wants more details on your point of view18:23
weshayI'm in my blue if anyone wants to chat18:24
rlandyit works as it - just the expected path did not18:25
weshayI'll continue poking at it today18:25
rlandyweshay: imho, I don;t think it worth so much more of our time - if someone else can point out what we missed, great - we'll change it18:26
rlandyif not, it's working code18:27
rlandymerge it and change afterwards18:27
rlandylots of reproducer code as merged directly witout being perfect18:27
rlandywe can all attest to that18:27
rlandyit doesn't break CI18:28
rlandylet's look at cost/benefit here18:28
weshayrlandy, top_working_dir is that a zuul native or we need to define it?18:36
rlandyweshay: not zuul native - zuul uses zuul.project.src_dir18:37
rlandyweshay: we are using to allow for both situations and switch between playbook_dir on a user system vs what zull uses18:38
rlandyon paul's advice18:38
rlandynowhere else does th code try to set up on both  a user box and zuul executor18:39
rlandythat is our complication18:39
*** agopi has joined #oooq18:45
weshayrlandy, if you are not busy w/  panda join my blue18:49
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci-centos-7-scenario004-multinode-oooq- (2 more messages)18:52
zbrweshay: look at https://review.openstack.org/#/c/633948/13/roles/install-docker/tasks/main.yaml -- sorting the group without touching config.18:53
zbralready merged upstream18:53
*** amoralej is now known as amoralej|off18:54
*** dsneddon__ is now known as dsneddon19:00
weshay2019-02-07 18:51:43.120683 | rdo-centos-7 | [0;34mplaybooks/tripleo-ci-reproducer/pre.yaml[0m:[0;36m45[0m: [[1;31mE301[0m] [0;31mCommands should not change things if nothing needs doing[0m19:02
*** trown|lunch is now known as trown19:15
weshayzbr, can you rejoin for a hot sec19:32
*** saneax has joined #oooq19:38
*** vinaykns has quit IRC19:40
*** vinaykns has joined #oooq19:43
*** dtrainor_ has joined #oooq19:45
*** dtrainor__ has joined #oooq19:46
*** dtrainor_ has quit IRC19:49
*** dtrainor__ has quit IRC19:52
weshay  - name: read the passwd file19:57
weshay    shell: cat /etc/passwd19:57
weshay    register: user_accts19:57
weshay  - name: a task that only happens if the user exists19:57
weshay    when: user_accts.stdout.find('hillsy') != -119:57
weshay    debug: msg="user hillsy exists"19:57
weshay  - shell: grep username /etc/passwd19:58
weshayrlandy, https://github.com/ansible/ansible-lint/blob/master/lib/ansiblelint/rules/UseCommandInsteadOfShellRule.py20:15
*** saneax has quit IRC20:17
*** ccamacho has quit IRC20:28
weshayrlandy, https://review.openstack.org/#/c/631067/20:29
*** sshnaidm is now known as sshnaidm|off20:29
weshayhttps://review.openstack.org/#/c/635591/20:39
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo- (2 more messages)20:52
weshayrlandy, more fun zuul news.. I think https://review.openstack.org/#/c/635550/ is failing because zuul is trying to install bindep.txt21:01
weshaythe linters failed again21:02
rlandylooking21:02
weshay2019-02-07 20:39:58.093290 | ubuntu-xenial |   "msg": "No package matching 'libffi-devel' is available"21:02
weshayhttp://logs.openstack.org/50/635550/2/check/openstack-tox-docs/adfe38f/job-output.txt.gz#_2019-02-07_20_39_58_09329021:02
rlandyugh21:02
weshayhttp://logs.openstack.org/50/635550/2/check/openstack-tox-linters/2c6e5fe/job-output.txt.gz#_2019-02-07_20_42_08_74599021:03
rlandydistro21:05
weshayoh21:05
weshaygood point21:05
rlandywhere is that needed - really?21:06
weshay[platform:rpm]21:06
weshayrlandy, thanks!21:08
rlandyfor what?21:08
* rlandy didn't do anything21:08
weshaysparking joy https://review.openstack.org/#/c/635550/3..2/bindep.txt21:08
rlandynot sure that will be enough to fix it - may have to be more specific - but will see what ci says21:10
weshayrlandy, https://review.rdoproject.org/r/#/c/18664/ merging21:12
rlandyweshay: cool - thanks21:15
weshayrlandy, that fixed it :)21:22
rlandyoh good21:22
rlandy Trim Trailing Whitespace.................................................Failed21:32
rlandyweshay; I'll lend you my bash rc file :)21:32
rlandyZuul CI21:46
rlandyChange has been successfully merged by Zuul CI21:46
rlandywoohoo21:46
rlandywe did it!!!21:46
rlandyweshay: ^^21:46
weshay:)21:49
weshayrlandy++21:49
hubbot1weshay: rlandy's karma is now 4421:49
rlandyweshay++21:50
hubbot1rlandy: weshay's karma is now 1521:50
rlandyyou merged it ")21:50
*** weshay has quit IRC21:58
*** holser_ has quit IRC22:33
*** agopi is now known as agopi|pto22:42
*** agopi|pto has quit IRC22:48
*** holser_ has joined #oooq22:51
*** tosky has quit IRC22:52
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo- (2 more messages)22:52
rlandyCurrently, synchronize is limited to elevating permissions via passwordless sudo. This is because rsync itself is connecting to the remote machine and rsync doesn’t give us a way to pass sudo credentials in.22:57
pandadamn rsync22:59
*** panda is now known as panda|off22:59
*** jtomasek has joined #oooq23:00
*** rascasoft has quit IRC23:08
*** jtomasek has quit IRC23:16
*** holser_ has quit IRC23:31
*** weshay has joined #oooq23:37
weshayrlandy, bah.. totally blew up my long running server w/ my bouncer23:38
weshaytrying to upgrade from fedora2323:38
weshayrlandy, did you see my other comments?23:38
*** vinaykns has quit IRC23:38
weshaymeh.. for tomorrow23:38
*** weshay has quit IRC23:43
*** weshay has joined #oooq23:48
*** weshay is now known as weshay_PTO23:50

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!