| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ (1 more message) | 00:39 |
|---|---|---|
| *** pliu_ has joined #oooq | 02:05 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates- (1 more message) | 02:39 |
| *** skramaja has joined #oooq | 03:23 | |
| *** gkadam has joined #oooq | 03:30 | |
| *** yolanda_ has joined #oooq | 03:38 | |
| *** links has joined #oooq | 03:39 | |
| *** udesale has joined #oooq | 03:40 | |
| *** yolanda has quit IRC | 03:41 | |
| *** links has quit IRC | 04:15 | |
| *** chkumar|trekk is now known as chandankumar | 04:30 | |
| *** jaganathan has joined #oooq | 04:30 | |
| *** links has joined #oooq | 04:33 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy- (1 more message) | 04:39 |
| *** ykarel| has joined #oooq | 05:20 | |
| *** pliu_ has quit IRC | 05:22 | |
| *** ykarel| is now known as ykarel | 05:35 | |
| *** holser_ has joined #oooq | 05:50 | |
| *** jaosorior has joined #oooq | 05:51 | |
| *** ratailor has joined #oooq | 06:14 | |
| chandankumar | panda|rover|off: rfolco|off container builds are failing on horizon https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-centos-7-master-containers-build/f36b526/logs/kolla/logs/000_FAILED_horizon.log | 06:14 |
| *** jtomasek has joined #oooq | 06:18 | |
| *** quiquell has joined #oooq | 06:19 | |
| *** jfrancoa has joined #oooq | 06:19 | |
| *** brault has quit IRC | 06:19 | |
| quiquell | sshnaidm: https://review.rdoproject.org/r/#/c/15041/ rebased | 06:26 |
| *** agopi has quit IRC | 06:35 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy- (1 more message) | 06:39 |
| *** jfrancoa has quit IRC | 06:43 | |
| *** jfrancoa has joined #oooq | 06:43 | |
| *** ccamacho has joined #oooq | 06:44 | |
| *** holser_ has quit IRC | 06:44 | |
| quiquell | panda|rover|off, rfolco|off: Have open a bug, with new issue for scenario001 that stop the fix of CI | 06:53 |
| quiquell | panda|rover|off, rfolco|off: https://bugs.launchpad.net/tripleo/+bug/1784307 | 06:53 |
| openstack | Launchpad bug 1784307 in tripleo "tripleomaster/centos-binary-collectd:current-tripleo-updated-20180730001257 \"kolla_start\" Restarting" [Undecided,New] - Assigned to Rafael Folco (rafaelfolco) | 06:53 |
| *** zoli is now known as zoli|wfh | 06:57 | |
| *** zoli|wfh is now known as zoli | 06:57 | |
| *** brault has joined #oooq | 06:59 | |
| *** tesseract has joined #oooq | 07:04 | |
| *** amoralej|off is now known as amoralej | 07:08 | |
| *** yolanda_ is now known as yolanda | 07:21 | |
| *** florianf has joined #oooq | 07:23 | |
| *** dalvarez has joined #oooq | 07:25 | |
| *** jtomasek_ has joined #oooq | 07:34 | |
| *** jtomasek has quit IRC | 07:34 | |
| *** ykarel is now known as ykarel|lunch | 07:38 | |
| *** amoralej_ has joined #oooq | 07:47 | |
| chandankumar | arxcruz: Hello | 07:55 |
| chandankumar | arxcruz: Can I add you as a QE for this card https://trello.com/c/MX0ZJVtI/889-update-python-stestr-to-210 ? | 07:55 |
| arxcruz | chandankumar: hi, done | 07:56 |
| chandankumar | arxcruz: thanks :-) | 07:56 |
| *** bogdando has joined #oooq | 08:03 | |
| *** ykarel|lunch is now known as ykarel | 08:22 | |
| *** skramaja_ has joined #oooq | 08:24 | |
| *** skramaja is now known as Guest77205 | 08:25 | |
| *** skramaja_ is now known as skramaja | 08:25 | |
| *** holser_ has joined #oooq | 08:29 | |
| *** d0ugal has joined #oooq | 08:31 | |
| *** d0ugal has quit IRC | 08:31 | |
| *** d0ugal has joined #oooq | 08:31 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ (1 more message) | 08:39 |
| *** skramaja_ has joined #oooq | 08:40 | |
| *** skramaja has quit IRC | 08:41 | |
| *** skramaja_ is now known as skramaja | 08:41 | |
| *** skramaja_ has joined #oooq | 08:46 | |
| *** skramaja is now known as Guest22393 | 08:46 | |
| *** skramaja_ is now known as skramaja | 08:46 | |
| *** Guest22393 has quit IRC | 08:46 | |
| *** sshnaidm has quit IRC | 09:07 | |
| *** sshnaidm has joined #oooq | 09:09 | |
| *** sshnaidm is now known as sshnaidm|afk | 09:12 | |
| *** jfrancoa has quit IRC | 09:13 | |
| *** jfrancoa has joined #oooq | 09:15 | |
| *** quiquell is now known as quiquell|mtg | 09:15 | |
| *** dtantsur|afk is now known as dtantsur | 09:35 | |
| *** quiquell|mtg is now known as quiquell | 09:37 | |
| *** ratailor has quit IRC | 09:44 | |
| *** sshnaidm|afk has quit IRC | 09:47 | |
| quiquell | ykarel, arxcruz, marios, ssbarnea: Did you guys have this in the reproducer ? Inappropriate ioctl for device | 09:49 |
| arxcruz | nopnope | 09:49 |
| arxcruz | nope* | 09:50 |
| ssbarnea | i seen this in the past, send me link. | 09:50 |
| quiquell | damn... | 09:50 |
| quiquell | ssbarnea: It's at my little fedora | 09:50 |
| ssbarnea | i think is ansible bug if i remember well, but last update should have fixed it. | 09:50 |
| quiquell | ssbarnea: Maybe downgrade ansible will help ? | 09:50 |
| ssbarnea | if you redirect(ansibke) stdout the pause module was choking with something like this, but they fixed it. | 09:50 |
| quiquell | ssbarnea: I am using |tee | 09:51 |
| quiquell | ssbarnea: Maybe is that ? | 09:51 |
| ssbarnea | i think it was fixed in both ansible branches 2.6/2.5 and released few days ago. | 09:51 |
| ssbarnea | almost sure, not you fault. let me send a link to bug | 09:52 |
| quiquell | ssbarnea: Ok going to update | 09:52 |
| quiquell | ssbarnea: Is not a bug, is just a execution at my laptop | 09:52 |
| ssbarnea | or better first update ansible and tell me. | 09:52 |
| quiquell | ssbarnea: Was going too, let me try | 09:52 |
| quiquell | ssbarnea: Thanks man ! | 09:52 |
| marios | quiquell: not something i've seen yet | 09:53 |
| ykarel | quiquell, i also not aware of it | 09:54 |
| quiquell | ok | 09:54 |
| *** ratailor has joined #oooq | 10:00 | |
| quiquell | Puff Failed to retrieve repo file from https://trunk.rdoproject.org/centos7-master/current-tripleo/delorean.repo after 10 retries | 10:05 |
| quiquell | is appearing again | 10:05 |
| *** brault has quit IRC | 10:10 | |
| quiquell | ssh | 10:22 |
| *** jaosorior has quit IRC | 10:26 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ (1 more message) | 10:39 |
| quiquell | ssbarnea: Working!! thanks man, ansible 2.5.7 works | 10:45 |
| *** udesale has quit IRC | 10:47 | |
| *** panda|rover|off is now known as panda|rover | 10:50 | |
| ssbarnea | quiquell: glad to hear that. it seems that being subscribed to all ansible release notes pays off. | 10:53 |
| quiquell | ssbarnea: Shure it pays off | 10:54 |
| ssbarnea | regarding ansible versions and fixes/bugs. What if we would have a non-voting gate that uses next-ansible version (one that we do not yet support). Having this would clearly help upgrading code. | 10:55 |
| *** ratailor has quit IRC | 10:56 | |
| ssbarnea | there is something i learn regarding ansible team: is like 5-10x faster to have a bug fixed before their release than after. they do care a lot to avoid regressions, but they lack extensive testing. but if we catch bugs early, we would prevent ansible from releasiong breaking changes. | 10:57 |
| ssbarnea | quiquell: read and review https://review.openstack.org/#/c/583965/ -- that is why I knew about the bug. | 11:01 |
| panda|rover | sounds familiar | 11:01 |
| quiquell | ssbarnea: Damn... Have to do more reviewing :-/ | 11:03 |
| ssbarnea | while ansible fixed it, the fix is still vaid, working with all versions and cutting one task. | 11:03 |
| *** sshnaidm|afk has joined #oooq | 11:05 | |
| ssbarnea | panda|rover: quiquell : another one that should be very easy to review: https://review.openstack.org/#/c/582599/ | 11:05 |
| panda|rover | ssbarnea: we're on fire at the moment, you may get delay on any reviews | 11:06 |
| quiquell | ssbarnea: Yep, sorry about delays man, we will get into it later on | 11:06 |
| ssbarnea | sure. and again let me know if I can help with something. don't forget about me. | 11:07 |
| ssbarnea | i am still following the RHU courses on OSP,... boring but needed. | 11:09 |
| quiquell | panda|rover: My first PTO was 13th not 15th so I am going to be 2 days less in the sprint | 11:13 |
| panda|rover | quiquell: fantastic | 11:16 |
| quiquell | panda|rover: I just merge broken stuff so it's good for the sprint | 11:17 |
| panda|rover | ssbarnea: Best talk to weshay about your tasks at the present, when you have a rough idea, come back to me and we can distribute across sprint. I guess you'll shadow ruck and rover for quite a while, follow what's happening in #tripleo right now | 11:23 |
| ssbarnea | panda|rover: ok! | 11:24 |
| ssbarnea | quiquell: i think the same applies to me regarding PTO, i will be away the week August 13-17. | 11:25 |
| panda|rover | ssbarnea: I don't count it in sprint until weshay says so. | 11:26 |
| panda|rover | you* | 11:26 |
| *** zoli is now known as zoli|doctor | 11:26 | |
| *** jaosorior has joined #oooq | 11:32 | |
| *** quiquell is now known as quiquell|lunch | 11:33 | |
| *** rfolco|off is now known as rfolco|ruck | 11:36 | |
| panda|rover | marios: you're woring on https://trello.com/c/lrSUMaqw/885-translate-tripleosh-bootstrap-subnodes-into-a-series-of-tasks-s17 ? | 11:45 |
| panda|rover | marios: can you modify the description and the definition of done ? | 11:47 |
| marios | panda|rover: yeah ack i posted something will update the card with that as well | 11:49 |
| *** panda|rover is now known as panda|lunch | 11:51 | |
| rfolco|ruck | panda|lunch, should I assign this one to you ? https://bugs.launchpad.net/tripleo/+bug/1772807 | 11:59 |
| openstack | Launchpad bug 1772807 in tripleo "default containerized undercloud install with local CA fails with "Error org.freedesktop.DBus.Error.TimedOut"" [High,Triaged] | 11:59 |
| rfolco|ruck | panda|lunch, morning | 11:59 |
| weshay | panda|lunch, rfolco|ruck greetings guys.. let me know if there is anything I can do for you | 12:02 |
| panda|lunch | thought and prayers | 12:02 |
| panda|lunch | rfolco|ruck: morning | 12:02 |
| weshay | ssbarnea, let me get settled in and we'll chat k? | 12:02 |
| weshay | panda|lunch, let me see if I'm reading things correctly... rdo jobs are very red.. and master is at 9 days.. | 12:04 |
| weshay | everything else seems ok | 12:04 |
| panda|lunch | rfolco|ruck: doesn't seem to be ci or quickstart fault | 12:04 |
| rfolco|ruck | panda|lunch, k... another question, | 12:06 |
| panda|lunch | weshay: define "everything else". Scenarios are broken, upgrades are missing a tag, and we are currently reporting fasle positives | 12:06 |
| rfolco|ruck | https://ci.centos.org/job/tripleo-quickstart-promote-pike-rdo_trunk-minimal/ --> pike promotion | 12:06 |
| weshay | panda|lunch, scenario01 / 03? | 12:06 |
| weshay | panda|lunch, hrm ... ok ping me when you have a sec and lets review where we stand re: ruck / rover before the escalation mtg | 12:07 |
| weshay | panda|lunch, you have a patch for the upgrade tag issue? | 12:08 |
| panda|lunch | weshay: I can talk now, and yes, we have a patch, but was blocked by the scenario failures | 12:08 |
| panda|lunch | weshay: you wioll se me eating ... | 12:08 |
| weshay | k.. | 12:08 |
| * weshay gets headset | 12:08 | |
| panda|lunch | I think I'll join ci escalation too | 12:09 |
| weshay | panda|lunch, rfolco|ruck in my blue | 12:11 |
| *** quiquell|lunch is now known as quiquell | 12:11 | |
| quiquell | weshay: welcome back sir | 12:11 |
| panda|lunch | weshay: https://review.openstack.org/585528 | 12:11 |
| rfolco|ruck | weshay, please paste me link, I f* my browser cache | 12:12 |
| panda|lunch | weshay: https://review.openstack.org/587006 | 12:15 |
| weshay | quiquell, panda|lunch nice work thus far.. let's start by getting a bug in the commit message for https://review.openstack.org/#/c/587006/ and start knocking these off one by one | 12:19 |
| weshay | starting w/ https://review.openstack.org/#/c/587006/ | 12:19 |
| *** ratailor has joined #oooq | 12:19 | |
| *** gkadam has quit IRC | 12:19 | |
| panda|lunch | weshay: ok | 12:19 |
| quiquell | weshay: ack | 12:19 |
| quiquell | weshay: Admin question, do I have to put my PTO at Orange HRM and appears in the calendar ? | 12:20 |
| panda|lunch | quiquell: weshay commit message updates | 12:20 |
| panda|lunch | d | 12:20 |
| quiquell | panda|lunch, weshay: Jiris change is failing now at tripleo-ci-centos-7-containers-multinode | 12:22 |
| weshay | quiquell, yes first put it in the pto cal on google so the organization knows ur out, then in orange | 12:24 |
| quiquell | weshay: I can modify the calendar, I think you need to give me permissions | 12:24 |
| quiquell | s/can/can't/ | 12:25 |
| weshay | quiquell, hrm.. k.. /me sees the 3 node multinode error | 12:25 |
| quiquell | weshay: 3nodes ? | 12:25 |
| quiquell | rfolco|ruck: is this http://logs.openstack.org/99/586499/1/check/tripleo-ci-centos-7-containers-multinode/34499b8/logs/undercloud/home/zuul/tempest.log.txt.gz#_2018-07-30_11_58_09 | 12:26 |
| quiquell | rfolco|ruck: related to this https://bugs.launchpad.net/tripleo/+bug/1784017 ? | 12:26 |
| weshay | quiquell, sorry was looking at Emiliens patch | 12:26 |
| openstack | Launchpad bug 1784017 in tripleo "TestNetworkBasicOps.test_network_basic_ops failures" [Critical,Triaged] | 12:26 |
| quiquell | weshay: Ahh ok, we have to merge Jiri's one first, it has being like a perfect storm so far :-( | 12:27 |
| weshay | quiquell, hrm.. jiri's patch failed in tempest | 12:28 |
| quiquell | weshay: Bug above feels related :-/ | 12:28 |
| quiquell | weshay: Going to add the Depends-On to disable health check, now that has failed, It's like impossible to merge this | 12:29 |
| rfolco|ruck | quiquell, not sure, traceback looks diff | 12:29 |
| quiquell | Ok we have the +1w in the revert | 12:29 |
| quiquell | rfolco|ruck: Damn... | 12:30 |
| *** rlandy has joined #oooq | 12:31 | |
| chandankumar | panda|lunch: weshay I need some help here https://review.openstack.org/#/c/584368/4/roles/validate-tempest/defaults/main.yml@27 | 12:32 |
| rfolco|ruck | quiquell, I am not discarding the hypothesis | 12:32 |
| chandankumar | it is giving [undercloud]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'undercloud_enable_tempest' is undefined"} | 12:32 |
| *** panda|lunch is now known as panda|rover | 12:32 | |
| quiquell | panda|rover, weshay: We have +1w at revert it's just a matter of time and luck | 12:33 |
| quiquell | panda|rover: I see there is no scenario001 running at the tht heatl check revert... | 12:38 |
| quiquell | panda|rover: If they want to rre-revert it it should run it I think | 12:38 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates- (1 more message) | 12:39 |
| *** links has quit IRC | 12:44 | |
| rlandy | rook: hi | 12:44 |
| weshay | EmilienM, bogdando I may need to understand how we can exercise the containers w/ something like a chaos monkey in the promotion jobs to make sure we catch intermittent issues w/ health checks | 12:45 |
| weshay | bah.. wrong channel | 12:45 |
| rlandy | rook: nice work from you and agopi - https://review.openstack.org/#/c/583717/ | 12:45 |
| rlandy | 053 passing :) | 12:45 |
| *** zoli|doctor is now known as zoli | 12:45 | |
| weshay | panda|rover, rfolco|ruck is there a bug on the collectd kolla issue? | 12:45 |
| *** zoli is now known as zoli|wfh | 12:46 | |
| *** zoli|wfh is now known as zoli | 12:46 | |
| rfolco|ruck | yes | 12:46 |
| rfolco|ruck | sec | 12:46 |
| bogdando | weshay: not sure promotion jobs are good fit for destructive tests | 12:46 |
| rlandy | rook: where do I look for the logs for broebeat specific stuff? | 12:46 |
| bogdando | it should be periodic , IMO | 12:46 |
| weshay | bogdando, ya.. not sure either tbh | 12:46 |
| rlandy | browbeat | 12:46 |
| rfolco|ruck | weshay, | 12:46 |
| rfolco|ruck | https://bugs.launchpad.net/tripleo/+bug/1784307 --> tripleomaster/centos-binary-collectd:current-tripleo-updated-20180730001257 \"kolla_start\" Restarting | 12:46 |
| openstack | Launchpad bug 1784307 in tripleo "tripleomaster/centos-binary-collectd:current-tripleo-updated-20180730001257 \"kolla_start\" Restarting" [Critical,In progress] - Assigned to Gabriele Cerami (gcerami) | 12:46 |
| rfolco|ruck | https://bugs.launchpad.net/tripleo/+bug/1784233 --> Kolla fails to build horizon container in periodic master job | 12:46 |
| openstack | Launchpad bug 1784233 in tripleo "Kolla fails to build horizon container in periodic master job" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:46 |
| weshay | rfolco|ruck, ya.. thanks | 12:47 |
| bogdando | It's better to have tires on fire periodically then blocking promotions :D | 12:47 |
| bogdando | when* | 12:47 |
| quiquell | bogdando: Make sense to exercise scenari001 at tht with commons from ^common/* ? | 12:48 |
| quiquell | bogdando: The healtcheck is there | 12:48 |
| quiquell | s/ commons / changes / | 12:48 |
| *** agopi has joined #oooq | 12:49 | |
| bogdando | quiquell: was that for my periodic proposal for wes's chaos monkeys? | 12:49 |
| rlandy | marios: still not able to get your patch through? :( we didn't have a good day with the gates on friday | 12:49 |
| bogdando | I'm not sure I'm following right | 12:49 |
| rook | rlandy: /home/<user>/browbeat/log/ | 12:50 |
| quiquell | bogdando: review coming | 12:50 |
| quiquell | bogdando: mean this https://review.openstack.org/587051 | 12:50 |
| *** amoralej is now known as amoralej|lunch | 12:51 | |
| rlandy | rook: https://logs.rdoproject.org/17/583717/20/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/298303d/logs/undercloud/home/zuul/ - so then we are not collecting that - will modify log collection list | 12:51 |
| quiquell | bogdando: Don't know if the other scenarios are needed too | 12:51 |
| bogdando | I think they do | 12:51 |
| rook | rlandy: it sends the most relevant information to stdout | 12:51 |
| * rlandy checks log collection | 12:51 | |
| rook | rlandy: do we capture that in the console? | 12:51 |
| bogdando | common contains the very DF core | 12:51 |
| quiquell | bogdando: So the review makes sense, was weird for me to not see the scenarios exercised | 12:52 |
| bogdando | so any change to it worth the maximum possible coverage | 12:52 |
| bogdando | yupp | 12:52 |
| rlandy | rook: yep ... https://logs.rdoproject.org/17/583717/20/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/298303d/job-output.txt.gz#_2018-07-28_02_52_03_759513 | 12:52 |
| rlandy | but I don't see any specific info on the run browbeat line | 12:52 |
| rlandy | rook: it's important for debugging that we collect *all* browbeat logs/info | 12:53 |
| rook | rlandy: ack | 12:53 |
| rlandy | otherwise debug will be very difficult | 12:53 |
| rook | rlandy: agreed. | 12:53 |
| rlandy | rook: ok - so we are juts missing /home/user/browbeat/*? | 12:53 |
| rlandy | anything else? | 12:53 |
| rlandy | while I am modifying log collection | 12:54 |
| rook | one sec i was reviewing what agopi changed.. /me looks | 12:54 |
| rlandy | rook: going in to scrum meeting - so take your time - will ping you when I get out of meeting | 12:54 |
| marios | rlandy: ack yeah looks like latest also failed on that pike job seems consistent | 12:55 |
| rlandy | marios: consistent with your job or consistent everywhere? | 12:55 |
| rook | agopi: going to make another minor change | 12:56 |
| *** links has joined #oooq | 12:56 | |
| marios | rlandy: didn't dig yet so don't know | 12:57 |
| *** amoralej|lunch is now known as amoralej | 12:57 | |
| panda|rover | weshay: https://trello.com/c/2WLBEuEc/589-cixlp1771233tripleociproa-te-broker-errors-in-tripleo-3rd-party-jobs we need to sync on this, there are news. | 12:58 |
| weshay | ssbarnea, please join the status mtg | 13:04 |
| weshay | ssbarnea, let's sync up after the mtg | 13:04 |
| *** myoung has joined #oooq | 13:04 | |
| myoung | triplo-ci scrum: o/ | 13:05 |
| rasca | rlandy, hey hi! I saw your work on the reviews, I've tried to test it but unfortunately I'm still hitting the dbus problem | 13:05 |
| myoung | quiquell, ssbarnea ^^ | 13:05 |
| quiquell | myoung: Damn sorry | 13:05 |
| quiquell | Going there | 13:05 |
| *** sshnaidm|afk is now known as sshnaidm | 13:06 | |
| rlandy | rasca: in scrum - will ping you when out | 13:06 |
| rasca | rlandy, sure no worries and thanks | 13:06 |
| agopi | sure thing rook | 13:09 |
| rook | agopi: patch up | 13:10 |
| rook | https://review.openstack.org/#/c/583717/21/ansible/oooq/roles/browbeat-run/tasks/main.yml agopi | 13:12 |
| agopi | understood rook want to make sure we can look at how browbeat ran | 13:13 |
| agopi | looks good | 13:13 |
| agopi | but the patch isnt ready yet, i've to make changes and test it as changes made to gen_tripleo_hostfile isnt running in the internal CI | 13:14 |
| marios | panda|rover: rlandy: weshay there are no logs :/ http://logs.openstack.org/95/583195/12/check/tripleo-ci-centos-7-containers-multinode-pike/b8eca56/logs/ | 13:19 |
| weshay | marios, ya.. saw that | 13:20 |
| weshay | need to see if the job before also had that happen | 13:20 |
| rlandy | we need also to see if it fails elsewhere | 13:21 |
| panda|rover | we've seen some other instance of this problem | 13:23 |
| rook | agopi: ok | 13:24 |
| *** agopi is now known as agopi|brb | 13:25 | |
| panda|rover | sorry guys I keep disconnecting | 13:27 |
| *** agopi|brb has quit IRC | 13:30 | |
| *** udesale has joined #oooq | 13:30 | |
| *** udesale has quit IRC | 13:35 | |
| *** skramaja has quit IRC | 13:36 | |
| *** Goneri has joined #oooq | 13:38 | |
| *** agopi has joined #oooq | 13:54 | |
| rook | rlandy: added to the commit to get more output | 13:54 |
| rook | thanks for brining that up | 13:55 |
| rook | how can I check the status of featureset053 | 13:55 |
| rook | ie, the queue | 13:55 |
| *** ratailor has quit IRC | 13:55 | |
| weshay | ssbarnea, sorry.. this went way over, going to have to sync w/ you tomorrow probably | 14:02 |
| ssbarnea | weshay: sure, i have enough to do. | 14:03 |
| quiquell | panda|rover, rfolco|ruck: Let me know if you find issues at dashboard-ci.tripleo.org | 14:08 |
| weshay | quiquell, check the newbie doc re: pto cal | 14:09 |
| quiquell | weshay: I have it added, but cannot change it | 14:09 |
| quiquell | weshay: Cannot add my days | 14:09 |
| weshay | quiquell, k.. I think you need to email eliska | 14:09 |
| weshay | should have that in the doc I think | 14:09 |
| quiquell | weshay: Ahh ok found it at the end of the doc... sorry mate | 14:10 |
| weshay | no prob | 14:10 |
| rlandy | rook: sorry - out of scrum | 14:11 |
| rlandy | rook: to check the status of fs053, https://review.rdoproject.org/zuul/status.html - use search box - add your review number | 14:11 |
| * rlandy works on review now to fix collect logs | 14:12 | |
| rlandy | marios: ^^ pls ping me before you go so I know where to pick up the investigation/job watching. thanks | 14:13 |
| panda|rover | myoung: want to chat about the isolated ansible tests ? I have a few minutes to spare | 14:16 |
| *** quiquell has quit IRC | 14:17 | |
| myoung | panda|rover, sure, but i'm in tempest squad sprint 17 planning for an hour | 14:21 |
| panda|rover | myoung: ok, ping me when you're ready | 14:24 |
| marios | rlandy: ack still on this call for a bit | 14:25 |
| marios | rlandy: sorry | 14:25 |
| rlandy | no rush | 14:27 |
| *** links has quit IRC | 14:27 | |
| panda|rover | marios: rlandy 018-07-30 12:38:51.008910 | primary | ok: 4 changed: 4 unreachable: 0 failed: 0 during post-run. THe nodes were unreachable, so we didn't get any log | 14:29 |
| panda|rover | the job timed out | 14:30 |
| *** udesale has joined #oooq | 14:34 | |
| rlandy | rook: https://review.openstack.org/#/c/583576/10/toci-quickstart/config/collect-logs.yml | 14:37 |
| *** dtantsur is now known as dtantsur|brb | 14:38 | |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates- (1 more message) | 14:39 |
| *** ykarel is now known as ykarel|away | 14:41 | |
| *** bogdando has quit IRC | 14:45 | |
| rook | rlandy++ | 14:51 |
| hubbot | rook: rlandy's karma is now 17 | 14:51 |
| rook | yup | 14:51 |
| rook | i added to the ci script to save stdout to logs | 14:51 |
| rook | log* | 14:51 |
| *** ykarel|away has quit IRC | 14:51 | |
| *** vinaykns has joined #oooq | 14:52 | |
| rlandy | http://logs.openstack.org/95/583195/12/check/tripleo-ci-centos-7-containers-multinode-pike/650309f/job-output.txt.gz#_2018-07-29_19_41_26_947888 | 15:08 |
| rlandy | marios: ^^ previous run from the same job | 15:08 |
| marios | rlandy: ack also here http://logs.openstack.org/95/583195/12/check/tripleo-ci-centos-7-containers-multinode-pike/b8eca56/ara-report/ | 15:09 |
| marios | rlandy: but without any more info why | 15:09 |
| rlandy | hmm ... | 15:16 |
| marios | rlandy: i have no idea why but it seems like it must be related to this change, or at least, i can see green runs on other reviews like | 15:18 |
| marios | green run @ https://review.openstack.org/#/c/585528/ 29th July https://review.openstack.org/#/c/570892/ today | 15:18 |
| marios | for tripleo-ci-centos-7-containers-multinode-pike | 15:18 |
| *** atoth has quit IRC | 15:21 | |
| *** atoth has joined #oooq | 15:22 | |
| rlandy | http://38.145.35.97/d/cEEjGFFmz/cockpit?orgId=1&panelId=61&fullscreen&var-launchpad_tags=alert&var-promotion_names=current-tripleo&var-promotion_names=current-tripleo-rdo&var-promotion_names=current-tripleo-rdo-testing&var-releases=master&var-releases=queens&var-releases=pike&var-releases=ocata&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-7-containers-multinode-pike | 15:27 |
| rlandy | shows no failing jobs | 15:27 |
| rlandy | weird | 15:27 |
| rlandy | even if I look back 7 days | 15:28 |
| rlandy | sova doesn't track this job | 15:28 |
| rlandy | marios: yep _ I do see green runs in other reviews | 15:34 |
| rlandy | not sure if that is an exact diagnosis though | 15:34 |
| marios | rlandy: yeah. i was also checking the job definition for clues like https://github.com/openstack-infra/tripleo-ci/blob/833c9d2a814c15011c43f25021bdfde19e308afd/zuul.d/multinode-jobs.yaml#L136 but other job with featureset 010 passes like tripleo-ci-centos-7-containers-multinode | 15:36 |
| rlandy | and we git a fairly consistent timeout here | 15:36 |
| rlandy | got | 15:36 |
| rlandy | marios: I think out best bet is a reproducer to see where the timeout is and why | 15:37 |
| *** dtantsur|brb is now known as dtantsur | 15:37 | |
| marios | rlandy: ack i will try that tomorrow but we don't even get the reproducer here | 15:42 |
| marios | rlandy: so not sure where to get it from :) I mean it doesn't make sense to get it from a green job | 15:43 |
| rlandy | marios: we can edit one from another fs | 15:43 |
| marios | rlandy: ah ok | 15:43 |
| rlandy | or job, right | 15:43 |
| marios | rlandy:right i can edit the zuul changes etc | 15:43 |
| rlandy | marios: ack - ok, I'll try a reproducer | 15:44 |
| *** myoung has quit IRC | 15:44 | |
| *** tesseract has quit IRC | 15:49 | |
| *** zoli is now known as zoli|gone | 15:57 | |
| *** zoli|gone is now known as zoli | 15:58 | |
| marios | rlandy: thanks, am gonna call it a day in a sec. i will have a look tomorrow for any updates and otherwise also try the reproducer | 16:02 |
| marios | rlandy: thanks for your help | 16:02 |
| marios | as always | 16:02 |
| *** jfrancoa has quit IRC | 16:05 | |
| panda|rover | rlandy: do you have a patch for collect logs ? | 16:05 |
| rlandy | panda|rover: https://review.openstack.org/#/c/583576/10/toci-quickstart/config/collect-logs.yml | 16:06 |
| rlandy | panda|rover: ^^ part of tripleo-ci patch | 16:07 |
| panda|rover | rlandy: ok, we have another bug for logs collection, the one that causes pike job to not collect any logs | 16:08 |
| panda|rover | rlandy: or any other job that times out | 16:08 |
| panda|rover | rlandy: do you have something for that too ?\ | 16:08 |
| *** myoung has joined #oooq | 16:14 | |
| panda|rover | rlandy: https://review.openstack.org/587103 | 16:17 |
| panda|rover | marios: ^ that's why the logs were not showing up | 16:17 |
| rlandy | panda|rover:yep - that's the timeout issue we are looking at | 16:18 |
| rlandy | panda|rover: that doesn't fix the timeout, right? just the log collection if timeout happens? | 16:19 |
| panda|rover | rlandy: yep, just the collection | 16:19 |
| panda|rover | maybe it's bettter if I create the bug | 16:21 |
| rlandy | yeah , ok - still need to debug the timeout then | 16:21 |
| rlandy | brb | 16:21 |
| *** rlandy is now known as rlandy|brb | 16:21 | |
| *** ccamacho has quit IRC | 16:26 | |
| panda|rover | rfolco|ruck: FYI https://bugs.launchpad.net/tripleo/+bug/1784417 | 16:35 |
| openstack | Launchpad bug 1784417 in tripleo "TripleO-CI workflow fails to collect logs after a time out" [Critical,Fix committed] - Assigned to Gabriele Cerami (gcerami) | 16:35 |
| *** panda|rover is now known as panda|rover|off | 16:36 | |
| myoung | panda|rover|off: sorry i missed yua | 16:37 |
| panda|rover|off | myoung: no you didn't, wanna chat 5 minutes ? | 16:38 |
| myoung | sure | 16:38 |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-3nodes- (1 more message) | 16:39 |
| myoung | panda|rover|off: https://bluejeans.com/7050859455 | 16:39 |
| *** jfrancoa has joined #oooq | 16:41 | |
| *** udesale has quit IRC | 16:43 | |
| weshay | rfolco|ruck, you online? | 16:47 |
| rfolco|ruck | weshay, o/ | 16:47 |
| weshay | rfolco|ruck, join my blue for a minute | 16:47 |
| *** jfrancoa has quit IRC | 16:48 | |
| weshay | rfolco|ruck, panda|rover|off this can be closed right? https://bugs.launchpad.net/tripleo/+bug/1783540 | 16:49 |
| openstack | Launchpad bug 1783540 in tripleo "RDO cloud is not in operational state" [Critical,Triaged] - Assigned to chandan kumar (chkumar246) | 16:49 |
| weshay | rfolco|ruck, | 16:49 |
| weshay | hubbot check jobs: | 16:49 |
| hubbot | weshay: Error: "check" is not a valid command. | 16:49 |
| weshay | TQE, https://review.openstack.org/#/c/560445, I214272a6f25feb75496e44eb0a16269c6ee4cfe2 | 16:49 |
| weshay | THT, https://review.openstack.org/#/c/567224, I0cbf9ffb8552411e4dd891c38702ff8d1f6db5b1, stable/queens | 16:49 |
| weshay | THT, https://review.openstack.org/#/c/564285, If12c8fe9bd0bea98a4842f279399285344f22246, stable/pike | 16:49 |
| weshay | THT, https://review.openstack.org/#/c/564291, I4c5bdf00ce8cf7eabf669b248b99cb8443e82fab, stable/ocata | 16:49 |
| panda|rover|off | weshay: rhos-ops confirmed it's working | 16:55 |
| weshay | panda|rover|off, aye | 16:55 |
| weshay | panda|rover|off, this scares me | 16:56 |
| weshay | http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1 | 16:56 |
| weshay | panda|rover|off, rfolco|ruck http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=63&fullscreen | 16:56 |
| weshay | panda|rover|off, that seems wrong to me | 16:57 |
| weshay | panda|rover|off, based on what I see from the no-op check jobs | 16:57 |
| *** rlandy|brb is now known as rlandy | 16:59 | |
| panda|rover|off | why the dashboard says NODE_FAILURE ? | 16:59 |
| weshay | panda|rover|off, well.. if the provisioning fails | 17:00 |
| panda|rover|off | oh I see | 17:00 |
| weshay | panda|rover|off, rfolco|ruck I'm going to run cleanup on the tenant unless | 17:00 |
| weshay | one of you are | 17:00 |
| weshay | rlandy, you ready? | 17:01 |
| rlandy | yep - joining | 17:01 |
| panda|rover|off | weshay: in http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=63&fullscreen, what does time stand for ? If you look at https://review.openstack.org/#/c/586285/ for example, the last check was in 27th of July, not today | 17:02 |
| weshay | hrm.. | 17:03 |
| weshay | I'm looking at recent changes | 17:04 |
| panda|rover|off | I still see some NODE_FAILURE yes, but the time column is really confusing | 17:05 |
| panda|rover|off | NODE_FAILURE anyway means that we're even unable to get the undercloud | 17:08 |
| panda|rover|off | we have 349 instances in nodepool tenant currently | 17:09 |
| panda|rover|off | I've seen much more ... | 17:09 |
| panda|rover|off | but yeah we have instances older than 1 day | 17:10 |
| panda|rover|off | but not many | 17:10 |
| panda|rover|off | weshay: I don't think the cleanup script is going to solve much, there are like 10 instances older than 1 day | 17:11 |
| panda|rover|off | weshay: 16 instances. | 17:11 |
| rfolco|ruck | weshay, panda|rover|off I have a doctor appt. will be back in one hour, asap | 17:13 |
| panda|rover|off | yeah, I'm off for real now too. | 17:15 |
| weshay | k | 17:19 |
| weshay | rfolco|ruck, ping me when you are back, let's chat w/ Ronelle | 17:23 |
| *** amoralej is now known as amoralej|off | 17:26 | |
| rfolco|ruck | weshay, ok | 17:28 |
| *** vinaykns has quit IRC | 17:31 | |
| *** ykarel|away has joined #oooq | 17:45 | |
| *** jaganathan has quit IRC | 17:50 | |
| weshay | interesting https://review.openstack.org/#/c/586843/1 | 17:59 |
| *** vinaykns has joined #oooq | 18:14 | |
| panda|rover|off | intersting indeed https://review.openstack.org/#/c/548005/ | 18:20 |
| weshay | panda|rover|off, do we have a bug on the master container build? | 18:20 |
| *** myoung is now known as myoung|lunch | 18:22 | |
| panda|rover|off | weshay: we have one that fails to build horizon in kolla | 18:23 |
| weshay | I don't even see logs :( | 18:24 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-centos-7-master-containers-build/a213a15/ | 18:24 |
| panda|rover|off | weshay I think it's in the ruck rover etherpad | 18:24 |
| weshay | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/?C=M;O=D | 18:24 |
| panda|rover|off | mmhh | 18:25 |
| weshay | rfolco|ruck, panda|rover|off how do we not have a bug opened on this ? | 18:25 |
| weshay | panda|rover|off, I'm not sure if that job is running | 18:27 |
| weshay | panda|rover|off, rfolco|ruck guess I'll wait until the next set kicks | 18:27 |
| *** dtantsur is now known as dtantsur|afk | 18:31 | |
| weshay | panda|rover|off, rfolco|ruck I'll watch it.. no worries | 18:34 |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-3nodes- (1 more message) | 18:39 |
| *** holser_ has quit IRC | 18:45 | |
| panda|rover|off | weshay we had this https://bugs.launchpad.net/tripleo/+bug/1784233 yesterday. | 18:46 |
| openstack | Launchpad bug 1784233 in tripleo "Kolla fails to build horizon container in periodic master job" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 18:46 |
| weshay | panda|rover|off, aye.. saw that | 18:46 |
| panda|rover|off | weshay it was merged 3 hours ago. maybe it's related if now is timing out | 18:46 |
| weshay | panda|rover|off, /me is just worried about the lack of visibility of the tripleo promotions jobs now | 18:46 |
| weshay | that we have zuulv3 | 18:46 |
| weshay | no sova | 18:46 |
| weshay | and it looks likes the containers build job is not kicking or logging at all | 18:47 |
| weshay | panda|rover|off, but I'll wait to see what happens when the next set kicks | 18:47 |
| weshay | panda|rover|off, the ruck/rover cockpit is also not reporting the status correctly | 18:47 |
| panda|rover|off | I usually keep the window open on the logs page when I'm ruck | 18:49 |
| rfolco|ruck | weshay, back | 18:49 |
| rfolco|ruck | weshay, sagi opened the bug for container build job | 18:50 |
| weshay | panda++ | 18:50 |
| weshay | rfolco|ruck, diff issue | 18:50 |
| rfolco|ruck | weshay, missing logs ? panda|rover|off did | 18:52 |
| rfolco|ruck | oh I see your comment in https://review.rdoproject.org/etherpad/p/ruckrover-sprint17 | 18:53 |
| weshay | rfolco|ruck, rlandy let's chat on my blue when you both are avail | 18:55 |
| weshay | re: rdo zuul config | 18:55 |
| rfolco|ruck | weshay, I can chat now | 18:55 |
| weshay | rfolco|ruck, k.. waiting on rlandy | 18:56 |
| weshay | rfolco|ruck, /me looking at https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-centos-7-master-containers-build/ | 18:56 |
| weshay | wth? | 18:56 |
| panda|rover|off | mmnh logs on rdo was broken last week too | 18:56 |
| panda|rover|off | for a while | 18:57 |
| weshay | panda|rover|off, rfolco|ruck so.. if I don't see a set of those jobs kick by the end of my day.. I'm opening a blocker | 18:58 |
| panda|rover|off | maybe it's relate to the NODE_FAILURE | 19:00 |
| sshnaidm | Seems like there is still problem with logs in rdo | 19:00 |
| sshnaidm | logs that I opened 1 hour before just disappeared | 19:00 |
| sshnaidm | maybe somebody is cleaning them now | 19:00 |
| weshay | hrm.. k | 19:01 |
| * weshay asking in #rdo | 19:01 | |
| rfolco|ruck | yep, the bug sshnaidm opened on Sunday had the logs and disappeared looks lile | 19:01 |
| rfolco|ruck | like | 19:01 |
| weshay | rfolco|ruck, where is that bug? | 19:01 |
| rfolco|ruck | https://bugs.launchpad.net/tripleo/+bug/1784233 | 19:02 |
| openstack | Launchpad bug 1784233 in tripleo "Kolla fails to build horizon container in periodic master job" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 19:02 |
| rlandy | weshay: rfolco|ruck: here - sorry | 19:02 |
| agopi | rlandy: ping | 19:02 |
| rfolco|ruck | trying to find what still remains at https://review.rdoproject.org/zuul/builds.html?pipeline=openstack-periodic | 19:03 |
| rlandy | weshay: rfolco|ruck: logged onto bj | 19:05 |
| rfolco|ruck | ok going | 19:05 |
| weshay | rlandy, ok.. joining | 19:05 |
| weshay | rfolco|ruck, my blue | 19:05 |
| weshay | rfolco|ruck, https://bluejeans.com/4113567798/ | 19:07 |
| *** holser_ has joined #oooq | 19:15 | |
| *** holser_ has quit IRC | 19:29 | |
| *** holser_ has joined #oooq | 19:29 | |
| *** ykarel|away has quit IRC | 19:30 | |
| *** myoung|lunch is now known as myoung | 19:34 | |
| *** holser_ has quit IRC | 19:46 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-3nodes- (1 more message) | 20:39 |
| rlandy | ha- reproducer passed for marios' review ? | 20:44 |
| rlandy | agopi: hi - responding to ping above | 20:45 |
| rlandy | rook: agopi: if you want to merge your changes to browbeat I can remove the depends-on jobs | 20:46 |
| *** holser_ has joined #oooq | 20:47 | |
| sshnaidm | rlandy, weshay can you review those please? https://review.openstack.org/#/c/580214/ https://review.openstack.org/#/c/583198/ | 20:49 |
| sshnaidm | weshay, can you merge this please? https://review.openstack.org/#/c/576816/ | 20:50 |
| weshay | sshnaidm, /me looks | 20:50 |
| agopi | hello rlandy, https://review.openstack.org/#/c/583717/ new patch didn't trigger a zuul build | 20:51 |
| agopi | https://review.rdoproject.org/zuul/builds.html?job_name=legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master | 20:51 |
| agopi | checked in status as well, nothing queued up either | 20:51 |
| agopi | https://review.rdoproject.org/zuul/status.html | 20:51 |
| rlandy | agopi: can I recheck? | 20:52 |
| agopi | yes rlandy | 20:52 |
| rlandy | actually rebse | 20:52 |
| rlandy | rebase | 20:53 |
| agopi | okay done just now rebased | 20:53 |
| rlandy | it triggered now | 20:53 |
| agopi | okay thats weird, it didn't trigger on previous patch :/ | 20:53 |
| rlandy | ok - let's watch it | 20:53 |
| *** hamzy_ is now known as hamzy | 20:59 | |
| weshay | sshnaidm, rlandy panda|rover|off fyi.. be sure to make sure https://ci.centos.org/job/tripleo-quickstart-gate-master-delorean-quick-basic/ is passing on your reviews :) | 21:04 |
| weshay | the job is working fairly well now | 21:04 |
| weshay | that installs the undercloud and does all the prep, everything except the overcloud install and test | 21:05 |
| rlandy | rfolco|ruck: panda|rover|off: do I need to edit both playbooks/tripleo-ci/vars/environment_type/ovb.yaml and playbooks/tripleo-ci/templates/toci_gate_test.sh.j2 | 21:06 |
| rlandy | to add the browbeat playbook | 21:06 |
| rfolco|ruck | looking | 21:06 |
| rlandy | does not seem to reference the ansible vars | 21:06 |
| rlandy | rfolco|ruck: ^^ | 21:06 |
| *** agopi is now known as agopi|brb | 21:07 | |
| rfolco|ruck | rlandy, well, I think we are in a strange point where things tend to be in the new way but still are in the old way, even in the run-v3 workflow | 21:09 |
| rlandy | rfolco|ruck: I edited both | 21:09 |
| rfolco|ruck | rlandy, IMHO you can do both | 21:09 |
| rfolco|ruck | and then we cleanup old | 21:09 |
| rlandy | let's just not merge any of this | 21:09 |
| rfolco|ruck | rlandy, I'll be happy to review this change | 21:10 |
| rfolco|ruck | count on me | 21:11 |
| rfolco|ruck | even if its DNM change, testing only | 21:11 |
| *** agopi|brb has quit IRC | 21:12 | |
| rlandy | rfolco|ruck: the demise begins ... https://review.openstack.org/#/c/587228/ | 21:13 |
| rlandy | rook: agopi: just FYI ^^ - we are using the browbeat job to test the new zuulv3 configuration | 21:15 |
| rlandy | you may see another fs053 test start running | 21:15 |
| rlandy | and we may need to rekick testing | 21:16 |
| rlandy | pls be patient with us :) | 21:16 |
| *** florianf has quit IRC | 21:21 | |
| rlandy | rfolco|ruck: panda|rover|off: If I copy over the runv3 playbook, I think I will also need everything in playbooks/tripleo-ci | 21:27 |
| rlandy | templates, vars, etc. | 21:28 |
| rlandy | all referenced in run v3 | 21:28 |
| panda|rover|off | yep | 21:28 |
| rlandy | unless it's just the playbook itself | 21:28 |
| rlandy | ok - copying the lot over | 21:28 |
| rlandy | minus post and run | 21:28 |
| rlandy | hmmm ... maybe post - will need to modify | 21:29 |
| *** dsneddon has quit IRC | 21:29 | |
| rlandy | panda|rover|off: actually that would mean everything in https://github.com/openstack-infra/tripleo-ci/tree/master/playbooks | 21:31 |
| rlandy | images and all? | 21:31 |
| *** dsneddon has joined #oooq | 21:31 | |
| rfolco|ruck | rlandy, I know you are still working on it... missing stuff like the featureset etc | 21:33 |
| rfolco|ruck | left a few comments | 21:33 |
| rfolco|ruck | leaving for today | 21:33 |
| rfolco|ruck | imagine all the people, leaving for today | 21:33 |
| *** rfolco|ruck is now known as rfolco|off | 21:34 | |
| rlandy | rfolco|off: yep, missing lots of stuff, we 're just getting started - "and miles to go before I sleep ... and mile to go before I sleep" | 21:34 |
| rfolco|off | will be back l8er | 21:35 |
| *** Goneri has quit IRC | 21:36 | |
| weshay | sshnaidm, I did not really understand your -1 here https://review.openstack.org/#/c/583547/ | 21:50 |
| weshay | the packaging for ceph is built such that the rpms must be removed and the proper version installed | 21:51 |
| sshnaidm | weshay, well, maybe was better to talk with me before approving.. | 21:51 |
| sshnaidm | weshay, we discussed it with marios in irc | 21:51 |
| weshay | sshnaidm, I can cancel my worflow if you want | 21:52 |
| sshnaidm | weshay, all this patch doesn't make sense at all | 21:52 |
| sshnaidm | weshay, it completely doesn't matter what to configure there, because in next minute everything will be rewritten by quickstart | 21:53 |
| sshnaidm | weshay, and this can't be reason for bug of course | 21:53 |
| weshay | I hit ceph issues prior to quickstart.sh running | 21:53 |
| sshnaidm | weshay, if so - not because of that | 21:53 |
| weshay | it happens in pre | 21:53 |
| sshnaidm | weshay, ok, let's see the log | 21:54 |
| weshay | it was local w/ the reproducer | 21:54 |
| sshnaidm | weshay, pre- is not related to tripleo.sh | 21:54 |
| weshay | so I don't have any ci logs | 21:54 |
| sshnaidm | weshay, ok, please add full logs to bug if you reproduce it again | 21:54 |
| weshay | aye | 21:55 |
| weshay | sshnaidm, ok.. review is updated | 21:55 |
| *** myoung has quit IRC | 21:59 | |
| *** holser_ has quit IRC | 22:00 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, legacy-tripleo-ci-centos-7-multinode-1ctlr-featureset037-updates-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-3nodes- (1 more message) | 22:39 |
| *** tcw has quit IRC | 22:42 | |
| *** tcw has joined #oooq | 22:42 | |
| *** sshnaidm is now known as sshnaidm|afk | 23:23 | |
| *** vinaykns has quit IRC | 23:24 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!