Thursday, 2018-04-26

hubbot`FAILING CHECK JOBS on stable/queens: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/560452, master: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master @ https://review.openstack.org/56044500:40
*** ykarel has joined #oooq00:51
*** agopi|zzz has joined #oooq01:06
*** ykarel is now known as ykarel|afk01:15
*** yolanda__ has joined #oooq01:40
*** yolanda_ has quit IRC01:44
*** ykarel|afk has quit IRC01:52
*** moguimar_ has joined #oooq02:11
*** moguimar has quit IRC02:12
*** d0ugal has quit IRC02:12
*** faceman has quit IRC02:12
*** faceman has joined #oooq02:13
*** d0ugal has joined #oooq02:14
*** bandini has quit IRC02:14
*** bandini has joined #oooq02:17
*** gkadam has joined #oooq02:26
hubbot`FAILING CHECK JOBS on stable/queens: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/560452, master: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master @ https://review.openstack.org/56044502:40
*** ykarel|afk has joined #oooq03:38
*** ykarel|afk is now known as ykarel03:39
*** skramaja has joined #oooq04:00
*** apetrich has quit IRC04:29
*** apetrich has joined #oooq04:30
hubbot`FAILING CHECK JOBS on master: gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master @ https://review.openstack.org/56044504:41
*** gkadam has quit IRC04:45
*** ccamacho has quit IRC04:48
*** jaganathan has quit IRC04:49
*** links has joined #oooq04:53
*** jaganathan has joined #oooq05:01
ykarelchandankumar, https://ci.centos.org/view/rdo/view/weirdo-pipelines/view/weirdo-promote-puppet/job/weirdo-generic-puppet-openstack-scenario003/5001/consoleText05:10
ykarel+ /usr/bin/tempest run --whitelist_file=/tmp/openstack/tempest/test-whitelist.txt --concurrency=205:10
ykarel  @test.services('compute')05:10
ykarel+ RESULT=10005:10
ykarelany idea in what cases return code can be 10005:10
*** jaganathan has quit IRC05:24
*** gkadam has joined #oooq05:29
*** ratailor has joined #oooq05:32
*** quiquell|off is now known as quiquell|rover05:35
quiquell|roverGood morning05:36
quiquell|rover!gatestatus05:36
openstackquiquell|rover: Error: "gatestatus" is not a valid command.05:36
quiquell|rover%gatestatus05:39
hubbot`All check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.05:40
ykarel^^ ocata twice?missing pike05:40
quiquell|roverykarel: You are reactivating the segments tests ?05:42
ykarelafter backported to queens is merged05:42
*** yolanda__ is now known as yolanda05:45
ykarelquiquell|rover, https://review.openstack.org/#/c/564378/05:45
*** d0ugal has quit IRC05:46
*** marios has joined #oooq05:49
*** jaganathan has joined #oooq05:49
quiquell|roverthe kolla problem is also in place with the hack reverted05:51
quiquell|rovercool cool05:51
ykarelyup all fixes are in place05:51
ykarelqueens is expected to fail in this run, would be clear by next run05:51
quiquell|roverhumm05:52
quiquell|rovernot all05:52
quiquell|roverocata will fail too05:53
ykarelyes :( and pike already failed05:53
ykarellooks like still bug not created for pike05:53
ykarelpike is red since 9 days05:54
ykarelhttp://rhos-release.virt.bos.redhat.com:3030/rhosp05:54
ykarelIt would be good to create bug and escalate it05:54
quiquell|roverykarel: There were unstabilities yesterday in the openstack infra05:55
ykarelquiquell|rover, pike jobs running in rdo cloud05:56
ykareland As far as i checked, jobs have real issue05:56
ykarelNow we have some issue with te-broker :(05:58
ykarel+(/opt/stack/new/tripleo-ci/toci_gate_test.sh:257): ./testenv-client -b 192.168.103.254:4730 -t 17400 --envsize 4 --ucinstance 41057063-4290-45e2-81bd-5dd9b590b344 --net-iso multi-nic -- ./toci_quickstart.sh05:58
ykarel+(/opt/stack/new/tripleo-ci/toci_gate_test.sh:251): sleep 120005:58
ykarel2018-04-26 05:39:19,418 - testenv-client - INFO - Received job : Couldn't retrieve env05:58
quiquell|roverykarel: Give me a minute05:58
quiquell|roverWill go back with a bug or two05:58
ykarelall OVB jobs failed in master, so we have to wait for promotion05:59
quiquell|rovermaster is pretty screw too05:59
ykarelHoping to get this(te-broker thing) fixed by next run06:00
ykarelmay be quota issue06:00
quiquell|roverwe have been without running the promotion jobs for 4 days...06:00
quiquell|roverNow we see the status of it06:00
quiquell|roverThey where blocked at container build job06:00
ykarelquiquell|rover, talking about pike?06:01
quiquell|rovermaster06:01
ykarelOkk06:01
*** jfrancoa has joined #oooq06:03
ykarelAll OVB check/promotion jobs red:- https://review.rdoproject.org/zuul/06:03
ykarellast time panda|ruck|off fixed it06:03
*** ykarel is now known as ykarel|afk06:08
*** d0ugal has joined #oooq06:14
*** jtomasek has joined #oooq06:18
*** jaganathan has quit IRC06:22
*** kopecmartin has joined #oooq06:24
*** jaganathan has joined #oooq06:38
hubbot`All check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.06:41
*** amoralej|off is now known as amoralej06:55
*** tesseract has joined #oooq06:58
*** tesseract has quit IRC07:00
*** tesseract has joined #oooq07:02
*** ccamacho has joined #oooq07:03
*** udesale has joined #oooq07:03
*** jtomasek has quit IRC07:05
*** quiquell|rover is now known as quique|rover|afk07:06
*** holser__ has joined #oooq07:07
quique|rover|afkarxcruz, chandankumar: https://bugs.launchpad.net/tripleo/+bug/176704907:12
openstackLaunchpad bug 1767049 in tripleo "Error generating tempest.conf 'network_feature_enabled': NoSectionError: No section: 'network_feature_enabled'" [Critical,New] - Assigned to Quique Llorente (quiquell)07:12
quique|rover|afkShit have to change the tittle07:12
*** ykarel|afk is now known as ykarel07:12
*** bogdando has joined #oooq07:14
ykarelquique|rover|afk, added comment to the bug07:19
quique|rover|afkok07:20
*** tosky has joined #oooq07:20
*** udesale has quit IRC07:24
quique|rover|afkykarel: It doesn't affect pike or ocata ?07:39
ykarelnope, as tempest plugins are introduced since queens07:40
*** ykarel is now known as ykarel|lunch07:42
*** gkadam has quit IRC07:56
quique|rover|afkykarel|lunch: Doing a nodepool cleanup08:08
*** ccamacho has quit IRC08:08
*** ccamacho has joined #oooq08:11
*** lucas-afk is now known as lucasagomes08:15
*** florianf has joined #oooq08:15
*** quique|rover|afk is now known as quiquell|rover08:19
*** sshnaidm|off is now known as sshnaidm08:19
*** ykarel|lunch is now known as ykarel08:20
ykarelquiquell|rover, cool08:20
*** gkadam has joined #oooq08:20
*** moguimar_ has quit IRC08:20
ykarelquiquell|rover, do you know why these are not cleaned up automatically08:21
ykarelas we are seeing it again and again and it affects both gate/promotion jobs08:21
chandankumarykarel: quiquell|rover new package gabbi is built in fedora08:21
chandankumaramoralej: is working on getting it imported in cbs08:21
ykarelyup we need that08:22
quiquell|roverykarel: Somatinas the cleanup itself fails08:22
ykarelahh08:22
quiquell|roverbut normally there is a cleanup at the end08:22
*** florianf has quit IRC08:22
arxcruzquiquell|rover: hi do i need to take a look on that or chandankumar already did it? sorry, went to office today, pretty hard to find a seat08:22
quiquell|roverarxcruz: I think chandankumar already did it08:23
chandankumararxcruz: gabbi is already in progress08:23
arxcruzchandankumar: let me know what you did :)08:23
chandankumarykarel: does pinning does not fixed the issue?08:23
arxcruzis it packaging issue ?08:23
ykarelchandankumar, jobs are passing08:23
chandankumararxcruz: we need a higher version of gabbi08:24
*** jaosorior has joined #oooq08:24
arxcruzchandankumar: want to walk me through ?08:24
chandankumararxcruz: more info is here https://trello.com/c/e0f3aGDn/721-build-python-gabbi-to-1421-for-heat-tempest-plugin08:24
chandankumararxcruz: in order to avoid breakage for other plugins in queens, we pinned all the tempest plugins at known commit which were passed in ci08:26
arxcruzchandankumar: got it08:26
chandankumararxcruz: we will make you packager in future so be prepared :-)08:27
arxcruzchandankumar: I born ready08:27
chandankumar:-)08:28
chandankumarquiquell|rover: panda|ruck|off this review might be helpful for you in debugging tempest container issues https://review.openstack.org/#/c/564156/08:30
chandankumarfor future08:30
arxcruzadarazs: https://review.openstack.org/#/c/561920/ pleeeeeeeeaaaaaaaaaaseeeeee?@?!?!?! :D08:32
quiquell|roverykarel: nodepool is clean now08:34
ykarelquiquell|rover, Good08:35
quiquell|roverLet's see if it helps with the testenv timeout08:35
ykarelyes now queens fixes are also in place, let's see how next run goes08:37
quiquell|roverlooking at the pingtest_stack problem in queens08:38
arxcruzsshnaidm: around ?08:39
hubbot`All check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.08:41
quiquell|roverpike's pingtest stack problem08:45
quiquell|roverResourceInError: resources.server1: Went to status ERROR due to "Message: No valid host was found. , Code: 500"08:45
*** saneax has quit IRC08:45
quiquell|roverCould be a quota problem ?08:46
*** udesale has joined #oooq08:46
*** florianf has joined #oooq08:50
*** udesale has quit IRC08:51
weshay%gatestatus08:52
hubbot`All check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.08:52
*** udesale has joined #oooq08:55
quiquell|roverpike's blocker09:01
quiquell|roverhttps://bugs.launchpad.net/tripleo/+bug/176707609:01
openstackLaunchpad bug 1767076 in tripleo "Creating pingtest_sack fails: Failed to schedule instances: NoValidHost_Remote: No valid host was found" [Critical,New] - Assigned to Quique Llorente (quiquell)09:01
*** panda|ruck|off is now known as panda|ruck09:10
*** moguimar has joined #oooq09:11
*** florianf has quit IRC09:12
quiquell|roverFound " Test ping across vxlan tunnel (and optionally restart ovs)] **"09:16
quiquell|roverAt one of the periodics at master...09:16
quiquell|roverThe same stuff I got yesterday in a reproducer...09:16
quiquell|rovergood morning panda|ruck09:20
panda|ruckno no09:21
panda|ruckI took a look at the dashboard. I'm going back to bed09:21
quiquell|roverTake him take him quick !!!09:21
quiquell|roverDon't know where to start09:22
quiquell|roverDo you mind some bj ?09:22
quiquell|roverWE have also te-broker timeouts09:23
quiquell|roverafter doing a cleanup09:23
panda|ruckquiquell|rover: ok bj09:23
* quiquell|rover openning09:23
quiquell|roverhttps://bluejeans.com/789106523209:24
arxcruzpanda|ruck: https://review.openstack.org/#/c/561920/ :D09:27
*** chem has joined #oooq09:27
*** zoli is now known as zoli|lunch09:32
*** dtantsur|afk is now known as dtantsur09:38
*** jtomasek has joined #oooq09:42
*** jaosorior has quit IRC09:49
panda|ruckis anyone still seeing testenv timeout after the cleanup ?09:53
*** udesale has quit IRC09:54
panda|ruckykarel: are you working on anything like this ? https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master/a120df1/console.txt.gz#_2018-04-26_06_40_50_17810:05
panda|ruckykarel: I'm going to create a bug10:06
ykarelpanda|ruck, can't this be covered on the bug you created yesterday10:06
ykarelnetwork random issues one10:06
quiquell|roverykarel: This new one is on RDO infra10:06
*** ccamacho has quit IRC10:07
ykarelOkk, btw i have seen this locally last week10:08
ykarelusing reproducer script10:08
ykarelbut didn't investigated it, just saw that ping was working after some time10:08
*** ccamacho has joined #oooq10:08
*** ccamacho has quit IRC10:08
*** ccamacho has joined #oooq10:08
adarazsarxcruz: I'd rather not merge stuff anymore that can break things :)10:09
*** hubbot` is now known as hubbot10:09
arxcruzadarazs: np <310:09
*** saneax-_-|AFK has joined #oooq10:09
*** saneax-_-|AFK is now known as saneax10:09
ykarelalso zuul queue is large, so good not to approve non critical fixes until it settles down :)10:10
panda|ruckykarel: do you know when config download was introduced as default for the undercloud installation ?10:12
ykarelpanda|ruck, no idea10:13
ykareleven i don't know that config download is used for undercloud, i was in impression that config download is used for overcloud installation10:13
ykarelonly10:13
panda|ruckykarel: when you said "i have seen this locally"  what did you mean  ?10:16
ykarelpanda|ruck, reproducing a job on rdocloud using reproducer script10:17
quiquell|roverykarel: Me too let's if we both have the issue10:18
*** apetrich has quit IRC10:18
panda|ruckok so this is less random10:18
*** holser__ has quit IRC10:26
quiquell|roverpanda|ruck: Doing a mnual delete at my RDO cloud10:31
quiquell|roverailed: (500) Request to http: //172.16.0.9: 9696/v2.0/ports/94582093-72a6-4499-98e4-3764f1b5fdb3.json timed out" |10:31
panda|ruckquiquell|rover: https://bugs.launchpad.net/tripleo/+bug/176709910:32
openstackLaunchpad bug 1767099 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset030-master vxlan tunnel fails randomly" [Critical,Triaged] - Assigned to Quique Llorente (quiquell)10:32
quiquell|roverpanda|ruck: ack10:32
panda|ruckquiquell|rover: mmmhh10:33
*** udesale has joined #oooq10:37
hubbotAll check jobs are working fine on stable/queens, stable/ocata, stable/ocata, master.10:41
*** zoli|lunch is now known as zoli10:42
*** udesale_ has joined #oooq10:45
*** udesale has quit IRC10:47
panda|ruck%stop-lying10:47
hubbotpanda|ruck: Error: "stop-lying" is not a valid command.10:47
adarazspanda|ruck: the new dummy gate jobs were not rechecked during the night10:48
panda|ruckadarazs: so they are filtered out by the time limit10:48
adarazsyep.10:48
adarazspanda|ruck: because they were still trying to query stuff with my username, for which the machine didn't have the key.10:48
adarazspanda|ruck: now I reenabled the ansible-pull on the machine, it took the merged changes that fix the username, regenerated the gate-check.sh script and it should work.10:49
adarazsdetails here: https://trello.com/c/iWAz4ONC/718-hubbot-bot-add-two-dummy-changes-on-tht-to-watch :)10:49
panda|ruckadarazs: yea I was watching exactly that10:50
panda|ruckadarazs: can we close this card then ?10:50
adarazspanda|ruck: let's wait for myoung to get in, it's his card. I closed the main hubbot card yesterday10:50
*** yolanda_ has joined #oooq10:53
*** yolanda__ has joined #oooq10:54
*** yolanda has quit IRC10:56
*** yolanda_ has quit IRC10:58
*** udesale_ has quit IRC11:01
*** holser__ has joined #oooq11:02
quiquell|rovervxlan ping failed11:09
quiquell|roverykarel: Did you have the same issue ?11:09
panda|ruckquiquell|rover: do you have a live environment we could inspect ?11:11
*** holser__ has quit IRC11:11
quiquell|roversure11:11
*** holser__ has joined #oooq11:11
quiquell|roverzuul@38.145.33.16311:12
quiquell|roverbj ?11:12
panda|ruckuuuh zuul queue is at 3 hours now11:12
quiquell|rovergo go go go11:13
panda|ruckquiquell|rover: permission denied11:13
quiquell|roverLet me add you public key11:13
quiquell|roverredpanda11:14
quiquell|rover:-)11:14
quiquell|roverTry now11:14
quiquell|roverdo a tmux attach11:15
quiquell|roverthere is already one going on11:15
panda|ruckquiquell|rover: you put the one available in rdo-infa/ci-config ?11:16
quiquell|rovernope11:16
quiquell|roverJust ssh promoter11:16
quiquell|roverAnd they the one there11:16
quiquell|roverI don't even know where they are11:16
quiquell|roveryou in ?11:17
panda|ruckno11:17
quiquell|roverhttps://bluejeans.com/789106523211:18
*** udesale_ has joined #oooq11:20
sshnaidmpanda|ruck, trown|outtypewww please review, just rebased: https://review.openstack.org/#/c/563455/11:22
*** lucasagomes is now known as lucas-hungry11:24
*** dtantsur has quit IRC11:39
*** dtantsur has joined #oooq11:44
*** dtantsur has quit IRC11:44
*** dtantsur has joined #oooq11:44
*** dtantsur has quit IRC11:44
*** dtantsur has joined #oooq11:49
*** dtantsur has quit IRC11:49
*** dtantsur has joined #oooq11:49
*** dtantsur has quit IRC11:49
*** dtantsur has joined #oooq11:54
chandankumarsshnaidm: EmilienM arxcruz it is the current set of tests which runs on undercloud http://logs.openstack.org/18/564118/3/check/tripleo-ci-centos-7-undercloud-containers-tempest/fd3ba29/logs/tempest.html.gz11:59
chandankumarwithout tweak11:59
arxcruzchandankumar: cool11:59
sshnaidmchandankumar++12:00
adarazshmm, why no karma plugin, hubbot?12:01
adarazs%karma most12:02
hubbotadarazs: (karma most [<channel>] {increased,decreased,active}) -- Returns the most increased, the most decreased, or the most active (the sum of increased and decreased) karma things. <channel> is only necessary if the message isn't sent in the channel itself.12:02
adarazs%karma most #oooq12:02
hubbotadarazs: (karma most [<channel>] {increased,decreased,active}) -- Returns the most increased, the most decreased, or the most active (the sum of increased and decreased) karma things. <channel> is only necessary if the message isn't sent in the channel itself.12:02
adarazshmm.12:02
adarazs%karma most #oooq active12:02
hubbotadarazs: "chandankumar": 112:02
adarazsah okay, so it did keep score, just not being vocal about it12:02
sshnaidmadarazs, where to fix? ::)12:03
adarazs%supybot.plugins.Karma.response true12:04
hubbotadarazs: Error: "supybot.plugins.Karma.response" is not a valid command.12:04
adarazs%config supybot.plugins.Karma.response true12:04
hubbotadarazs: The operation succeeded.12:04
adarazssshnaidm++12:04
hubbotadarazs: sshnaidm's karma is now 112:04
adarazsyay :)12:04
adarazs%flush12:04
hubbotadarazs: The operation succeeded.12:04
chandankumarsshnaidm: I have one question how we can make a featureset to run different tempest tests once we gate fs052 against different upstream project?12:04
sshnaidmadarazs, cool!12:05
chandankumarsshnaidm: for example if fs052 is used against heat it will run only heat tests , if gated against mistral it will trigger only mistral tempest tests12:05
panda|ruckpanda--12:05
hubbotpanda|ruck: panda's karma is now -112:05
*** atoth has joined #oooq12:06
sshnaidmchandankumar, good question.. maybe we can add tests list as a variable in job definition and check it in pre.yml playbook12:06
adarazspanda++ :)12:07
hubbotadarazs: panda's karma is now 012:07
quiquell|roverhubbot++ :-)12:07
hubbotquiquell|rover: Error: ":-)" is not a valid command.12:07
quiquell|roverThat's a feature12:07
adarazs%config supybot.plugins.Karma.onlyNicks true12:08
hubbotadarazs: The operation succeeded.12:08
sshnaidmi++12:08
adarazsnow it will only allow changing karma for currently present nicks :D12:08
*** tcw has quit IRC12:08
adarazssshnaidm: just a second late :D12:08
chandankumarhubbot+-/12:08
sshnaidmgood12:08
sshnaidmsshnaidm++--12:08
sshnaidmsshnaidm++ sshnaidm--12:09
hubbotsshnaidm: Error: You're not allowed to adjust your own karma.12:09
adarazsbut this is interesting actually, not sure how I hacked in the inline karma change...12:10
adarazssshnaidm++ sshnaidm--12:10
hubbotadarazs: sshnaidm's karma is now 212:10
hubbotadarazs: sshnaidm's karma is now 112:10
adarazsheh, works as expected :)12:10
*** sshnaidm is now known as notsshnaidm12:10
notsshnaidmsshnaidm++ sshnaidm--12:10
*** notsshnaidm is now known as nosshnaidm12:10
*** nosshnaidm is now known as sshnaidm12:10
adarazsnow this is relatively foolproof :)12:10
sshnaidmadarazs++++12:11
*** tcw has joined #oooq12:13
quiquell|roverpanda|ruck: Testing it with the review12:14
panda|ruckquiquell|rover: cool12:15
chandankumarhubbot++,--12:15
*** apetrich has joined #oooq12:16
quiquell|roverAlso the +2 in the sunode doesn't look good12:17
* adarazs is happy about hubbot's resilience to karma abuse. :)12:19
chandankumarhubbot+212:19
chandankumarit should complain but hubbot is not.12:20
quiquell|roverpanda|ruck: Going to ask Bogdan Dobrelya about the +212:21
panda|ruckquiquell|rover: ok , where in he code is that + 2 ?12:21
panda|ruckcan't find it12:21
quiquell|rovertq: roles/tripleo-inventory/tasks/inventory.yml12:22
quiquell|roverhttps://github.com/openstack/tripleo-quickstart/blob/master/roles/tripleo-inventory/tasks/inventory.yml#L5112:22
panda|ruckquiquell|rover: be sure to read commit fdaa9748e6dd6a3372b63ec8b0d31c6ccd89fa6912:22
panda|ruckquiquell|rover: anyway bogdando here is the droid you're looking for12:23
quiquell|roverbogdando++12:24
hubbotquiquell|rover: bogdando's karma is now 112:24
quiquell|roverbogdando: Do you know why we have a +2 here ? https://github.com/openstack/tripleo-quickstart/blob/master/roles/tripleo-inventory/tasks/inventory.yml#L5112:24
quiquell|roverIt's creating bad ansible inventiroy names for multinode (It's just confusing in the log not an issue)12:25
chandankumarsshnaidm: you mean something like http://git.openstack.org/cgit/openstack-infra/zuul/tree/.zuul.yaml#n2012:28
chandankumarin the gated project itself?12:29
*** lucas-hungry is now known as lucasagomes12:30
sshnaidmchandankumar, worth to check if we can pass vars to job, then we can save list of tests in tested projects12:32
*** udesale_ has quit IRC12:32
bogdandoquiquell|rover: nope, I don12:33
bogdandodo not12:33
*** trown|outtypewww is now known as trown12:34
quiquell|roverDamn the reproducer doesn't use the zuul_changes12:35
*** rlandy has joined #oooq12:35
panda|ruckquiquell|rover: yeah, I've just opened a card for that12:35
quiquell|roverI think I will take it, it's a feasible task for me12:36
*** kopecmartin has quit IRC12:38
*** kopecmartin has joined #oooq12:39
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/56429112:41
quiquell|roverbogdando: So there is no relation between the inventory name and the multinode hostnames ?12:48
quiquell|roverjust the word subnode was used in both12:49
bogdandoI'm not authored these, I can't answer that question tbh12:49
quiquell|roverAhhh, ok, will find the correct author sorry12:50
*** quiquell|rover is now known as quique|rover|foo12:50
*** quique|rover|foo is now known as quique|rover|lch12:55
arxcruzsshnaidm: https://review.openstack.org/#/c/564429/13:05
*** zoli is now known as zoli|afk13:06
*** Goneri has joined #oooq13:08
myoungchandankumar: do you have a moment to discuss tempest with DCI folk?13:10
myoungchandankumar: https://bluejeans.com/4113567798 if you're available13:10
chandankumarmyoung: yup13:11
*** ratailor has quit IRC13:17
*** udesale_ has joined #oooq13:27
*** quique|rover|lch is now known as quiquell|rover13:29
quiquell|roverOcata stopper merged13:30
quiquell|roverhttps://review.openstack.org/#/c/563843/13:30
*** jaosorior has joined #oooq13:35
quiquell|roverI have an ansible command13:44
quiquell|roverAnd a question13:44
quiquell|roverUsing --skip-tag and a playbook with a "add_host" fails creating the host13:46
quiquell|roverAny clue ?13:46
*** skramaja has quit IRC13:47
rascarlandy, myoung, panda|ruck hey folks, can you help me in giving a push to the rhos-13 pipelines https://code.engineering.redhat.com/gerrit/#/c/136144/ and to fix the RDOPhase2 deployments https://code.engineering.redhat.com/gerrit/#/c/137005/ ?13:47
myoungo/ scrum time14:00
myoungrasca: will look after scrum14:00
myoungsshnaidm, rlandy, weshay: scrum14:02
panda|ruckrfolco:14:02
myoungrfolco:14:02
myoungquiquell|rover: if you have a moment please also join for a min14:02
quiquell|roverok14:03
quiquell|roverGive me a min14:03
rfolcoo/ joined myoung14:03
*** jaosorior has quit IRC14:12
quiquell|roverpanda|ruck: vxlan ping working after stack change14:12
quiquell|roverThe patch should fix the issue14:12
*** jaosorior has joined #oooq14:13
*** zoli|afk is now known as zoli14:15
*** zoli is now known as zoli|wfh14:15
*** zoli|wfh is now known as zoli14:15
panda|ruck\o/14:15
arxcruzmyoung: chandankumar internet here in the office is down, i'm heading to home, hopefully i'll be back in time for the meeting14:24
*** holser__ has quit IRC14:40
sshnaidmhrybacki, send me please your last reproducer script you use14:40
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/56429114:41
chandankumararxcruz: ack!14:43
*** holser__ has joined #oooq14:45
hrybackisshnaidm: ack: http://pastebin.test.redhat.com/58276214:45
*** quiquell|rover is now known as quiquell|off14:47
rlandytrown: myoung: panda|ruck: one pother point - there is one addition in the snapshot reproducer review we may want to keep - and that is the option to define the libvirt_volume_path as a direct option within the reproducer14:48
rlandyit's enough of a issue that we are all using custom dirs in our workflows14:48
jfrancoapanda|ruck: o/ do you have a moment for a small concern?14:48
rlandywe can pick that up next week14:48
*** apetrich has quit IRC14:49
trownrlandy: ya makes sense14:49
jfrancoapanda|ruck: I've got a liter of gazpacho to payback :-D14:49
myoungrlandy: ack, I was debating pulling that into the "make POC nicer" review(s) anyway14:50
panda|ruckrlandy: trown cmoved the review under https://trello.com/c/pc1GEwpH/722-integrate-the-snapshotting-role-to-the-reproducer-script14:50
*** udesale_ has quit IRC14:50
rlandymyoung: ok - the review is out there - we can decide what exactly to do with it next week14:50
myoungrlandy, trown: or, just keeping the defaults (removing the override) so it's {{working_dir}}/pool like normal TQ14:51
panda|ruckjfrancoa: shoot14:51
rlandymyoung: pls comment on the review - makes it easier to see14:51
myoungrlandy: will do (after tempest squad)14:51
jfrancoapanda|ruck: so it called my attention the result of the tripleo-ci-centos-7-undercloud-upgrades job in this patch from stable/ocata: https://review.openstack.org/#/c/557353/14:51
* myoung is prepping for next scrum14:52
jfrancoapanda|ruck: the thing is that tripleo-upgrade only has three branches, stable/pike, stable/queens and master14:52
jfrancoapanda|ruck: so what is zuul downloading in this case? as the patch is from stable/ocata? seems to me that it's getting master release14:53
panda|ruckjfrancoa: master14:53
panda|ruckjfrancoa: it's the default fallback14:54
panda|ruckjfrancoa: the panic option for devstack-gate14:54
jfrancoapanda|ruck: I see, yes, makes sense. Then we'll have to improve the tasks in master so they don't fall this way14:54
jfrancoapanda|ruck: I mean in tripleo-upgrade14:55
jfrancoapanda|ruck: thanks for the info14:55
panda|ruckjfrancoa: or create stable/ocata branch14:55
*** jaosorior has quit IRC14:55
jfrancoapanda|ruck: well, yes, that could be an option...but how long more are we going to have ocata upstream?I think it's faster to "workaround" it and focus on the next releases14:56
myoungchandankumar, arxcruz, weshay: scrum15:01
-openstackstatus- NOTICE: We've successfully troubleshooted the issue that prevented paste.openstack.org from loading and it's now back online, thank you for your patience.15:04
*** links has quit IRC15:06
*** saneax is now known as saneax-_-|AFK15:16
*** holser__ has quit IRC15:17
*** kopecmartin has quit IRC15:21
panda|ruckno master promotion today15:36
ykarelpanda|ruck, seeing future?15:37
ykareljobs are still running, are they going to fail?15:37
panda|ruckykarel: see #rdo15:38
panda|ruckoh15:38
panda|ruckuhm15:38
panda|ruckwierd15:38
panda|ruckykarel: build 451 was run on the previous promotion pipeline15:39
ykarelpanda|ruck, ack15:39
panda|ruckykarel: but 453 is the one runnig with this promotion15:39
panda|ruck??15:39
panda|ruckwho run 452 ?15:40
ykarelno idea15:40
panda|ruckthat's the one taht failed with the slave error15:40
panda|ruckso I assumed it was for this promotion run15:40
ykarelOk, so we still have hope :)15:41
*** ykarel is now known as ykarel|away15:43
Gonerirlandy, I think the downstream branches are not imported anymore: https://code.engineering.redhat.com/gerrit/gitweb?p=tripleo-quickstart-extras.git;a=summary15:43
myoungrasca: https://code.engineering.redhat.com/gerrit/#/c/137005 merged, looking at updated PS for https://code.engineering.redhat.com/gerrit/#/c/136144/15:45
rascamyoung, ack thanks, at least with the first one we should have master/queens/pike rdophase2 bm jobs green again15:46
rascamyoung, about the second one I *think* we're good, but double check15:46
myoungrasca: for the second one looks great, just need to create rhos-13 branchs for internal TQ/TQE...want me to do that now/15:48
myounger...I'll do that now15:48
rascamyoung, what this creation is in practice?15:49
myoungrasca: only takes a second (just running that script) but you need push rights to ooo-env.  want to jump on my BJ or tmate and we can do together?  can show15:49
myoungor you can do and I'll be a cheerleader15:50
myoung:)15:50
rascamyoung, sure15:50
myoungbluejeans.com/matyoung15:50
*** ykarel|away has quit IRC15:50
arxcruzmyoung: just arrive at home, update the trello board15:52
*** gkadam has quit IRC15:53
*** hamzy has quit IRC15:59
*** d0ugal has quit IRC16:03
*** d0ugal has joined #oooq16:04
*** marios has quit IRC16:07
rlandyGoneri: hi - we have to import them if the weekly jobs fail16:08
rlandymyoung: ^^16:08
*** agopi|zzz has quit IRC16:11
*** jtomasek has quit IRC16:14
*** bogdando has quit IRC16:15
*** agopi|zzz has joined #oooq16:34
*** lucasagomes is now known as lucas-afk16:38
panda|ruckconfirmed no master promotion, at least fs020 failed16:40
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/56429116:41
sshnaidmcores, please take a look https://review.openstack.org/#/c/56447216:42
*** panda|ruck is now known as panda|ruck|off16:49
*** jaganathan has quit IRC16:53
myoungpanda|ruck|off: just missed ya.  looks like we have an issue with internal jenkins (where rdo2 and perf ci stuff runs)16:54
myoungpanda|ruck|off, quiquell|off, the push phase of https://code.engineering.redhat.com/gerrit/#/c/136144 failed with SSL error, jjb push happens from master node, and pushes to itself (over https).  as you folks are already offline I'll open a card, log the IT ticket, etc.  this was discovered just now on rasca's patch that adds osp13 pipelines that use TQ16:55
*** agopi|zzz has quit IRC16:55
* myoung goes hunting for calories and will be back in a bit16:56
*** agopi|zzz has joined #oooq17:10
*** dtantsur is now known as dtantsur|afk17:10
*** hamzy has joined #oooq17:12
*** trown is now known as trown|lunch17:13
*** ykarel|away has joined #oooq17:26
*** ykarel|away has quit IRC17:33
*** apetrich has joined #oooq17:42
*** zoli is now known as zoli|gone17:43
*** zoli|gone is now known as zoli17:43
*** amoralej is now known as amoralej|off17:46
*** jfrancoa has quit IRC17:51
rascamyoung, hey again, so we *might* need also this review here https://code.engineering.redhat.com/gerrit/#/c/137052/ to fix bm HA rdop2 jobs, but I'm testing it right now, if it's good, I'll ask you for a merge.18:08
rascamyoung, I'm still not sure if we need fields like DockerNamespace, DockerInsecureRegistryAddress and so on in net-env yaml, but I remember we distinctly need it before, so maybe that's the case also here. I'll let you know once I have the result of this last test job I'm running18:10
*** trown|lunch is now known as trown18:16
*** tesseract has quit IRC18:24
*** chem has quit IRC18:35
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/56429118:41
myoungrasca: ack19:00
rascamyoung, verified, if you can merge it would be great, thanks!19:06
myoungrasca: done19:11
rascamyoung, thanks man19:11
myoungrasca: nearly done with the ticket19:11
myoungwill put you on notify list along with ruck/rover (panda|ruck|off, quiquell|off)19:12
*** agopi|zzz has quit IRC19:14
rascamyoung, ack, thanks, tomorrow I'll do a check to test if we can omit the Docker stuff from the net-env yaml, but in the meantime we should be good19:14
rascamyoung, hopefully I'll start having green again on those pipelines and concentrate more on the tests19:14
myoungrasca: agreed, thanks!19:14
myoungrasca, quiquell|off, panda|ruck|off: PNT0229661 has been logged for the internal jenkins issue impacting rdo2 / osp0 / browbeat19:25
rlandymyoung: hello19:25
rlandyquestion re: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/gate-quickstart/library/jenkins_deps.py19:26
myoungrlandy: o/19:26
rlandymyoung: did you just copy that over from build_test_packages?19:26
myoungrlandy: I'm looking at this for (I think) the first time...19:26
myounghttps://github.com/openstack/tripleo-quickstart-extras/commits/master/roles/gate-quickstart/library/jenkins_deps.py19:27
rlandymyoung: sorry - mixed you up19:27
rlandyignore me19:27
rlandywring person19:27
rlandywrong person19:27
myoung'sok...what's the context?19:28
rlandymyoung: just trying to fix https://trello.com/c/caCmSItG/720-add-tq-and-tqe-as-changes-for-the-reproducer-to-test19:30
*** atoth has quit IRC20:29
*** atoth has joined #oooq20:29
*** agopi|zzz has joined #oooq20:37
*** yolanda__ has quit IRC20:38
*** sshnaidm is now known as sshnaidm|off20:39
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container @ https://review.openstack.org/56044520:41
*** yolanda__ has joined #oooq20:43
*** hamzy has quit IRC20:45
*** trown is now known as trown|outtypewww20:56
*** Goneri has quit IRC21:16
*** holser__ has joined #oooq21:51
*** agopi|zzz has quit IRC21:52
*** holser__ has quit IRC21:55
*** rlandy is now known as rlandy|afk22:19
*** agopi|zzz has joined #oooq22:31
hubbotFAILING CHECK JOBS on stable/ocata: openstack-tox-pep8, tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container @ https://review.openstack.org/56044522:41
*** agopi|zzz has quit IRC22:44
*** rlandy|afk is now known as rlandy23:48
*** tosky has quit IRC23:49

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!